2024-03-13T14:22:47.905 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-13T14:22:47.905 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --owner scheduled_vshankar@teuthology --verbose --archive /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645 --name vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645/orig.config.yaml 2024-03-13T14:22:47.960 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T14:22:48.049 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645 branch: wip-vshankar-testing-20240307.013758 description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: vshankar@redhat.com first_in_suite: false job_id: '7596645' kernel: flavor: default kdb: true sha1: a87fde6e633694345631fcca2fd8a5f190eef61c last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.1 ' name: vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240307.013758 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240307.013758 sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 owner: scheduled_vshankar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 4793 sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 sleep_before_teardown: 0 subset: 809/1024 suite: fs suite_branch: wip-vshankar-testing-20240307.013758 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 targets: smithi176.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYixE/TymHz03c2MtipOqrDOT/TE1wqB3Z88lRmj6FOWijYZm6wZF6ZbWYc2BzehhIX/LdXGmQ6uX7b3J+Hq/0= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6dkZT6B+RIDW36bsfZxql7s/g7tLqsKK84frAp/Kbp1jAMQfqcwGneZIsKaMmaG9O225hbvZCTcpVJz9sEoWU= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.1 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.1 roleless: true - print: '**** done end installing v18.2.1 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-13_13:59:32 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-03-13T14:22:48.049 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa; will attempt to use it 2024-03-13T14:22:48.050 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks 2024-03-13T14:22:48.050 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-13T14:22:48.052 INFO:teuthology.task.internal:Checking packages... 2024-03-13T14:22:48.076 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9a75c81fd074f3cb252248b860d5cc88e5f3fb55' 2024-03-13T14:22:48.076 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-13T14:22:48.076 INFO:teuthology.packaging:ref: None 2024-03-13T14:22:48.076 INFO:teuthology.packaging:tag: None 2024-03-13T14:22:48.076 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240307.013758 2024-03-13T14:22:48.076 INFO:teuthology.packaging:sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:22:48.076 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240307.013758 2024-03-13T14:22:48.221 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1916.g9a75c81f 2024-03-13T14:22:48.223 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-13T14:22:48.229 INFO:teuthology.task.internal:no buildpackages task found 2024-03-13T14:22:48.229 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-13T14:22:48.239 INFO:teuthology.task.internal:Saving configuration 2024-03-13T14:22:48.259 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-13T14:22:48.264 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-13T14:22:48.289 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 14:17:55.269420', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYixE/TymHz03c2MtipOqrDOT/TE1wqB3Z88lRmj6FOWijYZm6wZF6ZbWYc2BzehhIX/LdXGmQ6uX7b3J+Hq/0='} 2024-03-13T14:22:48.309 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 14:17:55.271188', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6dkZT6B+RIDW36bsfZxql7s/g7tLqsKK84frAp/Kbp1jAMQfqcwGneZIsKaMmaG9O225hbvZCTcpVJz9sEoWU='} 2024-03-13T14:22:48.310 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-13T14:22:48.315 INFO:teuthology.task.internal:roles: ubuntu@smithi176.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-03-13T14:22:48.315 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-03-13T14:22:48.315 INFO:teuthology.run_tasks:Running task console_log... 2024-03-13T14:22:48.394 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f343d5e9160>, signals=[15]) 2024-03-13T14:22:48.394 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-13T14:22:48.400 INFO:teuthology.task.internal:Opening connections... 2024-03-13T14:22:48.400 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:22:48.402 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:22:48.491 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-03-13T14:22:48.492 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:22:48.583 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-13T14:22:48.588 DEBUG:teuthology.orchestra.run.smithi176:> uname -m 2024-03-13T14:22:48.606 INFO:teuthology.orchestra.run.smithi176.stdout:x86_64 2024-03-13T14:22:48.606 DEBUG:teuthology.orchestra.run.smithi176:> cat /etc/os-release 2024-03-13T14:22:48.662 INFO:teuthology.orchestra.run.smithi176.stdout:NAME="CentOS Stream" 2024-03-13T14:22:48.662 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION="9" 2024-03-13T14:22:48.662 INFO:teuthology.orchestra.run.smithi176.stdout:ID="centos" 2024-03-13T14:22:48.662 INFO:teuthology.orchestra.run.smithi176.stdout:ID_LIKE="rhel fedora" 2024-03-13T14:22:48.662 INFO:teuthology.orchestra.run.smithi176.stdout:VERSION_ID="9" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:PLATFORM_ID="platform:el9" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:ANSI_COLOR="0;31" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:LOGO="fedora-logo-icon" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:HOME_URL="https://centos.org/" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T14:22:48.663 INFO:teuthology.orchestra.run.smithi176.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T14:22:48.664 INFO:teuthology.lock.ops:Updating smithi176.front.sepia.ceph.com on lock server 2024-03-13T14:22:48.683 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-03-13T14:22:48.701 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-03-13T14:22:48.702 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-13T14:22:48.759 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-03-13T14:22:48.760 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-13T14:22:48.760 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-13T14:22:48.760 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-13T14:22:48.760 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-03-13T14:22:48.783 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-13T14:22:48.792 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-13T14:22:48.797 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-13T14:22:48.797 DEBUG:teuthology.orchestra.run.smithi176:> test '!' -e /home/ubuntu/cephtest 2024-03-13T14:22:48.799 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-03-13T14:22:48.815 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-13T14:22:48.821 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-13T14:22:48.821 DEBUG:teuthology.orchestra.run.smithi176:> test -z $(ls -A /var/lib/ceph) 2024-03-13T14:22:48.856 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-03-13T14:22:48.878 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-13T14:22:48.936 INFO:teuthology.run_tasks:Running task kernel... 2024-03-13T14:22:48.951 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'a87fde6e633694345631fcca2fd8a5f190eef61c'} 2024-03-13T14:22:48.951 INFO:teuthology.task.kernel:config {'host.a': {'flavor': 'default', 'kdb': True, 'sha1': 'a87fde6e633694345631fcca2fd8a5f190eef61c'}, 'host.b': {'flavor': 'default', 'kdb': True, 'sha1': 'a87fde6e633694345631fcca2fd8a5f190eef61c'}}, timeout 300 2024-03-13T14:22:48.952 DEBUG:teuthology.orchestra.run.smithi176:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-13T14:22:48.952 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-13T14:22:48.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T14:22:48.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T14:22:48.969 DEBUG:teuthology.task.kernel:sha1 for host.a is a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:22:48.969 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "a87fde6e633694345631fcca2fd8a5f190eef61c"... 2024-03-13T14:22:48.969 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-03-13T14:22:48.970 DEBUG:teuthology.task.kernel:sha1 for host.b is a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:22:48.970 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "a87fde6e633694345631fcca2fd8a5f190eef61c"... 2024-03-13T14:22:48.970 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-03-13T14:22:49.025 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-361.el9.x86_64 2024-03-13T14:22:49.025 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:22:49.026 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-13T14:22:49.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:22:49.029 INFO:teuthology.orchestra.run.smithi176.stdout:5.14.0-361.el9.x86_64 2024-03-13T14:22:49.029 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:22:49.030 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-13T14:22:49.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:22:49.175 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum upgrade -y linux-firmware 2024-03-13T14:22:49.180 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum upgrade -y linux-firmware 2024-03-13T14:22:51.426 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.7 MB 00:00 2024-03-13T14:22:51.514 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - AppStream 8.9 MB/s | 6.7 MB 00:00 2024-03-13T14:22:52.914 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-03-13T14:22:53.068 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-13T14:22:53.663 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-03-13T14:22:54.383 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.0 MB 00:00 2024-03-13T14:22:58.364 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 5.0 MB/s | 19 MB 00:03 2024-03-13T14:23:02.423 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.1 MB 00:00 2024-03-13T14:23:03.974 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-13T14:23:05.766 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-13T14:23:10.691 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-03-13T14:23:10.849 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-03-13T14:23:13.019 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:23:13.029 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:23:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:23:13.029 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:23:13.029 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-03-13T14:23:13.029 INFO:teuthology.orchestra.run.smithi196.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-13T14:23:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:23:13.030 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:23:13.030 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:23:13.030 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 1 Package 2024-03-13T14:23:13.030 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:23:13.030 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 375 M 2024-03-13T14:23:13.030 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:23:16.574 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.1 MB 00:01 2024-03-13T14:23:18.124 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 15 kB 00:00 2024-03-13T14:23:19.526 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-13T14:23:24.392 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-03-13T14:23:25.566 INFO:teuthology.orchestra.run.smithi196.stdout:linux-firmware-20240219-143.el9.noarch.rpm 30 MB/s | 375 MB 00:12 2024-03-13T14:23:25.567 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:23:25.567 INFO:teuthology.orchestra.run.smithi196.stdout:Total 30 MB/s | 375 MB 00:12 2024-03-13T14:23:25.567 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:23:25.654 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:23:25.654 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:23:26.723 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:23:26.730 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrading: 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrade 1 Package 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 375 M 2024-03-13T14:23:26.731 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:23:28.097 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:23:28.097 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:23:34.304 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:23:34.551 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-13T14:23:34.639 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-13T14:23:35.989 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-13T14:23:35.989 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-13T14:23:36.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-13T14:23:36.383 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:23:36.383 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-03-13T14:23:36.383 INFO:teuthology.orchestra.run.smithi196.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-13T14:23:36.383 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:23:36.383 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:23:36.680 INFO:teuthology.task.kernel:Downloading kernel a87fde6e633694345631fcca2fd8a5f190eef61c on host.b... 2024-03-13T14:23:36.681 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:23:36.830 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/a87fde6e633694345631fcca2fd8a5f190eef61c/centos/9/flavors/default/x86_64/ 2024-03-13T14:23:36.831 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_ga87fde6e6336-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/a87fde6e633694345631fcca2fd8a5f190eef61c/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-13T14:23:36.834 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi196.front.sepia.ceph.com to complete... 2024-03-13T14:23:38.859 INFO:teuthology.orchestra.run.smithi176.stdout:linux-firmware-20240219-143.el9.noarch.rpm 31 MB/s | 375 MB 00:12 2024-03-13T14:23:38.860 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:23:38.860 INFO:teuthology.orchestra.run.smithi176.stdout:Total 31 MB/s | 375 MB 00:12 2024-03-13T14:23:38.860 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:23:38.953 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:23:38.953 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:23:41.351 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:23:41.351 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:23:47.772 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:23:48.043 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-13T14:23:48.124 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-13T14:23:48.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13 14:23:48 URL:https://4.chacra.ceph.com/r/kernel/testing/a87fde6e633694345631fcca2fd8a5f190eef61c/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_ga87fde6e6336-1.x86_64.rpm [336437659/336437659] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-13T14:23:48.682 INFO:teuthology.orchestra.run.smithi196.stderr:FINISHED --2024-03-13 14:23:48-- 2024-03-13T14:23:48.682 INFO:teuthology.orchestra.run.smithi196.stderr:Total wall clock time: 12s 2024-03-13T14:23:48.682 INFO:teuthology.orchestra.run.smithi196.stderr:Downloaded: 1 files, 321M in 12s (27.8 MB/s) 2024-03-13T14:23:48.686 INFO:teuthology.task.kernel:Installing kernel a87fde6e633694345631fcca2fd8a5f190eef61c on host.b... 2024-03-13T14:23:48.686 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-13T14:23:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-13T14:23:49.339 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-13T14:23:49.718 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-13T14:23:49.718 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:23:49.718 INFO:teuthology.orchestra.run.smithi176.stdout:Upgraded: 2024-03-13T14:23:49.718 INFO:teuthology.orchestra.run.smithi176.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-13T14:23:49.718 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:23:49.718 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:23:50.023 INFO:teuthology.task.kernel:Downloading kernel a87fde6e633694345631fcca2fd8a5f190eef61c on host.a... 2024-03-13T14:23:50.023 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a87fde6e633694345631fcca2fd8a5f190eef61c 2024-03-13T14:23:50.173 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/a87fde6e633694345631fcca2fd8a5f190eef61c/centos/9/flavors/default/x86_64/ 2024-03-13T14:23:50.174 DEBUG:teuthology.orchestra.run.smithi176:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_ga87fde6e6336-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/a87fde6e633694345631fcca2fd8a5f190eef61c/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-13T14:23:50.176 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi176.front.sepia.ceph.com to complete... 2024-03-13T14:23:50.616 INFO:teuthology.orchestra.run.smithi196.stdout:Verifying... ######################################## 2024-03-13T14:23:50.740 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing... ######################################## 2024-03-13T14:23:50.819 INFO:teuthology.orchestra.run.smithi196.stdout:Updating / installing... 2024-03-13T14:23:58.289 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-6.8.0_rc7_ga87fde6e6336-1 ######################################## 2024-03-13T14:24:08.602 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13 14:24:08 URL:https://4.chacra.ceph.com/r/kernel/testing/a87fde6e633694345631fcca2fd8a5f190eef61c/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_ga87fde6e6336-1.x86_64.rpm [336437659/336437659] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-13T14:24:08.603 INFO:teuthology.orchestra.run.smithi176.stderr:FINISHED --2024-03-13 14:24:08-- 2024-03-13T14:24:08.603 INFO:teuthology.orchestra.run.smithi176.stderr:Total wall clock time: 18s 2024-03-13T14:24:08.603 INFO:teuthology.orchestra.run.smithi176.stderr:Downloaded: 1 files, 321M in 18s (17.8 MB/s) 2024-03-13T14:24:08.606 INFO:teuthology.task.kernel:Installing kernel a87fde6e633694345631fcca2fd8a5f190eef61c on host.a... 2024-03-13T14:24:08.610 DEBUG:teuthology.orchestra.run.smithi176:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-13T14:24:10.543 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying... ######################################## 2024-03-13T14:24:10.682 INFO:teuthology.orchestra.run.smithi176.stdout:Preparing... ######################################## 2024-03-13T14:24:10.761 INFO:teuthology.orchestra.run.smithi176.stdout:Updating / installing... 2024-03-13T14:24:14.179 INFO:teuthology.orchestra.run.smithi196.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-13T14:24:18.261 INFO:teuthology.orchestra.run.smithi176.stdout:kernel-6.8.0_rc7_ga87fde6e6336-1 ######################################## 2024-03-13T14:24:34.035 INFO:teuthology.orchestra.run.smithi176.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-13T14:24:36.838 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-13T14:24:36.838 DEBUG:teuthology.orchestra.run.smithi196:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/System.map 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/config 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-13T14:24:36.915 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-13T14:24:36.916 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel/cpu 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel/cpu/mce 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm/kvm.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/ansi_cprng.ko 2024-03-13T14:24:36.917 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/anubis.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/arc4.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_memcpy.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_pq.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_tx.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_xor.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/raid6test.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/authenc.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/authencesn.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/blake2b_generic.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/blowfish_common.ko 2024-03-13T14:24:36.918 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/blowfish_generic.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/camellia_generic.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cast5_generic.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cast6_generic.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cast_common.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/ccm.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cmac.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/crc32_generic.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/crypto_engine.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/crypto_user.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cts.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/des_generic.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/dh_generic.ko 2024-03-13T14:24:36.919 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/echainiv.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/essiv.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/fcrypt.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/khazad.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/lz4.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/lz4hc.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/md4.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/michael_mic.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/pcbc.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/pcrypt.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/rmd160.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/seed.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/serpent_generic.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/tcrypt.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/tea.ko 2024-03-13T14:24:36.920 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/twofish_common.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/twofish_generic.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/vmac.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/wp512.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/xcbc.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/xor.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/xxhash_generic.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/acpi_pad.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/custom_method.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/ec_sys.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/platform_profile.ko 2024-03-13T14:24:36.921 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/sbs.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/sbshc.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/video.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/acard-ahci.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/ahci_platform.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/ata_generic.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/libahci_platform.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_acpi.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ali.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_amd.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_artop.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_atiixp.ko 2024-03-13T14:24:36.922 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_atp867x.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_cmd640.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_cmd64x.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_cypress.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_efar.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt366.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt37x.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_it8213.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_it821x.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_jmicron.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_marvell.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_mpiix.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_netcell.ko 2024-03-13T14:24:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ninja32.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ns87410.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ns87415.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_oldpiix.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_opti.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_optidma.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_pcmcia.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_piccolo.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_rdc.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sch.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_serverworks.ko 2024-03-13T14:24:36.924 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sil680.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sis.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sl82c105.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_triflex.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_via.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pdc_adma.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_inic162x.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_mv.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_nv.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_promise.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_qstor.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sil.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sil24.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sis.ko 2024-03-13T14:24:36.925 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_svw.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sx4.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_uli.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_via.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_vsc.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/auxdisplay 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/auxdisplay/ks0108.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base/regmap 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/bcma 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/bcma/bcma.ko 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block 2024-03-13T14:24:36.926 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/aoe 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/aoe/aoe.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/brd.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/drbd 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/drbd/drbd.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/floppy.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/loop.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/mtip32xx 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/nbd.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/null_blk 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/null_blk/null_blk.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/pktcdvd.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/rbd.ko 2024-03-13T14:24:36.927 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/virtio_blk.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/xen-blkback 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/xen-blkfront.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/zram 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/zram/zram.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hangcheck-timer.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/via-rng.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-13T14:24:36.928 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/lp.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/mwave 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/mwave/mwave.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ppdev.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tlclk.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm.ko 2024-03-13T14:24:36.929 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/uv_mmtimer.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/virtio_console.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-13T14:24:36.930 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/ccp 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/ccp/ccp.ko 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_common 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/padlock-aes.ko 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/padlock-sha.ko 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/virtio 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-13T14:24:36.931 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dca 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dca/dca.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/devfreq 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/dw 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/ioat 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/amd64_edac.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/e752x_edac.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/edac_mce_amd.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i3000_edac.ko 2024-03-13T14:24:36.932 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i3200_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i5100_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i5400_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i7300_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i7core_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i82975x_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/ie31200_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/sb_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/x38_edac.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/firmware 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/firmware/edd.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/gpio 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/gpio/gpio-ich.ko 2024-03-13T14:24:36.933 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-appleir.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-aureal.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-axff.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-dr.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-elecom.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-elo.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-emsff.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-gaff.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-gt683r.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-gyration.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-13T14:24:36.934 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-holtekff.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-hyperv.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-icade.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-keytouch.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-kye.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-lcpower.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-led.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-lenovo.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-multitouch.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-ortek.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-penmount.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-petalynx.ko 2024-03-13T14:24:36.935 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-picolcd.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-pl.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-plantronics.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-primax.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-rmi.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-common.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-13T14:24:36.936 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-saitek.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-samsung.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sjoy.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sony.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-speedlink.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-steelseries.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sunplus.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-tivo.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-tmff.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-topseed.ko 2024-03-13T14:24:36.937 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-twinhan.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-uclogic.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-waltop.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-wiimote.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-xinmo.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-zpff.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-zydacron.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/uhid.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/wacom.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv/hv_balloon.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv/hv_utils.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv/hv_vmbus.ko 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon 2024-03-13T14:24:36.938 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/abituguru.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/abituguru3.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ad7414.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ad7418.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adc128d818.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1025.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1026.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1029.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1031.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm9240.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ads7828.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7410.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7411.ko 2024-03-13T14:24:36.939 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7462.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7470.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7475.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7x10.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/amc6821.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/applesmc.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/asb100.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/asc7621.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/atxp1.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/coretemp.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/dme1737.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ds1621.ko 2024-03-13T14:24:36.940 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ds620.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/emc1403.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/emc6w201.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/f71805f.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/f71882fg.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/f75375s.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/fam15h_power.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/fschmd.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/g760a.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/g762.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/gl518sm.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/gl520sm.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/i5500_temp.ko 2024-03-13T14:24:36.941 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/i5k_amb.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ibmaem.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ibmpex.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ina209.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ina2xx.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/it87.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/k10temp.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/k8temp.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lineage-pem.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm63.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm73.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm75.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm77.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm78.ko 2024-03-13T14:24:36.942 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm80.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm83.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm85.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm87.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm90.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm92.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm93.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm95234.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm95241.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm95245.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc2945.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4151.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4215.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4222.ko 2024-03-13T14:24:36.943 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4245.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4260.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4261.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max16065.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max1619.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max1668.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max197.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max6639.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max6650.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max6697.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/mcp3021.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct6683.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct6775-core.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct6775.ko 2024-03-13T14:24:36.944 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct7802.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pc87360.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pc87427.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pcf8591.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-13T14:24:36.945 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/powr1220.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sch5627.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sch5636.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sht15.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sht21.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/shtc1.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sis5595.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/smsc47b397.ko 2024-03-13T14:24:36.946 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/smsc47m1.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/smsc47m192.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/thmc50.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp102.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp103.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp401.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp421.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/via-cputemp.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/via686a.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/vt1211.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/vt8231.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83627ehf.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83627hf.ko 2024-03-13T14:24:36.947 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83781d.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83791d.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83792d.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83793.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83795.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83l785ts.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83l786ng.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/algos 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-13T14:24:36.948 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-13T14:24:36.949 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-dev.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-mux.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-smbus.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-stub.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-13T14:24:36.950 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_core.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/cxgb4 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx4 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx5 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-13T14:24:36.951 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mthca 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/ipoib 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/iser 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/isert 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srp 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srpt 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport 2024-03-13T14:24:36.952 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport/gameport.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/joydev.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/keyboard 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/appletouch.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/bcm5974.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/cyapatp.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/sermouse.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-13T14:24:36.953 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/rmi4 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/altera_ps2.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/arc_ps2.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/serio_raw.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/sparse-keymap.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-blinkm.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-lm3530.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-lp3944.ko 2024-03-13T14:24:36.954 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-ss4200.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/bcache 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/bcache/bcache.ko 2024-03-13T14:24:36.955 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-bio-prison.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-cache-smq.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-cache.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-crypt.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-delay.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-flakey.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-log-userspace.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-multipath.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-queue-length.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-raid.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-round-robin.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-service-time.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-switch.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-thin-pool.ko 2024-03-13T14:24:36.956 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-verity.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/persistent-data 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid0.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid1.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid10.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid456.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptbase.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptctl.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptfc.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptlan.ko 2024-03-13T14:24:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptsas.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptscsih.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptspi.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/lpc_ich.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/lpc_sch.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/mfd-core.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/sm501.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/viperboard.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/vx855.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/wl1273-core.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/altera-stapl 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-13T14:24:36.958 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/apds9802als.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/apds990x.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/bh1770glc.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/cb710 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/cb710/cb710.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom/at24.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom/max6875.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/enclosure.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/hpilo.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/isl29003.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/isl29020.ko 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/lis3lv02d 2024-03-13T14:24:36.959 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei/mei-me.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei/mei-txe.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei/mei.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-gru 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/tifm_7xx1.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/tifm_core.ko 2024-03-13T14:24:36.960 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/tsl2550.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/vmw_balloon.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/vmw_vmci 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/bonding 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/bonding/bonding.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/b53 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/mv88e6xxx 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dummy.ko 2024-03-13T14:24:36.961 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/eql.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/adaptec 2024-03-13T14:24:36.962 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/agere 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/alteon 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/altera 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/alx 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-13T14:24:36.963 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atlx 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/genet 2024-03-13T14:24:36.964 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/brocade 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/brocade/bna 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-13T14:24:36.965 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/cisco 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/cisco/enic 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-13T14:24:36.966 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dlink 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dnet.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/emulex 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/emulex/benet 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/ethoc.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/fealnx.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000e 2024-03-13T14:24:36.967 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/fm10k 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/i40e 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/iavf 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igb 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igbvf 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-13T14:24:36.968 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/jme.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/micrel 2024-03-13T14:24:36.969 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/myricom 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/natsemi 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/neterion 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/nvidia 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/packetengines 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-13T14:24:36.970 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rdc 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-13T14:24:36.971 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rocker 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sfc 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/silan 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sis 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-13T14:24:36.972 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/tehuti 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/ti 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-13T14:24:36.973 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/via 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/wiznet 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/xircom 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/geneve.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/hyperv 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ieee802154 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ifb.ko 2024-03-13T14:24:36.974 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ipvlan 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/macvlan.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/macvtap.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mii.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/net_failover.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/netconsole.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/nlmon.ko 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/pcs 2024-03-13T14:24:36.975 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/amd.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm87xx.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/broadcom.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/cicada.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/davicom.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/dp83640.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/et1011c.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/icplus.ko 2024-03-13T14:24:36.976 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/lxt.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/marvell.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/micrel.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/national.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/phylink.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/qsemi.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/realtek.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/smsc.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/ste10Xp.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/vitesse.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_async.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-13T14:24:36.977 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/pppoe.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/pppox.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/pptp.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/slip 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/slip/slhc.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/slip/slip.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/sungem_phy.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/tap.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-13T14:24:36.978 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_random.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/tun.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/veth.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/virtio_net.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vmxnet3 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vxlan 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vxlan/vxlan.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/xen-netback 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/xen-netfront.ko 2024-03-13T14:24:36.979 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ntb 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ntb/ntb.ko 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme/host 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme/host/nvme-core.ko 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme/host/nvme.ko 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport.ko 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport_cs.ko 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport_pc.ko 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport_serial.ko 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/hotplug 2024-03-13T14:24:36.980 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/pcie 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/xen-pcifront.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia/i82092.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia/pd6729.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pinctrl 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pinctrl/intel 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-13T14:24:36.981 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/acerhdf.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/hdaps.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/intel 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/intel_ips.ko 2024-03-13T14:24:36.982 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/wmi.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/powercap 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps 2024-03-13T14:24:36.983 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients/pps_parport.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/pps_core.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ptp 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ptp/ptp.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ptp/ptp_kvm.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-13T14:24:36.984 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-em3027.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-max6900.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-13T14:24:36.985 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-x1205.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/3w-9xxx.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/3w-sas.ko 2024-03-13T14:24:36.986 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/3w-xxxx.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/BusLogic.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/a100u2w.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aacraid 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/advansys.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aic7xxx 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/am53c974.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/arcmsr 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/atp870u.ko 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/be2iscsi 2024-03-13T14:24:36.987 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bfa 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bfa/bfa.ko 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2fc 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2i 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ch.ko 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/csiostor 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-13T14:24:36.988 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/dc395x.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/dmx3191d.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/esas2r 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/esp_scsi.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fcoe 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-13T14:24:36.989 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fnic 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fnic/fnic.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/hpsa.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/hptiop.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/hv_storvsc.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/initio.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ipr.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ips.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/isci 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/isci/isci.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libfc 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libfc/libfc.ko 2024-03-13T14:24:36.990 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libiscsi.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libsas 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libsas/libsas.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/lpfc 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mpt3sas 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mvsas 2024-03-13T14:24:36.991 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mvumi.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/pm8001 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/pmcraid.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla1280.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla2xxx 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla4xxx 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/raid_class.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_debug.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-13T14:24:36.992 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ses.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/st.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/stex.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/sym53c8xx_2 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/virtio_scsi.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/wd719x.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ssb 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ssb/ssb.ko 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target 2024-03-13T14:24:36.993 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/iscsi 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/loopback 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_file.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_iblock.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_mod.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_pscsi.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_user.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/tcm_fc 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal 2024-03-13T14:24:36.994 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-13T14:24:36.995 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/ipwireless 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/n_gsm.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/n_hdlc.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/nozomi.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/8250 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/arc_uart.ko 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/jsm 2024-03-13T14:24:36.996 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/core 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/host 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_aec.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_cif.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_pci_generic.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_sercos3.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/cdc-acm.ko 2024-03-13T14:24:36.997 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/usblp.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/usbtmc.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/host 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/image 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/image/mdc800.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/image/microtek.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/adutux.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/appledisplay.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/emi26.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/emi62.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/ezusb.ko 2024-03-13T14:24:36.998 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/idmouse.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/iowarrior.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/ldusb.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/legousbtower.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/sisusbvga 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/usb3503.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/usblcd.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/uss720.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/yurex.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/phy 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/phy/phy-generic.ko 2024-03-13T14:24:36.999 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/aircable.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ark3116.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ch341.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/cp210x.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/cyberjack.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/empeg.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/io_ti.ko 2024-03-13T14:24:37.000 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ipaq.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ipw.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ir-usb.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/keyspan.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/mct_u232.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/mos7720.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/mos7840.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/navman.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/omninet.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/opticon.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/option.ko 2024-03-13T14:24:37.001 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/oti6858.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/pl2303.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/qcaux.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/qcserial.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/quatech2.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/safe_serial.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/sierra.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ssu100.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/symbolserial.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/usb_debug.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-13T14:24:37.002 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/visor.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/whiteheat.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/uas.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-karma.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-13T14:24:37.003 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/usb-storage.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/pci 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/vfio.ko 2024-03-13T14:24:37.004 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost_net.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost_scsi.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight/apple_bl.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight/lcd.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_balloon.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_mmio.ko 2024-03-13T14:24:37.005 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_pci.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_ring.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/masters 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/masters/ds2482.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/masters/ds2490.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-13T14:24:37.006 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/wire.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/hpwdt.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/i6300esb.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-13T14:24:37.007 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/ib700wdt.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/ibmasr.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/it87_wdt.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/machzwd.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/nv_tco.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/softdog.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/via_wdt.ko 2024-03-13T14:24:37.008 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/wdt_pci.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/xen_wdt.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-evtchn.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-gntalloc.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-gntdev.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-pciback 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-privcmd.ko 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xenfs 2024-03-13T14:24:37.009 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/9p 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/9p/9p.ko 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/affs 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/affs/affs.ko 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/befs 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/befs/befs.ko 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/binfmt_misc.ko 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/btrfs 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/btrfs/btrfs.ko 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cachefiles 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cachefiles/cachefiles.ko 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ceph 2024-03-13T14:24:37.010 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ceph/ceph.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/coda 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/coda/coda.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cramfs 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cramfs/cramfs.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/dlm 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/dlm/dlm.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ecryptfs 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/f2fs 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/f2fs/f2fs.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat/fat.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat/msdos.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat/vfat.ko 2024-03-13T14:24:37.011 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fuse 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fuse/cuse.ko 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fuse/fuse.ko 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/gfs2 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/gfs2/gfs2.ko 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfs 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfs/hfs.ko 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfsplus 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfsplus/hfsplus.ko 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/isofs 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/isofs/isofs.ko 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/jfs 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/jfs/jfs.ko 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/lockd 2024-03-13T14:24:37.012 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/lockd/lockd.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/minix 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/minix/minix.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/netfs 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/netfs/netfs.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/blocklayout 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/filelayout 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/flexfilelayout 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/nfs.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/nfsv3.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/nfsv4.ko 2024-03-13T14:24:37.013 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs_common 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs_common/grace.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs_common/nfs_acl.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfsd 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfsd/nfsd.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nilfs2 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nilfs2/nilfs2.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-celtic.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-centeuro.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-croatian.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-cyrillic.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-gaelic.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-greek.ko 2024-03-13T14:24:37.014 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-iceland.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-inuit.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-roman.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-romanian.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-turkish.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp1250.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp1251.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp1255.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp737.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp775.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp850.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp852.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp855.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp857.ko 2024-03-13T14:24:37.015 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp860.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp861.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp862.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp863.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp864.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp865.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp866.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp869.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp874.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp932.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp936.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp949.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp950.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_euc-jp.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-1.ko 2024-03-13T14:24:37.016 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-13.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-14.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-15.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-2.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-3.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-4.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-5.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-6.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-7.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-9.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_koi8-r.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_koi8-ru.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_koi8-u.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-13T14:24:37.017 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_utf8.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/cluster 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlm 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlmfs 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/overlayfs 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/overlayfs/overlay.ko 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/pstore 2024-03-13T14:24:37.018 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/pstore/ramoops.ko 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/reiserfs 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/reiserfs/reiserfs.ko 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/romfs 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/romfs/romfs.ko 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/client 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/client/cifs.ko 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/common 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/common/cifs_arc4.ko 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/common/cifs_md4.ko 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/squashfs 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/squashfs/squashfs.ko 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/sysv 2024-03-13T14:24:37.019 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/sysv/sysv.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/udf 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/udf/udf.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ufs 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ufs/ufs.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/xfs 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/xfs/xfs.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/rcu 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/rcu/rcutorture.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/torture.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/trace 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/asn1_encoder.ko 2024-03-13T14:24:37.020 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crc-itu-t.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crc8.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crypto 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crypto/libarc4.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crypto/libdes.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/libcrc32c.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lru_cache.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lz4 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lz4/lz4_compress.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lz4/lz4hc_compress.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/math 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/math/cordic.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/raid6 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/raid6/raid6_pq.ko 2024-03-13T14:24:37.021 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/reed_solomon 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/ts_bm.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/ts_fsm.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/ts_kmp.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/zstd 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/zstd/zstd_compress.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/mm 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/mm/hwpoison-inject.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/6lowpan.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_dest.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_fragment.ko 2024-03-13T14:24:37.022 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_hop.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_mobility.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_routing.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_udp.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/garp.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/mrp.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/p8022.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/psnap.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/stp.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/8021q 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/8021q/8021q.ko 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p 2024-03-13T14:24:37.023 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet_fd.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet_rdma.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet_virtio.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/batman-adv 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/batman-adv/batman-adv.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/br_netfilter.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/bridge.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-13T14:24:37.024 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-13T14:24:37.025 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtables.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ceph 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ceph/libceph.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/core 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/core/failover.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/core/pktgen.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp_diag.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp_ipv4.ko 2024-03-13T14:24:37.026 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp_ipv6.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dns_resolver 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dns_resolver/dns_resolver.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/dsa_core.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_brcm.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_dsa.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_none.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_trailer.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/6lowpan 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/ieee802154.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4 2024-03-13T14:24:37.027 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ah4.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/esp4.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/fou.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/gre.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/inet_diag.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ip_gre.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ip_tunnel.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ip_vti.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ipcomp.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ipip.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-13T14:24:37.028 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-13T14:24:37.029 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_bic.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_dctcp.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_diag.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_highspeed.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_htcp.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_hybla.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_illinois.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_lp.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_scalable.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_vegas.ko 2024-03-13T14:24:37.030 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_veno.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_westwood.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_yeah.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tunnel4.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/udp_diag.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/udp_tunnel.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ah6.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/esp6.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/fou6.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ip6_tunnel.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ip6_vti.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ipcomp6.ko 2024-03-13T14:24:37.031 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-13T14:24:37.032 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/sit.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/tunnel6.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-13T14:24:37.033 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/key 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/key/af_key.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_core.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_eth.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_ip.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_ip6.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_netlink.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_ppp.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/llc 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/llc/llc.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mac802154 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mac802154/mac802154.ko 2024-03-13T14:24:37.034 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mpls 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mpls/mpls_gso.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-13T14:24:37.035 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-13T14:24:37.036 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conncount.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-13T14:24:37.037 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_flow_table.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_log_syslog.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_irc.ko 2024-03-13T14:24:37.038 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_sip.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_tables.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_log.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_chain_nat.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_compat.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_ct.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_fib.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-13T14:24:37.039 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_hash.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_limit.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_log.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_masq.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_nat.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_numgen.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_queue.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_quota.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_redir.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_reject.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_reject_inet.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_socket.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_tunnel.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_AUDIT.ko 2024-03-13T14:24:37.040 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CT.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_DSCP.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_HL.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_HMARK.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_LED.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_LOG.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_NETMAP.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_NFLOG.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_RATEEST.ko 2024-03-13T14:24:37.041 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_SECMARK.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TEE.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TPROXY.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TRACE.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_addrtype.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_bpf.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_cgroup.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_cluster.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_comment.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connbytes.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connlabel.ko 2024-03-13T14:24:37.042 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connlimit.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connmark.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_conntrack.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_cpu.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_dccp.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_devgroup.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_dscp.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_ecn.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_esp.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_hashlimit.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_helper.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_hl.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_ipcomp.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_iprange.ko 2024-03-13T14:24:37.043 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_ipvs.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_l2tp.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_length.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_limit.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_mac.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_mark.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_multiport.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_nat.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_nfacct.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_osf.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_owner.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_physdev.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_pkttype.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_policy.ko 2024-03-13T14:24:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_quota.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_rateest.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_realm.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_recent.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_sctp.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_set.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_socket.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_state.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_statistic.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_string.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_tcpmss.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_time.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_u32.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netlink 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netlink/netlink_diag.ko 2024-03-13T14:24:37.045 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/nsh 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/nsh/nsh.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/openvswitch.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/vport-geneve.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/vport-gre.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/vport-vxlan.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/packet 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/packet/af_packet_diag.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds/rds.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds/rds_rdma.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds/rds_tcp.ko 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched 2024-03-13T14:24:37.046 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_csum.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_gact.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_mirred.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_nat.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_pedit.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_police.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_simple.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_skbedit.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_vlan.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_basic.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_bpf.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_flow.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_fw.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_route.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_u32.ko 2024-03-13T14:24:37.047 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_cmp.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_ipset.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_meta.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_nbyte.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_text.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_u32.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_choke.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_codel.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_drr.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_fq.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_gred.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_hfsc.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_hhf.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_htb.ko 2024-03-13T14:24:37.048 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_ingress.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_mqprio.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_mqprio_lib.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_multiq.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_netem.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_pie.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_plug.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_prio.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_qfq.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_red.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_sfb.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_sfq.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_tbf.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_teql.ko 2024-03-13T14:24:37.049 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sctp 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sctp/sctp.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sctp/sctp_diag.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/auth_gss 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/sunrpc.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/xprtrdma 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/tipc 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/tipc/diag.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/tipc/tipc.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/unix 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/unix/unix_diag.ko 2024-03-13T14:24:37.050 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vsock.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/xfrm 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/encrypted-keys 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/trusted-keys 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/trusted-keys/trusted.ko 2024-03-13T14:24:37.051 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/virt 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/virt/lib 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/virt/lib/irqbypass.ko 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.alias 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.alias.bin 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin.alias.bin 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin.bin 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin.modinfo 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.dep 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.dep.bin 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.devname 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.order 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.softdep 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.symbols 2024-03-13T14:24:37.052 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.symbols.bin 2024-03-13T14:24:37.053 INFO:teuthology.orchestra.run.smithi196.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/vmlinuz 2024-03-13T14:24:37.057 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:37.057 DEBUG:teuthology.orchestra.run.smithi196:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-13T14:24:37.079 INFO:teuthology.orchestra.run.smithi196.stdout:postinstall scriptlet (using /bin/sh): 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout: /usr/bin/kernel-install add 6.8.0-rc7-ga87fde6e6336 /lib/modules/6.8.0-rc7-ga87fde6e6336/vmlinuz 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:fi 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:for file in vmlinuz System.map config; do 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-ga87fde6e6336/${file}" "/boot/${file}-6.8.0-rc7-ga87fde6e6336"; then 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout: cp "/lib/modules/6.8.0-rc7-ga87fde6e6336/${file}" "/boot/${file}-6.8.0-rc7-ga87fde6e6336" 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout: fi 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:done 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:new-kernel-pkg --remove 6.8.0-rc7-ga87fde6e6336 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-ga87fde6e6336.img 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-13T14:24:37.080 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-install remove 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:37.081 INFO:teuthology.orchestra.run.smithi196.stdout:fi 2024-03-13T14:24:37.081 INFO:teuthology.orchestra.run.smithi196.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-13T14:24:37.081 INFO:teuthology.orchestra.run.smithi196.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-13T14:24:37.081 INFO:teuthology.orchestra.run.smithi196.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:37.081 INFO:teuthology.orchestra.run.smithi196.stdout:fi 2024-03-13T14:24:37.082 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-03-13T14:24:37.156 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-03-13T14:24:37.156 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-03-13T14:24:37.156 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 68.el9 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8264254 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-13T14:24:37.157 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:24:37.158 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-03-13T14:24:37.159 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-13T14:24:37.159 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:37.159 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-13T14:24:37.771 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-03-13T14:24:39.120 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-13T14:24:39.155 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-03-13T14:24:39.158 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-03-13T14:24:39.174 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.MPAqOEVVnd 2024-03-13T14:24:39.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MPAqOEVVnd 2024-03-13T14:24:39.243 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.MPAqOEVVnd 2024-03-13T14:24:39.380 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.MPAqOEVVnd is 6KB 2024-03-13T14:24:39.428 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.MPAqOEVVnd 2024-03-13T14:24:39.442 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-03-13T14:24:39.506 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-13T14:24:39.507 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-13T14:24:39.507 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-13T14:24:39.507 INFO:teuthology.orchestra.run.smithi196.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-ga87fde6e6336.conf 2024-03-13T14:24:39.508 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:39.670 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-03-13T14:24:56.380 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi176.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-13T14:24:56.381 DEBUG:teuthology.orchestra.run.smithi176:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/System.map 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/config 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-13T14:24:56.458 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-13T14:24:56.459 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel/cpu 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel/cpu/mce 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/arch/x86/kvm/kvm.ko 2024-03-13T14:24:56.460 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/ansi_cprng.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/anubis.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/arc4.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_memcpy.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_pq.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_tx.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/async_xor.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/async_tx/raid6test.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/authenc.ko 2024-03-13T14:24:56.461 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/authencesn.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/blake2b_generic.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/blowfish_common.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/blowfish_generic.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/camellia_generic.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cast5_generic.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cast6_generic.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cast_common.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/ccm.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cmac.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/crc32_generic.ko 2024-03-13T14:24:56.462 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/crypto_engine.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/crypto_user.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/cts.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/des_generic.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/dh_generic.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/echainiv.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/essiv.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/fcrypt.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/khazad.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/lz4.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/lz4hc.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/md4.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/michael_mic.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/pcbc.ko 2024-03-13T14:24:56.463 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/pcrypt.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/rmd160.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/seed.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/serpent_generic.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/tcrypt.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/tea.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/twofish_common.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/twofish_generic.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/vmac.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/wp512.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/xcbc.ko 2024-03-13T14:24:56.464 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/xor.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/crypto/xxhash_generic.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/acpi_pad.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/custom_method.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/ec_sys.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/platform_profile.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/sbs.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/sbshc.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/acpi/video.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/acard-ahci.ko 2024-03-13T14:24:56.465 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/ahci_platform.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/ata_generic.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/libahci_platform.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_acpi.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ali.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_amd.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_artop.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_atiixp.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_atp867x.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_cmd640.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_cmd64x.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_cypress.ko 2024-03-13T14:24:56.466 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_efar.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt366.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt37x.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_it8213.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_it821x.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_jmicron.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_marvell.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_mpiix.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_netcell.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ninja32.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ns87410.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_ns87415.ko 2024-03-13T14:24:56.467 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_oldpiix.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_opti.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_optidma.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_pcmcia.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_piccolo.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_rdc.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sch.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_serverworks.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sil680.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sis.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_sl82c105.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_triflex.ko 2024-03-13T14:24:56.468 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pata_via.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/pdc_adma.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_inic162x.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_mv.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_nv.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_promise.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_qstor.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sil.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sil24.ko 2024-03-13T14:24:56.469 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sis.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_svw.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_sx4.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_uli.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_via.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ata/sata_vsc.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/auxdisplay 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/auxdisplay/ks0108.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base/regmap 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/bcma 2024-03-13T14:24:56.470 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/bcma/bcma.ko 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/aoe 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/aoe/aoe.ko 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/brd.ko 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/drbd 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/drbd/drbd.ko 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/floppy.ko 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/loop.ko 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/mtip32xx 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-13T14:24:56.471 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/nbd.ko 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/null_blk 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/null_blk/null_blk.ko 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/pktcdvd.ko 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/rbd.ko 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/virtio_blk.ko 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/xen-blkback 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/xen-blkfront.ko 2024-03-13T14:24:56.472 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/zram 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/block/zram/zram.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hangcheck-timer.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/via-rng.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-13T14:24:56.473 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/lp.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/mwave 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/mwave/mwave.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/ppdev.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tlclk.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm.ko 2024-03-13T14:24:56.474 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/uv_mmtimer.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/char/virtio_console.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-13T14:24:56.475 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/ccp 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/ccp/ccp.ko 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_common 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-13T14:24:56.476 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/padlock-aes.ko 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/padlock-sha.ko 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/virtio 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dca 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dca/dca.ko 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/devfreq 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma 2024-03-13T14:24:56.477 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/dw 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/ioat 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/amd64_edac.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/e752x_edac.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/edac_mce_amd.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i3000_edac.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i3200_edac.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i5100_edac.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i5400_edac.ko 2024-03-13T14:24:56.478 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i7300_edac.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i7core_edac.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/i82975x_edac.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/ie31200_edac.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/sb_edac.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/edac/x38_edac.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/firmware 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/firmware/edd.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-13T14:24:56.479 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/gpio 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/gpio/gpio-ich.ko 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-appleir.ko 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-aureal.ko 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-axff.ko 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-dr.ko 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-elecom.ko 2024-03-13T14:24:56.480 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-elo.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-emsff.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-gaff.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-gt683r.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-gyration.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-holtekff.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-hyperv.ko 2024-03-13T14:24:56.481 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-icade.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-keytouch.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-kye.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-lcpower.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-led.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-lenovo.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-multitouch.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-ortek.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-penmount.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-petalynx.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-picolcd.ko 2024-03-13T14:24:56.482 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-pl.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-plantronics.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-primax.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-rmi.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-common.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-13T14:24:56.483 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-roccat.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-saitek.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-samsung.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sjoy.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sony.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-speedlink.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-steelseries.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-sunplus.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-tivo.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-tmff.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-topseed.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-twinhan.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-uclogic.ko 2024-03-13T14:24:56.484 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-waltop.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-wiimote.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-xinmo.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-zpff.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/hid-zydacron.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/uhid.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hid/wacom.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv/hv_balloon.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv/hv_utils.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hv/hv_vmbus.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/abituguru.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/abituguru3.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-13T14:24:56.485 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ad7414.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ad7418.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adc128d818.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1025.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1026.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1029.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm1031.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adm9240.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ads7828.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7410.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7411.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7462.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7470.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7475.ko 2024-03-13T14:24:56.486 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/adt7x10.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/amc6821.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/applesmc.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/asb100.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/asc7621.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/atxp1.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/coretemp.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/dme1737.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ds1621.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ds620.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/emc1403.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/emc6w201.ko 2024-03-13T14:24:56.487 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/f71805f.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/f71882fg.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/f75375s.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/fam15h_power.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/fschmd.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/g760a.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/g762.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/gl518sm.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/gl520sm.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/i5500_temp.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/i5k_amb.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ibmaem.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ibmpex.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ina209.ko 2024-03-13T14:24:56.488 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ina2xx.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/it87.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/k10temp.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/k8temp.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lineage-pem.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm63.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm73.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm75.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm77.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm78.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm80.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm83.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm85.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm87.ko 2024-03-13T14:24:56.489 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm90.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm92.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm93.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm95234.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm95241.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/lm95245.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc2945.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4151.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4215.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4222.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4245.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4260.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/ltc4261.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max16065.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max1619.ko 2024-03-13T14:24:56.490 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max1668.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max197.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max6639.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max6650.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/max6697.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/mcp3021.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct6683.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct6775-core.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct6775.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/nct7802.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pc87360.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pc87427.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pcf8591.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-13T14:24:56.491 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/powr1220.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sch5627.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sch5636.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-13T14:24:56.492 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sht15.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sht21.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/shtc1.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/sis5595.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/smsc47b397.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/smsc47m1.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/smsc47m192.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/thmc50.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp102.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp103.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp401.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/tmp421.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/via-cputemp.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/via686a.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/vt1211.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/vt8231.ko 2024-03-13T14:24:56.493 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83627ehf.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83627hf.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83781d.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83791d.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83792d.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83793.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83795.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83l785ts.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/hwmon/w83l786ng.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/algos 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-13T14:24:56.494 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-13T14:24:56.495 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-dev.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-mux.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-smbus.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/i2c/i2c-stub.ko 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core 2024-03-13T14:24:56.496 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_core.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/cxgb4 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx4 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx5 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mthca 2024-03-13T14:24:56.497 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/ipoib 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/iser 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/isert 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srp 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srpt 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport 2024-03-13T14:24:56.498 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/gameport/gameport.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/joydev.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/keyboard 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/appletouch.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/bcm5974.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/cyapatp.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/sermouse.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-13T14:24:56.499 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/rmi4 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/altera_ps2.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/arc_ps2.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/serio/serio_raw.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/input/sparse-keymap.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-blinkm.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-lm3530.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-lp3944.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/leds-ss4200.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-13T14:24:56.500 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/bcache 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/bcache/bcache.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-bio-prison.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-cache-smq.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-cache.ko 2024-03-13T14:24:56.501 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-crypt.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-delay.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-flakey.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-log-userspace.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-multipath.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-queue-length.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-raid.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-round-robin.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-service-time.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-switch.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-thin-pool.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/dm-verity.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/persistent-data 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-13T14:24:56.502 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid0.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid1.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid10.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/md/raid456.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptbase.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptctl.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptfc.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptlan.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptsas.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptscsih.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/message/fusion/mptspi.ko 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd 2024-03-13T14:24:56.503 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/lpc_ich.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/lpc_sch.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/mfd-core.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/sm501.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/viperboard.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/vx855.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/mfd/wl1273-core.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/altera-stapl 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/apds9802als.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/apds990x.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/bh1770glc.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/cb710 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/cb710/cb710.ko 2024-03-13T14:24:56.504 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom/at24.ko 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/eeprom/max6875.ko 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/enclosure.ko 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/hpilo.ko 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/isl29003.ko 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/isl29020.ko 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/lis3lv02d 2024-03-13T14:24:56.505 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei/mei-me.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei/mei-txe.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/mei/mei.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-gru 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/tifm_7xx1.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/tifm_core.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/tsl2550.ko 2024-03-13T14:24:56.506 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/vmw_balloon.ko 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/vmw_vmci 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/bonding 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/bonding/bonding.ko 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/b53 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-13T14:24:56.507 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/mv88e6xxx 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/dummy.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/eql.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-13T14:24:56.508 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/adaptec 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/agere 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/alteon 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/altera 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-13T14:24:56.509 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/alx 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atlx 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-13T14:24:56.510 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/genet 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/brocade 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/brocade/bna 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-13T14:24:56.511 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-13T14:24:56.512 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/cisco 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/cisco/enic 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dlink 2024-03-13T14:24:56.513 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/dnet.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/emulex 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/emulex/benet 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/ethoc.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/fealnx.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000e 2024-03-13T14:24:56.514 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-13T14:24:56.515 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/fm10k 2024-03-13T14:24:56.517 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-13T14:24:56.517 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/i40e 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/iavf 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igb 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igbvf 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/jme.ko 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell 2024-03-13T14:24:56.518 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/micrel 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/myricom 2024-03-13T14:24:56.519 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/natsemi 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/neterion 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/nvidia 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/packetengines 2024-03-13T14:24:56.520 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rdc 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-13T14:24:56.521 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rocker 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sfc 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/silan 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sis 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-13T14:24:56.522 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/tehuti 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/ti 2024-03-13T14:24:56.523 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/via 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/wiznet 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/xircom 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/geneve.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/hyperv 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ieee802154 2024-03-13T14:24:56.524 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ifb.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ipvlan 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/macvlan.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/macvtap.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/mii.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/net_failover.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/netconsole.ko 2024-03-13T14:24:56.525 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/nlmon.ko 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/pcs 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/amd.ko 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-13T14:24:56.526 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/bcm87xx.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/broadcom.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/cicada.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/davicom.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/dp83640.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/et1011c.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/icplus.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/lxt.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/marvell.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/micrel.ko 2024-03-13T14:24:56.527 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/national.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/phylink.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/qsemi.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/realtek.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/smsc.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/ste10Xp.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/phy/vitesse.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_async.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/pppoe.ko 2024-03-13T14:24:56.528 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/pppox.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/ppp/pptp.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/slip 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/slip/slhc.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/slip/slip.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/sungem_phy.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/tap.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_random.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-13T14:24:56.529 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/tun.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/veth.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/virtio_net.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vmxnet3 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vxlan 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/vxlan/vxlan.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/xen-netback 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/net/xen-netfront.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ntb 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ntb/ntb.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme/host 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme/host/nvme-core.ko 2024-03-13T14:24:56.530 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/nvme/host/nvme.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport_cs.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport_pc.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/parport/parport_serial.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/hotplug 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/pcie 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pci/xen-pcifront.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia/i82092.ko 2024-03-13T14:24:56.531 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia/pd6729.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pinctrl 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pinctrl/intel 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/acerhdf.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-13T14:24:56.532 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/hdaps.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/intel 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/intel_ips.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-13T14:24:56.533 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/platform/x86/wmi.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/powercap 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/clients/pps_parport.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/pps/pps_core.ko 2024-03-13T14:24:56.534 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ptp 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ptp/ptp.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ptp/ptp_kvm.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-13T14:24:56.535 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-em3027.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-max6900.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-13T14:24:56.536 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/rtc/rtc-x1205.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/3w-9xxx.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/3w-sas.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/3w-xxxx.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/BusLogic.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/a100u2w.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aacraid 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/advansys.ko 2024-03-13T14:24:56.537 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aic7xxx 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/am53c974.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/arcmsr 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/atp870u.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/be2iscsi 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bfa 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bfa/bfa.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2fc 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2i 2024-03-13T14:24:56.538 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ch.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/csiostor 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/dc395x.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-13T14:24:56.539 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/dmx3191d.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/esas2r 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/esp_scsi.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fcoe 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fnic 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/fnic/fnic.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/hpsa.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/hptiop.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/hv_storvsc.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/initio.ko 2024-03-13T14:24:56.540 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ipr.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ips.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/isci 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/isci/isci.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libfc 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libfc/libfc.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libiscsi.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libsas 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/libsas/libsas.ko 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/lpfc 2024-03-13T14:24:56.541 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mpt3sas 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mvsas 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/mvumi.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/pm8001 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-13T14:24:56.542 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/pmcraid.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla1280.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla2xxx 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla4xxx 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/raid_class.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_debug.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-13T14:24:56.543 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/ses.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/st.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/stex.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/sym53c8xx_2 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/virtio_scsi.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/scsi/wd719x.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ssb 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ssb/ssb.ko 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/iscsi 2024-03-13T14:24:56.544 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/loopback 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_file.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_iblock.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_mod.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_pscsi.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/target_core_user.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/tcm_fc 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-13T14:24:56.545 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-13T14:24:56.546 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/ipwireless 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/n_gsm.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/n_hdlc.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/nozomi.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/8250 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/arc_uart.ko 2024-03-13T14:24:56.547 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/jsm 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/core 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/host 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_aec.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_cif.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_pci_generic.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/uio/uio_sercos3.ko 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb 2024-03-13T14:24:56.548 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/cdc-acm.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/usblp.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/class/usbtmc.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/host 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/image 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/image/mdc800.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/image/microtek.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/adutux.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/appledisplay.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/emi26.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/emi62.ko 2024-03-13T14:24:56.549 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/ezusb.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/idmouse.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/iowarrior.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/ldusb.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/legousbtower.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/sisusbvga 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/usb3503.ko 2024-03-13T14:24:56.550 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/usblcd.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/uss720.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/misc/yurex.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/phy 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/phy/phy-generic.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/aircable.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ark3116.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ch341.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/cp210x.ko 2024-03-13T14:24:56.551 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/cyberjack.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/empeg.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/io_ti.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ipaq.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ipw.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ir-usb.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/keyspan.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-13T14:24:56.552 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/mct_u232.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/mos7720.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/mos7840.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/navman.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/omninet.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/opticon.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/option.ko 2024-03-13T14:24:56.553 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/oti6858.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/pl2303.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/qcaux.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/qcserial.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/quatech2.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/safe_serial.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/sierra.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ssu100.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/symbolserial.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/usb_debug.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-13T14:24:56.554 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/visor.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/whiteheat.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/uas.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-13T14:24:56.555 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-karma.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/storage/usb-storage.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-13T14:24:56.556 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/pci 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/vfio.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost_net.ko 2024-03-13T14:24:56.557 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/vhost/vhost_scsi.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight/apple_bl.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight/lcd.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_balloon.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_mmio.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_pci.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-13T14:24:56.558 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/virtio/virtio_ring.ko 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/masters 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/masters/ds2482.ko 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/masters/ds2490.ko 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-13T14:24:56.559 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/w1/wire.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/hpwdt.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/i6300esb.ko 2024-03-13T14:24:56.560 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/ib700wdt.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/ibmasr.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/it87_wdt.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/machzwd.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/nv_tco.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-13T14:24:56.561 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/softdog.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/via_wdt.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/wdt_pci.ko 2024-03-13T14:24:56.562 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/watchdog/xen_wdt.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-evtchn.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-gntalloc.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-gntdev.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-pciback 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xen-privcmd.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xenfs 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs 2024-03-13T14:24:56.563 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/9p 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/9p/9p.ko 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/affs 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/affs/affs.ko 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/befs 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/befs/befs.ko 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/binfmt_misc.ko 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/btrfs 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/btrfs/btrfs.ko 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cachefiles 2024-03-13T14:24:56.564 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cachefiles/cachefiles.ko 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ceph 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ceph/ceph.ko 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/coda 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/coda/coda.ko 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cramfs 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/cramfs/cramfs.ko 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/dlm 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/dlm/dlm.ko 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ecryptfs 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/f2fs 2024-03-13T14:24:56.565 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/f2fs/f2fs.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat/fat.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat/msdos.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fat/vfat.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fuse 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fuse/cuse.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/fuse/fuse.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/gfs2 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/gfs2/gfs2.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfs 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfs/hfs.ko 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfsplus 2024-03-13T14:24:56.566 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/hfsplus/hfsplus.ko 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/isofs 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/isofs/isofs.ko 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/jfs 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/jfs/jfs.ko 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/lockd 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/lockd/lockd.ko 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/minix 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/minix/minix.ko 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/netfs 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/netfs/netfs.ko 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/blocklayout 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-13T14:24:56.567 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/filelayout 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/flexfilelayout 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/nfs.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/nfsv3.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs/nfsv4.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs_common 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs_common/grace.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfs_common/nfs_acl.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfsd 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nfsd/nfsd.ko 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nilfs2 2024-03-13T14:24:56.568 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nilfs2/nilfs2.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-celtic.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-centeuro.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-croatian.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-cyrillic.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-gaelic.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-greek.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-iceland.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-inuit.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-roman.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-romanian.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/mac-turkish.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp1250.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp1251.ko 2024-03-13T14:24:56.569 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp1255.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp737.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp775.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp850.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp852.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp855.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp857.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp860.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp861.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp862.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp863.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp864.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp865.ko 2024-03-13T14:24:56.570 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp866.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp869.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp874.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp932.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp936.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp949.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_cp950.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_euc-jp.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-1.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-13.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-14.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-15.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-2.ko 2024-03-13T14:24:56.571 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-3.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-4.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-5.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-6.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-7.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_iso8859-9.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_koi8-r.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_koi8-ru.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_koi8-u.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/nls/nls_utf8.ko 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/cluster 2024-03-13T14:24:56.572 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlm 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlmfs 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/overlayfs 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/overlayfs/overlay.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/pstore 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/pstore/ramoops.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/reiserfs 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/reiserfs/reiserfs.ko 2024-03-13T14:24:56.573 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/romfs 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/romfs/romfs.ko 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/client 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/client/cifs.ko 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/common 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/common/cifs_arc4.ko 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/smb/common/cifs_md4.ko 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/squashfs 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/squashfs/squashfs.ko 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/sysv 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/sysv/sysv.ko 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/udf 2024-03-13T14:24:56.574 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/udf/udf.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ufs 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/ufs/ufs.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/xfs 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/fs/xfs/xfs.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/rcu 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/rcu/rcutorture.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/torture.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/trace 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/asn1_encoder.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crc-itu-t.ko 2024-03-13T14:24:56.575 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crc8.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crypto 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crypto/libarc4.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/crypto/libdes.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/libcrc32c.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lru_cache.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lz4 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lz4/lz4_compress.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/lz4/lz4hc_compress.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/math 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/math/cordic.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/raid6 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/raid6/raid6_pq.ko 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/reed_solomon 2024-03-13T14:24:56.576 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/ts_bm.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/ts_fsm.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/ts_kmp.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/zstd 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/lib/zstd/zstd_compress.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/mm 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/mm/hwpoison-inject.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/6lowpan.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_dest.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_fragment.ko 2024-03-13T14:24:56.577 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_hop.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_mobility.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_routing.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/6lowpan/nhc_udp.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/garp.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/mrp.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/p8022.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/psnap.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/802/stp.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/8021q 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/8021q/8021q.ko 2024-03-13T14:24:56.578 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet_fd.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet_rdma.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/9p/9pnet_virtio.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/batman-adv 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/batman-adv/batman-adv.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/br_netfilter.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/bridge.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-13T14:24:56.579 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-13T14:24:56.580 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/ebtables.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ceph 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ceph/libceph.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/core 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/core/failover.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/core/pktgen.ko 2024-03-13T14:24:56.581 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp_diag.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp_ipv4.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dccp/dccp_ipv6.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dns_resolver 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dns_resolver/dns_resolver.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/dsa_core.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_brcm.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_dsa.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_none.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/dsa/tag_trailer.ko 2024-03-13T14:24:56.582 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/6lowpan 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/ieee802154.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ah4.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/esp4.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/fou.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/gre.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/inet_diag.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ip_gre.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ip_tunnel.ko 2024-03-13T14:24:56.583 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ip_vti.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ipcomp.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/ipip.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-13T14:24:56.584 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_bic.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_dctcp.ko 2024-03-13T14:24:56.585 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_diag.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_highspeed.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_htcp.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_hybla.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_illinois.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_lp.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_scalable.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_vegas.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_veno.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_westwood.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tcp_yeah.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/tunnel4.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/udp_diag.ko 2024-03-13T14:24:56.586 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/udp_tunnel.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ah6.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/esp6.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/fou6.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ip6_tunnel.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ip6_vti.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/ipcomp6.ko 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter 2024-03-13T14:24:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-13T14:24:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/sit.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/tunnel6.ko 2024-03-13T14:24:56.589 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/key 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/key/af_key.ko 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_core.ko 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_eth.ko 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_ip.ko 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_ip6.ko 2024-03-13T14:24:56.590 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_netlink.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/l2tp/l2tp_ppp.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/llc 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/llc/llc.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mac802154 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mac802154/mac802154.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mpls 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/mpls/mpls_gso.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-13T14:24:56.591 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-13T14:24:56.592 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-13T14:24:56.593 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conncount.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-13T14:24:56.594 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_flow_table.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-13T14:24:56.595 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_log_syslog.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_irc.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_sip.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nf_tables.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-13T14:24:56.596 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_log.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_chain_nat.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_compat.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_ct.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_fib.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_hash.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_limit.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_log.ko 2024-03-13T14:24:56.597 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_masq.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_nat.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_numgen.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_queue.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_quota.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_redir.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_reject.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_reject_inet.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_socket.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/nft_tunnel.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_AUDIT.ko 2024-03-13T14:24:56.598 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_CT.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_DSCP.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_HL.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_HMARK.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_LED.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_LOG.ko 2024-03-13T14:24:56.599 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_NETMAP.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_NFLOG.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_RATEEST.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_SECMARK.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TEE.ko 2024-03-13T14:24:56.600 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TPROXY.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_TRACE.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_addrtype.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_bpf.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_cgroup.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_cluster.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_comment.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connbytes.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connlabel.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connlimit.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_connmark.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_conntrack.ko 2024-03-13T14:24:56.601 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_cpu.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_dccp.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_devgroup.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_dscp.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_ecn.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_esp.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_hashlimit.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_helper.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_hl.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_ipcomp.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_iprange.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_ipvs.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_l2tp.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_length.ko 2024-03-13T14:24:56.602 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_limit.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_mac.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_mark.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_multiport.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_nat.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_nfacct.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_osf.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_owner.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_physdev.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_pkttype.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_policy.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_quota.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_rateest.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_realm.ko 2024-03-13T14:24:56.603 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_recent.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_sctp.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_set.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_socket.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_state.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_statistic.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_string.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_tcpmss.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_time.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netfilter/xt_u32.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netlink 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/netlink/netlink_diag.ko 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/nsh 2024-03-13T14:24:56.604 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/nsh/nsh.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/openvswitch.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/vport-geneve.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/vport-gre.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/openvswitch/vport-vxlan.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/packet 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/packet/af_packet_diag.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds/rds.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds/rds_rdma.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/rds/rds_tcp.ko 2024-03-13T14:24:56.605 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_csum.ko 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_gact.ko 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_mirred.ko 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_nat.ko 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_pedit.ko 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_police.ko 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_simple.ko 2024-03-13T14:24:56.606 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_skbedit.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/act_vlan.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_basic.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_bpf.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_flow.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_fw.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_route.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/cls_u32.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_cmp.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_ipset.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_meta.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_nbyte.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_text.ko 2024-03-13T14:24:56.607 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/em_u32.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_choke.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_codel.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_drr.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_fq.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_gred.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_hfsc.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_hhf.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_htb.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_ingress.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_mqprio.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_mqprio_lib.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_multiq.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_netem.ko 2024-03-13T14:24:56.608 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_pie.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_plug.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_prio.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_qfq.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_red.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_sfb.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_sfq.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_tbf.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sched/sch_teql.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sctp 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sctp/sctp.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sctp/sctp_diag.ko 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/auth_gss 2024-03-13T14:24:56.609 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/sunrpc.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/xprtrdma 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/tipc 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/tipc/diag.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/tipc/tipc.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/unix 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/unix/unix_diag.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vsock.ko 2024-03-13T14:24:56.610 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/xfrm 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/encrypted-keys 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/trusted-keys 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/security/keys/trusted-keys/trusted.ko 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/virt 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/virt/lib 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/kernel/virt/lib/irqbypass.ko 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.alias 2024-03-13T14:24:56.611 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.alias.bin 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin.alias.bin 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin.bin 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.builtin.modinfo 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.dep 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.dep.bin 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.devname 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.order 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.softdep 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.symbols 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/modules.symbols.bin 2024-03-13T14:24:56.612 INFO:teuthology.orchestra.run.smithi176.stdout:/lib/modules/6.8.0-rc7-ga87fde6e6336/vmlinuz 2024-03-13T14:24:56.616 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:56.617 DEBUG:teuthology.orchestra.run.smithi176:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-13T14:24:56.638 INFO:teuthology.orchestra.run.smithi176.stdout:postinstall scriptlet (using /bin/sh): 2024-03-13T14:24:56.638 INFO:teuthology.orchestra.run.smithi176.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout: /usr/bin/kernel-install add 6.8.0-rc7-ga87fde6e6336 /lib/modules/6.8.0-rc7-ga87fde6e6336/vmlinuz 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout:fi 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout:for file in vmlinuz System.map config; do 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-ga87fde6e6336/${file}" "/boot/${file}-6.8.0-rc7-ga87fde6e6336"; then 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout: cp "/lib/modules/6.8.0-rc7-ga87fde6e6336/${file}" "/boot/${file}-6.8.0-rc7-ga87fde6e6336" 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout: fi 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout:done 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout:new-kernel-pkg --remove 6.8.0-rc7-ga87fde6e6336 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-ga87fde6e6336.img 2024-03-13T14:24:56.639 INFO:teuthology.orchestra.run.smithi176.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-13T14:24:56.640 INFO:teuthology.orchestra.run.smithi176.stdout:kernel-install remove 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:56.640 INFO:teuthology.orchestra.run.smithi176.stdout:fi 2024-03-13T14:24:56.640 INFO:teuthology.orchestra.run.smithi176.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-13T14:24:56.640 INFO:teuthology.orchestra.run.smithi176.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-13T14:24:56.640 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:56.640 INFO:teuthology.orchestra.run.smithi176.stdout:fi 2024-03-13T14:24:56.641 DEBUG:teuthology.orchestra.run.smithi176:> sudo rpm -qi grub2-tools 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Name : grub2-tools 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Epoch : 1 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Version : 2.06 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Release : 68.el9 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Architecture: x86_64 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Group : Unspecified 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Size : 8264254 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:License : GPLv3+ 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-13T14:24:56.716 INFO:teuthology.orchestra.run.smithi176.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:Packager : builder@centos.org 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:Vendor : CentOS 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:Summary : Support tools for GRUB. 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:Description : 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:hardware devices. 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:24:56.717 INFO:teuthology.orchestra.run.smithi176.stdout:This subpackage provides tools for support of all platforms. 2024-03-13T14:24:56.718 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-13T14:24:56.721 INFO:teuthology.task.kernel:Updating grub on smithi176 to boot 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:56.721 DEBUG:teuthology.orchestra.run.smithi176:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-13T14:24:57.318 INFO:teuthology.orchestra.run.smithi176.stderr:Generating grub configuration file ... 2024-03-13T14:24:58.662 INFO:teuthology.orchestra.run.smithi176.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-13T14:24:58.692 INFO:teuthology.orchestra.run.smithi176.stderr:done 2024-03-13T14:24:58.694 DEBUG:teuthology.orchestra.run.smithi176:> mktemp 2024-03-13T14:24:58.709 INFO:teuthology.orchestra.run.smithi176.stdout:/tmp/tmp.lHc8N5YbUv 2024-03-13T14:24:58.709 DEBUG:teuthology.orchestra.run.smithi176:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lHc8N5YbUv 2024-03-13T14:24:58.795 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0666 /tmp/tmp.lHc8N5YbUv 2024-03-13T14:24:58.889 DEBUG:teuthology.orchestra.remote:smithi176:/tmp/tmp.lHc8N5YbUv is 6KB 2024-03-13T14:24:58.937 DEBUG:teuthology.orchestra.run.smithi176:> rm -fr /tmp/tmp.lHc8N5YbUv 2024-03-13T14:24:58.951 DEBUG:teuthology.orchestra.run.smithi176:> sudo /bin/ls /boot/loader/entries || true 2024-03-13T14:24:59.017 INFO:teuthology.orchestra.run.smithi176.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-13T14:24:59.018 INFO:teuthology.orchestra.run.smithi176.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-13T14:24:59.018 INFO:teuthology.orchestra.run.smithi176.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-13T14:24:59.018 INFO:teuthology.orchestra.run.smithi176.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-ga87fde6e6336.conf 2024-03-13T14:24:59.019 DEBUG:teuthology.orchestra.run.smithi176:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-ga87fde6e6336 2024-03-13T14:24:59.179 DEBUG:teuthology.orchestra.run.smithi176:> sudo shutdown -r now 2024-03-13T14:25:09.682 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-03-13T14:25:09.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-13T14:25:09.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:25:29.198 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-03-13T14:25:29.199 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-03-13T14:25:29.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:26:09.727 DEBUG:teuthology.orchestra.remote:timed out 2024-03-13T14:26:18.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-13T14:26:18.737 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:26:26.411 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-13T14:26:27.015 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-13T14:26:27.015 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "6.8.0-rc7-ga87fde6e6336"... 2024-03-13T14:26:27.015 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-03-13T14:26:27.055 INFO:teuthology.orchestra.run.smithi196.stdout:6.8.0-rc7-ga87fde6e6336 2024-03-13T14:26:27.055 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-ga87fde6e6336 vs 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:26:27.055 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-13T14:26:27.056 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-13T14:26:28.056 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-03-13T14:26:28.057 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-13T14:26:28.169 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-03-13T14:26:28.220 DEBUG:teuthology.parallel:result is None 2024-03-13T14:26:29.201 DEBUG:teuthology.orchestra.remote:timed out 2024-03-13T14:26:38.202 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-03-13T14:26:38.203 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:26:39.603 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-03-13T14:26:40.221 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-03-13T14:26:40.222 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "6.8.0-rc7-ga87fde6e6336"... 2024-03-13T14:26:40.222 DEBUG:teuthology.orchestra.run.smithi176:> uname -r 2024-03-13T14:26:40.263 INFO:teuthology.orchestra.run.smithi176.stdout:6.8.0-rc7-ga87fde6e6336 2024-03-13T14:26:40.263 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-ga87fde6e6336 vs 6.8.0-rc7-ga87fde6e6336 2024-03-13T14:26:40.263 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-13T14:26:40.264 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-13T14:26:41.265 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-03-13T14:26:41.265 DEBUG:teuthology.orchestra.run.smithi176:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-13T14:26:41.408 INFO:teuthology.orchestra.run.smithi176.stdout:ttyS1 2024-03-13T14:26:41.462 DEBUG:teuthology.parallel:result is None 2024-03-13T14:26:41.462 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-13T14:26:41.471 INFO:teuthology.task.internal:Creating test directory... 2024-03-13T14:26:41.471 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-13T14:26:41.476 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-13T14:26:41.528 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-13T14:26:41.535 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-13T14:26:41.545 INFO:teuthology.task.internal:Creating archive directory... 2024-03-13T14:26:41.546 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-13T14:26:41.564 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-13T14:26:41.634 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-13T14:26:41.640 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-13T14:26:41.641 DEBUG:teuthology.orchestra.run.smithi176:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-13T14:26:41.646 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-13T14:26:41.731 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T14:26:41.750 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T14:26:41.766 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T14:26:41.790 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-13T14:26:41.795 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-13T14:26:41.803 INFO:teuthology.task.internal:Configuring sudo... 2024-03-13T14:26:41.803 DEBUG:teuthology.orchestra.run.smithi176:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-13T14:26:41.808 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-13T14:26:41.899 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-13T14:26:41.911 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-13T14:26:41.911 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-13T14:26:41.915 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-13T14:26:41.971 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T14:26:42.062 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T14:26:42.149 DEBUG:teuthology.orchestra.run.smithi176:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T14:26:42.217 DEBUG:teuthology.orchestra.run.smithi176:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T14:26:42.302 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:26:42.302 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-13T14:26:42.371 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T14:26:42.460 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-13T14:26:42.564 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T14:26:42.650 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-13T14:26:42.740 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:26:42.740 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-13T14:26:42.830 DEBUG:teuthology.orchestra.run.smithi176:> sudo service rsyslog restart 2024-03-13T14:26:42.835 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-03-13T14:26:42.929 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T14:26:43.006 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T14:26:43.277 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-13T14:26:43.285 INFO:teuthology.task.internal:Starting timer... 2024-03-13T14:26:43.285 INFO:teuthology.run_tasks:Running task pcp... 2024-03-13T14:26:43.295 INFO:teuthology.run_tasks:Running task selinux... 2024-03-13T14:26:43.305 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-13T14:26:43.306 DEBUG:teuthology.orchestra.run.smithi176:> sudo service auditd rotate 2024-03-13T14:26:43.410 INFO:teuthology.orchestra.run.smithi176.stdout:Rotating logs: 2024-03-13T14:26:43.414 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-03-13T14:26:43.563 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-03-13T14:26:43.568 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-13T14:26:43.568 DEBUG:teuthology.orchestra.run.smithi176:> /usr/sbin/getenforce 2024-03-13T14:26:43.614 INFO:teuthology.orchestra.run.smithi176.stdout:Permissive 2024-03-13T14:26:43.615 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-03-13T14:26:43.686 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-03-13T14:26:43.686 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi176.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-03-13T14:26:43.687 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T14:26:43.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T14:26:43.767 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T14:26:43.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T14:26:43.863 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-13T14:26:43.864 DEBUG:teuthology.orchestra.run.smithi176:> sudo /usr/sbin/setenforce permissive 2024-03-13T14:26:43.944 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-03-13T14:26:44.050 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-13T14:26:44.060 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-13T14:26:44.081 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-13T14:26:44.083 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi176.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-13T14:32:40.461 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi176.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-03-13T14:32:40.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-03-13T14:32:40.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi176.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:32:40.615 DEBUG:teuthology.orchestra.run.smithi176:> true 2024-03-13T14:32:40.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi176.front.sepia.ceph.com' 2024-03-13T14:32:40.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-13T14:32:40.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-13T14:32:40.906 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-03-13T14:32:41.052 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-03-13T14:32:41.052 INFO:teuthology.run_tasks:Running task clock... 2024-03-13T14:32:41.064 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-13T14:32:41.065 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-13T14:32:41.065 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T14:32:41.073 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-13T14:32:41.073 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T14:32:41.169 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-13T14:32:41.173 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-13T14:32:41.220 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-13T14:32:41.224 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-13T14:32:41.310 INFO:teuthology.orchestra.run.smithi176.stderr:sudo: ntpd: command not found 2024-03-13T14:32:41.318 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-03-13T14:32:41.352 INFO:teuthology.orchestra.run.smithi176.stdout:506 Cannot talk to daemon 2024-03-13T14:32:41.361 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-03-13T14:32:41.405 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-13T14:32:41.412 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-13T14:32:41.462 INFO:teuthology.orchestra.run.smithi176.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-13T14:32:41.462 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-13T14:32:41.585 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-03-13T14:32:41.587 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-03-13T14:32:41.594 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T14:32:41.594 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-03-13T14:32:41.594 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.594 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.594 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.594 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.595 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T14:32:41.595 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-03-13T14:32:41.595 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.595 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.595 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.595 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T14:32:41.597 INFO:teuthology.run_tasks:Running task install... 2024-03-13T14:32:41.606 DEBUG:teuthology.task.install:project ceph 2024-03-13T14:32:41.606 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9a75c81fd074f3cb252248b860d5cc88e5f3fb55'}} 2024-03-13T14:32:41.606 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.1', 'flavor': 'default', 'sha1': '9a75c81fd074f3cb252248b860d5cc88e5f3fb55'} 2024-03-13T14:32:41.606 INFO:teuthology.task.install:Using flavor: default 2024-03-13T14:32:41.613 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-13T14:32:41.613 INFO:teuthology.task.install:extra packages: [] 2024-03-13T14:32:41.613 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a75c81fd074f3cb252248b860d5cc88e5f3fb55', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-03-13T14:32:41.613 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-13T14:32:41.614 INFO:teuthology.packaging:ref: None 2024-03-13T14:32:41.614 INFO:teuthology.packaging:tag: v18.2.1 2024-03-13T14:32:41.614 INFO:teuthology.packaging:branch: None 2024-03-13T14:32:41.614 INFO:teuthology.packaging:sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:32:41.996 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.1^{} -> None 2024-03-13T14:32:41.997 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-13T14:32:44.222 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.1^{} -> 7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-13T14:32:44.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-13T14:32:44.226 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9a75c81fd074f3cb252248b860d5cc88e5f3fb55', 'tag': 'v18.2.1', 'wait_for_package': False} 2024-03-13T14:32:44.226 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-13T14:32:44.226 INFO:teuthology.packaging:ref: None 2024-03-13T14:32:44.226 INFO:teuthology.packaging:tag: v18.2.1 2024-03-13T14:32:44.226 INFO:teuthology.packaging:branch: None 2024-03-13T14:32:44.227 INFO:teuthology.packaging:sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:32:44.227 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-13T14:32:44.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-13T14:32:44.372 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-03-13T14:32:44.372 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-03-13T14:32:44.407 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-13T14:32:44.407 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:32:44.407 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-13T14:32:44.454 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/ 2024-03-13T14:32:44.455 INFO:teuthology.task.install.rpm:Package version is 18.2.1-0 2024-03-13T14:32:44.483 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/7fe91d5d5842e04be3b4f514d6dd990c54b29c76/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-13T14:32:44.483 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:32:44.483 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-13T14:32:44.511 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-13T14:32:44.511 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-13T14:32:44.512 INFO:teuthology.packaging:ref: None 2024-03-13T14:32:44.512 INFO:teuthology.packaging:tag: v18.2.1 2024-03-13T14:32:44.512 INFO:teuthology.packaging:branch: None 2024-03-13T14:32:44.512 INFO:teuthology.packaging:sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:32:44.512 DEBUG:teuthology.orchestra.run.smithi176:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-13T14:32:44.580 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-13T14:32:44.580 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-13T14:32:44.580 INFO:teuthology.packaging:ref: None 2024-03-13T14:32:44.581 INFO:teuthology.packaging:tag: v18.2.1 2024-03-13T14:32:44.581 INFO:teuthology.packaging:branch: None 2024-03-13T14:32:44.581 INFO:teuthology.packaging:sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:32:44.581 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-13T14:32:44.645 DEBUG:teuthology.orchestra.run.smithi176:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-13T14:32:44.724 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-13T14:32:44.774 DEBUG:teuthology.orchestra.run.smithi176:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-13T14:32:44.849 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-13T14:32:44.865 INFO:teuthology.orchestra.run.smithi176.stdout:check_obsoletes = 1 2024-03-13T14:32:44.869 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum clean all 2024-03-13T14:32:44.941 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-03-13T14:32:44.946 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-03-13T14:32:45.228 INFO:teuthology.orchestra.run.smithi176.stdout:74 files removed 2024-03-13T14:32:45.270 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-radosgw 2024-03-13T14:32:45.296 INFO:teuthology.orchestra.run.smithi196.stdout:74 files removed 2024-03-13T14:32:45.339 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw 2024-03-13T14:32:45.988 INFO:teuthology.orchestra.run.smithi176.stdout:ceph packages for x86_64 368 kB/s | 76 kB 00:00 2024-03-13T14:32:46.014 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 460 kB/s | 76 kB 00:00 2024-03-13T14:32:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 56 kB/s | 9.4 kB 00:00 2024-03-13T14:32:46.263 INFO:teuthology.orchestra.run.smithi176.stdout:ceph noarch packages 44 kB/s | 9.4 kB 00:00 2024-03-13T14:32:46.482 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 10 kB/s | 2.2 kB 00:00 2024-03-13T14:32:46.488 INFO:teuthology.orchestra.run.smithi176.stdout:ceph source packages 13 kB/s | 2.2 kB 00:00 2024-03-13T14:32:47.135 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-13T14:32:47.204 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-03-13T14:32:48.726 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-13T14:32:48.812 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-03-13T14:32:49.783 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-13T14:32:50.431 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-03-13T14:32:52.093 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-03-13T14:32:52.424 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-13T14:32:56.693 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.1 MB 00:00 2024-03-13T14:32:58.563 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2024-03-13T14:33:05.957 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - CRB 642 kB/s | 6.1 MB 00:09 2024-03-13T14:33:07.616 INFO:teuthology.orchestra.run.smithi176.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 15 kB 00:00 2024-03-13T14:33:08.584 INFO:teuthology.orchestra.run.smithi176.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-03-13T14:33:14.367 INFO:teuthology.orchestra.run.smithi176.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-03-13T14:33:17.169 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:33:17.176 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:17.176 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:33:17.176 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:17.176 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:33:17.176 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-03-13T14:33:17.176 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrading: 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-13T14:33:17.177 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-13T14:33:17.178 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:17.179 INFO:teuthology.orchestra.run.smithi176.stdout:Install 28 Packages 2024-03-13T14:33:17.180 INFO:teuthology.orchestra.run.smithi176.stdout:Upgrade 2 Packages 2024-03-13T14:33:17.180 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:17.180 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 53 M 2024-03-13T14:33:17.180 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:33:18.027 INFO:teuthology.orchestra.run.smithi176.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 8.8 MB/s | 5.2 MB 00:00 2024-03-13T14:33:18.077 INFO:teuthology.orchestra.run.smithi176.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 491 kB/s | 24 kB 00:00 2024-03-13T14:33:18.227 INFO:teuthology.orchestra.run.smithi176.stdout:(3/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 9.7 MB/s | 7.7 MB 00:00 2024-03-13T14:33:18.269 INFO:teuthology.orchestra.run.smithi176.stdout:(4/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 3.4 MB/s | 658 kB 00:00 2024-03-13T14:33:18.319 INFO:teuthology.orchestra.run.smithi176.stdout:(5/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 5.1 MB/s | 474 kB 00:00 2024-03-13T14:33:18.378 INFO:teuthology.orchestra.run.smithi176.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 783 kB/s | 45 kB 00:00 2024-03-13T14:33:18.436 INFO:teuthology.orchestra.run.smithi176.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 2.1 MB/s | 124 kB 00:00 2024-03-13T14:33:18.503 INFO:teuthology.orchestra.run.smithi176.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 2.4 MB/s | 161 kB 00:00 2024-03-13T14:33:18.604 INFO:teuthology.orchestra.run.smithi176.stdout:(9/30): librgw2-18.2.1-0.el9.x86_64.rpm 13 MB/s | 4.5 MB 00:00 2024-03-13T14:33:18.645 INFO:teuthology.orchestra.run.smithi176.stdout:(10/30): python3-rados-18.2.1-0.el9.x86_64.rpm 2.2 MB/s | 321 kB 00:00 2024-03-13T14:33:18.713 INFO:teuthology.orchestra.run.smithi176.stdout:(11/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 2.7 MB/s | 297 kB 00:00 2024-03-13T14:33:18.746 INFO:teuthology.orchestra.run.smithi176.stdout:(12/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 996 kB/s | 99 kB 00:00 2024-03-13T14:33:19.096 INFO:teuthology.orchestra.run.smithi176.stdout:(13/30): ceph-common-18.2.1-0.el9.x86_64.rpm 11 MB/s | 18 MB 00:01 2024-03-13T14:33:19.138 INFO:teuthology.orchestra.run.smithi176.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 249 kB/s | 106 kB 00:00 2024-03-13T14:33:19.155 INFO:teuthology.orchestra.run.smithi176.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 824 kB/s | 44 kB 00:00 2024-03-13T14:33:19.197 INFO:teuthology.orchestra.run.smithi176.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 418 kB/s | 159 kB 00:00 2024-03-13T14:33:19.389 INFO:teuthology.orchestra.run.smithi176.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 661 kB 00:00 2024-03-13T14:33:19.514 INFO:teuthology.orchestra.run.smithi176.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 790 kB/s | 282 kB 00:00 2024-03-13T14:33:19.531 INFO:teuthology.orchestra.run.smithi176.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 124 kB/s | 41 kB 00:00 2024-03-13T14:33:19.990 INFO:teuthology.orchestra.run.smithi176.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 185 kB/s | 111 kB 00:00 2024-03-13T14:33:20.107 INFO:teuthology.orchestra.run.smithi176.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-03-13T14:33:20.148 INFO:teuthology.orchestra.run.smithi176.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 488 kB/s | 309 kB 00:00 2024-03-13T14:33:20.291 INFO:teuthology.orchestra.run.smithi176.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-03-13T14:33:20.641 INFO:teuthology.orchestra.run.smithi176.stdout:(24/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-03-13T14:33:20.749 INFO:teuthology.orchestra.run.smithi176.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-03-13T14:33:20.800 INFO:teuthology.orchestra.run.smithi176.stdout:(26/30): liboath-2.6.7-2.el9.x86_64.rpm 70 kB/s | 49 kB 00:00 2024-03-13T14:33:20.992 INFO:teuthology.orchestra.run.smithi176.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 17 MB/s | 3.3 MB 00:00 2024-03-13T14:33:21.056 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 945 kB/s | 21 MB 00:22 2024-03-13T14:33:21.159 INFO:teuthology.orchestra.run.smithi176.stdout:(28/30): librbd1-18.2.1-0.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-03-13T14:33:21.201 INFO:teuthology.orchestra.run.smithi176.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 3.5 MB/s | 1.6 MB 00:00 2024-03-13T14:33:21.792 INFO:teuthology.orchestra.run.smithi176.stdout:(30/30): libarrow-9.0.0-10.el9.x86_64.rpm 2.0 MB/s | 4.4 MB 00:02 2024-03-13T14:33:21.801 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:33:21.802 INFO:teuthology.orchestra.run.smithi176.stdout:Total 11 MB/s | 53 MB 00:04 2024-03-13T14:33:21.803 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:33:21.877 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:33:21.877 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:33:22.382 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:33:22.383 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:33:23.889 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:33:23.986 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-13T14:33:24.024 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-13T14:33:24.126 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-13T14:33:24.519 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-13T14:33:24.563 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-13T14:33:24.661 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-13T14:33:24.757 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-03-13T14:33:24.781 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-13T14:33:24.847 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-13T14:33:24.888 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-13T14:33:24.945 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-03-13T14:33:25.045 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-13T14:33:25.075 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-13T14:33:25.166 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-13T14:33:25.343 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-13T14:33:25.389 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-13T14:33:25.449 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-13T14:33:25.484 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-13T14:33:25.524 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-13T14:33:25.574 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-13T14:33:25.622 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-13T14:33:26.008 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-13T14:33:26.039 INFO:teuthology.orchestra.run.smithi176.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-13T14:33:26.136 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-13T14:33:26.170 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-03-13T14:33:26.260 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-13T14:33:26.769 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-13T14:33:26.808 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-13T14:33:26.883 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-03-13T14:33:26.899 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-13T14:33:27.000 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-03-13T14:33:27.036 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-03-13T14:33:29.584 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-13T14:33:29.646 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-13T14:33:29.689 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:33:29.704 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:18.2.1-0.el9 ceph 7.7 M 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:18.2.1-0.el9 ceph 3.3 M 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-13T14:33:29.705 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:18.2.1-0.el9 ceph 5.2 M 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:18.2.1-0.el9 ceph 24 k 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:18.2.1-0.el9 ceph 658 k 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-13T14:33:29.706 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:18.2.1-0.el9 ceph 474 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:18.2.1-0.el9 ceph 4.5 M 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:18.2.1-0.el9 ceph 45 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:18.2.1-0.el9 ceph 124 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:18.2.1-0.el9 ceph 161 k 2024-03-13T14:33:29.707 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:18.2.1-0.el9 ceph 321 k 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:18.2.1-0.el9 ceph 297 k 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:18.2.1-0.el9 ceph 99 k 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout:Install 28 Packages 2024-03-13T14:33:29.708 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 2 Packages 2024-03-13T14:33:29.709 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:33:29.709 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 53 M 2024-03-13T14:33:29.709 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:33:30.010 INFO:teuthology.orchestra.run.smithi196.stdout:(1/30): ceph-base-18.2.1-0.el9.x86_64.rpm 21 MB/s | 5.2 MB 00:00 2024-03-13T14:33:30.034 INFO:teuthology.orchestra.run.smithi196.stdout:(2/30): ceph-selinux-18.2.1-0.el9.x86_64.rpm 983 kB/s | 24 kB 00:00 2024-03-13T14:33:30.102 INFO:teuthology.orchestra.run.smithi196.stdout:(3/30): libcephfs2-18.2.1-0.el9.x86_64.rpm 9.7 MB/s | 658 kB 00:00 2024-03-13T14:33:30.144 INFO:teuthology.orchestra.run.smithi196.stdout:(4/30): libradosstriper1-18.2.1-0.el9.x86_64.rp 11 MB/s | 474 kB 00:00 2024-03-13T14:33:30.275 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-13T14:33:30.310 INFO:teuthology.orchestra.run.smithi196.stdout:(5/30): ceph-radosgw-18.2.1-0.el9.x86_64.rpm 14 MB/s | 7.7 MB 00:00 2024-03-13T14:33:30.318 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-13T14:33:30.327 INFO:teuthology.orchestra.run.smithi196.stdout:(6/30): python3-ceph-argparse-18.2.1-0.el9.x86_ 2.8 MB/s | 45 kB 00:00 2024-03-13T14:33:30.361 INFO:teuthology.orchestra.run.smithi196.stdout:(7/30): python3-ceph-common-18.2.1-0.el9.x86_64 3.7 MB/s | 124 kB 00:00 2024-03-13T14:33:30.395 INFO:teuthology.orchestra.run.smithi196.stdout:(8/30): python3-cephfs-18.2.1-0.el9.x86_64.rpm 4.9 MB/s | 161 kB 00:00 2024-03-13T14:33:30.436 INFO:teuthology.orchestra.run.smithi196.stdout:(9/30): python3-rados-18.2.1-0.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-13T14:33:30.469 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-13T14:33:30.469 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-13T14:33:30.470 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-13T14:33:30.470 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:30.478 INFO:teuthology.orchestra.run.smithi196.stdout:(10/30): python3-rbd-18.2.1-0.el9.x86_64.rpm 7.1 MB/s | 297 kB 00:00 2024-03-13T14:33:30.503 INFO:teuthology.orchestra.run.smithi196.stdout:(11/30): python3-rgw-18.2.1-0.el9.x86_64.rpm 4.0 MB/s | 99 kB 00:00 2024-03-13T14:33:30.519 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-13T14:33:30.663 INFO:teuthology.orchestra.run.smithi196.stdout:(12/30): librgw2-18.2.1-0.el9.x86_64.rpm 8.6 MB/s | 4.5 MB 00:00 2024-03-13T14:33:30.729 INFO:teuthology.orchestra.run.smithi196.stdout:(13/30): boost-program-options-1.75.0-8.el9.x86 471 kB/s | 106 kB 00:00 2024-03-13T14:33:30.746 INFO:teuthology.orchestra.run.smithi196.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-13T14:33:30.772 INFO:teuthology.orchestra.run.smithi196.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2024-03-13T14:33:30.905 INFO:teuthology.orchestra.run.smithi196.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 661 kB 00:00 2024-03-13T14:33:30.939 INFO:teuthology.orchestra.run.smithi196.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 282 kB 00:00 2024-03-13T14:33:30.964 INFO:teuthology.orchestra.run.smithi196.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 740 kB/s | 41 kB 00:00 2024-03-13T14:33:31.231 INFO:teuthology.orchestra.run.smithi196.stdout:(19/30): ceph-common-18.2.1-0.el9.x86_64.rpm 13 MB/s | 18 MB 00:01 2024-03-13T14:33:31.456 INFO:teuthology.orchestra.run.smithi196.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 215 kB/s | 111 kB 00:00 2024-03-13T14:33:31.565 INFO:teuthology.orchestra.run.smithi196.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-03-13T14:33:31.715 INFO:teuthology.orchestra.run.smithi196.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 325 kB/s | 49 kB 00:00 2024-03-13T14:33:31.882 INFO:teuthology.orchestra.run.smithi196.stdout:(23/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 337 kB/s | 309 kB 00:00 2024-03-13T14:33:31.907 INFO:teuthology.orchestra.run.smithi196.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 352 kB/s | 67 kB 00:00 2024-03-13T14:33:32.766 INFO:teuthology.orchestra.run.smithi196.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 223 kB/s | 192 kB 00:00 2024-03-13T14:33:35.466 INFO:teuthology.orchestra.run.smithi196.stdout:(26/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 234 kB/s | 838 kB 00:03 2024-03-13T14:33:35.591 INFO:teuthology.orchestra.run.smithi196.stdout:(27/30): librados2-18.2.1-0.el9.x86_64.rpm 26 MB/s | 3.3 MB 00:00 2024-03-13T14:33:35.699 INFO:teuthology.orchestra.run.smithi196.stdout:(28/30): librbd1-18.2.1-0.el9.x86_64.rpm 28 MB/s | 3.0 MB 00:00 2024-03-13T14:33:40.449 INFO:teuthology.orchestra.run.smithi196.stdout:(29/30): thrift-0.15.0-2.el9.x86_64.rpm 211 kB/s | 1.6 MB 00:07 2024-03-13T14:33:41.356 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-13T14:33:41.357 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /sys 2024-03-13T14:33:41.357 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /proc 2024-03-13T14:33:41.357 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /mnt 2024-03-13T14:33:41.357 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /var/tmp 2024-03-13T14:33:41.357 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /home 2024-03-13T14:33:41.357 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /root 2024-03-13T14:33:41.358 INFO:teuthology.orchestra.run.smithi176.stdout:skipping the directory /tmp 2024-03-13T14:33:41.358 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:42.223 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-13T14:33:42.319 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-13T14:33:42.319 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:33:42.319 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-13T14:33:42.319 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T14:33:42.319 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T14:33:42.320 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:42.324 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-13T14:33:42.444 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-13T14:33:42.444 INFO:teuthology.orchestra.run.smithi176.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-13T14:33:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-13T14:33:43.850 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-13T14:33:43.851 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-03-13T14:33:43.852 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-13T14:33:43.853 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-03-13T14:33:44.560 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout:Upgraded: 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-13T14:33:44.561 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.562 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: re2-1:20211101-3.el9.x86_64 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:44.563 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:33:44.952 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-test 2024-03-13T14:33:45.234 INFO:teuthology.orchestra.run.smithi196.stdout:(30/30): libarrow-9.0.0-10.el9.x86_64.rpm 323 kB/s | 4.4 MB 00:14 2024-03-13T14:33:45.241 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:33:45.241 INFO:teuthology.orchestra.run.smithi196.stdout:Total 3.4 MB/s | 53 MB 00:15 2024-03-13T14:33:45.242 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:33:45.321 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:33:45.321 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:33:45.807 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:31 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:33:45.828 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:33:45.829 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:33:45.974 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-13T14:33:45.975 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout:Install 6 Packages 2024-03-13T14:33:45.976 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:45.977 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 41 M 2024-03-13T14:33:45.977 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 167 M 2024-03-13T14:33:45.977 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:33:46.327 INFO:teuthology.orchestra.run.smithi176.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 871 kB/s | 165 kB 00:00 2024-03-13T14:33:46.394 INFO:teuthology.orchestra.run.smithi176.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 847 kB/s | 217 kB 00:00 2024-03-13T14:33:46.436 INFO:teuthology.orchestra.run.smithi176.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 300 kB 00:00 2024-03-13T14:33:46.461 INFO:teuthology.orchestra.run.smithi176.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 973 kB/s | 63 kB 00:00 2024-03-13T14:33:46.604 INFO:teuthology.orchestra.run.smithi176.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-13T14:33:47.400 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:33:47.487 INFO:teuthology.orchestra.run.smithi176.stdout:(6/6): ceph-test-18.2.1-0.el9.x86_64.rpm 29 MB/s | 40 MB 00:01 2024-03-13T14:33:47.493 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:33:47.494 INFO:teuthology.orchestra.run.smithi176.stdout:Total 27 MB/s | 41 MB 00:01 2024-03-13T14:33:47.502 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-03-13T14:33:47.534 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:33:47.556 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-03-13T14:33:47.588 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:33:47.588 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:33:47.654 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-03-13T14:33:47.979 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:33:47.979 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:33:48.064 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-03-13T14:33:48.111 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-13T14:33:48.243 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:18.2.1-0.el9.x86_64 5/32 2024-03-13T14:33:48.341 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:18.2.1-0.el9.x86_64 6/32 2024-03-13T14:33:48.370 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-13T14:33:48.453 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:18.2.1-0.el9.x86_64 7/32 2024-03-13T14:33:48.498 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-03-13T14:33:48.570 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 9/32 2024-03-13T14:33:48.638 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:33:48.672 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-13T14:33:48.706 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-13T14:33:48.729 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-13T14:33:48.774 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-13T14:33:48.798 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:18.2.1-0.el9.x86_64 11/32 2024-03-13T14:33:48.808 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T14:33:48.879 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-13T14:33:48.907 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-13T14:33:48.972 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-03-13T14:33:49.009 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-03-13T14:33:49.062 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-03-13T14:33:49.098 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-03-13T14:33:49.169 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-03-13T14:33:49.224 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-03-13T14:33:49.270 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-03-13T14:33:49.662 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-03-13T14:33:49.695 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-13T14:33:49.808 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:18.2.1-0.el9.x86_64 20/32 2024-03-13T14:33:49.843 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:18.2.1-0.el9.x86_64 21/32 2024-03-13T14:33:49.912 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-03-13T14:33:50.395 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-03-13T14:33:50.423 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-13T14:33:50.514 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:18.2.1-0.el9.x86_64 24/32 2024-03-13T14:33:50.615 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:18.2.1-0.el9.x86_64 25/32 2024-03-13T14:33:50.650 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:18.2.1-0.el9.x86_64 26/32 2024-03-13T14:33:53.066 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-13T14:33:53.088 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-13T14:33:53.130 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-13T14:33:53.180 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-13T14:33:53.756 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:18.2.1-0.el9.x86_64 27/32 2024-03-13T14:33:53.799 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-13T14:33:53.943 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:18.2.1-0.el9.x86_64 28/32 2024-03-13T14:33:53.943 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-13T14:33:53.943 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-13T14:33:53.943 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:33:54.021 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-13T14:33:55.353 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-13T14:33:55.353 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-03-13T14:33:55.353 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-03-13T14:33:55.353 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T14:33:55.353 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-13T14:33:55.353 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-13T14:33:55.661 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-13T14:33:55.662 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:55.662 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:33:55.662 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-13T14:33:55.662 INFO:teuthology.orchestra.run.smithi176.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-13T14:33:55.662 INFO:teuthology.orchestra.run.smithi176.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-13T14:33:55.663 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:55.663 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:33:55.980 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph 2024-03-13T14:33:56.831 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:42 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:33:56.996 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:33:56.999 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:33:56.999 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:56.999 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:33:56.999 INFO:teuthology.orchestra.run.smithi176.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-13T14:33:57.000 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-13T14:33:57.001 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-13T14:33:57.002 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-13T14:33:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-13T14:33:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:57.003 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:33:57.003 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:33:57.003 INFO:teuthology.orchestra.run.smithi176.stdout:Install 33 Packages 2024-03-13T14:33:57.003 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:33:57.003 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 30 M 2024-03-13T14:33:57.004 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 110 M 2024-03-13T14:33:57.004 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:33:57.397 INFO:teuthology.orchestra.run.smithi176.stdout:(1/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 10 MB/s | 2.1 MB 00:00 2024-03-13T14:33:57.431 INFO:teuthology.orchestra.run.smithi176.stdout:(2/33): ceph-18.2.1-0.el9.x86_64.rpm 27 kB/s | 6.4 kB 00:00 2024-03-13T14:33:57.523 INFO:teuthology.orchestra.run.smithi176.stdout:(3/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 4.4 MB/s | 1.4 MB 00:00 2024-03-13T14:33:57.623 INFO:teuthology.orchestra.run.smithi176.stdout:(4/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-03-13T14:33:57.657 INFO:teuthology.orchestra.run.smithi176.stdout:(5/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 1.8 MB/s | 242 kB 00:00 2024-03-13T14:33:57.824 INFO:teuthology.orchestra.run.smithi176.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 296 kB/s | 32 kB 00:00 2024-03-13T14:33:57.849 INFO:teuthology.orchestra.run.smithi176.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 642 kB/s | 144 kB 00:00 2024-03-13T14:33:57.874 INFO:teuthology.orchestra.run.smithi176.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 957 kB/s | 47 kB 00:00 2024-03-13T14:33:57.899 INFO:teuthology.orchestra.run.smithi176.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 780 kB/s | 39 kB 00:00 2024-03-13T14:33:57.933 INFO:teuthology.orchestra.run.smithi176.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-13T14:33:58.000 INFO:teuthology.orchestra.run.smithi176.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-03-13T14:33:58.042 INFO:teuthology.orchestra.run.smithi176.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 969 kB/s | 103 kB 00:00 2024-03-13T14:33:58.059 INFO:teuthology.orchestra.run.smithi176.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.1 MB/s | 124 kB 00:00 2024-03-13T14:33:58.084 INFO:teuthology.orchestra.run.smithi176.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.5 MB/s | 187 kB 00:00 2024-03-13T14:33:58.185 INFO:teuthology.orchestra.run.smithi176.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 439 kB/s | 43 kB 00:00 2024-03-13T14:33:58.235 INFO:teuthology.orchestra.run.smithi176.stdout:(16/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-03-13T14:33:58.262 INFO:teuthology.orchestra.run.smithi176.stdout:(17/33): python3-requests-2.25.1-8.el9.noarch.r 620 kB/s | 125 kB 00:00 2024-03-13T14:33:58.296 INFO:teuthology.orchestra.run.smithi176.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 5.8 MB/s | 358 kB 00:00 2024-03-13T14:33:58.505 INFO:teuthology.orchestra.run.smithi176.stdout:(19/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 16 MB/s | 18 MB 00:01 2024-03-13T14:33:58.530 INFO:teuthology.orchestra.run.smithi176.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 40 kB/s | 11 kB 00:00 2024-03-13T14:33:58.555 INFO:teuthology.orchestra.run.smithi176.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 68 kB/s | 18 kB 00:00 2024-03-13T14:33:58.580 INFO:teuthology.orchestra.run.smithi176.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 310 kB/s | 23 kB 00:00 2024-03-13T14:33:58.606 INFO:teuthology.orchestra.run.smithi176.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 260 kB/s | 19 kB 00:00 2024-03-13T14:33:58.631 INFO:teuthology.orchestra.run.smithi176.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 263 kB/s | 20 kB 00:00 2024-03-13T14:33:58.656 INFO:teuthology.orchestra.run.smithi176.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 619 kB/s | 46 kB 00:00 2024-03-13T14:33:58.681 INFO:teuthology.orchestra.run.smithi176.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-03-13T14:33:58.706 INFO:teuthology.orchestra.run.smithi176.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 333 kB/s | 16 kB 00:00 2024-03-13T14:33:58.732 INFO:teuthology.orchestra.run.smithi176.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-03-13T14:33:58.757 INFO:teuthology.orchestra.run.smithi176.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 720 kB/s | 36 kB 00:00 2024-03-13T14:33:58.791 INFO:teuthology.orchestra.run.smithi176.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-03-13T14:33:58.816 INFO:teuthology.orchestra.run.smithi176.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-13T14:33:58.841 INFO:teuthology.orchestra.run.smithi176.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 403 kB/s | 20 kB 00:00 2024-03-13T14:33:58.875 INFO:teuthology.orchestra.run.smithi176.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-03-13T14:33:58.882 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:33:58.882 INFO:teuthology.orchestra.run.smithi176.stdout:Total 16 MB/s | 30 MB 00:01 2024-03-13T14:33:58.923 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:33:59.008 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:33:59.008 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:33:59.442 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:33:59.442 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:00.778 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:00.837 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-13T14:34:00.875 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-13T14:34:00.918 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-13T14:34:01.055 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-13T14:34:01.155 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-13T14:34:01.209 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-13T14:34:01.312 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-13T14:34:01.372 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-13T14:34:01.414 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-13T14:34:01.480 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-13T14:34:01.557 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-13T14:34:01.813 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-13T14:34:01.868 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-13T14:34:01.906 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-13T14:34:01.950 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-13T14:34:01.988 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-13T14:34:02.094 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-13T14:34:02.129 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-13T14:34:02.156 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-13T14:34:02.194 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-13T14:34:02.248 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-13T14:34:02.284 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-13T14:34:02.412 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-13T14:34:02.455 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-13T14:34:02.557 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-13T14:34:02.732 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-13T14:34:02.889 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-13T14:34:03.066 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-03-13T14:34:03.106 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-13T14:34:03.207 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-13T14:34:03.207 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:03.207 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-13T14:34:03.207 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T14:34:03.207 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T14:34:03.207 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:18.2.1-0.el9.x86_64 29/32 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-03-13T14:34:04.569 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:04.840 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-13T14:34:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-13T14:34:04.945 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:04.945 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-13T14:34:04.945 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T14:34:04.945 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T14:34:04.945 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:05.432 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-13T14:34:05.444 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-13T14:34:05.513 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-0.el9.x86_64 30/32 2024-03-13T14:34:05.513 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:05.514 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-13T14:34:05.514 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T14:34:05.514 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-13T14:34:05.514 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:05.519 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-13T14:34:05.527 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-13T14:34:05.527 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:05.527 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-13T14:34:05.527 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T14:34:05.527 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T14:34:05.527 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:05.629 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-03-13T14:34:05.630 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-13T14:34:05.750 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-13T14:34:05.843 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-13T14:34:05.843 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:05.843 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-13T14:34:05.843 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T14:34:05.843 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T14:34:05.843 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:05.934 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-13T14:34:06.068 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-13T14:34:06.069 INFO:teuthology.orchestra.run.smithi176.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-13T14:34:06.069 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:07.152 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-03-13T14:34:07.152 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:18.2.1-0.el9.x86_64 1/32 2024-03-13T14:34:07.152 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:18.2.1-0.el9.x86_64 2/32 2024-03-13T14:34:07.152 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:18.2.1-0.el9.x86_64 3/32 2024-03-13T14:34:07.152 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:18.2.1-0.el9.x86_64 4/32 2024-03-13T14:34:07.152 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:18.2.1-0.el9.x86_64 5/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:18.2.1-0.el9.x86_64 6/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:18.2.1-0.el9.x86_64 7/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:18.2.1-0.el9.x86_64 8/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:18.2.1-0.el9.x86_64 9/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:18.2.1-0.el9.x86_64 10/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:18.2.1-0.el9.x86_64 11/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:18.2.1-0.el9.x86_64 12/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:18.2.1-0.el9.x86_64 13/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-03-13T14:34:07.153 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-03-13T14:34:07.154 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-03-13T14:34:07.155 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-03-13T14:34:07.155 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:18.2.1-0.el9.x86_64 29/32 2024-03-13T14:34:07.155 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-03-13T14:34:07.155 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:18.2.1-0.el9.x86_64 31/32 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-13T14:34:07.815 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-13T14:34:07.816 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-13T14:34:07.817 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-13T14:34:07.817 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-13T14:34:07.817 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-13T14:34:07.817 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-13T14:34:07.817 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-13T14:34:07.817 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-13T14:34:07.818 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-13T14:34:07.818 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-13T14:34:07.818 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-13T14:34:07.993 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-03-13T14:34:07.993 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:18.2.1-0.el9.x86_64 librbd1-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: fmt-8.1.1-5.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-13T14:34:07.994 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-13T14:34:07.995 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.996 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.996 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:07.996 INFO:teuthology.orchestra.run.smithi196.stdout: re2-1:20211101-3.el9.x86_64 2024-03-13T14:34:07.996 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-13T14:34:07.996 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:07.996 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:08.417 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-test 2024-03-13T14:34:08.866 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-13T14:34:08.866 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-13T14:34:08.867 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-ply-3.11-14.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-13T14:34:08.868 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:08.869 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:09.239 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-base 2024-03-13T14:34:09.242 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:43 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:09.402 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:09.403 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:09.403 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:34:09.403 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:09.404 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:34:09.404 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:18.2.1-0.el9 ceph 40 M 2024-03-13T14:34:09.404 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:34:09.404 INFO:teuthology.orchestra.run.smithi196.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-13T14:34:09.404 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:18.2.1-0.el9 ceph 165 k 2024-03-13T14:34:09.404 INFO:teuthology.orchestra.run.smithi196.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-13T14:34:09.404 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-13T14:34:09.405 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-13T14:34:09.405 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:09.405 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:34:09.405 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:09.405 INFO:teuthology.orchestra.run.smithi196.stdout:Install 6 Packages 2024-03-13T14:34:09.405 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:09.407 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 41 M 2024-03-13T14:34:09.407 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 167 M 2024-03-13T14:34:09.407 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:34:09.579 INFO:teuthology.orchestra.run.smithi196.stdout:(1/6): libcephsqlite-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 165 kB 00:00 2024-03-13T14:34:09.645 INFO:teuthology.orchestra.run.smithi196.stdout:(2/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 217 kB 00:00 2024-03-13T14:34:09.671 INFO:teuthology.orchestra.run.smithi196.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 300 kB 00:00 2024-03-13T14:34:09.696 INFO:teuthology.orchestra.run.smithi196.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-13T14:34:10.069 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:56 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:10.171 INFO:teuthology.orchestra.run.smithi176.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:10.224 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:10.225 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:34:10.225 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:10.238 INFO:teuthology.orchestra.run.smithi196.stdout:(5/6): ceph-test-18.2.1-0.el9.x86_64.rpm 51 MB/s | 40 MB 00:00 2024-03-13T14:34:10.289 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install cephadm 2024-03-13T14:34:10.505 INFO:teuthology.orchestra.run.smithi196.stdout:(6/6): jq-1.6-16.el9.x86_64.rpm 225 kB/s | 187 kB 00:00 2024-03-13T14:34:10.510 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:10.511 INFO:teuthology.orchestra.run.smithi196.stdout:Total 37 MB/s | 41 MB 00:01 2024-03-13T14:34:10.551 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:34:10.606 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:34:10.606 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:34:11.017 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:34:11.017 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:34:11.107 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:00:57 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:11.262 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:11.263 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-03-13T14:34:11.264 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:11.264 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 221 k 2024-03-13T14:34:11.264 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 213 k 2024-03-13T14:34:11.264 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:11.357 INFO:teuthology.orchestra.run.smithi176.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.3 MB/s | 221 kB 00:00 2024-03-13T14:34:11.357 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:11.358 INFO:teuthology.orchestra.run.smithi176.stdout:Total 2.3 MB/s | 221 kB 00:00 2024-03-13T14:34:11.358 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:11.365 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:11.408 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:11.409 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:11.409 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:11.461 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:11.699 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:34:11.797 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-13T14:34:11.839 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-13T14:34:11.876 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T14:34:11.938 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-13T14:34:11.966 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-13T14:34:12.113 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:12.236 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:12.682 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:12.986 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:12.987 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:12.987 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:12.987 INFO:teuthology.orchestra.run.smithi176.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-03-13T14:34:12.987 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:12.987 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:13.231 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-immutable-object-cache 2024-03-13T14:34:14.084 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:00 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:14.242 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:14.242 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:14.242 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repo Size 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 142 k 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 395 k 2024-03-13T14:34:14.243 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:14.337 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 1.5 MB/s | 142 kB 00:00 2024-03-13T14:34:14.338 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:14.338 INFO:teuthology.orchestra.run.smithi176.stdout:Total 1.5 MB/s | 142 kB 00:00 2024-03-13T14:34:14.338 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:14.348 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:14.348 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:14.414 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:14.414 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:14.573 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:14.646 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:14.718 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:14.718 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:14.718 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-13T14:34:14.718 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:16.012 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:16.012 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:16.012 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:16.012 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:16.012 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:16.012 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:16.118 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:18.2.1-0.el9.x86_64 5/6 2024-03-13T14:34:16.244 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-13T14:34:16.332 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr 2024-03-13T14:34:17.160 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:03 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:17.262 INFO:teuthology.orchestra.run.smithi176.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:17.316 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:17.317 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:34:17.317 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:17.379 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-dashboard 2024-03-13T14:34:18.210 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:04 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:18.367 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout:Installing weak dependencies: 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-13T14:34:18.368 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:18.369 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:18.369 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:18.369 INFO:teuthology.orchestra.run.smithi176.stdout:Install 7 Packages 2024-03-13T14:34:18.369 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:18.370 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 2.1 M 2024-03-13T14:34:18.370 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 8.5 M 2024-03-13T14:34:18.370 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:18.495 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-test-2:18.2.1-0.el9.x86_64 6/6 2024-03-13T14:34:18.495 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:18.2.1-0.el9.x86_64 1/6 2024-03-13T14:34:18.495 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:18.2.1-0.el9.x86_64 2/6 2024-03-13T14:34:18.495 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-13T14:34:18.495 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-13T14:34:18.495 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-13T14:34:18.723 INFO:teuthology.orchestra.run.smithi176.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 123 kB/s | 23 kB 00:00 2024-03-13T14:34:18.790 INFO:teuthology.orchestra.run.smithi176.stdout:(2/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 6.9 MB/s | 1.8 MB 00:00 2024-03-13T14:34:18.815 INFO:teuthology.orchestra.run.smithi176.stdout:(3/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 52 kB/s | 15 kB 00:00 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:18.2.1-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:18.2.1-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:18.860 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:19.057 INFO:teuthology.orchestra.run.smithi176.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 27 kB/s | 9.0 kB 00:00 2024-03-13T14:34:19.091 INFO:teuthology.orchestra.run.smithi176.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 181 kB/s | 31 kB 00:00 2024-03-13T14:34:19.116 INFO:teuthology.orchestra.run.smithi176.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 126 kB/s | 41 kB 00:00 2024-03-13T14:34:19.166 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph 2024-03-13T14:34:19.225 INFO:teuthology.orchestra.run.smithi176.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-03-13T14:34:19.235 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:19.235 INFO:teuthology.orchestra.run.smithi176.stdout:Total 2.4 MB/s | 2.1 MB 00:00 2024-03-13T14:34:19.236 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:19.262 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:19.262 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:19.395 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:19.397 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:19.778 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:19.834 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-13T14:34:19.882 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-13T14:34:19.974 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-13T14:34:20.007 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-13T14:34:20.019 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:54 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:20.052 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-03-13T14:34:20.178 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:20.180 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:20.180 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:18.2.1-0.el9 ceph 6.4 k 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:18.2.1-0.el9 ceph 2.1 M 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:18.2.1-0.el9 ceph 1.4 M 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:18.2.1-0.el9 ceph-noarch 242 k 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:18.2.1-0.el9 ceph 4.4 M 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:18.2.1-0.el9 ceph 18 M 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-13T14:34:20.181 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-13T14:34:20.182 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout:Install 33 Packages 2024-03-13T14:34:20.183 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:20.184 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 30 M 2024-03-13T14:34:20.184 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 110 M 2024-03-13T14:34:20.184 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:34:20.360 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-03-13T14:34:20.404 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-13T14:34:20.766 INFO:teuthology.orchestra.run.smithi196.stdout:(1/33): ceph-18.2.1-0.el9.x86_64.rpm 37 kB/s | 6.4 kB 00:00 2024-03-13T14:34:20.841 INFO:teuthology.orchestra.run.smithi196.stdout:(2/33): ceph-mgr-18.2.1-0.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-03-13T14:34:20.908 INFO:teuthology.orchestra.run.smithi196.stdout:(3/33): ceph-mds-18.2.1-0.el9.x86_64.rpm 6.7 MB/s | 2.1 MB 00:00 2024-03-13T14:34:20.942 INFO:teuthology.orchestra.run.smithi196.stdout:(4/33): ceph-mgr-modules-core-18.2.1-0.el9.noar 7.2 MB/s | 242 kB 00:00 2024-03-13T14:34:21.018 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-13T14:34:21.018 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-03-13T14:34:21.018 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-03-13T14:34:21.018 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-03-13T14:34:21.018 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-13T14:34:21.018 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-13T14:34:21.018 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-13T14:34:21.109 INFO:teuthology.orchestra.run.smithi196.stdout:(5/33): ceph-mon-18.2.1-0.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-03-13T14:34:21.159 INFO:teuthology.orchestra.run.smithi196.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 670 kB/s | 32 kB 00:00 2024-03-13T14:34:21.184 INFO:teuthology.orchestra.run.smithi196.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 597 kB/s | 144 kB 00:00 2024-03-13T14:34:21.210 INFO:teuthology.orchestra.run.smithi196.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 964 kB/s | 47 kB 00:00 2024-03-13T14:34:21.235 INFO:teuthology.orchestra.run.smithi196.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 783 kB/s | 39 kB 00:00 2024-03-13T14:34:21.268 INFO:teuthology.orchestra.run.smithi196.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-13T14:34:21.336 INFO:teuthology.orchestra.run.smithi196.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-03-13T14:34:21.352 INFO:teuthology.orchestra.run.smithi196.stdout:(12/33): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-13T14:34:21.386 INFO:teuthology.orchestra.run.smithi196.stdout:(13/33): python3-pycparser-2.20-6.el9.noarch.rp 2.5 MB/s | 124 kB 00:00 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-13T14:34:21.396 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-13T14:34:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-13T14:34:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-13T14:34:21.397 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:21.397 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:21.420 INFO:teuthology.orchestra.run.smithi196.stdout:(14/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.8 MB/s | 187 kB 00:00 2024-03-13T14:34:21.551 INFO:teuthology.orchestra.run.smithi196.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 773 kB/s | 125 kB 00:00 2024-03-13T14:34:21.579 INFO:teuthology.orchestra.run.smithi196.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 276 kB/s | 43 kB 00:00 2024-03-13T14:34:21.684 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-13T14:34:21.821 INFO:teuthology.orchestra.run.smithi196.stdout:(17/33): ceph-osd-18.2.1-0.el9.x86_64.rpm 18 MB/s | 18 MB 00:00 2024-03-13T14:34:21.913 INFO:teuthology.orchestra.run.smithi196.stdout:(18/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.0 MB/s | 358 kB 00:00 2024-03-13T14:34:21.938 INFO:teuthology.orchestra.run.smithi196.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 92 kB/s | 11 kB 00:00 2024-03-13T14:34:21.972 INFO:teuthology.orchestra.run.smithi196.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 408 kB/s | 172 kB 00:00 2024-03-13T14:34:21.997 INFO:teuthology.orchestra.run.smithi196.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 212 kB/s | 18 kB 00:00 2024-03-13T14:34:22.056 INFO:teuthology.orchestra.run.smithi196.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 198 kB/s | 23 kB 00:00 2024-03-13T14:34:22.081 INFO:teuthology.orchestra.run.smithi196.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 180 kB/s | 19 kB 00:00 2024-03-13T14:34:22.106 INFO:teuthology.orchestra.run.smithi196.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 182 kB/s | 20 kB 00:00 2024-03-13T14:34:22.140 INFO:teuthology.orchestra.run.smithi196.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 559 kB/s | 46 kB 00:00 2024-03-13T14:34:22.190 INFO:teuthology.orchestra.run.smithi196.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 955 kB/s | 79 kB 00:00 2024-03-13T14:34:22.215 INFO:teuthology.orchestra.run.smithi196.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 220 kB/s | 16 kB 00:00 2024-03-13T14:34:22.282 INFO:teuthology.orchestra.run.smithi196.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 986 kB/s | 90 kB 00:00 2024-03-13T14:34:22.307 INFO:teuthology.orchestra.run.smithi196.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 391 kB/s | 36 kB 00:00 2024-03-13T14:34:22.374 INFO:teuthology.orchestra.run.smithi196.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-03-13T14:34:22.416 INFO:teuthology.orchestra.run.smithi196.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-03-13T14:34:22.442 INFO:teuthology.orchestra.run.smithi196.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 299 kB/s | 20 kB 00:00 2024-03-13T14:34:22.522 INFO:teuthology.orchestra.run.smithi196.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.0 MB/s | 427 kB 00:00 2024-03-13T14:34:22.529 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:22.530 INFO:teuthology.orchestra.run.smithi196.stdout:Total 13 MB/s | 30 MB 00:02 2024-03-13T14:34:22.537 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:08 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:22.571 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:34:22.657 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:34:22.657 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:34:22.700 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:22.701 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:22.701 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:34:22.701 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:22.701 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-13T14:34:22.702 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout:Install 12 Packages 2024-03-13T14:34:22.703 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:22.704 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 37 M 2024-03-13T14:34:22.704 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 209 M 2024-03-13T14:34:22.704 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:23.115 INFO:teuthology.orchestra.run.smithi176.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 175 kB/s | 32 kB 00:00 2024-03-13T14:34:23.136 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:34:23.137 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:34:23.148 INFO:teuthology.orchestra.run.smithi176.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 531 kB/s | 17 kB 00:00 2024-03-13T14:34:23.190 INFO:teuthology.orchestra.run.smithi176.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 843 kB/s | 34 kB 00:00 2024-03-13T14:34:23.282 INFO:teuthology.orchestra.run.smithi176.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.6 MB/s | 3.0 MB 00:00 2024-03-13T14:34:23.458 INFO:teuthology.orchestra.run.smithi176.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2024-03-13T14:34:23.566 INFO:teuthology.orchestra.run.smithi176.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-03-13T14:34:23.591 INFO:teuthology.orchestra.run.smithi176.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 217 kB 00:00 2024-03-13T14:34:23.784 INFO:teuthology.orchestra.run.smithi176.stdout:(8/12): ceph-mgr-diskprediction-local-18.2.1-0. 8.7 MB/s | 7.4 MB 00:00 2024-03-13T14:34:23.992 INFO:teuthology.orchestra.run.smithi176.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-03-13T14:34:24.530 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:34:24.567 INFO:teuthology.orchestra.run.smithi176.stdout:(10/12): python3-devel-3.9.17-2.el9.x86_64.rpm 428 kB/s | 245 kB 00:00 2024-03-13T14:34:24.603 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-03-13T14:34:24.642 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-03-13T14:34:24.685 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-03-13T14:34:24.809 INFO:teuthology.orchestra.run.smithi176.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 192 kB/s | 195 kB 00:01 2024-03-13T14:34:24.826 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-03-13T14:34:24.921 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-03-13T14:34:24.985 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-03-13T14:34:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-03-13T14:34:25.164 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-03-13T14:34:25.206 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-03-13T14:34:25.274 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-03-13T14:34:25.355 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-03-13T14:34:25.509 INFO:teuthology.orchestra.run.smithi176.stdout:(12/12): libgfortran-11.4.1-3.el9.x86_64.rpm 422 kB/s | 803 kB 00:01 2024-03-13T14:34:25.516 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:25.516 INFO:teuthology.orchestra.run.smithi176.stdout:Total 13 MB/s | 37 MB 00:02 2024-03-13T14:34:25.614 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-03-13T14:34:25.633 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:25.677 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-03-13T14:34:25.697 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:25.698 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:25.725 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-03-13T14:34:25.768 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-03-13T14:34:25.806 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-03-13T14:34:25.925 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-03-13T14:34:25.965 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-13T14:34:25.999 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-03-13T14:34:26.048 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-03-13T14:34:26.105 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-03-13T14:34:26.144 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-03-13T14:34:26.222 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:26.222 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:26.285 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-03-13T14:34:26.331 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-03-13T14:34:26.439 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-03-13T14:34:26.630 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-03-13T14:34:26.783 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-13T14:34:26.960 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 28/33 2024-03-13T14:34:26.991 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-13T14:34:27.049 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:27.067 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:18.2.1-0.el9.x86_64 29/33 2024-03-13T14:34:27.068 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:27.068 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-13T14:34:27.068 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T14:34:27.068 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-13T14:34:27.068 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:27.157 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-13T14:34:27.191 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-13T14:34:27.342 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-13T14:34:27.402 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-13T14:34:27.659 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T14:34:27.691 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T14:34:27.806 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-13T14:34:28.638 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-13T14:34:28.701 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-13T14:34:28.767 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-13T14:34:28.779 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:18.2.1-0.el9.x86_64 30/33 2024-03-13T14:34:28.779 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:28.779 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-13T14:34:28.779 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T14:34:28.779 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-13T14:34:28.779 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:29.268 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-13T14:34:29.342 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:18.2.1-0.el9.x86_64 31/33 2024-03-13T14:34:29.343 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:29.343 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-13T14:34:29.343 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T14:34:29.343 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-13T14:34:29.343 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:29.560 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-13T14:34:29.630 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:18.2.1-0.el9.x86_64 32/33 2024-03-13T14:34:29.630 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:29.630 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-13T14:34:29.630 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T14:34:29.630 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-13T14:34:29.630 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:29.692 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-13T14:34:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-2:18.2.1-0.el9.x86_64 33/33 2024-03-13T14:34:29.838 INFO:teuthology.orchestra.run.smithi196.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-13T14:34:29.838 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:30.289 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-13T14:34:31.436 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-13T14:34:31.473 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:18.2.1-0.el9.x86_64 1/33 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:18.2.1-0.el9.x86_64 2/33 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:18.2.1-0.el9.x86_64 3/33 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:18.2.1-0.el9.x86_64 4/33 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:18.2.1-0.el9.x86_64 5/33 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 6/33 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-03-13T14:34:31.709 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 15/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-03-13T14:34:31.710 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-03-13T14:34:31.711 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-03-13T14:34:31.712 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-03-13T14:34:31.712 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:18.2.1-0.el9.noarch 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:32.821 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-13T14:34:32.822 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-13T14:34:32.823 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-13T14:34:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-13T14:34:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-13T14:34:32.824 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:32.824 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:33.279 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-base 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-13T14:34:34.030 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-13T14:34:34.031 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-13T14:34:34.031 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-13T14:34:34.121 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:08 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:34.225 INFO:teuthology.orchestra.run.smithi196.stdout:Package ceph-base-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:34.278 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:34.280 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:34:34.280 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:34.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install cephadm 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-13T14:34:34.469 INFO:teuthology.orchestra.run.smithi176.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:34.470 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:34.906 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-rook 2024-03-13T14:34:35.130 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:09 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:35.286 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:35.287 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:35.287 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-13T14:34:35.287 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:35.287 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:18.2.1-0.el9 ceph-noarch 221 k 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 221 k 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 213 k 2024-03-13T14:34:35.288 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:34:35.369 INFO:teuthology.orchestra.run.smithi196.stdout:cephadm-18.2.1-0.el9.noarch.rpm 2.7 MB/s | 221 kB 00:00 2024-03-13T14:34:35.370 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:35.370 INFO:teuthology.orchestra.run.smithi196.stdout:Total 2.6 MB/s | 221 kB 00:00 2024-03-13T14:34:35.370 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:34:35.376 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:34:35.376 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:34:35.379 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:34:35.380 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:34:35.470 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:34:35.761 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:21 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:35.915 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:35.916 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:34:35.916 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:35.916 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:35.916 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-03-13T14:34:35.916 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-13T14:34:35.917 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-13T14:34:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:35.918 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:35.918 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:35.918 INFO:teuthology.orchestra.run.smithi176.stdout:Install 13 Packages 2024-03-13T14:34:35.918 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:35.918 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 2.0 M 2024-03-13T14:34:35.918 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 26 M 2024-03-13T14:34:35.919 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:36.125 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:36.246 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 149 kB/s | 24 kB 00:00 2024-03-13T14:34:36.274 INFO:teuthology.orchestra.run.smithi176.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 96 kB/s | 18 kB 00:00 2024-03-13T14:34:36.299 INFO:teuthology.orchestra.run.smithi176.stdout:(3/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 234 kB/s | 50 kB 00:00 2024-03-13T14:34:36.325 INFO:teuthology.orchestra.run.smithi176.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 174 kB 00:00 2024-03-13T14:34:36.358 INFO:teuthology.orchestra.run.smithi176.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-13T14:34:36.384 INFO:teuthology.orchestra.run.smithi176.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-03-13T14:34:36.409 INFO:teuthology.orchestra.run.smithi176.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 530 kB/s | 43 kB 00:00 2024-03-13T14:34:36.526 INFO:teuthology.orchestra.run.smithi176.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 107 kB/s | 15 kB 00:00 2024-03-13T14:34:36.551 INFO:teuthology.orchestra.run.smithi176.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 194 kB/s | 32 kB 00:00 2024-03-13T14:34:36.643 INFO:teuthology.orchestra.run.smithi176.stdout:(10/13): python3-google-auth-2.28.0-1.el9.noarc 870 kB/s | 203 kB 00:00 2024-03-13T14:34:36.668 INFO:teuthology.orchestra.run.smithi176.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 508 kB/s | 59 kB 00:00 2024-03-13T14:34:36.695 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:36.710 INFO:teuthology.orchestra.run.smithi176.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-03-13T14:34:36.785 INFO:teuthology.orchestra.run.smithi176.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.0 MB/s | 1.0 MB 00:00 2024-03-13T14:34:36.790 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:36.791 INFO:teuthology.orchestra.run.smithi176.stdout:Total 2.3 MB/s | 2.0 MB 00:00 2024-03-13T14:34:36.792 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:36.831 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:36.831 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:36.984 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:36.985 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:37.043 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:18.2.1-0.el9.noarch 1/1 2024-03-13T14:34:37.043 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:34:37.044 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:18.2.1-0.el9.noarch 2024-03-13T14:34:37.044 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:37.044 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:37.346 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-immutable-object-cache 2024-03-13T14:34:37.641 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:37.854 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-13T14:34:37.920 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-13T14:34:37.986 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-13T14:34:38.024 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-13T14:34:38.070 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-13T14:34:38.185 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:12 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:38.215 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-13T14:34:38.346 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:38.347 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:38.347 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repo Size 2024-03-13T14:34:38.347 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:38.347 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-0.el9 ceph 142 k 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 142 k 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 395 k 2024-03-13T14:34:38.348 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:34:38.361 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-13T14:34:38.408 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-immutable-object-cache-18.2.1-0.el9.x86_64 2.3 MB/s | 142 kB 00:00 2024-03-13T14:34:38.409 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:38.409 INFO:teuthology.orchestra.run.smithi196.stdout:Total 2.3 MB/s | 142 kB 00:00 2024-03-13T14:34:38.410 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:34:38.418 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:34:38.418 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:34:38.430 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-13T14:34:38.476 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:34:38.476 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:34:38.630 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:34:38.726 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:38.797 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:38.797 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:34:38.798 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-13T14:34:38.798 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:39.345 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-13T14:34:39.413 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-13T14:34:39.455 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-13T14:34:39.524 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-13T14:34:39.548 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-13T14:34:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:40.026 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:40.026 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:34:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:40.027 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:40.027 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:40.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr 2024-03-13T14:34:40.380 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-13T14:34:40.380 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-03-13T14:34:40.380 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-13T14:34:40.380 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-13T14:34:40.381 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-13T14:34:40.825 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-13T14:34:40.826 INFO:teuthology.orchestra.run.smithi176.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-13T14:34:40.826 INFO:teuthology.orchestra.run.smithi176.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-13T14:34:40.826 INFO:teuthology.orchestra.run.smithi176.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-13T14:34:40.826 INFO:teuthology.orchestra.run.smithi176.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-13T14:34:40.826 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:40.826 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:41.158 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-mgr-cephadm 2024-03-13T14:34:41.176 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:15 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:41.278 INFO:teuthology.orchestra.run.smithi196.stdout:Package ceph-mgr-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:41.331 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:41.333 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:34:41.333 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:41.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-dashboard 2024-03-13T14:34:42.029 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:28 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:42.187 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:42.187 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: Package Arch Version Repository Size 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout:Install 6 Packages 2024-03-13T14:34:42.188 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:42.189 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 6.8 M 2024-03-13T14:34:42.189 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 33 M 2024-03-13T14:34:42.189 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:42.195 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:16 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:42.353 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:18.2.1-0.el9 ceph-noarch 1.8 M 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:18.2.1-0.el9 ceph-noarch 23 k 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:18.2.1-0.el9 ceph-noarch 15 k 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-03-13T14:34:42.354 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout:Install 7 Packages 2024-03-13T14:34:42.355 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:42.356 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 2.1 M 2024-03-13T14:34:42.356 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 8.5 M 2024-03-13T14:34:42.356 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:34:42.546 INFO:teuthology.orchestra.run.smithi196.stdout:(1/7): ceph-grafana-dashboards-18.2.1-0.el9.noa 177 kB/s | 23 kB 00:00 2024-03-13T14:34:42.570 INFO:teuthology.orchestra.run.smithi196.stdout:(2/7): ceph-prometheus-alerts-18.2.1-0.el9.noar 94 kB/s | 15 kB 00:00 2024-03-13T14:34:42.612 INFO:teuthology.orchestra.run.smithi176.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 830 kB/s | 132 kB 00:00 2024-03-13T14:34:42.629 INFO:teuthology.orchestra.run.smithi196.stdout:(3/7): ceph-mgr-dashboard-18.2.1-0.el9.noarch.r 8.2 MB/s | 1.8 MB 00:00 2024-03-13T14:34:42.813 INFO:teuthology.orchestra.run.smithi196.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 34 kB/s | 9.0 kB 00:00 2024-03-13T14:34:42.855 INFO:teuthology.orchestra.run.smithi196.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 144 kB/s | 41 kB 00:00 2024-03-13T14:34:42.880 INFO:teuthology.orchestra.run.smithi196.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 173 kB/s | 31 kB 00:00 2024-03-13T14:34:42.913 INFO:teuthology.orchestra.run.smithi176.stdout:(2/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.8 MB/s | 548 kB 00:00 2024-03-13T14:34:42.971 INFO:teuthology.orchestra.run.smithi176.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-03-13T14:34:43.004 INFO:teuthology.orchestra.run.smithi176.stdout:(4/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 450 kB/s | 248 kB 00:00 2024-03-13T14:34:43.014 INFO:teuthology.orchestra.run.smithi196.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 944 kB/s | 188 kB 00:00 2024-03-13T14:34:43.023 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:43.024 INFO:teuthology.orchestra.run.smithi196.stdout:Total 3.1 MB/s | 2.1 MB 00:00 2024-03-13T14:34:43.025 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:34:43.050 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:34:43.051 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:34:43.122 INFO:teuthology.orchestra.run.smithi176.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 8.6 MB/s | 5.8 MB 00:00 2024-03-13T14:34:43.146 INFO:teuthology.orchestra.run.smithi176.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 305 kB/s | 51 kB 00:00 2024-03-13T14:34:43.153 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:43.154 INFO:teuthology.orchestra.run.smithi176.stdout:Total 7.0 MB/s | 6.8 MB 00:00 2024-03-13T14:34:43.189 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:34:43.189 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:34:43.196 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:43.230 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:43.230 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:43.374 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:43.379 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:43.584 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:34:43.657 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-03-13T14:34:43.701 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-03-13T14:34:43.784 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-03-13T14:34:43.788 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:43.821 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-03-13T14:34:43.857 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 5/7 2024-03-13T14:34:43.993 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-13T14:34:44.070 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-13T14:34:44.165 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 6/7 2024-03-13T14:34:44.201 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-13T14:34:44.759 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-13T14:34:44.858 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-13T14:34:44.871 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 7/7 2024-03-13T14:34:44.871 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 1/7 2024-03-13T14:34:44.871 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2/7 2024-03-13T14:34:44.872 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 3/7 2024-03-13T14:34:44.872 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-03-13T14:34:44.872 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-03-13T14:34:44.872 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-03-13T14:34:44.945 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-13T14:34:44.977 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-13T14:34:45.276 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-03-13T14:34:45.276 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:18.2.1-0.el9.noarch 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:18.2.1-0.el9.noarch 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:18.2.1-0.el9.noarch 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:45.277 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:45.538 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-13T14:34:45.771 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-13T14:34:45.772 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-03-13T14:34:45.772 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-13T14:34:45.772 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-13T14:34:45.772 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-13T14:34:45.772 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-13T14:34:46.110 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:46.111 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:46.396 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:20 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:34:46.417 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install ceph-fuse 2024-03-13T14:34:46.565 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-0.el9 ceph-noarch 7.4 M 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-13T14:34:46.567 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout:Install 12 Packages 2024-03-13T14:34:46.568 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:46.569 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 37 M 2024-03-13T14:34:46.569 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 209 M 2024-03-13T14:34:46.569 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:34:47.276 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:33 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:47.421 INFO:teuthology.orchestra.run.smithi196.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.1-0. 24 MB/s | 7.4 MB 00:00 2024-03-13T14:34:47.436 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout:Installing dependencies: 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:47.437 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:47.438 INFO:teuthology.orchestra.run.smithi176.stdout:Install 2 Packages 2024-03-13T14:34:47.438 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:47.438 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 918 k 2024-03-13T14:34:47.438 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 2.7 M 2024-03-13T14:34:47.438 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:47.446 INFO:teuthology.orchestra.run.smithi196.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 96 kB/s | 32 kB 00:00 2024-03-13T14:34:47.505 INFO:teuthology.orchestra.run.smithi196.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 208 kB/s | 17 kB 00:00 2024-03-13T14:34:47.565 INFO:teuthology.orchestra.run.smithi176.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-03-13T14:34:47.572 INFO:teuthology.orchestra.run.smithi196.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 273 kB/s | 34 kB 00:00 2024-03-13T14:34:47.707 INFO:teuthology.orchestra.run.smithi176.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 293 kB/s | 79 kB 00:00 2024-03-13T14:34:47.710 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:47.711 INFO:teuthology.orchestra.run.smithi176.stdout:Total 3.3 MB/s | 918 kB 00:00 2024-03-13T14:34:47.712 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:47.781 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:47.781 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:47.875 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:47.875 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:48.149 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:48.172 INFO:teuthology.orchestra.run.smithi196.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.0 MB/s | 4.8 MB 00:00 2024-03-13T14:34:48.264 INFO:teuthology.orchestra.run.smithi196.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 6.2 MB/s | 4.7 MB 00:00 2024-03-13T14:34:48.298 INFO:teuthology.orchestra.run.smithi196.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 217 kB 00:00 2024-03-13T14:34:48.325 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-13T14:34:48.402 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-13T14:34:48.898 INFO:teuthology.orchestra.run.smithi196.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-03-13T14:34:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-13T14:34:49.307 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-03-13T14:34:49.357 INFO:teuthology.orchestra.run.smithi196.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 430 kB/s | 195 kB 00:00 2024-03-13T14:34:49.465 INFO:teuthology.orchestra.run.smithi196.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 697 kB/s | 803 kB 00:01 2024-03-13T14:34:49.582 INFO:teuthology.orchestra.run.smithi196.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-03-13T14:34:49.639 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-13T14:34:49.639 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:49.639 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:49.640 INFO:teuthology.orchestra.run.smithi176.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-13T14:34:49.640 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:49.640 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:49.857 INFO:teuthology.orchestra.run.smithi196.stdout:(12/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 1.1 MB/s | 3.0 MB 00:02 2024-03-13T14:34:49.862 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:49.863 INFO:teuthology.orchestra.run.smithi196.stdout:Total 11 MB/s | 37 MB 00:03 2024-03-13T14:34:49.950 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install librados-devel 2024-03-13T14:34:49.981 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:34:50.045 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:34:50.045 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:34:50.558 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:34:50.558 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:34:50.798 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:36 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:50.955 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:50.956 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:50.956 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-03-13T14:34:50.956 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:50.956 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:50.956 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-03-13T14:34:50.956 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:50.957 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:50.957 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:50.957 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-03-13T14:34:50.957 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:50.957 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 127 k 2024-03-13T14:34:50.957 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 456 k 2024-03-13T14:34:50.957 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:51.053 INFO:teuthology.orchestra.run.smithi176.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.3 MB/s | 127 kB 00:00 2024-03-13T14:34:51.053 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:51.054 INFO:teuthology.orchestra.run.smithi176.stdout:Total 1.3 MB/s | 127 kB 00:00 2024-03-13T14:34:51.054 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:51.061 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:51.061 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:51.129 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:51.130 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:51.288 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:51.388 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:51.436 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:34:51.510 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-13T14:34:51.546 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-13T14:34:51.689 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-13T14:34:51.733 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-13T14:34:51.922 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:51.982 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T14:34:52.014 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T14:34:52.123 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-13T14:34:52.240 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:52.241 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:52.241 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:52.241 INFO:teuthology.orchestra.run.smithi176.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:52.241 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:52.241 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:52.597 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install libcephfs2 2024-03-13T14:34:52.953 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-13T14:34:53.089 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-13T14:34:53.448 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:39 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:53.552 INFO:teuthology.orchestra.run.smithi176.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:53.608 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:53.609 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:34:53.609 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:53.674 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install libcephfs-devel 2024-03-13T14:34:54.462 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:40 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:54.584 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-13T14:34:54.620 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:54.621 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:54.621 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-03-13T14:34:54.622 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:54.622 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:34:54.622 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-03-13T14:34:54.622 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:54.622 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:34:54.623 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:34:54.623 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-03-13T14:34:54.623 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:54.623 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 31 k 2024-03-13T14:34:54.624 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 139 k 2024-03-13T14:34:54.624 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:34:54.723 INFO:teuthology.orchestra.run.smithi176.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 304 kB/s | 31 kB 00:00 2024-03-13T14:34:54.724 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:34:54.724 INFO:teuthology.orchestra.run.smithi176.stdout:Total 299 kB/s | 31 kB 00:00 2024-03-13T14:34:54.725 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:34:54.729 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:34:54.729 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:34:54.772 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:34:54.773 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:34:54.912 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:34:55.070 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:55.519 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:55.724 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-13T14:34:55.780 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-13T14:34:55.826 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:34:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:55.827 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:34:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-03-13T14:34:55.827 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:34:55.827 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:56.056 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install librados2 2024-03-13T14:34:56.891 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:42 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:56.994 INFO:teuthology.orchestra.run.smithi176.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:57.050 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:57.051 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:34:57.051 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:57.110 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install librbd1 2024-03-13T14:34:57.903 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:43 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:58.006 INFO:teuthology.orchestra.run.smithi176.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:58.062 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:58.063 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:34:58.063 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:58.137 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-rados 2024-03-13T14:34:58.280 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 12/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarc 1/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-13T14:34:58.281 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-13T14:34:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-13T14:34:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:58.728 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:34:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:18.2.1-0.el9.noarch 2024-03-13T14:34:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-13T14:34:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-13T14:34:58.728 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:34:58.729 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:34:58.962 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:44 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:34:59.063 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:34:59.118 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:34:59.119 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:34:59.119 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:34:59.141 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-rook 2024-03-13T14:34:59.190 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-rgw 2024-03-13T14:34:59.982 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:33 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:00.017 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:46 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:35:00.120 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:00.133 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:00.134 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:18.2.1-0.el9 ceph-noarch 50 k 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-13T14:35:00.135 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout:Install 13 Packages 2024-03-13T14:35:00.136 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:00.137 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 2.0 M 2024-03-13T14:35:00.137 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 26 M 2024-03-13T14:35:00.137 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:00.175 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:35:00.176 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:35:00.176 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:35:00.244 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-cephfs 2024-03-13T14:35:00.538 INFO:teuthology.orchestra.run.smithi196.stdout:(1/13): ceph-mgr-rook-18.2.1-0.el9.noarch.rpm 253 kB/s | 50 kB 00:00 2024-03-13T14:35:00.564 INFO:teuthology.orchestra.run.smithi196.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 110 kB/s | 24 kB 00:00 2024-03-13T14:35:00.599 INFO:teuthology.orchestra.run.smithi196.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 70 kB/s | 18 kB 00:00 2024-03-13T14:35:00.673 INFO:teuthology.orchestra.run.smithi196.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 174 kB 00:00 2024-03-13T14:35:00.698 INFO:teuthology.orchestra.run.smithi196.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 988 kB/s | 132 kB 00:00 2024-03-13T14:35:00.731 INFO:teuthology.orchestra.run.smithi196.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.5 MB/s | 211 kB 00:00 2024-03-13T14:35:00.757 INFO:teuthology.orchestra.run.smithi196.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 524 kB/s | 43 kB 00:00 2024-03-13T14:35:01.057 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:47 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:35:01.157 INFO:teuthology.orchestra.run.smithi196.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 38 kB/s | 15 kB 00:00 2024-03-13T14:35:01.158 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:01.214 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:35:01.215 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:35:01.215 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:35:01.285 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install python3-rbd 2024-03-13T14:35:01.337 INFO:teuthology.orchestra.run.smithi196.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 51 kB/s | 32 kB 00:00 2024-03-13T14:35:01.418 INFO:teuthology.orchestra.run.smithi196.stdout:(10/13): python3-google-auth-2.28.0-1.el9.noarc 308 kB/s | 203 kB 00:00 2024-03-13T14:35:01.516 INFO:teuthology.orchestra.run.smithi196.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 332 kB/s | 59 kB 00:00 2024-03-13T14:35:01.541 INFO:teuthology.orchestra.run.smithi196.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 719 kB/s | 90 kB 00:00 2024-03-13T14:35:01.608 INFO:teuthology.orchestra.run.smithi196.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.3 MB/s | 1.0 MB 00:00 2024-03-13T14:35:01.618 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:01.619 INFO:teuthology.orchestra.run.smithi196.stdout:Total 1.4 MB/s | 2.0 MB 00:01 2024-03-13T14:35:01.620 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:01.659 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:01.659 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:01.814 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:01.815 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:02.105 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:48 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:35:02.207 INFO:teuthology.orchestra.run.smithi176.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:35:02.264 INFO:teuthology.orchestra.run.smithi176.stdout:Nothing to do. 2024-03-13T14:35:02.264 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:35:02.325 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install rbd-fuse 2024-03-13T14:35:02.459 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:02.684 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-13T14:35:02.768 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-13T14:35:02.851 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-13T14:35:02.888 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-13T14:35:02.935 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-13T14:35:03.066 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-13T14:35:03.164 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:49 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:35:03.200 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 7/13 2024-03-13T14:35:03.269 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-13T14:35:03.334 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:35:03.334 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:03.334 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:03.334 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:03.334 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:35:03.334 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-03-13T14:35:03.335 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:03.335 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:35:03.335 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:03.335 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-03-13T14:35:03.335 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:03.335 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 86 k 2024-03-13T14:35:03.336 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 226 k 2024-03-13T14:35:03.336 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:35:03.444 INFO:teuthology.orchestra.run.smithi176.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 803 kB/s | 86 kB 00:00 2024-03-13T14:35:03.444 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:03.445 INFO:teuthology.orchestra.run.smithi176.stdout:Total 785 kB/s | 86 kB 00:00 2024-03-13T14:35:03.445 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:35:03.454 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:35:03.454 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:35:03.522 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:35:03.522 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:35:03.684 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:35:03.787 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:04.195 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-13T14:35:04.279 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-13T14:35:04.282 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:04.320 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-13T14:35:04.378 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-13T14:35:04.405 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-13T14:35:04.576 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:04.577 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:35:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:04.577 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:04.577 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:35:04.886 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install rbd-mirror 2024-03-13T14:35:05.242 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-0.el9.noarch 13/13 2024-03-13T14:35:05.242 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:18.2.1-0.el9.noarch 1/13 2024-03-13T14:35:05.242 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 10/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-13T14:35:05.243 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-13T14:35:05.710 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:51 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:35:05.736 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-13T14:35:05.736 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:05.736 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:05.736 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:18.2.1-0.el9.noarch 2024-03-13T14:35:05.736 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:05.737 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:05.867 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:35:05.868 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:05.869 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-03-13T14:35:05.869 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:05.869 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 3.0 M 2024-03-13T14:35:05.869 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 12 M 2024-03-13T14:35:05.869 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:35:06.015 INFO:teuthology.orchestra.run.smithi176.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-03-13T14:35:06.016 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:06.016 INFO:teuthology.orchestra.run.smithi176.stdout:Total 20 MB/s | 3.0 MB 00:00 2024-03-13T14:35:06.016 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:35:06.030 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:35:06.030 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:35:06.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-mgr-cephadm 2024-03-13T14:35:06.121 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:35:06.121 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:35:06.605 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:35:06.666 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:06.751 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:06.751 INFO:teuthology.orchestra.run.smithi176.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:35:06.751 INFO:teuthology.orchestra.run.smithi176.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-13T14:35:06.752 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T14:35:06.752 INFO:teuthology.orchestra.run.smithi176.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T14:35:06.752 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:06.912 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:40 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:07.076 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:07.077 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:07.077 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-03-13T14:35:07.077 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:18.2.1-0.el9 ceph-noarch 132 k 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout:Install 6 Packages 2024-03-13T14:35:07.078 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:07.079 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 6.8 M 2024-03-13T14:35:07.079 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 33 M 2024-03-13T14:35:07.079 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:07.586 INFO:teuthology.orchestra.run.smithi196.stdout:(1/6): ceph-mgr-cephadm-18.2.1-0.el9.noarch.rpm 860 kB/s | 132 kB 00:00 2024-03-13T14:35:08.141 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:08.141 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:08.141 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:35:08.141 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:08.141 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:08.142 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:35:08.228 INFO:teuthology.orchestra.run.smithi196.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.3 MB/s | 5.8 MB 00:00 2024-03-13T14:35:08.395 INFO:teuthology.orchestra.run.smithi196.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 258 kB/s | 248 kB 00:00 2024-03-13T14:35:08.436 INFO:teuthology.orchestra.run.smithi196.stdout:(4/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 646 kB/s | 548 kB 00:00 2024-03-13T14:35:08.459 DEBUG:teuthology.orchestra.run.smithi176:> sudo yum -y install rbd-nbd 2024-03-13T14:35:08.462 INFO:teuthology.orchestra.run.smithi196.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 819 kB/s | 51 kB 00:00 2024-03-13T14:35:08.570 INFO:teuthology.orchestra.run.smithi196.stdout:(6/6): python3-natsort-7.1.1-5.el9.noarch.rpm 169 kB/s | 58 kB 00:00 2024-03-13T14:35:08.577 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:08.577 INFO:teuthology.orchestra.run.smithi196.stdout:Total 4.5 MB/s | 6.8 MB 00:01 2024-03-13T14:35:08.619 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:08.654 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:08.654 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:08.797 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:08.798 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:09.196 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:09.329 INFO:teuthology.orchestra.run.smithi176.stdout:Last metadata expiration check: 0:01:55 ago on Wed 13 Mar 2024 02:33:14 PM UTC. 2024-03-13T14:35:09.412 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-13T14:35:09.478 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-03-13T14:35:09.489 INFO:teuthology.orchestra.run.smithi176.stdout:Dependencies resolved. 2024-03-13T14:35:09.490 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:09.490 INFO:teuthology.orchestra.run.smithi176.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:09.490 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:09.490 INFO:teuthology.orchestra.run.smithi176.stdout:Installing: 2024-03-13T14:35:09.490 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-03-13T14:35:09.490 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:09.491 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction Summary 2024-03-13T14:35:09.491 INFO:teuthology.orchestra.run.smithi176.stdout:================================================================================ 2024-03-13T14:35:09.491 INFO:teuthology.orchestra.run.smithi176.stdout:Install 1 Package 2024-03-13T14:35:09.491 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:09.491 INFO:teuthology.orchestra.run.smithi176.stdout:Total download size: 171 k 2024-03-13T14:35:09.492 INFO:teuthology.orchestra.run.smithi176.stdout:Installed size: 494 k 2024-03-13T14:35:09.492 INFO:teuthology.orchestra.run.smithi176.stdout:Downloading Packages: 2024-03-13T14:35:09.585 INFO:teuthology.orchestra.run.smithi176.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 1.8 MB/s | 171 kB 00:00 2024-03-13T14:35:09.585 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:09.586 INFO:teuthology.orchestra.run.smithi176.stdout:Total 1.8 MB/s | 171 kB 00:00 2024-03-13T14:35:09.586 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction check 2024-03-13T14:35:09.597 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction check succeeded. 2024-03-13T14:35:09.597 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction test 2024-03-13T14:35:09.667 INFO:teuthology.orchestra.run.smithi176.stdout:Transaction test succeeded. 2024-03-13T14:35:09.668 INFO:teuthology.orchestra.run.smithi176.stdout:Running transaction 2024-03-13T14:35:09.834 INFO:teuthology.orchestra.run.smithi176.stdout: Preparing : 1/1 2024-03-13T14:35:09.967 INFO:teuthology.orchestra.run.smithi176.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:10.170 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-13T14:35:10.274 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-13T14:35:10.362 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-13T14:35:10.392 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-13T14:35:10.517 INFO:teuthology.orchestra.run.smithi176.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:10.798 INFO:teuthology.orchestra.run.smithi176.stdout:Installed: 2024-03-13T14:35:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:10.798 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:35:10.798 INFO:teuthology.orchestra.run.smithi176.stdout:Complete! 2024-03-13T14:35:11.073 DEBUG:teuthology.parallel:result is None 2024-03-13T14:35:11.305 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 6/6 2024-03-13T14:35:11.306 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 1/6 2024-03-13T14:35:11.306 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-13T14:35:11.306 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-13T14:35:11.306 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-03-13T14:35:11.306 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:18.2.1-0.el9.noarch 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-13T14:35:11.659 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-13T14:35:11.660 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-13T14:35:11.660 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:11.660 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:12.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-fuse 2024-03-13T14:35:12.837 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:46 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:12.996 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:18.2.1-0.el9 ceph 839 k 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout:Install 2 Packages 2024-03-13T14:35:12.997 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:12.998 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 918 k 2024-03-13T14:35:12.998 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 2.7 M 2024-03-13T14:35:12.998 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:13.116 INFO:teuthology.orchestra.run.smithi196.stdout:(1/2): ceph-fuse-18.2.1-0.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-03-13T14:35:13.199 INFO:teuthology.orchestra.run.smithi196.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 393 kB/s | 79 kB 00:00 2024-03-13T14:35:13.202 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:13.203 INFO:teuthology.orchestra.run.smithi196.stdout:Total 4.4 MB/s | 918 kB 00:00 2024-03-13T14:35:13.203 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:13.268 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:13.268 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:13.361 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:13.361 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:13.603 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:13.796 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-13T14:35:13.869 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-13T14:35:14.813 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-fuse-2:18.2.1-0.el9.x86_64 2/2 2024-03-13T14:35:14.813 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:18.2.1-0.el9.x86_64 1/2 2024-03-13T14:35:15.170 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-13T14:35:15.170 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:15.170 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:15.170 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:18.2.1-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-13T14:35:15.170 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:15.170 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:15.501 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librados-devel 2024-03-13T14:35:16.356 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:50 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:16.524 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:16.524 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:16.524 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:16.524 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:16.524 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:16.524 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:18.2.1-0.el9 ceph 127 k 2024-03-13T14:35:16.524 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:16.525 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:16.525 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:16.525 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-13T14:35:16.525 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:16.525 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 127 k 2024-03-13T14:35:16.525 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 456 k 2024-03-13T14:35:16.525 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:16.613 INFO:teuthology.orchestra.run.smithi196.stdout:librados-devel-18.2.1-0.el9.x86_64.rpm 1.4 MB/s | 127 kB 00:00 2024-03-13T14:35:16.614 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:16.614 INFO:teuthology.orchestra.run.smithi196.stdout:Total 1.4 MB/s | 127 kB 00:00 2024-03-13T14:35:16.614 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:16.622 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:16.622 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:16.686 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:16.686 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:16.850 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:16.971 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:17.618 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:17.911 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:17.911 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:17.912 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:17.912 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:17.912 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:17.912 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:18.266 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install libcephfs2 2024-03-13T14:35:19.076 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:53 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:19.183 INFO:teuthology.orchestra.run.smithi196.stdout:Package libcephfs2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:19.238 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:19.239 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:35:19.240 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:19.306 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install libcephfs-devel 2024-03-13T14:35:20.105 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:54 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:20.261 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:20.261 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:20.261 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:18.2.1-0.el9 ceph 31 k 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 31 k 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 139 k 2024-03-13T14:35:20.262 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:20.334 INFO:teuthology.orchestra.run.smithi196.stdout:libcephfs-devel-18.2.1-0.el9.x86_64.rpm 433 kB/s | 31 kB 00:00 2024-03-13T14:35:20.335 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:20.335 INFO:teuthology.orchestra.run.smithi196.stdout:Total 423 kB/s | 31 kB 00:00 2024-03-13T14:35:20.335 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:20.339 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:20.339 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:20.386 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:20.386 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:20.519 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:20.689 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:21.106 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:21.391 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:21.391 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:21.391 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:21.391 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:21.391 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:21.391 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:21.612 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librados2 2024-03-13T14:35:22.444 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:56 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:22.546 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:22.603 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:22.604 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:35:22.604 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:22.675 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install librbd1 2024-03-13T14:35:23.498 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:57 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:23.600 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:23.656 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:23.658 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:35:23.658 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:23.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rados 2024-03-13T14:35:24.543 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:58 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:24.645 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rados-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:24.699 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:24.701 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:35:24.701 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:24.763 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rgw 2024-03-13T14:35:25.545 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:01:59 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:25.649 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rgw-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:25.706 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:25.707 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:35:25.707 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:25.774 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-cephfs 2024-03-13T14:35:26.567 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:00 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:26.667 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-cephfs-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:26.723 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:26.724 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:35:26.724 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:26.791 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install python3-rbd 2024-03-13T14:35:27.622 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:01 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:27.756 INFO:teuthology.orchestra.run.smithi196.stdout:Package python3-rbd-2:18.2.1-0.el9.x86_64 is already installed. 2024-03-13T14:35:27.815 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:27.817 INFO:teuthology.orchestra.run.smithi196.stdout:Nothing to do. 2024-03-13T14:35:27.817 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:27.887 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-fuse 2024-03-13T14:35:28.688 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:02 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:28.849 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:28.850 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:28.850 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:28.850 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:28.850 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:18.2.1-0.el9 ceph 86 k 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 86 k 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 226 k 2024-03-13T14:35:28.851 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:28.929 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-fuse-18.2.1-0.el9.x86_64.rpm 1.1 MB/s | 86 kB 00:00 2024-03-13T14:35:28.930 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:28.930 INFO:teuthology.orchestra.run.smithi196.stdout:Total 1.1 MB/s | 86 kB 00:00 2024-03-13T14:35:28.930 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:28.938 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:28.938 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:28.999 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:28.999 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:29.166 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:29.269 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:29.753 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:30.074 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:30.075 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:30.075 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:30.075 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:30.075 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:30.075 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:30.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-mirror 2024-03-13T14:35:31.239 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:05 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:31.398 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:31.399 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:31.399 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:18.2.1-0.el9 ceph 3.0 M 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 3.0 M 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 12 M 2024-03-13T14:35:31.400 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:31.516 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-mirror-18.2.1-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-03-13T14:35:31.517 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:31.517 INFO:teuthology.orchestra.run.smithi196.stdout:Total 25 MB/s | 3.0 MB 00:00 2024-03-13T14:35:31.518 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:31.528 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:31.529 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:31.609 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:31.609 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:32.067 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:32.134 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:32.206 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:32.206 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-13T14:35:32.206 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-13T14:35:32.206 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T14:35:32.206 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-13T14:35:32.206 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:33.646 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:33.646 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:33.646 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:33.646 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:33.646 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:33.646 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:33.925 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install rbd-nbd 2024-03-13T14:35:34.770 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:02:08 ago on Wed 13 Mar 2024 02:33:26 PM UTC. 2024-03-13T14:35:34.931 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-03-13T14:35:34.931 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:34.931 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-03-13T14:35:34.931 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:34.931 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-03-13T14:35:34.931 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:18.2.1-0.el9 ceph 171 k 2024-03-13T14:35:34.932 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:34.932 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-03-13T14:35:34.932 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-03-13T14:35:34.932 INFO:teuthology.orchestra.run.smithi196.stdout:Install 1 Package 2024-03-13T14:35:34.932 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:34.932 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 171 k 2024-03-13T14:35:34.932 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 494 k 2024-03-13T14:35:34.933 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-03-13T14:35:35.012 INFO:teuthology.orchestra.run.smithi196.stdout:rbd-nbd-18.2.1-0.el9.x86_64.rpm 2.1 MB/s | 171 kB 00:00 2024-03-13T14:35:35.013 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:35:35.013 INFO:teuthology.orchestra.run.smithi196.stdout:Total 2.1 MB/s | 171 kB 00:00 2024-03-13T14:35:35.013 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-03-13T14:35:35.022 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-03-13T14:35:35.022 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-03-13T14:35:35.083 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-03-13T14:35:35.083 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-03-13T14:35:35.249 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-03-13T14:35:35.360 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:35.889 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:36.237 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:18.2.1-0.el9.x86_64 1/1 2024-03-13T14:35:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:36.238 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-03-13T14:35:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:18.2.1-0.el9.x86_64 2024-03-13T14:35:36.238 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:35:36.238 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-03-13T14:35:36.564 DEBUG:teuthology.parallel:result is None 2024-03-13T14:35:36.564 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-13T14:35:36.564 INFO:teuthology.packaging:ref: None 2024-03-13T14:35:36.564 INFO:teuthology.packaging:tag: v18.2.1 2024-03-13T14:35:36.564 INFO:teuthology.packaging:branch: None 2024-03-13T14:35:36.564 INFO:teuthology.packaging:sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:35:36.565 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-13T14:35:36.565 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-13T14:35:36.715 DEBUG:teuthology.orchestra.run.smithi176:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-13T14:35:36.784 INFO:teuthology.orchestra.run.smithi176.stdout:18.2.1-0.el9 2024-03-13T14:35:36.785 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-03-13T14:35:36.785 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-03-13T14:35:36.787 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-03-13T14:35:36.787 INFO:teuthology.packaging:ref: None 2024-03-13T14:35:36.787 INFO:teuthology.packaging:tag: v18.2.1 2024-03-13T14:35:36.787 INFO:teuthology.packaging:branch: None 2024-03-13T14:35:36.787 INFO:teuthology.packaging:sha1: 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:35:36.787 INFO:teuthology.packaging:Tag 'v18.2.1' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-03-13T14:35:36.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7fe91d5d5842e04be3b4f514d6dd990c54b29c76 2024-03-13T14:35:37.022 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-13T14:35:37.090 INFO:teuthology.orchestra.run.smithi196.stdout:18.2.1-0.el9 2024-03-13T14:35:37.090 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-0.el9 2024-03-13T14:35:37.090 INFO:teuthology.task.install:The correct ceph version 18.2.1-0 is installed. 2024-03-13T14:35:37.092 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-13T14:35:37.092 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:35:37.092 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-13T14:35:37.190 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:35:37.190 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-13T14:35:37.288 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-13T14:35:37.288 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:35:37.288 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/daemon-helper 2024-03-13T14:35:37.384 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-13T14:35:37.477 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:35:37.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-03-13T14:35:37.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-13T14:35:37.674 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-13T14:35:37.675 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:35:37.675 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-13T14:35:37.760 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-13T14:35:37.888 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:35:37.888 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-13T14:35:37.977 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-13T14:35:38.112 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-13T14:35:38.113 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:35:38.113 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/usr/bin/stdin-killer 2024-03-13T14:35:38.189 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-13T14:35:38.317 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:35:38.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-03-13T14:35:38.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-13T14:35:38.570 INFO:teuthology.run_tasks:Running task print... 2024-03-13T14:35:38.581 INFO:teuthology.task.print:**** done install task... 2024-03-13T14:35:38.581 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-13T14:35:38.714 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.1', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '9a75c81fd074f3cb252248b860d5cc88e5f3fb55'} 2024-03-13T14:35:38.714 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.1 2024-03-13T14:35:38.715 INFO:tasks.cephadm:Cluster fsid is f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:35:38.715 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-13T14:35:38.715 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-03-13T14:35:38.715 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi176': '172.21.15.176', 'mon.smithi196': '172.21.15.196'} 2024-03-13T14:35:38.715 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-13T14:35:38.715 DEBUG:teuthology.orchestra.run.smithi176:> sudo hostname $(hostname -s) 2024-03-13T14:35:38.810 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-03-13T14:35:38.907 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-03-13T14:35:38.908 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:35:39.051 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9a75c81fd074f3cb252248b860d5cc88e5f3fb55', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77400/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1916-g9a75c81f', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1916.g9a75c81f'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240307.013758/9a75c81fd074f3cb252248b860d5cc88e5f3fb55/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-07 06:42:34.171931', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240307.013758', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240307.013758/9a75c81fd074f3cb252248b860d5cc88e5f3fb55/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-03-13T14:35:39.392 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 98f1558676b6aa7a05f392f0a58d982ad42dbfcc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-03-13T14:35:39.393 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/98f1558676b6aa7a05f392f0a58d982ad42dbfcc/centos/9/x86_64/flavors/default/cephadm 2024-03-13T14:35:39.394 DEBUG:teuthology.orchestra.run.smithi176:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/98f1558676b6aa7a05f392f0a58d982ad42dbfcc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-13T14:35:40.088 INFO:teuthology.orchestra.run.smithi176.stdout:-rw-r--r--. 1 ubuntu ubuntu 214740 Mar 13 14:35 /home/ubuntu/cephtest/cephadm 2024-03-13T14:35:40.089 DEBUG:teuthology.orchestra.run.smithi196:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/98f1558676b6aa7a05f392f0a58d982ad42dbfcc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-13T14:35:40.591 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 214740 Mar 13 14:35 /home/ubuntu/cephtest/cephadm 2024-03-13T14:35:40.592 DEBUG:teuthology.orchestra.run.smithi176:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-13T14:35:40.651 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-13T14:35:40.722 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.1 on all hosts... 2024-03-13T14:35:40.723 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-03-13T14:35:40.730 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 pull 2024-03-13T14:35:41.798 INFO:teuthology.orchestra.run.smithi176.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-13T14:35:41.870 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-13T14:36:07.430 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:36:07.431 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-13T14:36:07.431 INFO:teuthology.orchestra.run.smithi176.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-03-13T14:36:07.431 INFO:teuthology.orchestra.run.smithi176.stdout: "repo_digests": [ 2024-03-13T14:36:07.431 INFO:teuthology.orchestra.run.smithi176.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-03-13T14:36:07.431 INFO:teuthology.orchestra.run.smithi176.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-03-13T14:36:07.431 INFO:teuthology.orchestra.run.smithi176.stdout: ] 2024-03-13T14:36:07.431 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf", 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout: "quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3", 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout: "quay.io/ceph/ceph@sha256:e8e55db8b4fd270dbec25bc764437a2a3abb707971c4dba5f559fb83018049dc" 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-03-13T14:36:11.345 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-13T14:36:11.387 DEBUG:teuthology.orchestra.run.smithi176:> sudo mkdir -p /etc/ceph 2024-03-13T14:36:11.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-03-13T14:36:11.591 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 777 /etc/ceph 2024-03-13T14:36:11.689 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-03-13T14:36:11.783 INFO:tasks.cephadm:Writing seed config... 2024-03-13T14:36:11.784 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-03-13T14:36:11.784 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-03-13T14:36:11.784 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-03-13T14:36:11.784 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-03-13T14:36:11.784 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-13T14:36:11.785 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-13T14:36:11.786 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-13T14:36:11.787 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-13T14:36:11.788 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:36:11.788 DEBUG:teuthology.orchestra.run.smithi176:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-13T14:36:11.842 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f60e35da-e146-11ee-95c9-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-13T14:36:11.843 DEBUG:teuthology.orchestra.run.smithi176:mon.smithi176> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service 2024-03-13T14:36:11.889 INFO:tasks.cephadm:Bootstrapping... 2024-03-13T14:36:11.890 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 -v bootstrap --fsid f60e35da-e146-11ee-95c9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.176 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-13T14:36:12.123 INFO:teuthology.orchestra.run.smithi176.stdout:-------------------------------------------------------------------------------- 2024-03-13T14:36:12.123 INFO:teuthology.orchestra.run.smithi176.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.1', '-v', 'bootstrap', '--fsid', 'f60e35da-e146-11ee-95c9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.176', '--skip-admin-label'] 2024-03-13T14:36:12.183 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-13T14:36:12.183 INFO:teuthology.orchestra.run.smithi176.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-13T14:36:12.184 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying podman|docker is present... 2024-03-13T14:36:12.237 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-13T14:36:12.238 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying lvm2 is present... 2024-03-13T14:36:12.238 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying time synchronization is in place... 2024-03-13T14:36:12.256 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-13T14:36:12.256 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-13T14:36:12.276 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-13T14:36:12.276 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout inactive 2024-03-13T14:36:12.301 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout enabled 2024-03-13T14:36:12.323 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout active 2024-03-13T14:36:12.323 INFO:teuthology.orchestra.run.smithi176.stdout:Unit chronyd.service is enabled and running 2024-03-13T14:36:12.323 INFO:teuthology.orchestra.run.smithi176.stdout:Repeating the final host check... 2024-03-13T14:36:12.379 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-13T14:36:12.380 INFO:teuthology.orchestra.run.smithi176.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-13T14:36:12.380 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl is present 2024-03-13T14:36:12.380 INFO:teuthology.orchestra.run.smithi176.stdout:lvcreate is present 2024-03-13T14:36:12.394 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-13T14:36:12.394 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-13T14:36:12.412 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-13T14:36:12.412 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout inactive 2024-03-13T14:36:12.436 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout enabled 2024-03-13T14:36:12.452 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout active 2024-03-13T14:36:12.453 INFO:teuthology.orchestra.run.smithi176.stdout:Unit chronyd.service is enabled and running 2024-03-13T14:36:12.453 INFO:teuthology.orchestra.run.smithi176.stdout:Host looks OK 2024-03-13T14:36:12.453 INFO:teuthology.orchestra.run.smithi176.stdout:Cluster fsid: f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:12.454 INFO:teuthology.orchestra.run.smithi176.stdout:Acquiring lock 140274790365408 on /run/cephadm/f60e35da-e146-11ee-95c9-87774f69a715.lock 2024-03-13T14:36:12.454 INFO:teuthology.orchestra.run.smithi176.stdout:Lock 140274790365408 acquired on /run/cephadm/f60e35da-e146-11ee-95c9-87774f69a715.lock 2024-03-13T14:36:12.455 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying IP 172.21.15.176 port 3300 ... 2024-03-13T14:36:12.456 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying IP 172.21.15.176 port 6789 ... 2024-03-13T14:36:12.457 INFO:teuthology.orchestra.run.smithi176.stdout:Base mon IP(s) is [172.21.15.176:3300, 172.21.15.176:6789], mon addrv is [v2:172.21.15.176:3300,v1:172.21.15.176:6789] 2024-03-13T14:36:12.471 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.176 metric 100 2024-03-13T14:36:12.471 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.176 metric 100 2024-03-13T14:36:12.483 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-13T14:36:12.483 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-13T14:36:12.484 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-03-13T14:36:12.495 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-13T14:36:12.495 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-13T14:36:12.495 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-13T14:36:12.495 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-13T14:36:12.495 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6f61/64 scope link 2024-03-13T14:36:12.496 INFO:teuthology.orchestra.run.smithi176.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-13T14:36:12.497 INFO:teuthology.orchestra.run.smithi176.stdout:Mon IP `172.21.15.176` is in CIDR network `172.21.0.0/20` 2024-03-13T14:36:12.498 INFO:teuthology.orchestra.run.smithi176.stdout:Mon IP `172.21.15.176` is in CIDR network `172.21.0.0/20` 2024-03-13T14:36:12.498 INFO:teuthology.orchestra.run.smithi176.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-13T14:36:12.500 INFO:teuthology.orchestra.run.smithi176.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-13T14:36:12.503 INFO:teuthology.orchestra.run.smithi176.stdout:Pulling container image quay.io/ceph/ceph:v18.2.1... 2024-03-13T14:36:13.559 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stdout 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-03-13T14:36:13.559 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.1... 2024-03-13T14:36:13.559 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stderr Getting image source signatures 2024-03-13T14:36:13.559 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stderr Copying blob sha256:a733d3c618b71f19c168ebecd1953429dce2c1631835ca182e9551c36dce5989 2024-03-13T14:36:13.559 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stderr Copying blob sha256:7feca07754707458c3945cf0062cf4dabc512f6d90fe1a9a1370b362b6011124 2024-03-13T14:36:13.559 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stderr Copying config sha256:5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf 2024-03-13T14:36:13.559 INFO:teuthology.orchestra.run.smithi176.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-13T14:36:14.703 INFO:teuthology.orchestra.run.smithi176.stdout:ceph: stdout ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-03-13T14:36:14.703 INFO:teuthology.orchestra.run.smithi176.stdout:Ceph version: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable) 2024-03-13T14:36:14.703 INFO:teuthology.orchestra.run.smithi176.stdout:Extracting ceph user uid/gid from container image... 2024-03-13T14:36:15.823 INFO:teuthology.orchestra.run.smithi176.stdout:stat: stdout 167 167 2024-03-13T14:36:15.824 INFO:teuthology.orchestra.run.smithi176.stdout:Creating initial keys... 2024-03-13T14:36:16.887 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph-authtool: stdout AQDgufFlFlR6EBAAIaph0WrC0UZFRc+KMaJh1Q== 2024-03-13T14:36:18.057 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph-authtool: stdout AQDhufFlUhnlFhAAvp8Qt+LZ6AhtONIoK8ss5A== 2024-03-13T14:36:19.116 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph-authtool: stdout AQDiufFltyRiHRAA7vekjSAoOI5xFScwI/M1BA== 2024-03-13T14:36:19.118 INFO:teuthology.orchestra.run.smithi176.stdout:Creating initial monmap... 2024-03-13T14:36:20.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-13T14:36:20.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-03-13T14:36:20.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:20.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-13T14:36:20.096 INFO:teuthology.orchestra.run.smithi176.stdout:monmaptool for smithi176 [v2:172.21.15.176:3300,v1:172.21.15.176:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-13T14:36:20.096 INFO:teuthology.orchestra.run.smithi176.stdout:setting min_mon_release = pacific 2024-03-13T14:36:20.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/monmaptool: set fsid to f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:20.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-13T14:36:20.096 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:36:20.096 INFO:teuthology.orchestra.run.smithi176.stdout:Creating mon... 2024-03-13T14:36:29.955 INFO:teuthology.orchestra.run.smithi176.stdout:create mon.smithi176 on 2024-03-13T14:36:30.302 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-13T14:36:30.540 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-13T14:36:30.817 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f60e35da-e146-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-f60e35da-e146-11ee-95c9-87774f69a715.target. 2024-03-13T14:36:30.817 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f60e35da-e146-11ee-95c9-87774f69a715.target → /etc/systemd/system/ceph-f60e35da-e146-11ee-95c9-87774f69a715.target. 2024-03-13T14:36:31.140 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176 2024-03-13T14:36:31.141 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service: Unit ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service not loaded. 2024-03-13T14:36:31.412 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f60e35da-e146-11ee-95c9-87774f69a715.target.wants/ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service → /etc/systemd/system/ceph-f60e35da-e146-11ee-95c9-87774f69a715@.service. 2024-03-13T14:36:32.033 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:32 smithi176 podman[28668]: 2024-03-13 14:36:32.036922724 +0000 UTC m=+0.378428873 container init 4d76c6d4f81832be5ad23849f5784978825ebb4e9a6dc5d59d74722c0e54be9e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-13T14:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-13T14:36:32.207 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout disabled 2024-03-13T14:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-13T14:36:32.226 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout inactive 2024-03-13T14:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout:firewalld.service is not enabled 2024-03-13T14:36:32.227 INFO:teuthology.orchestra.run.smithi176.stdout:Not possible to enable service . firewalld.service is not available 2024-03-13T14:36:32.228 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for mon to start... 2024-03-13T14:36:32.229 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for mon... 2024-03-13T14:36:32.384 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:32 smithi176 podman[28668]: 2024-03-13 14:36:32.048028728 +0000 UTC m=+0.389534839 container start 4d76c6d4f81832be5ad23849f5784978825ebb4e9a6dc5d59d74722c0e54be9e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, RELEASE=HEAD) 2024-03-13T14:36:32.384 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:32 smithi176 ceph-mon[28687]: mkfs f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:32.384 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:32 smithi176 ceph-mon[28687]: mon.smithi176 is new leader, mons smithi176 in quorum (ranks 0) 2024-03-13T14:36:32.384 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:32 smithi176 bash[28668]: 4d76c6d4f81832be5ad23849f5784978825ebb4e9a6dc5d59d74722c0e54be9e 2024-03-13T14:36:32.384 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:32 smithi176 systemd[1]: Started Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:36:33.236 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:33 smithi176 ceph-mon[28687]: mon.smithi176 is new leader, mons smithi176 in quorum (ranks 0) 2024-03-13T14:36:33.237 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:33 smithi176 ceph-mon[28687]: monmap e1: 1 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:36:33.237 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:33 smithi176 ceph-mon[28687]: fsmap 2024-03-13T14:36:33.237 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:33 smithi176 ceph-mon[28687]: osdmap e1: 0 total, 0 up, 0 in 2024-03-13T14:36:33.237 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:33 smithi176 ceph-mon[28687]: mgrmap e1: no daemons active 2024-03-13T14:36:33.237 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:33 smithi176 ceph-mon[28687]: from='client.? 172.21.15.176:0/3645152530' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-13T14:36:33.472 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout cluster: 2024-03-13T14:36:33.472 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout id: f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout services: 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi176 (age 0.774157s) 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout data: 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout pgs: 2024-03-13T14:36:33.473 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.801+0000 7ff04e800700 1 Processor -- start 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.802+0000 7ff04e800700 1 -- start start 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.802+0000 7ff04e800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff048106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.802+0000 7ff04e800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff048106e10 con 0x7ff048106420 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.803+0000 7ff047400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff048106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.803+0000 7ff047400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff048106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57184/0 (socket says 172.21.15.176:57184) 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.803+0000 7ff047400700 1 -- 172.21.15.176:0/389793208 learned_addr learned my addr 172.21.15.176:0/389793208 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.804+0000 7ff047400700 1 -- 172.21.15.176:0/389793208 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048107620 con 0x7ff048106420 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.804+0000 7ff047400700 1 --2- 172.21.15.176:0/389793208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff048106840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff038009e80 tx=0x7ff03800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ce9ca369e3f81 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.805+0000 7ff046a00700 1 -- 172.21.15.176:0/389793208 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff03800e070 con 0x7ff048106420 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.805+0000 7ff046a00700 1 -- 172.21.15.176:0/389793208 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038003a20 con 0x7ff048106420 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.805+0000 7ff046a00700 1 -- 172.21.15.176:0/389793208 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff038003bd0 con 0x7ff048106420 2024-03-13T14:36:33.479 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.805+0000 7ff04e800700 1 -- 172.21.15.176:0/389793208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 msgr2=0x7ff048106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.805+0000 7ff04e800700 1 --2- 172.21.15.176:0/389793208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff048106840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff038009e80 tx=0x7ff03800b330 comp rx=0 tx=0).stop 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.806+0000 7ff04e800700 1 -- 172.21.15.176:0/389793208 shutdown_connections 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.806+0000 7ff04e800700 1 --2- 172.21.15.176:0/389793208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff048106840 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.806+0000 7ff04e800700 1 -- 172.21.15.176:0/389793208 >> 172.21.15.176:0/389793208 conn(0x7ff048101a00 msgr2=0x7ff048103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.806+0000 7ff04e800700 1 -- 172.21.15.176:0/389793208 shutdown_connections 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.806+0000 7ff04e800700 1 -- 172.21.15.176:0/389793208 wait complete. 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.807+0000 7ff04e800700 1 Processor -- start 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.807+0000 7ff04e800700 1 -- start start 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.807+0000 7ff04e800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff04807bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.807+0000 7ff04e800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04807c1a0 con 0x7ff048106420 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.807+0000 7ff047400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff04807bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:33.480 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.808+0000 7ff047400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff04807bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57200/0 (socket says 172.21.15.176:57200) 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.808+0000 7ff047400700 1 -- 172.21.15.176:0/3645152530 learned_addr learned my addr 172.21.15.176:0/3645152530 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.808+0000 7ff047400700 1 -- 172.21.15.176:0/3645152530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009b30 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.808+0000 7ff047400700 1 --2- 172.21.15.176:0/3645152530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff04807bc60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff038004000 tx=0x7ff038004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.809+0000 7ff044c00700 1 -- 172.21.15.176:0/3645152530 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff038004240 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.809+0000 7ff044c00700 1 -- 172.21.15.176:0/3645152530 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0380043e0 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.809+0000 7ff044c00700 1 -- 172.21.15.176:0/3645152530 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0380046e0 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.809+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04807a2b0 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.809+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04807a750 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.809+0000 7ff044c00700 1 -- 172.21.15.176:0/3645152530 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff038019050 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.809+0000 7ff044c00700 1 -- 172.21.15.176:0/3645152530 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff038025790 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.810+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0480623c0 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.813+0000 7ff044c00700 1 -- 172.21.15.176:0/3645152530 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7ff03802a060 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.891+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff048192d50 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.892+0000 7ff044c00700 1 -- 172.21.15.176:0/3645152530 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7ff038020030 con 0x7ff048106420 2024-03-13T14:36:33.481 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.893+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 msgr2=0x7ff04807bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.893+0000 7ff04e800700 1 --2- 172.21.15.176:0/3645152530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff04807bc60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff038004000 tx=0x7ff038004ad0 comp rx=0 tx=0).stop 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.893+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 shutdown_connections 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.893+0000 7ff04e800700 1 --2- 172.21.15.176:0/3645152530 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048106420 0x7ff04807bc60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.893+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 >> 172.21.15.176:0/3645152530 conn(0x7ff048101a00 msgr2=0x7ff04818dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.894+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 shutdown_connections 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:32.894+0000 7ff04e800700 1 -- 172.21.15.176:0/3645152530 wait complete. 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:mon is available 2024-03-13T14:36:33.482 INFO:teuthology.orchestra.run.smithi176.stdout:Assimilating anything we can from ceph.conf... 2024-03-13T14:36:34.385 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:34 smithi176 ceph-mon[28687]: from='client.? 172.21.15.176:0/359510087' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-13T14:36:34.385 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:34 smithi176 ceph-mon[28687]: from='client.? 172.21.15.176:0/359510087' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-13T14:36:36.238 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:36.239 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout [global] 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout fsid = f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.176:3300,v1:172.21.15.176:6789] 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-13T14:36:36.240 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-13T14:36:36.241 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:36.243 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout [osd] 2024-03-13T14:36:36.243 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-13T14:36:36.243 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.062+0000 7fd1a8c00700 1 Processor -- start 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.062+0000 7fd1a8c00700 1 -- start start 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.063+0000 7fd1a8c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a4079240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.063+0000 7fd1a8c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a4079810 con 0x7fd1a407ade0 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.063+0000 7fd1a1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a4079240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.063+0000 7fd1a1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a4079240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57202/0 (socket says 172.21.15.176:57202) 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.063+0000 7fd1a1e00700 1 -- 172.21.15.176:0/769698827 learned_addr learned my addr 172.21.15.176:0/769698827 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.064+0000 7fd1a1e00700 1 -- 172.21.15.176:0/769698827 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a4079950 con 0x7fd1a407ade0 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.064+0000 7fd1a1e00700 1 --2- 172.21.15.176:0/769698827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a4079240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd194009e80 tx=0x7fd19400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4216e6bed2b378f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.064+0000 7fd1a1400700 1 -- 172.21.15.176:0/769698827 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd19400e070 con 0x7fd1a407ade0 2024-03-13T14:36:36.244 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.064+0000 7fd1a1400700 1 -- 172.21.15.176:0/769698827 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd194003a20 con 0x7fd1a407ade0 2024-03-13T14:36:36.245 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.064+0000 7fd1a1400700 1 -- 172.21.15.176:0/769698827 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd194003bd0 con 0x7fd1a407ade0 2024-03-13T14:36:36.245 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.065+0000 7fd1a8c00700 1 -- 172.21.15.176:0/769698827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 msgr2=0x7fd1a4079240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:36.245 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.065+0000 7fd1a8c00700 1 --2- 172.21.15.176:0/769698827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a4079240 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd194009e80 tx=0x7fd19400b330 comp rx=0 tx=0).stop 2024-03-13T14:36:36.245 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.065+0000 7fd1a8c00700 1 -- 172.21.15.176:0/769698827 shutdown_connections 2024-03-13T14:36:36.245 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.065+0000 7fd1a8c00700 1 --2- 172.21.15.176:0/769698827 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a4079240 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:36.248 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.065+0000 7fd1a8c00700 1 -- 172.21.15.176:0/769698827 >> 172.21.15.176:0/769698827 conn(0x7fd1a41019b0 msgr2=0x7fd1a4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:36.248 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.065+0000 7fd1a8c00700 1 -- 172.21.15.176:0/769698827 shutdown_connections 2024-03-13T14:36:36.248 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.065+0000 7fd1a8c00700 1 -- 172.21.15.176:0/769698827 wait complete. 2024-03-13T14:36:36.248 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.066+0000 7fd1a8c00700 1 Processor -- start 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.067+0000 7fd1a8c00700 1 -- start start 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.067+0000 7fd1a8c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.067+0000 7fd1a8c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a419c010 con 0x7fd1a407ade0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.067+0000 7fd1a1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.067+0000 7fd1a1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a419bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57214/0 (socket says 172.21.15.176:57214) 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.067+0000 7fd1a1e00700 1 -- 172.21.15.176:0/359510087 learned_addr learned my addr 172.21.15.176:0/359510087 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.067+0000 7fd1a1e00700 1 -- 172.21.15.176:0/359510087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd194009b30 con 0x7fd1a407ade0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd1a1e00700 1 --2- 172.21.15.176:0/359510087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a419bad0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd194004000 tx=0x7fd194004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd194004240 con 0x7fd1a407ade0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1940043e0 con 0x7fd1a407ade0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a419c210 con 0x7fd1a407ade0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1940046e0 con 0x7fd1a407ade0 2024-03-13T14:36:36.249 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a419c6b0 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd194019050 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.068+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd194025790 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.069+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a4195930 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.072+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fd19402a060 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.147+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd1a402d0f0 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.157+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd194012d30 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.157+0000 7fd19b400700 1 -- 172.21.15.176:0/359510087 <== mon.0 v2:172.21.15.176:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fd1a402d0f0 con 0x7fd1a407ade0 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.159+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 msgr2=0x7fd1a419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.159+0000 7fd1a8c00700 1 --2- 172.21.15.176:0/359510087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a419bad0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd194004000 tx=0x7fd194004ad0 comp rx=0 tx=0).stop 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.159+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 shutdown_connections 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.159+0000 7fd1a8c00700 1 --2- 172.21.15.176:0/359510087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd1a407ade0 0x7fd1a419bad0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.159+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 >> 172.21.15.176:0/359510087 conn(0x7fd1a41019b0 msgr2=0x7fd1a41033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.159+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 shutdown_connections 2024-03-13T14:36:36.250 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:34.159+0000 7fd1a8c00700 1 -- 172.21.15.176:0/359510087 wait complete. 2024-03-13T14:36:36.251 INFO:teuthology.orchestra.run.smithi176.stdout:Generating new minimal ceph.conf... 2024-03-13T14:36:37.135 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:36 smithi176 ceph-mon[28687]: from='client.? 172.21.15.176:0/3784093300' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:36:37.532 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.785+0000 7feceac00700 1 Processor -- start 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.785+0000 7feceac00700 1 -- start start 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.785+0000 7feceac00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece4109320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.785+0000 7feceac00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece4074600 con 0x7fece4106f30 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.786+0000 7fece3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece4109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.786+0000 7fece3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece4109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57216/0 (socket says 172.21.15.176:57216) 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.786+0000 7fece3e00700 1 -- 172.21.15.176:0/3827240868 learned_addr learned my addr 172.21.15.176:0/3827240868 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.786+0000 7fece3e00700 1 -- 172.21.15.176:0/3827240868 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece4109860 con 0x7fece4106f30 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.786+0000 7fece3e00700 1 --2- 172.21.15.176:0/3827240868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece4109320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fecd40098a0 tx=0x7fecd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a2dadea5157fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.787+0000 7fece3400700 1 -- 172.21.15.176:0/3827240868 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd4004180 con 0x7fece4106f30 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.787+0000 7fece3400700 1 -- 172.21.15.176:0/3827240868 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fecd40042e0 con 0x7fece4106f30 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.787+0000 7fece3400700 1 -- 172.21.15.176:0/3827240868 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd4011a50 con 0x7fece4106f30 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.788+0000 7feceac00700 1 -- 172.21.15.176:0/3827240868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 msgr2=0x7fece4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:37.537 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.788+0000 7feceac00700 1 --2- 172.21.15.176:0/3827240868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece4109320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fecd40098a0 tx=0x7fecd4009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.788+0000 7feceac00700 1 -- 172.21.15.176:0/3827240868 shutdown_connections 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.788+0000 7feceac00700 1 --2- 172.21.15.176:0/3827240868 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece4109320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.788+0000 7feceac00700 1 -- 172.21.15.176:0/3827240868 >> 172.21.15.176:0/3827240868 conn(0x7fece41008c0 msgr2=0x7fece4102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.788+0000 7feceac00700 1 -- 172.21.15.176:0/3827240868 shutdown_connections 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.788+0000 7feceac00700 1 -- 172.21.15.176:0/3827240868 wait complete. 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.789+0000 7feceac00700 1 Processor -- start 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.789+0000 7feceac00700 1 -- start start 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.789+0000 7feceac00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.789+0000 7feceac00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece419a5f0 con 0x7fece4106f30 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece419a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57218/0 (socket says 172.21.15.176:57218) 2024-03-13T14:36:37.538 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece3e00700 1 -- 172.21.15.176:0/3784093300 learned_addr learned my addr 172.21.15.176:0/3784093300 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece3e00700 1 -- 172.21.15.176:0/3784093300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4009550 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece3e00700 1 --2- 172.21.15.176:0/3784093300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece419a0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fecd4003de0 tx=0x7fecd4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece1600700 1 -- 172.21.15.176:0/3784093300 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd4003930 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece1600700 1 -- 172.21.15.176:0/3784093300 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fecd4003a90 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7fece1600700 1 -- 172.21.15.176:0/3784093300 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fecd4019730 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.790+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece419a7f0 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.791+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece419ac90 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.791+0000 7fece1600700 1 -- 172.21.15.176:0/3784093300 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fecd4019c60 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.791+0000 7fece1600700 1 -- 172.21.15.176:0/3784093300 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fecd402c3a0 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.792+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece40623c0 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.795+0000 7fece1600700 1 -- 172.21.15.176:0/3784093300 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fecd401abc0 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.869+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fece419d970 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.869+0000 7fece1600700 1 -- 172.21.15.176:0/3784093300 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fecd401a430 con 0x7fece4106f30 2024-03-13T14:36:37.539 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.871+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 msgr2=0x7fece419a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:37.540 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.871+0000 7feceac00700 1 --2- 172.21.15.176:0/3784093300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece419a0b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fecd4003de0 tx=0x7fecd4011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:37.540 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.871+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 shutdown_connections 2024-03-13T14:36:37.540 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.871+0000 7feceac00700 1 --2- 172.21.15.176:0/3784093300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fece4106f30 0x7fece419a0b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:37.540 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.871+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 >> 172.21.15.176:0/3784093300 conn(0x7fece41008c0 msgr2=0x7fece4101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:37.540 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.871+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 shutdown_connections 2024-03-13T14:36:37.540 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:36.871+0000 7feceac00700 1 -- 172.21.15.176:0/3784093300 wait complete. 2024-03-13T14:36:37.540 INFO:teuthology.orchestra.run.smithi176.stdout:Restarting the monitor... 2024-03-13T14:36:37.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:37 smithi176 systemd[1]: Stopping Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:36:38.331 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[28683]: 2024-03-13T14:36:38.014+0000 7fce8a200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi176 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:36:38.331 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[28683]: 2024-03-13T14:36:38.014+0000 7fce8a200700 -1 mon.smithi176@0(leader) e1 *** Got Signal Terminated *** 2024-03-13T14:36:38.331 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 podman[28930]: 2024-03-13 14:36:38.066195943 +0000 UTC m=+0.179800978 container died 4d76c6d4f81832be5ad23849f5784978825ebb4e9a6dc5d59d74722c0e54be9e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS) 2024-03-13T14:36:38.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 podman[28930]: 2024-03-13 14:36:38.448904732 +0000 UTC m=+0.562509780 container cleanup 4d76c6d4f81832be5ad23849f5784978825ebb4e9a6dc5d59d74722c0e54be9e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-03-13T14:36:38.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 bash[28930]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176 2024-03-13T14:36:38.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 podman[28944]: 2024-03-13 14:36:38.632907696 +0000 UTC m=+0.555438560 container remove 4d76c6d4f81832be5ad23849f5784978825ebb4e9a6dc5d59d74722c0e54be9e (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, ceph=True, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:36:39.361 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service: Deactivated successfully. 2024-03-13T14:36:39.362 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 systemd[1]: Stopped Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:36:39.362 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:38 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service: Consumed 1.725s CPU time. 2024-03-13T14:36:39.362 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 systemd[1]: Starting Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:36:39.616 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 podman[29045]: 2024-03-13 14:36:39.367051382 +0000 UTC m=+0.120729066 container create 3e3173fcfbc8bed856c14c83f765b1b53cec73ff99b5fba4ec796561f4b77dff (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True) 2024-03-13T14:36:39.617 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 podman[29045]: 2024-03-13 14:36:39.282792168 +0000 UTC m=+0.036469818 image pull 5be31c24972a920012b90a9769e8313e2490c82aee752aefd49af9cf4c0f3fcf quay.io/ceph/ceph:v18.2.1 2024-03-13T14:36:39.732 INFO:teuthology.orchestra.run.smithi176.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 podman[29045]: 2024-03-13 14:36:39.621757516 +0000 UTC m=+0.375435222 container init 3e3173fcfbc8bed856c14c83f765b1b53cec73ff99b5fba4ec796561f4b77dff (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.29.1) 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 podman[29045]: 2024-03-13 14:36:39.632490524 +0000 UTC m=+0.386168167 container start 3e3173fcfbc8bed856c14c83f765b1b53cec73ff99b5fba4ec796561f4b77dff (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, GIT_BRANCH=HEAD) 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: set uid:gid to 167:167 (ceph:ceph) 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: pidfile_write: ignore empty --pid-file 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: load: jerasure load: lrc 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: RocksDB version: 7.9.2 2024-03-13T14:36:39.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Git sha 0 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Compile date 2023-12-11 22:07:34 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: DB SUMMARY 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: DB Session ID: 1258XM4ALR03IHWY5D1I 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: CURRENT file: CURRENT 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: IDENTITY file: IDENTITY 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi176/store.db dir, Total Num: 1, files: 000008.sst 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi176/store.db: 000009.log size: 98013 ; 2024-03-13T14:36:39.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.error_if_exists: 0 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.create_if_missing: 0 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.paranoid_checks: 1 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.env: 0x55aa88010720 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.fs: PosixFileSystem 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.info_log: 0x55aa89fc32e0 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_file_opening_threads: 16 2024-03-13T14:36:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.statistics: (nil) 2024-03-13T14:36:39.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.use_fsync: 0 2024-03-13T14:36:39.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_log_file_size: 0 2024-03-13T14:36:39.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-13T14:36:39.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-13T14:36:39.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.keep_log_file_num: 1000 2024-03-13T14:36:39.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.recycle_log_file_num: 0 2024-03-13T14:36:39.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.allow_fallocate: 1 2024-03-13T14:36:39.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.allow_mmap_reads: 0 2024-03-13T14:36:39.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.allow_mmap_writes: 0 2024-03-13T14:36:39.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.use_direct_reads: 0 2024-03-13T14:36:39.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-13T14:36:39.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.create_missing_column_families: 0 2024-03-13T14:36:39.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.db_log_dir: 2024-03-13T14:36:39.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.wal_dir: 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.advise_random_on_open: 1 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.db_write_buffer_size: 0 2024-03-13T14:36:39.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.write_buffer_manager: 0x55aa89234140 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.rate_limiter: (nil) 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.wal_recovery_mode: 2 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.enable_thread_tracking: 0 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.enable_pipelined_write: 0 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.unordered_write: 0 2024-03-13T14:36:39.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.row_cache: None 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.wal_filter: None 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.allow_ingest_behind: 0 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.two_write_queues: 0 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.manual_wal_flush: 0 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.wal_compression: 0 2024-03-13T14:36:39.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.atomic_flush: 0 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.log_readahead_size: 0 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.best_efforts_recovery: 0 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-13T14:36:39.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.allow_data_in_errors: 0 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.db_host_id: __hostname__ 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_background_jobs: 2 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_background_compactions: -1 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_subcompactions: 1 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-13T14:36:39.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_total_wal_size: 0 2024-03-13T14:36:39.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-13T14:36:39.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-13T14:36:39.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-13T14:36:39.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-13T14:36:39.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_open_files: -1 2024-03-13T14:36:39.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bytes_per_sync: 0 2024-03-13T14:36:39.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_readahead_size: 0 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_background_flushes: -1 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Compression algorithms supported: 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kZSTD supported: 0 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kXpressCompression supported: 0 2024-03-13T14:36:39.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kLZ4HCCompression supported: 1 2024-03-13T14:36:39.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kZlibCompression supported: 1 2024-03-13T14:36:39.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kSnappyCompression supported: 1 2024-03-13T14:36:39.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kLZ4Compression supported: 1 2024-03-13T14:36:39.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: kBZip2Compression supported: 0 2024-03-13T14:36:39.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-13T14:36:39.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-13T14:36:39.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi176/store.db/MANIFEST-000010 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.merge_operator: 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_filter: None 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_filter_factory: None 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.sst_partitioner_factory: None 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-13T14:36:39.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55aa89fc33e0) 2024-03-13T14:36:39.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: cache_index_and_filter_blocks: 1 2024-03-13T14:36:39.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-13T14:36:39.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-13T14:36:39.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: pin_top_level_index_and_filter: 1 2024-03-13T14:36:39.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_type: 0 2024-03-13T14:36:39.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_block_index_type: 0 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_shortening: 1 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: checksum: 4 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: no_block_cache: 0 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache: 0x55aa892d51f0 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_name: BinnedLRUCache 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_options: 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: capacity : 536870912 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: num_shard_bits : 4 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: strict_capacity_limit : 0 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: high_pri_pool_ratio: 0.000 2024-03-13T14:36:39.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_compressed: (nil) 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: persistent_cache: (nil) 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_size: 4096 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_size_deviation: 10 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_restart_interval: 16 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_block_restart_interval: 1 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: metadata_block_size: 4096 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: partition_filters: 0 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: use_delta_encoding: 1 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: filter_policy: bloomfilter 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: whole_key_filtering: 1 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: verify_compression: 0 2024-03-13T14:36:39.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: read_amp_bytes_per_bit: 0 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: format_version: 5 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: enable_index_compression: 1 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_align: 0 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_auto_readahead_size: 262144 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: prepopulate_block_cache: 0 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: initial_auto_readahead_size: 8192 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: num_file_reads_for_auto_readahead: 2 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.write_buffer_size: 33554432 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_write_buffer_number: 2 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression: NoCompression 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression: Disabled 2024-03-13T14:36:39.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.prefix_extractor: nullptr 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.num_levels: 7 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:36:39.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.level: 32767 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.strategy: 0 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-13T14:36:39.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.enabled: false 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-13T14:36:39.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.target_file_size_base: 67108864 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-13T14:36:39.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-13T14:36:39.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-13T14:36:39.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-13T14:36:39.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-13T14:36:39.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-13T14:36:39.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-13T14:36:39.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.arena_block_size: 1048576 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.disable_auto_compactions: 0 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-13T14:36:39.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.table_properties_collectors: 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.inplace_update_support: 0 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-13T14:36:39.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.bloom_locality: 0 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.max_successive_merges: 0 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.paranoid_file_checks: 0 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.force_consistency_checks: 1 2024-03-13T14:36:39.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.report_bg_io_stats: 0 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.ttl: 2592000 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.enable_blob_files: false 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.min_blob_size: 0 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.blob_file_size: 268435456 2024-03-13T14:36:39.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-13T14:36:39.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-13T14:36:39.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-13T14:36:39.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-13T14:36:39.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-13T14:36:39.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.blob_file_starting_level: 0 2024-03-13T14:36:39.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-13T14:36:39.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi176/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a030fa90-416d-4be9-b1b0-c6a192d0c156 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710340599686166, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710340599689278, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93120, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 319, "table_properties": {"data_size": 91137, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13372, "raw_average_key_size": 49, "raw_value_size": 83838, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 270, "num_filter_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710340599, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a030fa90-416d-4be9-b1b0-c6a192d0c156", "db_session_id": "1258XM4ALR03IHWY5D1I", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710340599689567, "job": 1, "event": "recovery_finished"} 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-13T14:36:39.909 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi176/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55aa89372000 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: DB pointer 0x55aa8935e000 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** DB Stats ** 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:36:39.910 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** Compaction Stats [default] ** 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: L0 2/0 92.81 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 42.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Sum 2/0 92.81 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 42.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 42.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-13T14:36:39.911 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** Compaction Stats [default] ** 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 42.9 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-13T14:36:39.912 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative compaction: 0.00 GB write, 8.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval compaction: 0.00 GB write, 8.85 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Block cache BinnedLRUCache@0x55aa892d51f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: starting mon.smithi176 rank 0 at public addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] at bind addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi176 fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???) e1 preinit fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???).mds e1 new map 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???).mds e1 print_map 2024-03-13T14:36:39.913 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: e1 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: legacy client fscid: -1 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: No filesystems configured 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-13T14:36:39.914 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 bash[29045]: 3e3173fcfbc8bed856c14c83f765b1b53cec73ff99b5fba4ec796561f4b77dff 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 systemd[1]: Started Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mon.smithi176 is new leader, mons smithi176 in quorum (ranks 0) 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: monmap e1: 1 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: fsmap 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: osdmap e1: 0 total, 0 up, 0 in 2024-03-13T14:36:39.915 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:39 smithi176 ceph-mon[29064]: mgrmap e1: no daemons active 2024-03-13T14:36:41.092 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.446+0000 7f42de800700 1 Processor -- start 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.446+0000 7f42de800700 1 -- start start 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.446+0000 7f42de800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d8106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.446+0000 7f42de800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8106e10 con 0x7f42d8106420 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.447+0000 7f42d7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d8106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.447+0000 7f42d7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d8106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57224/0 (socket says 172.21.15.176:57224) 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.447+0000 7f42d7400700 1 -- 172.21.15.176:0/3596881239 learned_addr learned my addr 172.21.15.176:0/3596881239 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.448+0000 7f42d7400700 1 -- 172.21.15.176:0/3596881239 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8107620 con 0x7f42d8106420 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.448+0000 7f42d7400700 1 --2- 172.21.15.176:0/3596881239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d8106840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f42c80098a0 tx=0x7f42c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9e30cb58b6022c server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.448+0000 7f42d6000700 1 -- 172.21.15.176:0/3596881239 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42c8004180 con 0x7f42d8106420 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.448+0000 7f42d6000700 1 -- 172.21.15.176:0/3596881239 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42c80042e0 con 0x7f42d8106420 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.448+0000 7f42d6000700 1 -- 172.21.15.176:0/3596881239 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42c8011a50 con 0x7f42d8106420 2024-03-13T14:36:41.093 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.449+0000 7f42de800700 1 -- 172.21.15.176:0/3596881239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 msgr2=0x7f42d8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.449+0000 7f42de800700 1 --2- 172.21.15.176:0/3596881239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d8106840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f42c80098a0 tx=0x7f42c8009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.449+0000 7f42de800700 1 -- 172.21.15.176:0/3596881239 shutdown_connections 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.449+0000 7f42de800700 1 --2- 172.21.15.176:0/3596881239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d8106840 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.449+0000 7f42de800700 1 -- 172.21.15.176:0/3596881239 >> 172.21.15.176:0/3596881239 conn(0x7f42d8101a00 msgr2=0x7f42d8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.449+0000 7f42de800700 1 -- 172.21.15.176:0/3596881239 shutdown_connections 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.449+0000 7f42de800700 1 -- 172.21.15.176:0/3596881239 wait complete. 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.450+0000 7f42de800700 1 Processor -- start 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.451+0000 7f42de800700 1 -- start start 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.451+0000 7f42de800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d819c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.451+0000 7f42de800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d819c7f0 con 0x7f42d8106420 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.451+0000 7f42d7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d819c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:41.094 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.451+0000 7f42d7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d819c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57226/0 (socket says 172.21.15.176:57226) 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.451+0000 7f42d7400700 1 -- 172.21.15.176:0/483289803 learned_addr learned my addr 172.21.15.176:0/483289803 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.452+0000 7f42d7400700 1 -- 172.21.15.176:0/483289803 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c8009550 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.452+0000 7f42d7400700 1 --2- 172.21.15.176:0/483289803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d819c2b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000c00 tx=0x7f42c8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.452+0000 7f42cfe00700 1 -- 172.21.15.176:0/483289803 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42c8003b90 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.452+0000 7f42cfe00700 1 -- 172.21.15.176:0/483289803 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f42c80248b0 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.452+0000 7f42cfe00700 1 -- 172.21.15.176:0/483289803 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f42c801a8e0 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.452+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d819c9f0 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.452+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d819ce90 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.453+0000 7f42cfe00700 1 -- 172.21.15.176:0/483289803 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f42c802d3f0 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.453+0000 7f42cfe00700 1 -- 172.21.15.176:0/483289803 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f42c8024440 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.453+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d80623c0 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.457+0000 7f42cfe00700 1 -- 172.21.15.176:0/483289803 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f42c8024600 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.526+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f42d810a820 con 0x7f42d8106420 2024-03-13T14:36:41.095 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.532+0000 7f42cfe00700 1 -- 172.21.15.176:0/483289803 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f42c802d7e0 con 0x7f42d8106420 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.533+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 msgr2=0x7f42d819c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.533+0000 7f42de800700 1 --2- 172.21.15.176:0/483289803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d819c2b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000c00 tx=0x7f42c8011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.534+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 shutdown_connections 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.534+0000 7f42de800700 1 --2- 172.21.15.176:0/483289803 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f42d8106420 0x7f42d819c2b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.534+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 >> 172.21.15.176:0/483289803 conn(0x7f42d8101a00 msgr2=0x7f42d81027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.534+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 shutdown_connections 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:40.534+0000 7f42de800700 1 -- 172.21.15.176:0/483289803 wait complete. 2024-03-13T14:36:41.096 INFO:teuthology.orchestra.run.smithi176.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-13T14:36:41.119 INFO:teuthology.orchestra.run.smithi176.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-13T14:36:41.119 INFO:teuthology.orchestra.run.smithi176.stdout:Creating mgr... 2024-03-13T14:36:41.120 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-13T14:36:41.121 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-13T14:36:41.121 INFO:teuthology.orchestra.run.smithi176.stdout:Verifying port 0.0.0.0:8443 ... 2024-03-13T14:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f60e35da-e146-11ee-95c9-87774f69a715@mgr.smithi176.tzfdwd 2024-03-13T14:36:41.551 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f60e35da-e146-11ee-95c9-87774f69a715@mgr.smithi176.tzfdwd.service: Unit ceph-f60e35da-e146-11ee-95c9-87774f69a715@mgr.smithi176.tzfdwd.service not loaded. 2024-03-13T14:36:41.797 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f60e35da-e146-11ee-95c9-87774f69a715.target.wants/ceph-f60e35da-e146-11ee-95c9-87774f69a715@mgr.smithi176.tzfdwd.service → /etc/systemd/system/ceph-f60e35da-e146-11ee-95c9-87774f69a715@.service. 2024-03-13T14:36:41.826 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:41 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/483289803' entity='client.admin' 2024-03-13T14:36:42.591 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-13T14:36:42.591 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout disabled 2024-03-13T14:36:42.607 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-13T14:36:42.608 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout inactive 2024-03-13T14:36:42.608 INFO:teuthology.orchestra.run.smithi176.stdout:firewalld.service is not enabled 2024-03-13T14:36:42.608 INFO:teuthology.orchestra.run.smithi176.stdout:Not possible to enable service . firewalld.service is not available 2024-03-13T14:36:42.630 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-13T14:36:42.630 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout disabled 2024-03-13T14:36:42.647 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-13T14:36:42.647 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout inactive 2024-03-13T14:36:42.647 INFO:teuthology.orchestra.run.smithi176.stdout:firewalld.service is not enabled 2024-03-13T14:36:42.647 INFO:teuthology.orchestra.run.smithi176.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-03-13T14:36:42.647 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for mgr to start... 2024-03-13T14:36:42.647 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for mgr... 2024-03-13T14:36:43.635 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:43 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1370874294' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsid": "f60e35da-e146-11ee-95c9-87774f69a715", 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "health": { 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-13T14:36:44.007 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 0 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "smithi176" 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-13T14:36:44.008 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:44.009 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-13T14:36:44.009 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-13T14:36:44.009 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-13T14:36:44.009 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-13T14:36:44.009 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-13T14:36:44.018 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-13T14:36:44.019 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "restful" 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:44.020 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modified": "2024-03-13T14:36:32.120524+0000", 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.191+0000 7f789e600700 1 Processor -- start 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.192+0000 7f789e600700 1 -- start start 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.192+0000 7f789e600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f78981090c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.192+0000 7f789e600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898074620 con 0x7f7898106cd0 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.193+0000 7f789d200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f78981090c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:44.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.193+0000 7f789d200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f78981090c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36232/0 (socket says 172.21.15.176:36232) 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.193+0000 7f789d200700 1 -- 172.21.15.176:0/3844121217 learned_addr learned my addr 172.21.15.176:0/3844121217 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.193+0000 7f789d200700 1 -- 172.21.15.176:0/3844121217 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898109600 con 0x7f7898106cd0 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.193+0000 7f789d200700 1 --2- 172.21.15.176:0/3844121217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f78981090c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8984a1a8a2d17 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.194+0000 7f7897400700 1 -- 172.21.15.176:0/3844121217 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f788c004180 con 0x7f7898106cd0 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.194+0000 7f7897400700 1 -- 172.21.15.176:0/3844121217 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f788c0042e0 con 0x7f7898106cd0 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.194+0000 7f7897400700 1 -- 172.21.15.176:0/3844121217 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f788c011a50 con 0x7f7898106cd0 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.194+0000 7f789e600700 1 -- 172.21.15.176:0/3844121217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 msgr2=0x7f78981090c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.194+0000 7f789e600700 1 --2- 172.21.15.176:0/3844121217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f78981090c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:44.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.195+0000 7f789e600700 1 -- 172.21.15.176:0/3844121217 shutdown_connections 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.195+0000 7f789e600700 1 --2- 172.21.15.176:0/3844121217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f78981090c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.195+0000 7f789e600700 1 -- 172.21.15.176:0/3844121217 >> 172.21.15.176:0/3844121217 conn(0x7f78981006a0 msgr2=0x7f7898102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.195+0000 7f789e600700 1 -- 172.21.15.176:0/3844121217 shutdown_connections 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.195+0000 7f789e600700 1 -- 172.21.15.176:0/3844121217 wait complete. 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.196+0000 7f789e600700 1 Processor -- start 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.196+0000 7f789e600700 1 -- start start 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.196+0000 7f789e600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f789819c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.196+0000 7f789e600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789819c630 con 0x7f7898106cd0 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.197+0000 7f789d200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f789819c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.197+0000 7f789d200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f789819c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36244/0 (socket says 172.21.15.176:36244) 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.197+0000 7f789d200700 1 -- 172.21.15.176:0/1370874294 learned_addr learned my addr 172.21.15.176:0/1370874294 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:44.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.197+0000 7f789d200700 1 -- 172.21.15.176:0/1370874294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c009550 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.197+0000 7f789d200700 1 --2- 172.21.15.176:0/1370874294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f789819c0f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f788c0119e0 tx=0x7f788c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.198+0000 7f7895600700 1 -- 172.21.15.176:0/1370874294 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f788c003b90 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.198+0000 7f7895600700 1 -- 172.21.15.176:0/1370874294 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f788c01a8a0 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.198+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789819c830 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.198+0000 7f7895600700 1 -- 172.21.15.176:0/1370874294 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f788c019930 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.198+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789819ccd0 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.198+0000 7f7895600700 1 -- 172.21.15.176:0/1370874294 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f788c0243f0 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.198+0000 7f7895600700 1 -- 172.21.15.176:0/1370874294 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f788c0249e0 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.199+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789810a320 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.202+0000 7f7895600700 1 -- 172.21.15.176:0/1370874294 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f788c01aa10 con 0x7f7898106cd0 2024-03-13T14:36:44.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.278+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f789810a510 con 0x7f7898106cd0 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.279+0000 7f7895600700 1 -- 172.21.15.176:0/1370874294 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f788c01abb0 con 0x7f7898106cd0 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.281+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 msgr2=0x7f789819c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.281+0000 7f789e600700 1 --2- 172.21.15.176:0/1370874294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f789819c0f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f788c0119e0 tx=0x7f788c011ac0 comp rx=0 tx=0).stop 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.281+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 shutdown_connections 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.281+0000 7f789e600700 1 --2- 172.21.15.176:0/1370874294 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7898106cd0 0x7f789819c0f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.281+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 >> 172.21.15.176:0/1370874294 conn(0x7f78981006a0 msgr2=0x7f7898101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.281+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 shutdown_connections 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:43.281+0000 7f789e600700 1 -- 172.21.15.176:0/1370874294 wait complete. 2024-03-13T14:36:44.025 INFO:teuthology.orchestra.run.smithi176.stdout:mgr not available, waiting (1/15)... 2024-03-13T14:36:47.021 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:46 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/4131987369' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-13T14:36:47.452 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:47.453 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsid": "f60e35da-e146-11ee-95c9-87774f69a715", 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "health": { 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-13T14:36:47.454 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 0 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "smithi176" 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-13T14:36:47.456 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:47.457 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-13T14:36:47.457 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:47.457 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-13T14:36:47.457 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-13T14:36:47.457 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-13T14:36:47.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-13T14:36:47.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-13T14:36:47.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-13T14:36:47.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:47.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-13T14:36:47.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-13T14:36:47.466 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "restful" 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:47.467 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modified": "2024-03-13T14:36:32.120524+0000", 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.658+0000 7fca44400700 1 Processor -- start 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.658+0000 7fca44400700 1 -- start start 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.658+0000 7fca44400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c079240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.658+0000 7fca44400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c079810 con 0x7fca3c07ade0 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.659+0000 7fca41a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c079240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.659+0000 7fca41a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c079240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36246/0 (socket says 172.21.15.176:36246) 2024-03-13T14:36:47.468 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.659+0000 7fca41a00700 1 -- 172.21.15.176:0/282036587 learned_addr learned my addr 172.21.15.176:0/282036587 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.659+0000 7fca41a00700 1 -- 172.21.15.176:0/282036587 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c079950 con 0x7fca3c07ade0 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.660+0000 7fca41a00700 1 --2- 172.21.15.176:0/282036587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c079240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca300098a0 tx=0x7fca30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e079d90e3d4ea070 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.660+0000 7fca3be00700 1 -- 172.21.15.176:0/282036587 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca30004180 con 0x7fca3c07ade0 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.660+0000 7fca3be00700 1 -- 172.21.15.176:0/282036587 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca300042e0 con 0x7fca3c07ade0 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.660+0000 7fca3be00700 1 -- 172.21.15.176:0/282036587 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca30011a50 con 0x7fca3c07ade0 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.661+0000 7fca44400700 1 -- 172.21.15.176:0/282036587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 msgr2=0x7fca3c079240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.661+0000 7fca44400700 1 --2- 172.21.15.176:0/282036587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c079240 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fca300098a0 tx=0x7fca30009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.661+0000 7fca44400700 1 -- 172.21.15.176:0/282036587 shutdown_connections 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.661+0000 7fca44400700 1 --2- 172.21.15.176:0/282036587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c079240 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.661+0000 7fca44400700 1 -- 172.21.15.176:0/282036587 >> 172.21.15.176:0/282036587 conn(0x7fca3c1019b0 msgr2=0x7fca3c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.661+0000 7fca44400700 1 -- 172.21.15.176:0/282036587 shutdown_connections 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.661+0000 7fca44400700 1 -- 172.21.15.176:0/282036587 wait complete. 2024-03-13T14:36:47.469 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.662+0000 7fca44400700 1 Processor -- start 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.662+0000 7fca44400700 1 -- start start 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.662+0000 7fca44400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.662+0000 7fca44400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c1a0c70 con 0x7fca3c07ade0 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.663+0000 7fca41a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.663+0000 7fca41a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c1a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36258/0 (socket says 172.21.15.176:36258) 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.663+0000 7fca41a00700 1 -- 172.21.15.176:0/4131987369 learned_addr learned my addr 172.21.15.176:0/4131987369 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.663+0000 7fca41a00700 1 -- 172.21.15.176:0/4131987369 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca30009550 con 0x7fca3c07ade0 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.663+0000 7fca41a00700 1 --2- 172.21.15.176:0/4131987369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c1a0730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca30003de0 tx=0x7fca30011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.664+0000 7fca3a000700 1 -- 172.21.15.176:0/4131987369 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca30003930 con 0x7fca3c07ade0 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.664+0000 7fca3a000700 1 -- 172.21.15.176:0/4131987369 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fca30003a90 con 0x7fca3c07ade0 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.664+0000 7fca3a000700 1 -- 172.21.15.176:0/4131987369 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca300196c0 con 0x7fca3c07ade0 2024-03-13T14:36:47.470 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.664+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c1a0e70 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.664+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c1a1310 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.664+0000 7fca3a000700 1 -- 172.21.15.176:0/4131987369 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fca30019c80 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.664+0000 7fca3a000700 1 -- 172.21.15.176:0/4131987369 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fca3001a430 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.665+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c0623c0 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.668+0000 7fca3a000700 1 -- 172.21.15.176:0/4131987369 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7fca3001abc0 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.745+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fca3c1a3e60 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.746+0000 7fca3a000700 1 -- 172.21.15.176:0/4131987369 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fca3001a5f0 con 0x7fca3c07ade0 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.748+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 msgr2=0x7fca3c1a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.748+0000 7fca44400700 1 --2- 172.21.15.176:0/4131987369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c1a0730 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fca30003de0 tx=0x7fca30011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.748+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 shutdown_connections 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.748+0000 7fca44400700 1 --2- 172.21.15.176:0/4131987369 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca3c07ade0 0x7fca3c1a0730 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.748+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 >> 172.21.15.176:0/4131987369 conn(0x7fca3c1019b0 msgr2=0x7fca3c1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:47.471 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.748+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 shutdown_connections 2024-03-13T14:36:47.472 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:46.748+0000 7fca44400700 1 -- 172.21.15.176:0/4131987369 wait complete. 2024-03-13T14:36:47.472 INFO:teuthology.orchestra.run.smithi176.stdout:mgr not available, waiting (2/15)... 2024-03-13T14:36:50.774 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:50 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/484109420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-13T14:36:50.802 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsid": "f60e35da-e146-11ee-95c9-87774f69a715", 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "health": { 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 0 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "smithi176" 2024-03-13T14:36:50.803 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-13T14:36:50.804 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-13T14:36:50.811 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-13T14:36:50.811 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-13T14:36:50.812 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "restful" 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:50.813 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modified": "2024-03-13T14:36:32.120524+0000", 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.076+0000 7f0bf0800700 1 Processor -- start 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.076+0000 7f0bf0800700 1 -- start start 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.077+0000 7f0bf0800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be8079220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.077+0000 7f0bf0800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be80797f0 con 0x7f0be807adc0 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.077+0000 7f0bede00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be8079220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.077+0000 7f0bede00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be8079220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36266/0 (socket says 172.21.15.176:36266) 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.077+0000 7f0bede00700 1 -- 172.21.15.176:0/3746526785 learned_addr learned my addr 172.21.15.176:0/3746526785 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:50.814 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.078+0000 7f0bede00700 1 -- 172.21.15.176:0/3746526785 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be8079930 con 0x7f0be807adc0 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.078+0000 7f0bede00700 1 --2- 172.21.15.176:0/3746526785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be8079220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0098a0 tx=0x7f0bdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f702839fb018bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.078+0000 7f0beca00700 1 -- 172.21.15.176:0/3746526785 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bdc004180 con 0x7f0be807adc0 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.078+0000 7f0beca00700 1 -- 172.21.15.176:0/3746526785 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bdc0042e0 con 0x7f0be807adc0 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.078+0000 7f0beca00700 1 -- 172.21.15.176:0/3746526785 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bdc011a50 con 0x7f0be807adc0 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.079+0000 7f0bf0800700 1 -- 172.21.15.176:0/3746526785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 msgr2=0x7f0be8079220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.079+0000 7f0bf0800700 1 --2- 172.21.15.176:0/3746526785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be8079220 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc0098a0 tx=0x7f0bdc009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.079+0000 7f0bf0800700 1 -- 172.21.15.176:0/3746526785 shutdown_connections 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.079+0000 7f0bf0800700 1 --2- 172.21.15.176:0/3746526785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be8079220 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.079+0000 7f0bf0800700 1 -- 172.21.15.176:0/3746526785 >> 172.21.15.176:0/3746526785 conn(0x7f0be81019b0 msgr2=0x7f0be8103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.079+0000 7f0bf0800700 1 -- 172.21.15.176:0/3746526785 shutdown_connections 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.079+0000 7f0bf0800700 1 -- 172.21.15.176:0/3746526785 wait complete. 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.080+0000 7f0bf0800700 1 Processor -- start 2024-03-13T14:36:50.815 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.080+0000 7f0bf0800700 1 -- start start 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.080+0000 7f0bf0800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.080+0000 7f0bf0800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be819c850 con 0x7f0be807adc0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.081+0000 7f0bede00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.081+0000 7f0bede00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be819c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36276/0 (socket says 172.21.15.176:36276) 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.081+0000 7f0bede00700 1 -- 172.21.15.176:0/484109420 learned_addr learned my addr 172.21.15.176:0/484109420 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.081+0000 7f0bede00700 1 -- 172.21.15.176:0/484109420 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc009550 con 0x7f0be807adc0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.081+0000 7f0bede00700 1 --2- 172.21.15.176:0/484109420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be819c310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc003de0 tx=0x7f0bdc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.082+0000 7f0be2a00700 1 -- 172.21.15.176:0/484109420 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bdc003930 con 0x7f0be807adc0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.082+0000 7f0be2a00700 1 -- 172.21.15.176:0/484109420 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0bdc003a90 con 0x7f0be807adc0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.082+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be819ca50 con 0x7f0be807adc0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.082+0000 7f0be2a00700 1 -- 172.21.15.176:0/484109420 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0bdc0196c0 con 0x7f0be807adc0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.082+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be819cef0 con 0x7f0be807adc0 2024-03-13T14:36:50.816 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.082+0000 7f0be2a00700 1 -- 172.21.15.176:0/484109420 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0bdc019c80 con 0x7f0be807adc0 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.082+0000 7f0be2a00700 1 -- 172.21.15.176:0/484109420 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0bdc02c3a0 con 0x7f0be807adc0 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.083+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be80623c0 con 0x7f0be807adc0 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.086+0000 7f0be2a00700 1 -- 172.21.15.176:0/484109420 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72446 (secure 0 0 0) 0x7f0bdc01abc0 con 0x7f0be807adc0 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.163+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0be819fa70 con 0x7f0be807adc0 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.164+0000 7f0be2a00700 1 -- 172.21.15.176:0/484109420 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f0bdc005a20 con 0x7f0be807adc0 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.166+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 msgr2=0x7f0be819c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.166+0000 7f0bf0800700 1 --2- 172.21.15.176:0/484109420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be819c310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc003de0 tx=0x7f0bdc011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.166+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 shutdown_connections 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.166+0000 7f0bf0800700 1 --2- 172.21.15.176:0/484109420 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0be807adc0 0x7f0be819c310 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.166+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 >> 172.21.15.176:0/484109420 conn(0x7f0be81019b0 msgr2=0x7f0be81030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.166+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 shutdown_connections 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:50.166+0000 7f0bf0800700 1 -- 172.21.15.176:0/484109420 wait complete. 2024-03-13T14:36:50.817 INFO:teuthology.orchestra.run.smithi176.stdout:mgr not available, waiting (3/15)... 2024-03-13T14:36:52.136 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: Activating manager daemon smithi176.tzfdwd 2024-03-13T14:36:52.136 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: mgrmap e2: smithi176.tzfdwd(active, starting, since 0.00989354s) 2024-03-13T14:36:52.136 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:36:52.137 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:36:52.137 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:36:52.137 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:36:52.137 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:36:52.137 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:36:52.137 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:36:52.138 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:36:52.138 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:36:52.138 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:36:52.138 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:36:52.138 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:36:52.139 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:36:52.139 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:51 smithi176 ceph-mon[29064]: from='mgr.14100 172.21.15.176:0/3263810157' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:36:53.298 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:53 smithi176 ceph-mon[29064]: mgrmap e3: smithi176.tzfdwd(active, since 1.01821s) 2024-03-13T14:36:54.384 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:54 smithi176 ceph-mon[29064]: mgrmap e4: smithi176.tzfdwd(active, since 2s) 2024-03-13T14:36:54.385 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:54 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2348722393' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsid": "f60e35da-e146-11ee-95c9-87774f69a715", 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "health": { 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-13T14:36:54.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 0 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "smithi176" 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-13T14:36:54.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-13T14:36:54.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-13T14:36:54.504 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "restful" 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ], 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "modified": "2024-03-13T14:36:32.120524+0000", 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout }, 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-13T14:36:54.505 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.463+0000 7f157a800700 1 Processor -- start 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.463+0000 7f157a800700 1 -- start start 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.463+0000 7f157a800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f1574108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.463+0000 7f157a800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1574109070 con 0x7f1574108680 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.464+0000 7f1573400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f1574108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.464+0000 7f1573400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f1574108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53358/0 (socket says 172.21.15.176:53358) 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.464+0000 7f1573400700 1 -- 172.21.15.176:0/3160836395 learned_addr learned my addr 172.21.15.176:0/3160836395 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.464+0000 7f1573400700 1 -- 172.21.15.176:0/3160836395 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1574109830 con 0x7f1574108680 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.464+0000 7f1573400700 1 --2- 172.21.15.176:0/3160836395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f1574108aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f15640098a0 tx=0x7f1564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1de50484d5b6429 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f1572a00700 1 -- 172.21.15.176:0/3160836395 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1564004180 con 0x7f1574108680 2024-03-13T14:36:54.511 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f1572a00700 1 -- 172.21.15.176:0/3160836395 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f15640042e0 con 0x7f1574108680 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f1572a00700 1 -- 172.21.15.176:0/3160836395 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1564011a50 con 0x7f1574108680 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f157a800700 1 -- 172.21.15.176:0/3160836395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 msgr2=0x7f1574108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f157a800700 1 --2- 172.21.15.176:0/3160836395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f1574108aa0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f15640098a0 tx=0x7f1564009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f157a800700 1 -- 172.21.15.176:0/3160836395 shutdown_connections 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f157a800700 1 --2- 172.21.15.176:0/3160836395 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f1574108aa0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f157a800700 1 -- 172.21.15.176:0/3160836395 >> 172.21.15.176:0/3160836395 conn(0x7f157407bb20 msgr2=0x7f15741064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.465+0000 7f157a800700 1 -- 172.21.15.176:0/3160836395 shutdown_connections 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.466+0000 7f157a800700 1 -- 172.21.15.176:0/3160836395 wait complete. 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.466+0000 7f157a800700 1 Processor -- start 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.467+0000 7f157a800700 1 -- start start 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.467+0000 7f157a800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f157419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.467+0000 7f157a800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157419c890 con 0x7f1574108680 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.467+0000 7f1573400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f157419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:54.512 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.467+0000 7f1573400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f157419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53360/0 (socket says 172.21.15.176:53360) 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.467+0000 7f1573400700 1 -- 172.21.15.176:0/2348722393 learned_addr learned my addr 172.21.15.176:0/2348722393 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.468+0000 7f1573400700 1 -- 172.21.15.176:0/2348722393 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564009550 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.468+0000 7f1573400700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f157419c350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1564003de0 tx=0x7f1564011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.468+0000 7f1570c00700 1 -- 172.21.15.176:0/2348722393 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1564003930 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.468+0000 7f1570c00700 1 -- 172.21.15.176:0/2348722393 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1564003a90 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.468+0000 7f1570c00700 1 -- 172.21.15.176:0/2348722393 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f15640196c0 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.468+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157419ca90 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.468+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157419cf30 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.469+0000 7f1570c00700 1 -- 172.21.15.176:0/2348722393 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f156401abc0 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.469+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1574196450 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.470+0000 7f1570c00700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1558038320 0x7f155803a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.470+0000 7f1570c00700 1 -- 172.21.15.176:0/2348722393 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f156404c010 con 0x7f1574108680 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.470+0000 7f156be00700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1558038320 0x7f155803a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:54.513 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.471+0000 7f156be00700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1558038320 0x7f155803a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f155c006fd0 tx=0x7f155c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.476+0000 7f1570c00700 1 -- 172.21.15.176:0/2348722393 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1564010790 con 0x7f1574108680 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.767+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f157402d090 con 0x7f1574108680 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.767+0000 7f1570c00700 1 -- 172.21.15.176:0/2348722393 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f1564010380 con 0x7f1574108680 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.770+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1558038320 msgr2=0x7f155803a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.770+0000 7f157a800700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1558038320 0x7f155803a7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f155c006fd0 tx=0x7f155c006e40 comp rx=0 tx=0).stop 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.770+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 msgr2=0x7f157419c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.770+0000 7f157a800700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f157419c350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f1564003de0 tx=0x7f1564011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.771+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 shutdown_connections 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.771+0000 7f157a800700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1558038320 0x7f155803a7e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.771+0000 7f157a800700 1 --2- 172.21.15.176:0/2348722393 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1574108680 0x7f157419c350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.771+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 >> 172.21.15.176:0/2348722393 conn(0x7f157407bb20 msgr2=0x7f1574106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.771+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 shutdown_connections 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:53.771+0000 7f157a800700 1 -- 172.21.15.176:0/2348722393 wait complete. 2024-03-13T14:36:54.514 INFO:teuthology.orchestra.run.smithi176.stdout:mgr is available 2024-03-13T14:36:55.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:55 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/409125407' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-13T14:36:55.956 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:55.956 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout [global] 2024-03-13T14:36:55.963 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout fsid = f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:36:55.963 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-13T14:36:55.963 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-13T14:36:55.963 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-13T14:36:55.963 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-13T14:36:55.963 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout [osd] 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.063+0000 7f5063e00700 1 Processor -- start 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.063+0000 7f5063e00700 1 -- start start 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.063+0000 7f5063e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c1092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.063+0000 7f5063e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c074680 con 0x7f505c106eb0 2024-03-13T14:36:55.964 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.064+0000 7f5061400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c1092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.064+0000 7f5061400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c1092a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53370/0 (socket says 172.21.15.176:53370) 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.064+0000 7f5061400700 1 -- 172.21.15.176:0/3647541001 learned_addr learned my addr 172.21.15.176:0/3647541001 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.064+0000 7f5061400700 1 -- 172.21.15.176:0/3647541001 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c1097e0 con 0x7f505c106eb0 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.065+0000 7f5061400700 1 --2- 172.21.15.176:0/3647541001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c1092a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f50500098a0 tx=0x7f5050009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3996b8ce4623d590 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.065+0000 7f505be00700 1 -- 172.21.15.176:0/3647541001 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5050004180 con 0x7f505c106eb0 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.065+0000 7f505be00700 1 -- 172.21.15.176:0/3647541001 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f50500042e0 con 0x7f505c106eb0 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.065+0000 7f505be00700 1 -- 172.21.15.176:0/3647541001 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5050011a50 con 0x7f505c106eb0 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.066+0000 7f5063e00700 1 -- 172.21.15.176:0/3647541001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 msgr2=0x7f505c1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.066+0000 7f5063e00700 1 --2- 172.21.15.176:0/3647541001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c1092a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f50500098a0 tx=0x7f5050009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.066+0000 7f5063e00700 1 -- 172.21.15.176:0/3647541001 shutdown_connections 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.066+0000 7f5063e00700 1 --2- 172.21.15.176:0/3647541001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c1092a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.066+0000 7f5063e00700 1 -- 172.21.15.176:0/3647541001 >> 172.21.15.176:0/3647541001 conn(0x7f505c1008c0 msgr2=0x7f505c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:55.965 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.066+0000 7f5063e00700 1 -- 172.21.15.176:0/3647541001 shutdown_connections 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.066+0000 7f5063e00700 1 -- 172.21.15.176:0/3647541001 wait complete. 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.068+0000 7f5063e00700 1 Processor -- start 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.068+0000 7f5063e00700 1 -- start start 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.068+0000 7f5063e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.068+0000 7f5063e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c19a650 con 0x7f505c106eb0 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.069+0000 7f5061400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.069+0000 7f5061400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53378/0 (socket says 172.21.15.176:53378) 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.069+0000 7f5061400700 1 -- 172.21.15.176:0/409125407 learned_addr learned my addr 172.21.15.176:0/409125407 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.069+0000 7f5061400700 1 -- 172.21.15.176:0/409125407 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5050009550 con 0x7f505c106eb0 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.070+0000 7f5061400700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c19a110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5050003de0 tx=0x7f5050011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.070+0000 7f505a000700 1 -- 172.21.15.176:0/409125407 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5050003930 con 0x7f505c106eb0 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.070+0000 7f505a000700 1 -- 172.21.15.176:0/409125407 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5050003a90 con 0x7f505c106eb0 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.070+0000 7f505a000700 1 -- 172.21.15.176:0/409125407 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5050019730 con 0x7f505c106eb0 2024-03-13T14:36:55.966 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.070+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c19a850 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.070+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c19acf0 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.071+0000 7f505a000700 1 -- 172.21.15.176:0/409125407 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f505001a430 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.071+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c10a560 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.072+0000 7f505a000700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f5044038320 0x7f504403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.072+0000 7f505a000700 1 -- 172.21.15.176:0/409125407 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f505004c050 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.072+0000 7f5060a00700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f5044038320 0x7f504403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.073+0000 7f5060a00700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f5044038320 0x7f504403a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5048006fd0 tx=0x7f5048006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.077+0000 7f505a000700 1 -- 172.21.15.176:0/409125407 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5050010560 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.280+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f505c10a780 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.282+0000 7f505a000700 1 -- 172.21.15.176:0/409125407 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f50500281c0 con 0x7f505c106eb0 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.284+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f5044038320 msgr2=0x7f504403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.284+0000 7f5063e00700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f5044038320 0x7f504403a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5048006fd0 tx=0x7f5048006e40 comp rx=0 tx=0).stop 2024-03-13T14:36:55.967 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.284+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 msgr2=0x7f505c19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.284+0000 7f5063e00700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c19a110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5050003de0 tx=0x7f5050011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.285+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 shutdown_connections 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.285+0000 7f5063e00700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f5044038320 0x7f504403a7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.285+0000 7f5063e00700 1 --2- 172.21.15.176:0/409125407 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f505c106eb0 0x7f505c19a110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.285+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 >> 172.21.15.176:0/409125407 conn(0x7f505c1008c0 msgr2=0x7f505c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.285+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 shutdown_connections 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:55.285+0000 7f5063e00700 1 -- 172.21.15.176:0/409125407 wait complete. 2024-03-13T14:36:55.968 INFO:teuthology.orchestra.run.smithi176.stdout:Enabling cephadm module... 2024-03-13T14:36:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:56 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2816338429' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-13T14:36:57.644 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.568+0000 7fdba4e00700 1 Processor -- start 2024-03-13T14:36:57.644 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.568+0000 7fdba4e00700 1 -- start start 2024-03-13T14:36:57.644 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.568+0000 7fdba4e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba0108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:57.644 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.568+0000 7fdba4e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0109070 con 0x7fdba0108680 2024-03-13T14:36:57.644 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.569+0000 7fdb9de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba0108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:57.649 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.569+0000 7fdb9de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba0108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53382/0 (socket says 172.21.15.176:53382) 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.569+0000 7fdb9de00700 1 -- 172.21.15.176:0/3415080700 learned_addr learned my addr 172.21.15.176:0/3415080700 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.569+0000 7fdb9de00700 1 -- 172.21.15.176:0/3415080700 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0109830 con 0x7fdba0108680 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.570+0000 7fdb9de00700 1 --2- 172.21.15.176:0/3415080700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba0108aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4910aacc26391483 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.570+0000 7fdb9d400700 1 -- 172.21.15.176:0/3415080700 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb90004180 con 0x7fdba0108680 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.570+0000 7fdb9d400700 1 -- 172.21.15.176:0/3415080700 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb900042e0 con 0x7fdba0108680 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.570+0000 7fdb9d400700 1 -- 172.21.15.176:0/3415080700 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb90011a50 con 0x7fdba0108680 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.570+0000 7fdba4e00700 1 -- 172.21.15.176:0/3415080700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 msgr2=0x7fdba0108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.570+0000 7fdba4e00700 1 --2- 172.21.15.176:0/3415080700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba0108aa0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb900098a0 tx=0x7fdb90009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.571+0000 7fdba4e00700 1 -- 172.21.15.176:0/3415080700 shutdown_connections 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.571+0000 7fdba4e00700 1 --2- 172.21.15.176:0/3415080700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba0108aa0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:57.650 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.571+0000 7fdba4e00700 1 -- 172.21.15.176:0/3415080700 >> 172.21.15.176:0/3415080700 conn(0x7fdba007bb20 msgr2=0x7fdba01064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.571+0000 7fdba4e00700 1 -- 172.21.15.176:0/3415080700 shutdown_connections 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.571+0000 7fdba4e00700 1 -- 172.21.15.176:0/3415080700 wait complete. 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.572+0000 7fdba4e00700 1 Processor -- start 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.572+0000 7fdba4e00700 1 -- start start 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.572+0000 7fdba4e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.572+0000 7fdba4e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba019c820 con 0x7fdba0108680 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.573+0000 7fdb9de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.573+0000 7fdb9de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba019c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53388/0 (socket says 172.21.15.176:53388) 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.573+0000 7fdb9de00700 1 -- 172.21.15.176:0/2816338429 learned_addr learned my addr 172.21.15.176:0/2816338429 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.573+0000 7fdb9de00700 1 -- 172.21.15.176:0/2816338429 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb90009550 con 0x7fdba0108680 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.573+0000 7fdb9de00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba019c2e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdb90003de0 tx=0x7fdb90011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:57.651 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.574+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb90003930 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.574+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdb90003a90 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.574+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba019ca20 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.574+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdb900196c0 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.574+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba019cec0 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.575+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 4) v1 ==== 45272+0+0 (secure 0 0 0) 0x7fdb9001abc0 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.575+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba00623c0 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.575+0000 7fdb96a00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7fdb840382d0 0x7fdb8403a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.575+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdb9004c080 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.576+0000 7fdb97e00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7fdb840382d0 0x7fdb8403a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.576+0000 7fdb97e00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7fdb840382d0 0x7fdb8403a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006fd0 tx=0x7fdb88006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.581+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdb9007f0e0 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.854+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fdba010ca20 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.911+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fdb9001a7b0 con 0x7fdba0108680 2024-03-13T14:36:57.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.915+0000 7fdb96a00700 1 -- 172.21.15.176:0/2816338429 <== mon.0 v2:172.21.15.176:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fdb9004fbb0 con 0x7fdba0108680 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7fdb840382d0 msgr2=0x7fdb8403a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7fdb840382d0 0x7fdb8403a790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb88006fd0 tx=0x7fdb88006e40 comp rx=0 tx=0).stop 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 msgr2=0x7fdba019c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba019c2e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fdb90003de0 tx=0x7fdb90011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 shutdown_connections 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7fdb840382d0 0x7fdb8403a790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 --2- 172.21.15.176:0/2816338429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdba0108680 0x7fdba019c2e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 >> 172.21.15.176:0/2816338429 conn(0x7fdba007bb20 msgr2=0x7fdba0106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 shutdown_connections 2024-03-13T14:36:57.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:56.918+0000 7fdba4e00700 1 -- 172.21.15.176:0/2816338429 wait complete. 2024-03-13T14:36:58.204 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:57 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2816338429' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-13T14:36:58.205 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:57 smithi176 ceph-mon[29064]: mgrmap e5: smithi176.tzfdwd(active, since 5s) 2024-03-13T14:36:59.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:36:58 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3312353299' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-13T14:36:59.251 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:36:59.251 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-13T14:36:59.251 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-13T14:36:59.251 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "active_name": "smithi176.tzfdwd", 2024-03-13T14:36:59.251 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-13T14:36:59.251 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.254+0000 7f1c17e00700 1 Processor -- start 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.254+0000 7f1c17e00700 1 -- start start 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.254+0000 7f1c17e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c10106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.255+0000 7f1c17e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10106e20 con 0x7f1c10106430 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.255+0000 7f1c15400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c10106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.255+0000 7f1c15400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c10106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53402/0 (socket says 172.21.15.176:53402) 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.255+0000 7f1c15400700 1 -- 172.21.15.176:0/2303678166 learned_addr learned my addr 172.21.15.176:0/2303678166 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.255+0000 7f1c15400700 1 -- 172.21.15.176:0/2303678166 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c10107630 con 0x7f1c10106430 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.256+0000 7f1c15400700 1 --2- 172.21.15.176:0/2303678166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c10106850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c040098a0 tx=0x7f1c04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398aba2e1f605b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.256+0000 7f1c0fe00700 1 -- 172.21.15.176:0/2303678166 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c04004180 con 0x7f1c10106430 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.256+0000 7f1c0fe00700 1 -- 172.21.15.176:0/2303678166 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c040042e0 con 0x7f1c10106430 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.256+0000 7f1c0fe00700 1 -- 172.21.15.176:0/2303678166 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c04011a50 con 0x7f1c10106430 2024-03-13T14:36:59.252 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.257+0000 7f1c17e00700 1 -- 172.21.15.176:0/2303678166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 msgr2=0x7f1c10106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.257+0000 7f1c17e00700 1 --2- 172.21.15.176:0/2303678166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c10106850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c040098a0 tx=0x7f1c04009bb0 comp rx=0 tx=0).stop 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.257+0000 7f1c17e00700 1 -- 172.21.15.176:0/2303678166 shutdown_connections 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.257+0000 7f1c17e00700 1 --2- 172.21.15.176:0/2303678166 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c10106850 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.257+0000 7f1c17e00700 1 -- 172.21.15.176:0/2303678166 >> 172.21.15.176:0/2303678166 conn(0x7f1c101019b0 msgr2=0x7f1c10103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.257+0000 7f1c17e00700 1 -- 172.21.15.176:0/2303678166 shutdown_connections 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.257+0000 7f1c17e00700 1 -- 172.21.15.176:0/2303678166 wait complete. 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.258+0000 7f1c17e00700 1 Processor -- start 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.258+0000 7f1c17e00700 1 -- start start 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.259+0000 7f1c17e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c1019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.259+0000 7f1c17e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1019c880 con 0x7f1c10106430 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.259+0000 7f1c15400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c1019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.259+0000 7f1c15400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c1019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53418/0 (socket says 172.21.15.176:53418) 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.259+0000 7f1c15400700 1 -- 172.21.15.176:0/3312353299 learned_addr learned my addr 172.21.15.176:0/3312353299 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.259+0000 7f1c15400700 1 -- 172.21.15.176:0/3312353299 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c04009550 con 0x7f1c10106430 2024-03-13T14:36:59.253 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.259+0000 7f1c15400700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c1019c340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009520 tx=0x7f1c04011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.260+0000 7f1c0e000700 1 -- 172.21.15.176:0/3312353299 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c04003a80 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.260+0000 7f1c0e000700 1 -- 172.21.15.176:0/3312353299 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c04003be0 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.260+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1019ca80 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.260+0000 7f1c0e000700 1 -- 172.21.15.176:0/3312353299 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c040197b0 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.260+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1019cf20 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.261+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c100623c0 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.261+0000 7f1c0e000700 1 -- 172.21.15.176:0/3312353299 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1c0401a430 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.261+0000 7f1c0e000700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 0x7f1bf803a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.261+0000 7f1c0e000700 1 -- 172.21.15.176:0/3312353299 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1c0404c540 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.262+0000 7f1c14a00700 1 -- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 msgr2=0x7f1bf803a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/1711793734 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.262+0000 7f1c14a00700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 0x7f1bf803a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.266+0000 7f1c0e000700 1 -- 172.21.15.176:0/3312353299 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1c0401ac30 con 0x7f1c10106430 2024-03-13T14:36:59.254 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.463+0000 7f1c14a00700 1 -- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 msgr2=0x7f1bf803a790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/1711793734 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.463+0000 7f1c14a00700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 0x7f1bf803a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.536+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f1c1019fd00 con 0x7f1c10106430 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.537+0000 7f1c0e000700 1 -- 172.21.15.176:0/3312353299 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f1c040172b0 con 0x7f1c10106430 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 msgr2=0x7f1bf803a790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 0x7f1bf803a790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 msgr2=0x7f1c1019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c1019c340 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c04009520 tx=0x7f1c04011ab0 comp rx=0 tx=0).stop 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 shutdown_connections 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f1bf80382d0 0x7f1bf803a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 --2- 172.21.15.176:0/3312353299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1c10106430 0x7f1c1019c340 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.540+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 >> 172.21.15.176:0/3312353299 conn(0x7f1c101019b0 msgr2=0x7f1c10103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.541+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 shutdown_connections 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:58.541+0000 7f1c17e00700 1 -- 172.21.15.176:0/3312353299 wait complete. 2024-03-13T14:36:59.255 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for the mgr to restart... 2024-03-13T14:36:59.256 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for mgr epoch 5... 2024-03-13T14:37:05.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: Active manager daemon smithi176.tzfdwd restarted 2024-03-13T14:37:05.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: Activating manager daemon smithi176.tzfdwd 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: osdmap e2: 0 total, 0 up, 0 in 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: mgrmap e6: smithi176.tzfdwd(active, starting, since 0.00955987s) 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:05.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:05.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:37:05.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:05 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:37:07.248 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:06 smithi176 ceph-mon[29064]: Found migration_current of "None". Setting to last migration. 2024-03-13T14:37:07.248 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:06 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:07.248 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:06 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:07.248 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:06 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:07.248 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:06 smithi176 ceph-mon[29064]: mgrmap e7: smithi176.tzfdwd(active, since 1.01586s) 2024-03-13T14:37:07.547 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:37:07.547 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-13T14:37:07.547 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-13T14:37:07.547 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.828+0000 7f0bb1400700 1 Processor -- start 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.829+0000 7f0bb1400700 1 -- start start 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.829+0000 7f0bb1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.829+0000 7f0bb1400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac0797d0 con 0x7f0bac07ada0 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.829+0000 7f0babe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.829+0000 7f0babe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53420/0 (socket says 172.21.15.176:53420) 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.829+0000 7f0babe00700 1 -- 172.21.15.176:0/3619683156 learned_addr learned my addr 172.21.15.176:0/3619683156 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.830+0000 7f0babe00700 1 -- 172.21.15.176:0/3619683156 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac079910 con 0x7f0bac07ada0 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.830+0000 7f0babe00700 1 --2- 172.21.15.176:0/3619683156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac079200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0098a0 tx=0x7f0b9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bb1bd05b537ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.831+0000 7f0baaa00700 1 -- 172.21.15.176:0/3619683156 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b9c004180 con 0x7f0bac07ada0 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.831+0000 7f0baaa00700 1 -- 172.21.15.176:0/3619683156 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b9c0042e0 con 0x7f0bac07ada0 2024-03-13T14:37:07.554 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.831+0000 7f0baaa00700 1 -- 172.21.15.176:0/3619683156 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b9c011a50 con 0x7f0bac07ada0 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.831+0000 7f0bb1400700 1 -- 172.21.15.176:0/3619683156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 msgr2=0x7f0bac079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.831+0000 7f0bb1400700 1 --2- 172.21.15.176:0/3619683156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac079200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0098a0 tx=0x7f0b9c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.832+0000 7f0bb1400700 1 -- 172.21.15.176:0/3619683156 shutdown_connections 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.832+0000 7f0bb1400700 1 --2- 172.21.15.176:0/3619683156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac079200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.832+0000 7f0bb1400700 1 -- 172.21.15.176:0/3619683156 >> 172.21.15.176:0/3619683156 conn(0x7f0bac101800 msgr2=0x7f0bac103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.832+0000 7f0bb1400700 1 -- 172.21.15.176:0/3619683156 shutdown_connections 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.832+0000 7f0bb1400700 1 -- 172.21.15.176:0/3619683156 wait complete. 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.833+0000 7f0bb1400700 1 Processor -- start 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.833+0000 7f0bb1400700 1 -- start start 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.833+0000 7f0bb1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.833+0000 7f0bb1400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac19c6f0 con 0x7f0bac07ada0 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.833+0000 7f0babe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.833+0000 7f0babe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac19c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53434/0 (socket says 172.21.15.176:53434) 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.833+0000 7f0babe00700 1 -- 172.21.15.176:0/2193989932 learned_addr learned my addr 172.21.15.176:0/2193989932 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:07.555 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.834+0000 7f0babe00700 1 -- 172.21.15.176:0/2193989932 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b9c009550 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.834+0000 7f0babe00700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac19c1b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c003de0 tx=0x7f0b9c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.834+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b9c003930 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.834+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0b9c003a90 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.834+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b9c0196c0 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.834+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac19c8f0 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.835+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac19cd90 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.836+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 5) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f0b9c01abc0 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.836+0000 7f0ba8c00700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.836+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 --> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b9003af40 con 0x7f0b90038370 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.836+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0b9c04c040 con 0x7f0bac07ada0 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.837+0000 7f0bab400700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 msgr2=0x7f0b9003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/1711793734 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:36:59.837+0000 7f0bab400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:00.038+0000 7f0bab400700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 msgr2=0x7f0b9003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/1711793734 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:00.038+0000 7f0bab400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T14:37:07.556 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:00.439+0000 7f0bab400700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 msgr2=0x7f0b9003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/1711793734 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:00.439+0000 7f0bab400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:01.240+0000 7f0bab400700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 msgr2=0x7f0b9003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/1711793734 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:01.240+0000 7f0bab400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:02.842+0000 7f0bab400700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 msgr2=0x7f0b9003a830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/1711793734 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:02.842+0000 7f0bab400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:05.496+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mgrmap(e 6) v1 ==== 45050+0+0 (secure 0 0 0) 0x7f0b9c01a610 con 0x7f0bac07ada0 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:05.497+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 msgr2=0x7f0b9003a830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:05.497+0000 7f0ba8c00700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.500+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 7) v1 ==== 45177+0+0 (secure 0 0 0) 0x7f0b9c016d40 con 0x7f0bac07ada0 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.501+0000 7f0ba8c00700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f0b9003bc50 0x7f0b9003e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.501+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b9003af40 con 0x7f0b9003bc50 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.502+0000 7f0bab400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f0b9003bc50 0x7f0b9003e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:07.557 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.506+0000 7f0bab400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f0b9003bc50 0x7f0b9003e040 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b98003a10 tx=0x7f0b980092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.508+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f0b9003af40 con 0x7f0b9003bc50 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.519+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0bac19d4f0 con 0x7f0b9003bc50 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.519+0000 7f0ba8c00700 1 -- 172.21.15.176:0/2193989932 <== mgr.14120 v2:172.21.15.176:6800/922386027 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0bac19d4f0 con 0x7f0b9003bc50 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f0b9003bc50 msgr2=0x7f0b9003e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f0b9003bc50 0x7f0b9003e040 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0b98003a10 tx=0x7f0b980092b0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 msgr2=0x7f0bac19c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac19c1b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c003de0 tx=0x7f0b9c011ab0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 shutdown_connections 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f0b9003bc50 0x7f0b9003e040 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bac07ada0 0x7f0bac19c1b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 --2- 172.21.15.176:0/2193989932 >> [v2:172.21.15.176:6800/1711793734,v1:172.21.15.176:6801/1711793734] conn(0x7f0b90038370 0x7f0b9003a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:07.558 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.520+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 >> 172.21.15.176:0/2193989932 conn(0x7f0bac101800 msgr2=0x7f0bac102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:07.559 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.521+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 shutdown_connections 2024-03-13T14:37:07.559 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:06.521+0000 7f0bb1400700 1 -- 172.21.15.176:0/2193989932 wait complete. 2024-03-13T14:37:07.559 INFO:teuthology.orchestra.run.smithi176.stdout:mgr epoch 5 is available 2024-03-13T14:37:07.559 INFO:teuthology.orchestra.run.smithi176.stdout:Setting orchestrator backend to cephadm... 2024-03-13T14:37:08.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:07 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:06] ENGINE Bus STARTING 2024-03-13T14:37:08.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:07 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:06] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:37:08.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:07 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:06] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:37:08.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:07 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:06] ENGINE Bus STARTED 2024-03-13T14:37:08.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:07 smithi176 ceph-mon[29064]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-13T14:37:08.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:07 smithi176 ceph-mon[29064]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-13T14:37:09.087 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:08 smithi176 ceph-mon[29064]: mgrmap e8: smithi176.tzfdwd(active, since 2s) 2024-03-13T14:37:09.087 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:08 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:09.087 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:08 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:09.102 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.130+0000 7fc6cb200700 1 Processor -- start 2024-03-13T14:37:09.102 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.130+0000 7fc6cb200700 1 -- start start 2024-03-13T14:37:09.102 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.131+0000 7fc6cb200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c4108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:09.103 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.131+0000 7fc6cb200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4108e60 con 0x7fc6c4108470 2024-03-13T14:37:09.103 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.131+0000 7fc6c9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c4108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:09.103 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.131+0000 7fc6c9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c4108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54550/0 (socket says 172.21.15.176:54550) 2024-03-13T14:37:09.103 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.131+0000 7fc6c9e00700 1 -- 172.21.15.176:0/3831614895 learned_addr learned my addr 172.21.15.176:0/3831614895 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:09.104 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.131+0000 7fc6c9e00700 1 -- 172.21.15.176:0/3831614895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4109670 con 0x7fc6c4108470 2024-03-13T14:37:09.104 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.132+0000 7fc6c9e00700 1 --2- 172.21.15.176:0/3831614895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c4108890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80098a0 tx=0x7fc6b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b2110bb2349242 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:09.104 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.132+0000 7fc6c8a00700 1 -- 172.21.15.176:0/3831614895 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b8004180 con 0x7fc6c4108470 2024-03-13T14:37:09.104 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.132+0000 7fc6c8a00700 1 -- 172.21.15.176:0/3831614895 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6b80042e0 con 0x7fc6c4108470 2024-03-13T14:37:09.108 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.132+0000 7fc6c8a00700 1 -- 172.21.15.176:0/3831614895 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b8011a50 con 0x7fc6c4108470 2024-03-13T14:37:09.108 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.133+0000 7fc6cb200700 1 -- 172.21.15.176:0/3831614895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 msgr2=0x7fc6c4108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:09.108 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.133+0000 7fc6cb200700 1 --2- 172.21.15.176:0/3831614895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c4108890 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80098a0 tx=0x7fc6b8009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:09.108 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.133+0000 7fc6cb200700 1 -- 172.21.15.176:0/3831614895 shutdown_connections 2024-03-13T14:37:09.108 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.133+0000 7fc6cb200700 1 --2- 172.21.15.176:0/3831614895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c4108890 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:09.108 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.133+0000 7fc6cb200700 1 -- 172.21.15.176:0/3831614895 >> 172.21.15.176:0/3831614895 conn(0x7fc6c41039f0 msgr2=0x7fc6c4105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:09.108 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.133+0000 7fc6cb200700 1 -- 172.21.15.176:0/3831614895 shutdown_connections 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.133+0000 7fc6cb200700 1 -- 172.21.15.176:0/3831614895 wait complete. 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.134+0000 7fc6cb200700 1 Processor -- start 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.134+0000 7fc6cb200700 1 -- start start 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.134+0000 7fc6cb200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.134+0000 7fc6cb200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c419c6d0 con 0x7fc6c4108470 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.135+0000 7fc6c9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.135+0000 7fc6c9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c419c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54556/0 (socket says 172.21.15.176:54556) 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.135+0000 7fc6c9e00700 1 -- 172.21.15.176:0/2521856494 learned_addr learned my addr 172.21.15.176:0/2521856494 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.135+0000 7fc6c9e00700 1 -- 172.21.15.176:0/2521856494 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b8009550 con 0x7fc6c4108470 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.135+0000 7fc6c9e00700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c419c190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8003de0 tx=0x7fc6b8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:09.109 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.135+0000 7fc6bea00700 1 -- 172.21.15.176:0/2521856494 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b8003930 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.136+0000 7fc6bea00700 1 -- 172.21.15.176:0/2521856494 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc6b8003a90 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.136+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c419c8d0 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.136+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c419cd70 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.136+0000 7fc6bea00700 1 -- 172.21.15.176:0/2521856494 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc6b80196c0 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.137+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c40623c0 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.137+0000 7fc6bea00700 1 -- 172.21.15.176:0/2521856494 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fc6b801a430 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.137+0000 7fc6bea00700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fc6ac038370 0x7fc6ac03a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.137+0000 7fc6bea00700 1 -- 172.21.15.176:0/2521856494 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc6b804c380 con 0x7fc6c4108470 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.138+0000 7fc6c9400700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fc6ac038370 0x7fc6ac03a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:09.110 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.138+0000 7fc6c9400700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fc6ac038370 0x7fc6ac03a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4006fd0 tx=0x7fc6b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:09.111 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.142+0000 7fc6bea00700 1 -- 172.21.15.176:0/2521856494 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc6b8010560 con 0x7fc6c4108470 2024-03-13T14:37:09.111 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.371+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6c4105d20 con 0x7fc6ac038370 2024-03-13T14:37:09.111 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.384+0000 7fc6bea00700 1 -- 172.21.15.176:0/2521856494 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc6c4105d20 con 0x7fc6ac038370 2024-03-13T14:37:09.111 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.387+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fc6ac038370 msgr2=0x7fc6ac03a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:09.112 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.387+0000 7fc6cb200700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fc6ac038370 0x7fc6ac03a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4006fd0 tx=0x7fc6b4006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:09.112 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.387+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 msgr2=0x7fc6c419c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:09.112 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.387+0000 7fc6cb200700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c419c190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8003de0 tx=0x7fc6b8011ab0 comp rx=0 tx=0).stop 2024-03-13T14:37:09.113 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.388+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 shutdown_connections 2024-03-13T14:37:09.113 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.388+0000 7fc6cb200700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fc6ac038370 0x7fc6ac03a830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:09.113 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.388+0000 7fc6cb200700 1 --2- 172.21.15.176:0/2521856494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6c4108470 0x7fc6c419c190 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:09.113 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.388+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 >> 172.21.15.176:0/2521856494 conn(0x7fc6c41039f0 msgr2=0x7fc6c41050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:09.113 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.388+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 shutdown_connections 2024-03-13T14:37:09.113 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:08.389+0000 7fc6cb200700 1 -- 172.21.15.176:0/2521856494 wait complete. 2024-03-13T14:37:10.320 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:09 smithi176 ceph-mon[29064]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:10.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-13T14:37:10.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.731+0000 7f4a53c00700 1 Processor -- start 2024-03-13T14:37:10.652 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.732+0000 7f4a53c00700 1 -- start start 2024-03-13T14:37:10.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.732+0000 7f4a53c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:10.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.732+0000 7f4a53c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c109070 con 0x7f4a4c108680 2024-03-13T14:37:10.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.732+0000 7f4a51200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:10.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.733+0000 7f4a51200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54560/0 (socket says 172.21.15.176:54560) 2024-03-13T14:37:10.653 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.733+0000 7f4a51200700 1 -- 172.21.15.176:0/2087968929 learned_addr learned my addr 172.21.15.176:0/2087968929 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:10.654 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.733+0000 7f4a51200700 1 -- 172.21.15.176:0/2087968929 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c109830 con 0x7f4a4c108680 2024-03-13T14:37:10.654 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.733+0000 7f4a51200700 1 --2- 172.21.15.176:0/2087968929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c108aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0098a0 tx=0x7f4a3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221f6226ec4cb8ed server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:10.654 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.733+0000 7f4a4be00700 1 -- 172.21.15.176:0/2087968929 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a3c004180 con 0x7f4a4c108680 2024-03-13T14:37:10.654 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.734+0000 7f4a4be00700 1 -- 172.21.15.176:0/2087968929 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a3c0042e0 con 0x7f4a4c108680 2024-03-13T14:37:10.655 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.734+0000 7f4a4be00700 1 -- 172.21.15.176:0/2087968929 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a3c011a50 con 0x7f4a4c108680 2024-03-13T14:37:10.655 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.734+0000 7f4a53c00700 1 -- 172.21.15.176:0/2087968929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 msgr2=0x7f4a4c108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:10.655 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.734+0000 7f4a53c00700 1 --2- 172.21.15.176:0/2087968929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c108aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0098a0 tx=0x7f4a3c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:10.657 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.734+0000 7f4a53c00700 1 -- 172.21.15.176:0/2087968929 shutdown_connections 2024-03-13T14:37:10.657 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.734+0000 7f4a53c00700 1 --2- 172.21.15.176:0/2087968929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c108aa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:10.657 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.734+0000 7f4a53c00700 1 -- 172.21.15.176:0/2087968929 >> 172.21.15.176:0/2087968929 conn(0x7f4a4c07bb20 msgr2=0x7f4a4c1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:10.657 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.735+0000 7f4a53c00700 1 -- 172.21.15.176:0/2087968929 shutdown_connections 2024-03-13T14:37:10.657 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.735+0000 7f4a53c00700 1 -- 172.21.15.176:0/2087968929 wait complete. 2024-03-13T14:37:10.657 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.736+0000 7f4a53c00700 1 Processor -- start 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.736+0000 7f4a53c00700 1 -- start start 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.736+0000 7f4a53c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.736+0000 7f4a53c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19c890 con 0x7f4a4c108680 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.736+0000 7f4a51200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.736+0000 7f4a51200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c19c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54562/0 (socket says 172.21.15.176:54562) 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.736+0000 7f4a51200700 1 -- 172.21.15.176:0/1952432594 learned_addr learned my addr 172.21.15.176:0/1952432594 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.737+0000 7f4a51200700 1 -- 172.21.15.176:0/1952432594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c009550 con 0x7f4a4c108680 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.737+0000 7f4a51200700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c19c350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c003de0 tx=0x7f4a3c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.737+0000 7f4a4a000700 1 -- 172.21.15.176:0/1952432594 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a3c003930 con 0x7f4a4c108680 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.737+0000 7f4a4a000700 1 -- 172.21.15.176:0/1952432594 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4a3c003a90 con 0x7f4a4c108680 2024-03-13T14:37:10.658 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.737+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c19ca90 con 0x7f4a4c108680 2024-03-13T14:37:10.661 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.737+0000 7f4a4a000700 1 -- 172.21.15.176:0/1952432594 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4a3c0196c0 con 0x7f4a4c108680 2024-03-13T14:37:10.661 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.737+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c19cf30 con 0x7f4a4c108680 2024-03-13T14:37:10.661 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.739+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c196450 con 0x7f4a4c108680 2024-03-13T14:37:10.661 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.739+0000 7f4a4a000700 1 -- 172.21.15.176:0/1952432594 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f4a3c01abc0 con 0x7f4a4c108680 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.739+0000 7f4a4a000700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f4a30038370 0x7f4a3003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.739+0000 7f4a4a000700 1 -- 172.21.15.176:0/1952432594 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4a3c04c470 con 0x7f4a4c108680 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.740+0000 7f4a43e00700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f4a30038370 0x7f4a3003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.740+0000 7f4a43e00700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f4a30038370 0x7f4a3003a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a34006fd0 tx=0x7f4a34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.745+0000 7f4a4a000700 1 -- 172.21.15.176:0/1952432594 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f4a3c010790 con 0x7f4a4c108680 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.959+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a4c0611d0 con 0x7f4a30038370 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.960+0000 7f4a4a000700 1 -- 172.21.15.176:0/1952432594 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f4a4c0611d0 con 0x7f4a30038370 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.964+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f4a30038370 msgr2=0x7f4a3003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.964+0000 7f4a53c00700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f4a30038370 0x7f4a3003a830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a34006fd0 tx=0x7f4a34006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.964+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 msgr2=0x7f4a4c19c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.964+0000 7f4a53c00700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c19c350 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c003de0 tx=0x7f4a3c011ab0 comp rx=0 tx=0).stop 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.965+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 shutdown_connections 2024-03-13T14:37:10.662 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.965+0000 7f4a53c00700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f4a30038370 0x7f4a3003a830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:10.663 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.965+0000 7f4a53c00700 1 --2- 172.21.15.176:0/1952432594 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4a4c108680 0x7f4a4c19c350 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:10.663 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.965+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 >> 172.21.15.176:0/1952432594 conn(0x7f4a4c07bb20 msgr2=0x7f4a4c106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:10.663 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.965+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 shutdown_connections 2024-03-13T14:37:10.663 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:09.965+0000 7f4a53c00700 1 -- 172.21.15.176:0/1952432594 wait complete. 2024-03-13T14:37:10.663 INFO:teuthology.orchestra.run.smithi176.stdout:Generating ssh key... 2024-03-13T14:37:11.232 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:10 smithi176 ceph-mon[29064]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:12.318 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.238+0000 7f5931600700 1 Processor -- start 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.239+0000 7f5931600700 1 -- start start 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.239+0000 7f5931600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.239+0000 7f5931600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c106e10 con 0x7f592c106420 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.239+0000 7f592aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.239+0000 7f592aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54570/0 (socket says 172.21.15.176:54570) 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.239+0000 7f592aa00700 1 -- 172.21.15.176:0/3756717009 learned_addr learned my addr 172.21.15.176:0/3756717009 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.240+0000 7f592aa00700 1 -- 172.21.15.176:0/3756717009 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c107620 con 0x7f592c106420 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.240+0000 7f592aa00700 1 --2- 172.21.15.176:0/3756717009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c106840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f591c0098a0 tx=0x7f591c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bfe56ed9e180b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.240+0000 7f592a000700 1 -- 172.21.15.176:0/3756717009 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f591c004180 con 0x7f592c106420 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.240+0000 7f592a000700 1 -- 172.21.15.176:0/3756717009 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f591c0042e0 con 0x7f592c106420 2024-03-13T14:37:12.319 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.240+0000 7f592a000700 1 -- 172.21.15.176:0/3756717009 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f591c011a50 con 0x7f592c106420 2024-03-13T14:37:12.320 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.241+0000 7f5931600700 1 -- 172.21.15.176:0/3756717009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 msgr2=0x7f592c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:12.320 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.241+0000 7f5931600700 1 --2- 172.21.15.176:0/3756717009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c106840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f591c0098a0 tx=0x7f591c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.241+0000 7f5931600700 1 -- 172.21.15.176:0/3756717009 shutdown_connections 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.241+0000 7f5931600700 1 --2- 172.21.15.176:0/3756717009 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c106840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.241+0000 7f5931600700 1 -- 172.21.15.176:0/3756717009 >> 172.21.15.176:0/3756717009 conn(0x7f592c101a00 msgr2=0x7f592c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.241+0000 7f5931600700 1 -- 172.21.15.176:0/3756717009 shutdown_connections 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.241+0000 7f5931600700 1 -- 172.21.15.176:0/3756717009 wait complete. 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.242+0000 7f5931600700 1 Processor -- start 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.243+0000 7f5931600700 1 -- start start 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.243+0000 7f5931600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c19c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.243+0000 7f5931600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c19c7e0 con 0x7f592c106420 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.243+0000 7f592aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c19c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:12.324 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.243+0000 7f592aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c19c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54582/0 (socket says 172.21.15.176:54582) 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.243+0000 7f592aa00700 1 -- 172.21.15.176:0/3536748179 learned_addr learned my addr 172.21.15.176:0/3536748179 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.243+0000 7f592aa00700 1 -- 172.21.15.176:0/3536748179 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c009550 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.244+0000 7f592aa00700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c19c2a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f591c000c00 tx=0x7f591c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.244+0000 7f5923e00700 1 -- 172.21.15.176:0/3536748179 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f591c003b90 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.244+0000 7f5923e00700 1 -- 172.21.15.176:0/3536748179 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f591c0248b0 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.244+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c19c9e0 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.244+0000 7f5923e00700 1 -- 172.21.15.176:0/3536748179 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f591c01a8e0 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.244+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c19ce80 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.245+0000 7f5923e00700 1 -- 172.21.15.176:0/3536748179 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f591c024440 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.245+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c0623c0 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.246+0000 7f5923e00700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5910038370 0x7f591003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.246+0000 7f5923e00700 1 -- 172.21.15.176:0/3536748179 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f591c04c3b0 con 0x7f592c106420 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.246+0000 7f5922000700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5910038370 0x7f591003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.247+0000 7f5922000700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5910038370 0x7f591003a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5914006fd0 tx=0x7f5914006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:12.325 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.251+0000 7f5923e00700 1 -- 172.21.15.176:0/3536748179 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f591c018b40 con 0x7f592c106420 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.465+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f592c10a820 con 0x7f5910038370 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.697+0000 7f5923e00700 1 -- 172.21.15.176:0/3536748179 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f592c10a820 con 0x7f5910038370 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5910038370 msgr2=0x7f591003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5910038370 0x7f591003a830 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5914006fd0 tx=0x7f5914006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 msgr2=0x7f592c19c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c19c2a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f591c000c00 tx=0x7f591c011ab0 comp rx=0 tx=0).stop 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 shutdown_connections 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5910038370 0x7f591003a830 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 --2- 172.21.15.176:0/3536748179 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f592c106420 0x7f592c19c2a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 >> 172.21.15.176:0/3536748179 conn(0x7f592c101a00 msgr2=0x7f592c1027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 shutdown_connections 2024-03-13T14:37:12.326 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:11.700+0000 7f5931600700 1 -- 172.21.15.176:0/3536748179 wait complete. 2024-03-13T14:37:12.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:12 smithi176 ceph-mon[29064]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:12.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:12 smithi176 ceph-mon[29064]: Generating ssh key... 2024-03-13T14:37:12.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:12 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:12.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:12 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM6w9SmY/uZm/MhqTx30LsgAeaBlMz17hnsFOtvVM+8wiBdavRsbpozSMsAzY1gWiosu5BloSatJVSkxRA7Dwj/cc6YjyA55geevrCwz4QtCHImkxwsl81i8RPO+en9Y88RYeyzN3sZTqi07zdZnsuUTAIy6YgKotBWFTPzwMYGYEmlQmKior3ClgVYrXlU5g/DjCWNTahxEgET4Go2Vh62VMfrernf6tLz2RNmamGmOUeUyReyDrWkeoZT2MhN6IY+J9nbV26HIWl0e3pOeksHmS8dQaki9zaEbDeac5V4thPnik7uV4y/O5xV5YlIdH9Rxc1qU9uRHxoY4EVwUNNkEtkepsX73uYDIO+OvEzMwl0CV2ag94sCGa8MJct50KImvaQaT/aukWfDE/L1TiDSMRSRL+6y2budd21Tl2R8lObMq6tY7UJ4yIctkKjHsO1+rWH/pYH4g+yAETvwX0ilezVCf3PiLHu/pV+chiJ6OCtU+d+j3K5qntXwlg1oH8= ceph-f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.925+0000 7f1de0200700 1 Processor -- start 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.925+0000 7f1de0200700 1 -- start start 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.926+0000 7f1de0200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd8108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.926+0000 7f1de0200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8109070 con 0x7f1dd8108680 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.926+0000 7f1ddd800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd8108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.926+0000 7f1ddd800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd8108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45374/0 (socket says 172.21.15.176:45374) 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.926+0000 7f1ddd800700 1 -- 172.21.15.176:0/1630225466 learned_addr learned my addr 172.21.15.176:0/1630225466 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:13.827 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.927+0000 7f1ddd800700 1 -- 172.21.15.176:0/1630225466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd8109830 con 0x7f1dd8108680 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.927+0000 7f1ddd800700 1 --2- 172.21.15.176:0/1630225466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd8108aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0098a0 tx=0x7f1dcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b7bff7565b3b34 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.927+0000 7f1dd7e00700 1 -- 172.21.15.176:0/1630225466 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dcc004180 con 0x7f1dd8108680 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.927+0000 7f1dd7e00700 1 -- 172.21.15.176:0/1630225466 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dcc0042e0 con 0x7f1dd8108680 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.927+0000 7f1dd7e00700 1 -- 172.21.15.176:0/1630225466 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dcc011a50 con 0x7f1dd8108680 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.928+0000 7f1de0200700 1 -- 172.21.15.176:0/1630225466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 msgr2=0x7f1dd8108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.928+0000 7f1de0200700 1 --2- 172.21.15.176:0/1630225466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd8108aa0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc0098a0 tx=0x7f1dcc009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.928+0000 7f1de0200700 1 -- 172.21.15.176:0/1630225466 shutdown_connections 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.928+0000 7f1de0200700 1 --2- 172.21.15.176:0/1630225466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd8108aa0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.928+0000 7f1de0200700 1 -- 172.21.15.176:0/1630225466 >> 172.21.15.176:0/1630225466 conn(0x7f1dd807bb20 msgr2=0x7f1dd81064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.928+0000 7f1de0200700 1 -- 172.21.15.176:0/1630225466 shutdown_connections 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.928+0000 7f1de0200700 1 -- 172.21.15.176:0/1630225466 wait complete. 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.929+0000 7f1de0200700 1 Processor -- start 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.929+0000 7f1de0200700 1 -- start start 2024-03-13T14:37:13.828 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.929+0000 7f1de0200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.929+0000 7f1de0200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd819c890 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.930+0000 7f1ddd800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.930+0000 7f1ddd800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45386/0 (socket says 172.21.15.176:45386) 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.930+0000 7f1ddd800700 1 -- 172.21.15.176:0/329420455 learned_addr learned my addr 172.21.15.176:0/329420455 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.930+0000 7f1ddd800700 1 -- 172.21.15.176:0/329420455 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc009550 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.930+0000 7f1ddd800700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd819c350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc003de0 tx=0x7f1dcc011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.931+0000 7f1dd6000700 1 -- 172.21.15.176:0/329420455 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dcc003930 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.931+0000 7f1dd6000700 1 -- 172.21.15.176:0/329420455 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1dcc003a90 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.931+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd819ca90 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.931+0000 7f1dd6000700 1 -- 172.21.15.176:0/329420455 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1dcc0196c0 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.931+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd819cf30 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.932+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd8196450 con 0x7f1dd8108680 2024-03-13T14:37:13.829 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.932+0000 7f1dd6000700 1 -- 172.21.15.176:0/329420455 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f1dcc01abc0 con 0x7f1dd8108680 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.933+0000 7f1dd6000700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f1dc0038320 0x7f1dc003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.933+0000 7f1dd6000700 1 -- 172.21.15.176:0/329420455 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1dcc04c4d0 con 0x7f1dd8108680 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.933+0000 7f1ddce00700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f1dc0038320 0x7f1dc003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.934+0000 7f1ddce00700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f1dc0038320 0x7f1dc003a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4006fd0 tx=0x7f1dc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:12.938+0000 7f1dd6000700 1 -- 172.21.15.176:0/329420455 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1dcc010790 con 0x7f1dd8108680 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.155+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dd80611d0 con 0x7f1dc0038320 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.157+0000 7f1dd6000700 1 -- 172.21.15.176:0/329420455 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f1dd80611d0 con 0x7f1dc0038320 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.160+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f1dc0038320 msgr2=0x7f1dc003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.160+0000 7f1de0200700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f1dc0038320 0x7f1dc003a7e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4006fd0 tx=0x7f1dc4006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.160+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 msgr2=0x7f1dd819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.160+0000 7f1de0200700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd819c350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc003de0 tx=0x7f1dcc011ab0 comp rx=0 tx=0).stop 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.160+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 shutdown_connections 2024-03-13T14:37:13.830 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.161+0000 7f1de0200700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f1dc0038320 0x7f1dc003a7e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:13.831 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.161+0000 7f1de0200700 1 --2- 172.21.15.176:0/329420455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1dd8108680 0x7f1dd819c350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:13.831 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.161+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 >> 172.21.15.176:0/329420455 conn(0x7f1dd807bb20 msgr2=0x7f1dd8106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:13.831 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.161+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 shutdown_connections 2024-03-13T14:37:13.831 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:13.161+0000 7f1de0200700 1 -- 172.21.15.176:0/329420455 wait complete. 2024-03-13T14:37:13.831 INFO:teuthology.orchestra.run.smithi176.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-13T14:37:13.831 INFO:teuthology.orchestra.run.smithi176.stdout:Adding key to root@localhost authorized_keys... 2024-03-13T14:37:13.831 INFO:teuthology.orchestra.run.smithi176.stdout:Adding host smithi176... 2024-03-13T14:37:14.349 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:14 smithi176 ceph-mon[29064]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:16.000 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:15 smithi176 ceph-mon[29064]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi176", "addr": "172.21.15.176", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:16.989 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:16 smithi176 ceph-mon[29064]: Deploying cephadm binary to smithi176 2024-03-13T14:37:19.833 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Added host 'smithi176' with addr '172.21.15.176' 2024-03-13T14:37:19.833 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.420+0000 7f7665e00700 1 Processor -- start 2024-03-13T14:37:19.833 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.420+0000 7f7665e00700 1 -- start start 2024-03-13T14:37:19.833 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.421+0000 7f7665e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f766010e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:19.834 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.421+0000 7f7665e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f766010e9c0 con 0x7f766010dfd0 2024-03-13T14:37:19.834 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.421+0000 7f765ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f766010e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:19.840 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.421+0000 7f765ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f766010e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45392/0 (socket says 172.21.15.176:45392) 2024-03-13T14:37:19.840 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.421+0000 7f765ea00700 1 -- 172.21.15.176:0/2968605603 learned_addr learned my addr 172.21.15.176:0/2968605603 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:19.840 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.421+0000 7f765ea00700 1 -- 172.21.15.176:0/2968605603 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766010cbe0 con 0x7f766010dfd0 2024-03-13T14:37:19.840 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.422+0000 7f765ea00700 1 --2- 172.21.15.176:0/2968605603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f766010e3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76500098a0 tx=0x7f7650009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec79968713305cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:19.840 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.422+0000 7f765d600700 1 -- 172.21.15.176:0/2968605603 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7650004180 con 0x7f766010dfd0 2024-03-13T14:37:19.840 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.422+0000 7f765d600700 1 -- 172.21.15.176:0/2968605603 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76500042e0 con 0x7f766010dfd0 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.422+0000 7f765d600700 1 -- 172.21.15.176:0/2968605603 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7650011a50 con 0x7f766010dfd0 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.423+0000 7f7665e00700 1 -- 172.21.15.176:0/2968605603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 msgr2=0x7f766010e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.423+0000 7f7665e00700 1 --2- 172.21.15.176:0/2968605603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f766010e3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76500098a0 tx=0x7f7650009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.423+0000 7f7665e00700 1 -- 172.21.15.176:0/2968605603 shutdown_connections 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.423+0000 7f7665e00700 1 --2- 172.21.15.176:0/2968605603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f766010e3f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.423+0000 7f7665e00700 1 -- 172.21.15.176:0/2968605603 >> 172.21.15.176:0/2968605603 conn(0x7f766007bb20 msgr2=0x7f766007df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.423+0000 7f7665e00700 1 -- 172.21.15.176:0/2968605603 shutdown_connections 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.423+0000 7f7665e00700 1 -- 172.21.15.176:0/2968605603 wait complete. 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.424+0000 7f7665e00700 1 Processor -- start 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.424+0000 7f7665e00700 1 -- start start 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.424+0000 7f7665e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f76601a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.424+0000 7f7665e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76601a0cd0 con 0x7f766010dfd0 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.425+0000 7f765ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f76601a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:19.841 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.425+0000 7f765ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f76601a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45402/0 (socket says 172.21.15.176:45402) 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.425+0000 7f765ea00700 1 -- 172.21.15.176:0/1746199175 learned_addr learned my addr 172.21.15.176:0/1746199175 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.425+0000 7f765ea00700 1 -- 172.21.15.176:0/1746199175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7650009550 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.425+0000 7f765ea00700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f76601a0790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7650000c00 tx=0x7f7650011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.426+0000 7f7664a00700 1 -- 172.21.15.176:0/1746199175 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7650003b90 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.426+0000 7f7664a00700 1 -- 172.21.15.176:0/1746199175 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f76500248b0 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.426+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76601a0ed0 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.426+0000 7f7664a00700 1 -- 172.21.15.176:0/1746199175 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f765001a8e0 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.426+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76601a1370 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.427+0000 7f7664a00700 1 -- 172.21.15.176:0/1746199175 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f7650024a20 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.427+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76600623c0 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.427+0000 7f7664a00700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f7644038370 0x7f764403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.427+0000 7f7664a00700 1 -- 172.21.15.176:0/1746199175 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f765004c410 con 0x7f766010dfd0 2024-03-13T14:37:19.842 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.428+0000 7f765e000700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f7644038370 0x7f764403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.428+0000 7f765e000700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f7644038370 0x7f764403a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f764c006fd0 tx=0x7f764c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.432+0000 7f7664a00700 1 -- 172.21.15.176:0/1746199175 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7650018b80 con 0x7f766010dfd0 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:14.652+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi176", "addr": "172.21.15.176", "target": ["mon-mgr", ""]}) v1 -- 0x7f7660110df0 con 0x7f7644038370 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.181+0000 7f7664a00700 1 -- 172.21.15.176:0/1746199175 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7660110df0 con 0x7f7644038370 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f7644038370 msgr2=0x7f764403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f7644038370 0x7f764403a830 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f764c006fd0 tx=0x7f764c006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 msgr2=0x7f76601a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f76601a0790 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7650000c00 tx=0x7f7650011ab0 comp rx=0 tx=0).stop 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 shutdown_connections 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f7644038370 0x7f764403a830 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 --2- 172.21.15.176:0/1746199175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f766010dfd0 0x7f76601a0790 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 >> 172.21.15.176:0/1746199175 conn(0x7f766007bb20 msgr2=0x7f766007d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 shutdown_connections 2024-03-13T14:37:19.843 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:19.185+0000 7f7665e00700 1 -- 172.21.15.176:0/1746199175 wait complete. 2024-03-13T14:37:19.844 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying mon service with default placement... 2024-03-13T14:37:20.390 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:20 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:20.391 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:20 smithi176 ceph-mon[29064]: Added host smithi176 2024-03-13T14:37:20.391 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:20 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:21.347 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-13T14:37:21.348 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.459+0000 7fcf52c00700 1 Processor -- start 2024-03-13T14:37:21.348 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.459+0000 7fcf52c00700 1 -- start start 2024-03-13T14:37:21.348 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.460+0000 7fcf52c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:21.348 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.460+0000 7fcf52c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c0797d0 con 0x7fcf4c07ada0 2024-03-13T14:37:21.348 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.460+0000 7fcf4be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.460+0000 7fcf4be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45412/0 (socket says 172.21.15.176:45412) 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.460+0000 7fcf4be00700 1 -- 172.21.15.176:0/394913550 learned_addr learned my addr 172.21.15.176:0/394913550 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.461+0000 7fcf4be00700 1 -- 172.21.15.176:0/394913550 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c079910 con 0x7fcf4c07ada0 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.461+0000 7fcf4be00700 1 --2- 172.21.15.176:0/394913550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c079200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0098a0 tx=0x7fcf3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b1f82f95756a95 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf4aa00700 1 -- 172.21.15.176:0/394913550 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf3c004180 con 0x7fcf4c07ada0 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf4aa00700 1 -- 172.21.15.176:0/394913550 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf3c0042e0 con 0x7fcf4c07ada0 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf4aa00700 1 -- 172.21.15.176:0/394913550 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf3c011a50 con 0x7fcf4c07ada0 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf52c00700 1 -- 172.21.15.176:0/394913550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 msgr2=0x7fcf4c079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf52c00700 1 --2- 172.21.15.176:0/394913550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c079200 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0098a0 tx=0x7fcf3c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:21.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf52c00700 1 -- 172.21.15.176:0/394913550 shutdown_connections 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf52c00700 1 --2- 172.21.15.176:0/394913550 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c079200 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.462+0000 7fcf52c00700 1 -- 172.21.15.176:0/394913550 >> 172.21.15.176:0/394913550 conn(0x7fcf4c1019b0 msgr2=0x7fcf4c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.463+0000 7fcf52c00700 1 -- 172.21.15.176:0/394913550 shutdown_connections 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.463+0000 7fcf52c00700 1 -- 172.21.15.176:0/394913550 wait complete. 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.464+0000 7fcf52c00700 1 Processor -- start 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.464+0000 7fcf52c00700 1 -- start start 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.464+0000 7fcf52c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.464+0000 7fcf52c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c19c870 con 0x7fcf4c07ada0 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.464+0000 7fcf4be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.464+0000 7fcf4be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c19c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45416/0 (socket says 172.21.15.176:45416) 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.465+0000 7fcf4be00700 1 -- 172.21.15.176:0/2977457023 learned_addr learned my addr 172.21.15.176:0/2977457023 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.465+0000 7fcf4be00700 1 -- 172.21.15.176:0/2977457023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c009550 con 0x7fcf4c07ada0 2024-03-13T14:37:21.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.465+0000 7fcf4be00700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c19c330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c003de0 tx=0x7fcf3c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.465+0000 7fcf48c00700 1 -- 172.21.15.176:0/2977457023 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf3c003930 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.465+0000 7fcf48c00700 1 -- 172.21.15.176:0/2977457023 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcf3c003a90 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.465+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c19ca70 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.465+0000 7fcf48c00700 1 -- 172.21.15.176:0/2977457023 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf3c0196c0 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.466+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c19cf10 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.467+0000 7fcf48c00700 1 -- 172.21.15.176:0/2977457023 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fcf3c01abc0 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.467+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4c10c790 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.467+0000 7fcf48c00700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fcf30038320 0x7fcf3003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.467+0000 7fcf48c00700 1 -- 172.21.15.176:0/2977457023 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcf3c04c410 con 0x7fcf4c07ada0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.468+0000 7fcf4b400700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fcf30038320 0x7fcf3003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.468+0000 7fcf4b400700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fcf30038320 0x7fcf3003a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf38006fd0 tx=0x7fcf38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:21.355 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.473+0000 7fcf48c00700 1 -- 172.21.15.176:0/2977457023 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcf3c010770 con 0x7fcf4c07ada0 2024-03-13T14:37:21.356 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.705+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf4c103d20 con 0x7fcf30038320 2024-03-13T14:37:21.356 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.713+0000 7fcf48c00700 1 -- 172.21.15.176:0/2977457023 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcf4c103d20 con 0x7fcf30038320 2024-03-13T14:37:21.356 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fcf30038320 msgr2=0x7fcf3003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:21.356 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fcf30038320 0x7fcf3003a7e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf38006fd0 tx=0x7fcf38006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:21.356 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 msgr2=0x7fcf4c19c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:21.356 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c19c330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c003de0 tx=0x7fcf3c011ab0 comp rx=0 tx=0).stop 2024-03-13T14:37:21.357 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 shutdown_connections 2024-03-13T14:37:21.357 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fcf30038320 0x7fcf3003a7e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:21.357 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 --2- 172.21.15.176:0/2977457023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcf4c07ada0 0x7fcf4c19c330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:21.357 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 >> 172.21.15.176:0/2977457023 conn(0x7fcf4c1019b0 msgr2=0x7fcf4c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:21.357 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 shutdown_connections 2024-03-13T14:37:21.357 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:20.716+0000 7fcf52c00700 1 -- 172.21.15.176:0/2977457023 wait complete. 2024-03-13T14:37:21.357 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying mgr service with default placement... 2024-03-13T14:37:21.965 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:21 smithi176 ceph-mon[29064]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:21.966 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:21 smithi176 ceph-mon[29064]: Saving service mon spec with placement count:5 2024-03-13T14:37:21.966 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:21 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:23.634 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:23 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:23.634 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:23 smithi176 ceph-mon[29064]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:23.634 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:23 smithi176 ceph-mon[29064]: Saving service mgr spec with placement count:2 2024-03-13T14:37:23.635 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:23 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:23.635 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:23 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:23.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-13T14:37:23.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.736+0000 7f6350200700 1 Processor -- start 2024-03-13T14:37:23.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.736+0000 7f6350200700 1 -- start start 2024-03-13T14:37:23.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.737+0000 7f6350200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348105760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:23.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.737+0000 7f6350200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6348105d30 con 0x7f6348105340 2024-03-13T14:37:23.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.737+0000 7f634d800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348105760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:23.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.737+0000 7f634d800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348105760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48370/0 (socket says 172.21.15.176:48370) 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.737+0000 7f634d800700 1 -- 172.21.15.176:0/1088683718 learned_addr learned my addr 172.21.15.176:0/1088683718 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.738+0000 7f634d800700 1 -- 172.21.15.176:0/1088683718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348106540 con 0x7f6348105340 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.738+0000 7f634d800700 1 --2- 172.21.15.176:0/1088683718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348105760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f633c0098a0 tx=0x7f633c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb2a8ea361859a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.738+0000 7f6347e00700 1 -- 172.21.15.176:0/1088683718 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c004180 con 0x7f6348105340 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.739+0000 7f6347e00700 1 -- 172.21.15.176:0/1088683718 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f633c0042e0 con 0x7f6348105340 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.739+0000 7f6347e00700 1 -- 172.21.15.176:0/1088683718 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c011af0 con 0x7f6348105340 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.739+0000 7f6350200700 1 -- 172.21.15.176:0/1088683718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 msgr2=0x7f6348105760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.739+0000 7f6350200700 1 --2- 172.21.15.176:0/1088683718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348105760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f633c0098a0 tx=0x7f633c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.739+0000 7f6350200700 1 -- 172.21.15.176:0/1088683718 shutdown_connections 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.739+0000 7f6350200700 1 --2- 172.21.15.176:0/1088683718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348105760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:23.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.739+0000 7f6350200700 1 -- 172.21.15.176:0/1088683718 >> 172.21.15.176:0/1088683718 conn(0x7f63481008c0 msgr2=0x7f6348102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.740+0000 7f6350200700 1 -- 172.21.15.176:0/1088683718 shutdown_connections 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.740+0000 7f6350200700 1 -- 172.21.15.176:0/1088683718 wait complete. 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.741+0000 7f6350200700 1 Processor -- start 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.741+0000 7f6350200700 1 -- start start 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.741+0000 7f6350200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.741+0000 7f6350200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63481984b0 con 0x7f6348105340 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.741+0000 7f634d800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.741+0000 7f634d800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48378/0 (socket says 172.21.15.176:48378) 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.741+0000 7f634d800700 1 -- 172.21.15.176:0/2617008276 learned_addr learned my addr 172.21.15.176:0/2617008276 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.742+0000 7f634d800700 1 -- 172.21.15.176:0/2617008276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c009550 con 0x7f6348105340 2024-03-13T14:37:23.756 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.742+0000 7f634d800700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348197f70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f633c011670 tx=0x7f633c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.743+0000 7f6346000700 1 -- 172.21.15.176:0/2617008276 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c0039f0 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.743+0000 7f6346000700 1 -- 172.21.15.176:0/2617008276 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f633c003b50 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.743+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63481986b0 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.743+0000 7f6346000700 1 -- 172.21.15.176:0/2617008276 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f633c0247c0 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.743+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6348198b50 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.744+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6348191bd0 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.744+0000 7f6346000700 1 -- 172.21.15.176:0/2617008276 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f633c02d3f0 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.745+0000 7f6346000700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f633003a580 0x7f633003ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.745+0000 7f6346000700 1 -- 172.21.15.176:0/2617008276 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f633c04c3a0 con 0x7f6348105340 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.745+0000 7f634ce00700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f633003a580 0x7f633003ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.746+0000 7f634ce00700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f633003a580 0x7f633003ca40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6334006fd0 tx=0x7f6334006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:23.757 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.750+0000 7f6346000700 1 -- 172.21.15.176:0/2617008276 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f633c010970 con 0x7f6348105340 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.981+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f63480611d0 con 0x7f633003a580 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.988+0000 7f6346000700 1 -- 172.21.15.176:0/2617008276 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f63480611d0 con 0x7f633003a580 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f633003a580 msgr2=0x7f633003ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f633003a580 0x7f633003ca40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6334006fd0 tx=0x7f6334006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 msgr2=0x7f6348197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348197f70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f633c011670 tx=0x7f633c011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 shutdown_connections 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f633003a580 0x7f633003ca40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 --2- 172.21.15.176:0/2617008276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6348105340 0x7f6348197f70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 >> 172.21.15.176:0/2617008276 conn(0x7f63481008c0 msgr2=0x7f634818ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 shutdown_connections 2024-03-13T14:37:23.758 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:22.991+0000 7f6350200700 1 -- 172.21.15.176:0/2617008276 wait complete. 2024-03-13T14:37:23.759 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying crash service with default placement... 2024-03-13T14:37:25.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:25 smithi176 ceph-mon[29064]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:25.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:25 smithi176 ceph-mon[29064]: Saving service crash spec with placement * 2024-03-13T14:37:25.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:25 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:25.917 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-03-13T14:37:25.917 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.345+0000 7ffb59a00700 1 Processor -- start 2024-03-13T14:37:25.917 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.345+0000 7ffb59a00700 1 -- start start 2024-03-13T14:37:25.917 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.345+0000 7ffb59a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb54106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:25.917 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.345+0000 7ffb59a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54106e20 con 0x7ffb54106430 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.346+0000 7ffb52a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb54106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.346+0000 7ffb52a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb54106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48382/0 (socket says 172.21.15.176:48382) 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.346+0000 7ffb52a00700 1 -- 172.21.15.176:0/486593030 learned_addr learned my addr 172.21.15.176:0/486593030 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.346+0000 7ffb52a00700 1 -- 172.21.15.176:0/486593030 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54107630 con 0x7ffb54106430 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.347+0000 7ffb52a00700 1 --2- 172.21.15.176:0/486593030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb54106850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb440098a0 tx=0x7ffb44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecc691d60e2d90c server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.347+0000 7ffb51600700 1 -- 172.21.15.176:0/486593030 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb44004180 con 0x7ffb54106430 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.347+0000 7ffb51600700 1 -- 172.21.15.176:0/486593030 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb440042e0 con 0x7ffb54106430 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.347+0000 7ffb51600700 1 -- 172.21.15.176:0/486593030 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb44011af0 con 0x7ffb54106430 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.348+0000 7ffb59a00700 1 -- 172.21.15.176:0/486593030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 msgr2=0x7ffb54106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.348+0000 7ffb59a00700 1 --2- 172.21.15.176:0/486593030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb54106850 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb440098a0 tx=0x7ffb44009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.348+0000 7ffb59a00700 1 -- 172.21.15.176:0/486593030 shutdown_connections 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.348+0000 7ffb59a00700 1 --2- 172.21.15.176:0/486593030 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb54106850 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:25.918 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.348+0000 7ffb59a00700 1 -- 172.21.15.176:0/486593030 >> 172.21.15.176:0/486593030 conn(0x7ffb541019b0 msgr2=0x7ffb54103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.348+0000 7ffb59a00700 1 -- 172.21.15.176:0/486593030 shutdown_connections 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.348+0000 7ffb59a00700 1 -- 172.21.15.176:0/486593030 wait complete. 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.349+0000 7ffb59a00700 1 Processor -- start 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.349+0000 7ffb59a00700 1 -- start start 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.350+0000 7ffb59a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb5407ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.350+0000 7ffb59a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5407b090 con 0x7ffb54106430 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.350+0000 7ffb52a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb5407ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.350+0000 7ffb52a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb5407ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48392/0 (socket says 172.21.15.176:48392) 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.350+0000 7ffb52a00700 1 -- 172.21.15.176:0/1979486474 learned_addr learned my addr 172.21.15.176:0/1979486474 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.350+0000 7ffb52a00700 1 -- 172.21.15.176:0/1979486474 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb44009550 con 0x7ffb54106430 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.351+0000 7ffb52a00700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb5407ab50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb44011670 tx=0x7ffb44011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.351+0000 7ffb4b400700 1 -- 172.21.15.176:0/1979486474 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb440039f0 con 0x7ffb54106430 2024-03-13T14:37:25.919 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.351+0000 7ffb4b400700 1 -- 172.21.15.176:0/1979486474 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb44003b50 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.351+0000 7ffb4b400700 1 -- 172.21.15.176:0/1979486474 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb440247c0 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.351+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb540791a0 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.351+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb54079640 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.352+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb540623c0 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.353+0000 7ffb4b400700 1 -- 172.21.15.176:0/1979486474 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ffb4401ac00 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.353+0000 7ffb4b400700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ffb38038370 0x7ffb3803a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.353+0000 7ffb4b400700 1 -- 172.21.15.176:0/1979486474 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffb4404c850 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.354+0000 7ffb52000700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ffb38038370 0x7ffb3803a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.354+0000 7ffb52000700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ffb38038370 0x7ffb3803a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb40006fd0 tx=0x7ffb40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.358+0000 7ffb4b400700 1 -- 172.21.15.176:0/1979486474 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffb44018760 con 0x7ffb54106430 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.598+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb54103ce0 con 0x7ffb38038370 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.607+0000 7ffb4b400700 1 -- 172.21.15.176:0/1979486474 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7ffb54103ce0 con 0x7ffb38038370 2024-03-13T14:37:25.920 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ffb38038370 msgr2=0x7ffb3803a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ffb38038370 0x7ffb3803a830 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb40006fd0 tx=0x7ffb40006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 msgr2=0x7ffb5407ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb5407ab50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb44011670 tx=0x7ffb44011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 shutdown_connections 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ffb38038370 0x7ffb3803a830 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 --2- 172.21.15.176:0/1979486474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb54106430 0x7ffb5407ab50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.609+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 >> 172.21.15.176:0/1979486474 conn(0x7ffb541019b0 msgr2=0x7ffb541030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.610+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 shutdown_connections 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:24.610+0000 7ffb59a00700 1 -- 172.21.15.176:0/1979486474 wait complete. 2024-03-13T14:37:25.921 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying ceph-exporter service with default placement... 2024-03-13T14:37:26.780 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:26 smithi176 ceph-mon[29064]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:37:26.780 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:26 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:27.459 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-03-13T14:37:27.459 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.511+0000 7f980b600700 1 Processor -- start 2024-03-13T14:37:27.459 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.511+0000 7f980b600700 1 -- start start 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.511+0000 7f980b600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f9804106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.512+0000 7f980b600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9804106e20 con 0x7f9804106430 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.512+0000 7f9808c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f9804106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.512+0000 7f9808c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f9804106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48404/0 (socket says 172.21.15.176:48404) 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.512+0000 7f9808c00700 1 -- 172.21.15.176:0/2539839756 learned_addr learned my addr 172.21.15.176:0/2539839756 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.512+0000 7f9808c00700 1 -- 172.21.15.176:0/2539839756 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9804107630 con 0x7f9804106430 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.513+0000 7f9808c00700 1 --2- 172.21.15.176:0/2539839756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f9804106850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f97f80098a0 tx=0x7f97f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaeaecf21f7e69 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:27.460 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.513+0000 7f9803400700 1 -- 172.21.15.176:0/2539839756 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f8004180 con 0x7f9804106430 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.513+0000 7f9803400700 1 -- 172.21.15.176:0/2539839756 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97f80042e0 con 0x7f9804106430 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.513+0000 7f9803400700 1 -- 172.21.15.176:0/2539839756 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f8011af0 con 0x7f9804106430 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.514+0000 7f980b600700 1 -- 172.21.15.176:0/2539839756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 msgr2=0x7f9804106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.514+0000 7f980b600700 1 --2- 172.21.15.176:0/2539839756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f9804106850 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f97f80098a0 tx=0x7f97f8009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.514+0000 7f980b600700 1 -- 172.21.15.176:0/2539839756 shutdown_connections 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.514+0000 7f980b600700 1 --2- 172.21.15.176:0/2539839756 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f9804106850 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.514+0000 7f980b600700 1 -- 172.21.15.176:0/2539839756 >> 172.21.15.176:0/2539839756 conn(0x7f98041019b0 msgr2=0x7f9804103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.514+0000 7f980b600700 1 -- 172.21.15.176:0/2539839756 shutdown_connections 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.514+0000 7f980b600700 1 -- 172.21.15.176:0/2539839756 wait complete. 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.515+0000 7f980b600700 1 Processor -- start 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.515+0000 7f980b600700 1 -- start start 2024-03-13T14:37:27.461 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.516+0000 7f980b600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f980419a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.516+0000 7f980b600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980419a6b0 con 0x7f9804106430 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.516+0000 7f9808c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f980419a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.516+0000 7f9808c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f980419a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48412/0 (socket says 172.21.15.176:48412) 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.516+0000 7f9808c00700 1 -- 172.21.15.176:0/1850368216 learned_addr learned my addr 172.21.15.176:0/1850368216 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.516+0000 7f9808c00700 1 -- 172.21.15.176:0/1850368216 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8009550 con 0x7f9804106430 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.517+0000 7f9808c00700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f980419a170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f97f8011670 tx=0x7f97f8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.517+0000 7f9801600700 1 -- 172.21.15.176:0/1850368216 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f80039c0 con 0x7f9804106430 2024-03-13T14:37:27.462 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.517+0000 7f9801600700 1 -- 172.21.15.176:0/1850368216 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97f8003b20 con 0x7f9804106430 2024-03-13T14:37:27.463 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.517+0000 7f9801600700 1 -- 172.21.15.176:0/1850368216 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f97f80247f0 con 0x7f9804106430 2024-03-13T14:37:27.463 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.517+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980419a8b0 con 0x7f9804106430 2024-03-13T14:37:27.463 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.517+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980419ad50 con 0x7f9804106430 2024-03-13T14:37:27.463 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.518+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98040623c0 con 0x7f9804106430 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.519+0000 7f9801600700 1 -- 172.21.15.176:0/1850368216 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f97f801ac70 con 0x7f9804106430 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.519+0000 7f9801600700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f97ec0382e0 0x7f97ec03a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.519+0000 7f9801600700 1 -- 172.21.15.176:0/1850368216 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f97f804c290 con 0x7f9804106430 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.519+0000 7f9803e00700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f97ec0382e0 0x7f97ec03a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.520+0000 7f9803e00700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f97ec0382e0 0x7f97ec03a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97f0006fd0 tx=0x7f97f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.528+0000 7f9801600700 1 -- 172.21.15.176:0/1850368216 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f97f8018740 con 0x7f9804106430 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.786+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f9804103ce0 con 0x7f97ec0382e0 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.793+0000 7f9801600700 1 -- 172.21.15.176:0/1850368216 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f9804103ce0 con 0x7f97ec0382e0 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f97ec0382e0 msgr2=0x7f97ec03a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f97ec0382e0 0x7f97ec03a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97f0006fd0 tx=0x7f97f0006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 msgr2=0x7f980419a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f980419a170 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f97f8011670 tx=0x7f97f8011aa0 comp rx=0 tx=0).stop 2024-03-13T14:37:27.464 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 shutdown_connections 2024-03-13T14:37:27.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f97ec0382e0 0x7f97ec03a7a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:27.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 --2- 172.21.15.176:0/1850368216 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9804106430 0x7f980419a170 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:27.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 >> 172.21.15.176:0/1850368216 conn(0x7f98041019b0 msgr2=0x7f98041030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:27.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 shutdown_connections 2024-03-13T14:37:27.465 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:26.796+0000 7f980b600700 1 -- 172.21.15.176:0/1850368216 wait complete. 2024-03-13T14:37:27.465 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying prometheus service with default placement... 2024-03-13T14:37:28.024 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:27 smithi176 ceph-mon[29064]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:28.024 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:27 smithi176 ceph-mon[29064]: Saving service ceph-exporter spec with placement * 2024-03-13T14:37:28.024 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:27 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:28.024 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:27 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:29.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:28 smithi176 ceph-mon[29064]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:37:29.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:28 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:29.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-03-13T14:37:29.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.111+0000 7f376b200700 1 Processor -- start 2024-03-13T14:37:29.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.111+0000 7f376b200700 1 -- start start 2024-03-13T14:37:29.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.111+0000 7f376b200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c1091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:29.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.111+0000 7f376b200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c0745a0 con 0x7f376c106dd0 2024-03-13T14:37:29.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.112+0000 7f3769e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c1091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.112+0000 7f3769e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c1091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48416/0 (socket says 172.21.15.176:48416) 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.112+0000 7f3769e00700 1 -- 172.21.15.176:0/1439160612 learned_addr learned my addr 172.21.15.176:0/1439160612 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.112+0000 7f3769e00700 1 -- 172.21.15.176:0/1439160612 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c0746e0 con 0x7f376c106dd0 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.112+0000 7f3769e00700 1 --2- 172.21.15.176:0/1439160612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c1091c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f375c0098a0 tx=0x7f375c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f717cfe8a023a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.113+0000 7f3768a00700 1 -- 172.21.15.176:0/1439160612 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f375c004180 con 0x7f376c106dd0 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.113+0000 7f3768a00700 1 -- 172.21.15.176:0/1439160612 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f375c0042e0 con 0x7f376c106dd0 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.113+0000 7f3768a00700 1 -- 172.21.15.176:0/1439160612 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f375c011af0 con 0x7f376c106dd0 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.113+0000 7f376b200700 1 -- 172.21.15.176:0/1439160612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 msgr2=0x7f376c1091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.113+0000 7f376b200700 1 --2- 172.21.15.176:0/1439160612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c1091c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f375c0098a0 tx=0x7f375c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.114+0000 7f376b200700 1 -- 172.21.15.176:0/1439160612 shutdown_connections 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.114+0000 7f376b200700 1 --2- 172.21.15.176:0/1439160612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c1091c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:29.524 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.114+0000 7f376b200700 1 -- 172.21.15.176:0/1439160612 >> 172.21.15.176:0/1439160612 conn(0x7f376c1006e0 msgr2=0x7f376c102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.114+0000 7f376b200700 1 -- 172.21.15.176:0/1439160612 shutdown_connections 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.114+0000 7f376b200700 1 -- 172.21.15.176:0/1439160612 wait complete. 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.115+0000 7f376b200700 1 Processor -- start 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.115+0000 7f376b200700 1 -- start start 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.115+0000 7f376b200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.115+0000 7f376b200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c19a4d0 con 0x7f376c106dd0 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.115+0000 7f3769e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.116+0000 7f3769e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48428/0 (socket says 172.21.15.176:48428) 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.116+0000 7f3769e00700 1 -- 172.21.15.176:0/1355972607 learned_addr learned my addr 172.21.15.176:0/1355972607 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.116+0000 7f3769e00700 1 -- 172.21.15.176:0/1355972607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c009550 con 0x7f376c106dd0 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.116+0000 7f3769e00700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c199f90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f375c011670 tx=0x7f375c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.116+0000 7f3762a00700 1 -- 172.21.15.176:0/1355972607 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f375c003a40 con 0x7f376c106dd0 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.116+0000 7f3762a00700 1 -- 172.21.15.176:0/1355972607 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f375c003ba0 con 0x7f376c106dd0 2024-03-13T14:37:29.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.116+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376c19a6d0 con 0x7f376c106dd0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.117+0000 7f3762a00700 1 -- 172.21.15.176:0/1355972607 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f375c024790 con 0x7f376c106dd0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.117+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376c19ab70 con 0x7f376c106dd0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.118+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376c10a420 con 0x7f376c106dd0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.119+0000 7f3762a00700 1 -- 172.21.15.176:0/1355972607 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f375c024ab0 con 0x7f376c106dd0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.119+0000 7f3762a00700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f3750038320 0x7f375003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.119+0000 7f3762a00700 1 -- 172.21.15.176:0/1355972607 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f375c04ca90 con 0x7f376c106dd0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.120+0000 7f3769400700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f3750038320 0x7f375003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.120+0000 7f3769400700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f3750038320 0x7f375003a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3758006fd0 tx=0x7f3758006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.124+0000 7f3762a00700 1 -- 172.21.15.176:0/1355972607 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f375c018760 con 0x7f376c106dd0 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.379+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f376c102a00 con 0x7f3750038320 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.386+0000 7f3762a00700 1 -- 172.21.15.176:0/1355972607 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f376c102a00 con 0x7f3750038320 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f3750038320 msgr2=0x7f375003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:29.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f3750038320 0x7f375003a7e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3758006fd0 tx=0x7f3758006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 msgr2=0x7f376c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c199f90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f375c011670 tx=0x7f375c011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 shutdown_connections 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f3750038320 0x7f375003a7e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 --2- 172.21.15.176:0/1355972607 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f376c106dd0 0x7f376c199f90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 >> 172.21.15.176:0/1355972607 conn(0x7f376c1006e0 msgr2=0x7f376c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 shutdown_connections 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:28.389+0000 7f376b200700 1 -- 172.21.15.176:0/1355972607 wait complete. 2024-03-13T14:37:29.527 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying grafana service with default placement... 2024-03-13T14:37:30.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:29 smithi176 ceph-mon[29064]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:30.118 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:29 smithi176 ceph-mon[29064]: Saving service prometheus spec with placement count:1 2024-03-13T14:37:31.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:30 smithi176 ceph-mon[29064]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:37:31.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:30 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:31.490 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-03-13T14:37:31.490 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.548+0000 7f2b50800700 1 Processor -- start 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.548+0000 7f2b50800700 1 -- start start 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.548+0000 7f2b50800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4807a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.548+0000 7f2b50800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4807a900 con 0x7f2b4807bed0 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.549+0000 7f2b4de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4807a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.549+0000 7f2b4de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4807a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48440/0 (socket says 172.21.15.176:48440) 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.549+0000 7f2b4de00700 1 -- 172.21.15.176:0/1167443279 learned_addr learned my addr 172.21.15.176:0/1167443279 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.549+0000 7f2b4de00700 1 -- 172.21.15.176:0/1167443279 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4807aa40 con 0x7f2b4807bed0 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.550+0000 7f2b4de00700 1 --2- 172.21.15.176:0/1167443279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4807a330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0098a0 tx=0x7f2b3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9dd662b457c35f server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.550+0000 7f2b4ca00700 1 -- 172.21.15.176:0/1167443279 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b3c004180 con 0x7f2b4807bed0 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.550+0000 7f2b4ca00700 1 -- 172.21.15.176:0/1167443279 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b3c0042e0 con 0x7f2b4807bed0 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.550+0000 7f2b4ca00700 1 -- 172.21.15.176:0/1167443279 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b3c011af0 con 0x7f2b4807bed0 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.551+0000 7f2b50800700 1 -- 172.21.15.176:0/1167443279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 msgr2=0x7f2b4807a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.551+0000 7f2b50800700 1 --2- 172.21.15.176:0/1167443279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4807a330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0098a0 tx=0x7f2b3c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:31.491 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.551+0000 7f2b50800700 1 -- 172.21.15.176:0/1167443279 shutdown_connections 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.551+0000 7f2b50800700 1 --2- 172.21.15.176:0/1167443279 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4807a330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.551+0000 7f2b50800700 1 -- 172.21.15.176:0/1167443279 >> 172.21.15.176:0/1167443279 conn(0x7f2b48101a00 msgr2=0x7f2b48103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.551+0000 7f2b50800700 1 -- 172.21.15.176:0/1167443279 shutdown_connections 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.551+0000 7f2b50800700 1 -- 172.21.15.176:0/1167443279 wait complete. 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.552+0000 7f2b50800700 1 Processor -- start 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.552+0000 7f2b50800700 1 -- start start 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.552+0000 7f2b50800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.552+0000 7f2b50800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4819c880 con 0x7f2b4807bed0 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.553+0000 7f2b4de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.553+0000 7f2b4de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48446/0 (socket says 172.21.15.176:48446) 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.553+0000 7f2b4de00700 1 -- 172.21.15.176:0/895861130 learned_addr learned my addr 172.21.15.176:0/895861130 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.553+0000 7f2b4de00700 1 -- 172.21.15.176:0/895861130 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c009550 con 0x7f2b4807bed0 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.553+0000 7f2b4de00700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4819c340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c000c00 tx=0x7f2b3c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:31.492 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.553+0000 7f2b42a00700 1 -- 172.21.15.176:0/895861130 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b3c003a70 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.554+0000 7f2b42a00700 1 -- 172.21.15.176:0/895861130 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b3c003bd0 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.554+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4819ca80 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.554+0000 7f2b42a00700 1 -- 172.21.15.176:0/895861130 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b3c0238c0 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.554+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4819cf20 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.555+0000 7f2b42a00700 1 -- 172.21.15.176:0/895861130 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f2b3c023b30 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.555+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4810c7c0 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.555+0000 7f2b42a00700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f2b30038370 0x7f2b3003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.555+0000 7f2b42a00700 1 -- 172.21.15.176:0/895861130 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2b3c04c2c0 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.556+0000 7f2b4d400700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f2b30038370 0x7f2b3003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.556+0000 7f2b4d400700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f2b30038370 0x7f2b3003a830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b38006fd0 tx=0x7f2b38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.561+0000 7f2b42a00700 1 -- 172.21.15.176:0/895861130 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2b3c010560 con 0x7f2b4807bed0 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.785+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b48103cc0 con 0x7f2b30038370 2024-03-13T14:37:31.493 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.795+0000 7f2b42a00700 1 -- 172.21.15.176:0/895861130 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2b48103cc0 con 0x7f2b30038370 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.797+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f2b30038370 msgr2=0x7f2b3003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.797+0000 7f2b50800700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f2b30038370 0x7f2b3003a830 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b38006fd0 tx=0x7f2b38006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.797+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 msgr2=0x7f2b4819c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.797+0000 7f2b50800700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4819c340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c000c00 tx=0x7f2b3c011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.797+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 shutdown_connections 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.797+0000 7f2b50800700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f2b30038370 0x7f2b3003a830 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.798+0000 7f2b50800700 1 --2- 172.21.15.176:0/895861130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4807bed0 0x7f2b4819c340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.798+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 >> 172.21.15.176:0/895861130 conn(0x7f2b48101a00 msgr2=0x7f2b481030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.798+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 shutdown_connections 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:30.798+0000 7f2b50800700 1 -- 172.21.15.176:0/895861130 wait complete. 2024-03-13T14:37:31.494 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying node-exporter service with default placement... 2024-03-13T14:37:32.071 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:31 smithi176 ceph-mon[29064]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:32.072 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:31 smithi176 ceph-mon[29064]: Saving service grafana spec with placement count:1 2024-03-13T14:37:32.975 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:32 smithi176 ceph-mon[29064]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:37:32.975 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:32 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:33.015 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.077+0000 7ff5a3e00700 1 Processor -- start 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.078+0000 7ff5a3e00700 1 -- start start 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.078+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c108aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.078+0000 7ff5a3e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c109070 con 0x7ff59c108680 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.078+0000 7ff5a1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c108aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.078+0000 7ff5a1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c108aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57426/0 (socket says 172.21.15.176:57426) 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.078+0000 7ff5a1400700 1 -- 172.21.15.176:0/1262834922 learned_addr learned my addr 172.21.15.176:0/1262834922 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.079+0000 7ff5a1400700 1 -- 172.21.15.176:0/1262834922 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c109830 con 0x7ff59c108680 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.079+0000 7ff5a1400700 1 --2- 172.21.15.176:0/1262834922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c108aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5900098a0 tx=0x7ff590009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db9df3e34f9f0e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:33.021 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.079+0000 7ff59be00700 1 -- 172.21.15.176:0/1262834922 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff590004180 con 0x7ff59c108680 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff59be00700 1 -- 172.21.15.176:0/1262834922 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5900042e0 con 0x7ff59c108680 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff59be00700 1 -- 172.21.15.176:0/1262834922 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff590011af0 con 0x7ff59c108680 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff5a3e00700 1 -- 172.21.15.176:0/1262834922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 msgr2=0x7ff59c108aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff5a3e00700 1 --2- 172.21.15.176:0/1262834922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c108aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5900098a0 tx=0x7ff590009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff5a3e00700 1 -- 172.21.15.176:0/1262834922 shutdown_connections 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff5a3e00700 1 --2- 172.21.15.176:0/1262834922 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c108aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff5a3e00700 1 -- 172.21.15.176:0/1262834922 >> 172.21.15.176:0/1262834922 conn(0x7ff59c07bb20 msgr2=0x7ff59c1064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff5a3e00700 1 -- 172.21.15.176:0/1262834922 shutdown_connections 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.080+0000 7ff5a3e00700 1 -- 172.21.15.176:0/1262834922 wait complete. 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.081+0000 7ff5a3e00700 1 Processor -- start 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.082+0000 7ff5a3e00700 1 -- start start 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.082+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:33.022 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.082+0000 7ff5a3e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c19c8e0 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.082+0000 7ff5a1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.082+0000 7ff5a1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57436/0 (socket says 172.21.15.176:57436) 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.082+0000 7ff5a1400700 1 -- 172.21.15.176:0/3231852891 learned_addr learned my addr 172.21.15.176:0/3231852891 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.082+0000 7ff5a1400700 1 -- 172.21.15.176:0/3231852891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590009550 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.083+0000 7ff5a1400700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c19c3a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff590011670 tx=0x7ff590011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.083+0000 7ff59a000700 1 -- 172.21.15.176:0/3231852891 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5900039f0 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.083+0000 7ff59a000700 1 -- 172.21.15.176:0/3231852891 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff590003b50 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.083+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c19cae0 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.083+0000 7ff59a000700 1 -- 172.21.15.176:0/3231852891 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5900247c0 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.083+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c19cf80 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.084+0000 7ff59a000700 1 -- 172.21.15.176:0/3231852891 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7ff59001a430 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.084+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c196470 con 0x7ff59c108680 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.085+0000 7ff59a000700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ff584038370 0x7ff58403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:33.023 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.085+0000 7ff59a000700 1 -- 172.21.15.176:0/3231852891 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff59004c370 con 0x7ff59c108680 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.085+0000 7ff5a0a00700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ff584038370 0x7ff58403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.086+0000 7ff5a0a00700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ff584038370 0x7ff58403a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff588006fd0 tx=0x7ff588006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.090+0000 7ff59a000700 1 -- 172.21.15.176:0/3231852891 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff59001ac00 con 0x7ff59c108680 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.314+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7ff59c0611d0 con 0x7ff584038370 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.326+0000 7ff59a000700 1 -- 172.21.15.176:0/3231852891 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7ff59c0611d0 con 0x7ff584038370 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.329+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ff584038370 msgr2=0x7ff58403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.329+0000 7ff5a3e00700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ff584038370 0x7ff58403a830 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff588006fd0 tx=0x7ff588006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.329+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 msgr2=0x7ff59c19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.329+0000 7ff5a3e00700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c19c3a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff590011670 tx=0x7ff590011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.329+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 shutdown_connections 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.329+0000 7ff5a3e00700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7ff584038370 0x7ff58403a830 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.329+0000 7ff5a3e00700 1 --2- 172.21.15.176:0/3231852891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff59c108680 0x7ff59c19c3a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.330+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 >> 172.21.15.176:0/3231852891 conn(0x7ff59c07bb20 msgr2=0x7ff59c106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.330+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 shutdown_connections 2024-03-13T14:37:33.024 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:32.330+0000 7ff5a3e00700 1 -- 172.21.15.176:0/3231852891 wait complete. 2024-03-13T14:37:33.025 INFO:teuthology.orchestra.run.smithi176.stdout:Deploying alertmanager service with default placement... 2024-03-13T14:37:33.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:33 smithi176 ceph-mon[29064]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:33.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:33 smithi176 ceph-mon[29064]: Saving service node-exporter spec with placement * 2024-03-13T14:37:34.515 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-03-13T14:37:34.515 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.661+0000 7f81c5e00700 1 Processor -- start 2024-03-13T14:37:34.515 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.661+0000 7f81c5e00700 1 -- start start 2024-03-13T14:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.662+0000 7f81c5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c0079200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.662+0000 7f81c5e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00797d0 con 0x7f81c007ada0 2024-03-13T14:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.662+0000 7f81bea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c0079200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.662+0000 7f81bea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c0079200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57442/0 (socket says 172.21.15.176:57442) 2024-03-13T14:37:34.516 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.662+0000 7f81bea00700 1 -- 172.21.15.176:0/1330771028 learned_addr learned my addr 172.21.15.176:0/1330771028 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:34.517 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.662+0000 7f81bea00700 1 -- 172.21.15.176:0/1330771028 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c0079910 con 0x7f81c007ada0 2024-03-13T14:37:34.517 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.663+0000 7f81bea00700 1 --2- 172.21.15.176:0/1330771028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c0079200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098a0 tx=0x7f81b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ae0f88ac43ffe1 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:34.517 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.663+0000 7f81bd600700 1 -- 172.21.15.176:0/1330771028 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b0004180 con 0x7f81c007ada0 2024-03-13T14:37:34.517 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.663+0000 7f81bd600700 1 -- 172.21.15.176:0/1330771028 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f81b00042e0 con 0x7f81c007ada0 2024-03-13T14:37:34.519 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.663+0000 7f81bd600700 1 -- 172.21.15.176:0/1330771028 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b0011af0 con 0x7f81c007ada0 2024-03-13T14:37:34.519 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.664+0000 7f81c5e00700 1 -- 172.21.15.176:0/1330771028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 msgr2=0x7f81c0079200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.664+0000 7f81c5e00700 1 --2- 172.21.15.176:0/1330771028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c0079200 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81b00098a0 tx=0x7f81b0009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.664+0000 7f81c5e00700 1 -- 172.21.15.176:0/1330771028 shutdown_connections 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.664+0000 7f81c5e00700 1 --2- 172.21.15.176:0/1330771028 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c0079200 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.664+0000 7f81c5e00700 1 -- 172.21.15.176:0/1330771028 >> 172.21.15.176:0/1330771028 conn(0x7f81c01019b0 msgr2=0x7f81c0103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.664+0000 7f81c5e00700 1 -- 172.21.15.176:0/1330771028 shutdown_connections 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.664+0000 7f81c5e00700 1 -- 172.21.15.176:0/1330771028 wait complete. 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.665+0000 7f81c5e00700 1 Processor -- start 2024-03-13T14:37:34.520 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.666+0000 7f81c5e00700 1 -- start start 2024-03-13T14:37:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.666+0000 7f81c5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.666+0000 7f81c5e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c019c8b0 con 0x7f81c007ada0 2024-03-13T14:37:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.666+0000 7f81bea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.666+0000 7f81bea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57444/0 (socket says 172.21.15.176:57444) 2024-03-13T14:37:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.666+0000 7f81bea00700 1 -- 172.21.15.176:0/1865186062 learned_addr learned my addr 172.21.15.176:0/1865186062 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.667+0000 7f81bea00700 1 -- 172.21.15.176:0/1865186062 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b0009550 con 0x7f81c007ada0 2024-03-13T14:37:34.522 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.667+0000 7f81bea00700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c019c370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81b0011670 tx=0x7f81b0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.667+0000 7f81c4a00700 1 -- 172.21.15.176:0/1865186062 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b00039f0 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.667+0000 7f81c4a00700 1 -- 172.21.15.176:0/1865186062 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f81b0003b50 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.667+0000 7f81c5e00700 1 -- 172.21.15.176:0/1865186062 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c019cab0 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.667+0000 7f81c4a00700 1 -- 172.21.15.176:0/1865186062 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f81b00247c0 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.667+0000 7f81c5e00700 1 -- 172.21.15.176:0/1865186062 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c019cf50 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.669+0000 7f81c4a00700 1 -- 172.21.15.176:0/1865186062 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f81b001a430 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.669+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c010c7f0 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.669+0000 7f81c4a00700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f81a4038320 0x7f81a403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.669+0000 7f81c4a00700 1 -- 172.21.15.176:0/1865186062 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f81b004c4a0 con 0x7f81c007ada0 2024-03-13T14:37:34.523 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.669+0000 7f81be000700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f81a4038320 0x7f81a403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.670+0000 7f81be000700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f81a4038320 0x7f81a403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81ac006fd0 tx=0x7f81ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.674+0000 7f81c4a00700 1 -- 172.21.15.176:0/1865186062 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f81b0018b40 con 0x7f81c007ada0 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.904+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f81c010c980 con 0x7f81a4038320 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.913+0000 7f81c4a00700 1 -- 172.21.15.176:0/1865186062 <== mgr.14120 v2:172.21.15.176:6800/922386027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f81c010c980 con 0x7f81a4038320 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.916+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f81a4038320 msgr2=0x7f81a403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.916+0000 7f81b5600700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f81a4038320 0x7f81a403a7e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f81ac006fd0 tx=0x7f81ac006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.916+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 msgr2=0x7f81c019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.916+0000 7f81b5600700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c019c370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81b0011670 tx=0x7f81b0011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.917+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 shutdown_connections 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.917+0000 7f81b5600700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f81a4038320 0x7f81a403a7e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:34.525 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.917+0000 7f81b5600700 1 --2- 172.21.15.176:0/1865186062 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81c007ada0 0x7f81c019c370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:34.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.917+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 >> 172.21.15.176:0/1865186062 conn(0x7f81c01019b0 msgr2=0x7f81c0103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:34.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.917+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 shutdown_connections 2024-03-13T14:37:34.526 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:33.917+0000 7f81b5600700 1 -- 172.21.15.176:0/1865186062 wait complete. 2024-03-13T14:37:35.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:34 smithi176 ceph-mon[29064]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:37:35.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:34 smithi176 ceph-mon[29064]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:35.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:34 smithi176 ceph-mon[29064]: Saving service alertmanager spec with placement count:1 2024-03-13T14:37:35.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:34 smithi176 ceph-mon[29064]: from='mgr.14120 172.21.15.176:0/4027203539' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:35.997 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.144+0000 7fb844000700 1 Processor -- start 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.144+0000 7fb844000700 1 -- start start 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.144+0000 7fb844000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c07a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.144+0000 7fb844000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c07a920 con 0x7fb83c07bef0 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.145+0000 7fb841600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c07a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.145+0000 7fb841600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c07a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57448/0 (socket says 172.21.15.176:57448) 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.145+0000 7fb841600700 1 -- 172.21.15.176:0/653368560 learned_addr learned my addr 172.21.15.176:0/653368560 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.145+0000 7fb841600700 1 -- 172.21.15.176:0/653368560 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c07aa60 con 0x7fb83c07bef0 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.145+0000 7fb841600700 1 --2- 172.21.15.176:0/653368560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c07a350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8300098a0 tx=0x7fb830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dad9e516c3d96d server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.146+0000 7fb83be00700 1 -- 172.21.15.176:0/653368560 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb830004180 con 0x7fb83c07bef0 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.146+0000 7fb83be00700 1 -- 172.21.15.176:0/653368560 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb8300042e0 con 0x7fb83c07bef0 2024-03-13T14:37:35.998 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.146+0000 7fb83be00700 1 -- 172.21.15.176:0/653368560 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb830011af0 con 0x7fb83c07bef0 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.146+0000 7fb844000700 1 -- 172.21.15.176:0/653368560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 msgr2=0x7fb83c07a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.146+0000 7fb844000700 1 --2- 172.21.15.176:0/653368560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c07a350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb8300098a0 tx=0x7fb830009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.147+0000 7fb844000700 1 -- 172.21.15.176:0/653368560 shutdown_connections 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.147+0000 7fb844000700 1 --2- 172.21.15.176:0/653368560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c07a350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.147+0000 7fb844000700 1 -- 172.21.15.176:0/653368560 >> 172.21.15.176:0/653368560 conn(0x7fb83c101a00 msgr2=0x7fb83c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.147+0000 7fb844000700 1 -- 172.21.15.176:0/653368560 shutdown_connections 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.147+0000 7fb844000700 1 -- 172.21.15.176:0/653368560 wait complete. 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.148+0000 7fb844000700 1 Processor -- start 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.148+0000 7fb844000700 1 -- start start 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.148+0000 7fb844000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.148+0000 7fb844000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c109c90 con 0x7fb83c07bef0 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.148+0000 7fb841600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.148+0000 7fb841600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c109750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57454/0 (socket says 172.21.15.176:57454) 2024-03-13T14:37:35.999 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.148+0000 7fb841600700 1 -- 172.21.15.176:0/4000176888 learned_addr learned my addr 172.21.15.176:0/4000176888 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.149+0000 7fb841600700 1 -- 172.21.15.176:0/4000176888 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb830009550 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.149+0000 7fb841600700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c109750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb830006ba0 tx=0x7fb830011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.149+0000 7fb83a000700 1 -- 172.21.15.176:0/4000176888 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb830003b00 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.149+0000 7fb83a000700 1 -- 172.21.15.176:0/4000176888 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb83001a8e0 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.149+0000 7fb83a000700 1 -- 172.21.15.176:0/4000176888 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb830019990 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.149+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83c109e90 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.149+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c106400 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.150+0000 7fb83a000700 1 -- 172.21.15.176:0/4000176888 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fb830019c00 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.151+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83c0623c0 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.151+0000 7fb83a000700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fb824038320 0x7fb82403a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.151+0000 7fb83a000700 1 -- 172.21.15.176:0/4000176888 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb83004c610 con 0x7fb83c07bef0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.151+0000 7fb840c00700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fb824038320 0x7fb82403a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.152+0000 7fb840c00700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fb824038320 0x7fb82403a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:36.000 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.157+0000 7fb83a000700 1 -- 172.21.15.176:0/4000176888 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb83001aa50 con 0x7fb83c07bef0 2024-03-13T14:37:36.001 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.368+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fb83c10a020 con 0x7fb83c07bef0 2024-03-13T14:37:36.001 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.376+0000 7fb83a000700 1 -- 172.21.15.176:0/4000176888 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fb830018350 con 0x7fb83c07bef0 2024-03-13T14:37:36.001 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fb824038320 msgr2=0x7fb82403a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:36.001 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fb824038320 0x7fb82403a7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:36.001 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 msgr2=0x7fb83c109750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:36.001 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c109750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb830006ba0 tx=0x7fb830011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:36.002 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 shutdown_connections 2024-03-13T14:37:36.002 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fb824038320 0x7fb82403a7e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:36.002 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 --2- 172.21.15.176:0/4000176888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb83c07bef0 0x7fb83c109750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:36.002 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.379+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 >> 172.21.15.176:0/4000176888 conn(0x7fb83c101a00 msgr2=0x7fb83c1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:36.002 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.380+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 shutdown_connections 2024-03-13T14:37:36.002 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:35.380+0000 7fb844000700 1 -- 172.21.15.176:0/4000176888 wait complete. 2024-03-13T14:37:36.494 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:36 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/4000176888' entity='client.admin' 2024-03-13T14:37:36.494 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:36 smithi176 ceph-mon[29064]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:37:37.600 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.663+0000 7fa832c00700 1 Processor -- start 2024-03-13T14:37:37.600 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.663+0000 7fa832c00700 1 -- start start 2024-03-13T14:37:37.600 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.664+0000 7fa832c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:37.600 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.664+0000 7fa832c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c108e60 con 0x7fa82c108470 2024-03-13T14:37:37.600 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.664+0000 7fa831800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:37.600 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.664+0000 7fa831800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57470/0 (socket says 172.21.15.176:57470) 2024-03-13T14:37:37.600 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.664+0000 7fa831800700 1 -- 172.21.15.176:0/2053832871 learned_addr learned my addr 172.21.15.176:0/2053832871 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.664+0000 7fa831800700 1 -- 172.21.15.176:0/2053832871 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c109670 con 0x7fa82c108470 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.665+0000 7fa831800700 1 --2- 172.21.15.176:0/2053832871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c108890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8200098a0 tx=0x7fa820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49563bda53c9a4b server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.665+0000 7fa82be00700 1 -- 172.21.15.176:0/2053832871 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa820004180 con 0x7fa82c108470 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.665+0000 7fa82be00700 1 -- 172.21.15.176:0/2053832871 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa8200042e0 con 0x7fa82c108470 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.665+0000 7fa82be00700 1 -- 172.21.15.176:0/2053832871 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa820011af0 con 0x7fa82c108470 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.666+0000 7fa832c00700 1 -- 172.21.15.176:0/2053832871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 msgr2=0x7fa82c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.666+0000 7fa832c00700 1 --2- 172.21.15.176:0/2053832871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c108890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8200098a0 tx=0x7fa820009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:37.601 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.666+0000 7fa832c00700 1 -- 172.21.15.176:0/2053832871 shutdown_connections 2024-03-13T14:37:37.605 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.666+0000 7fa832c00700 1 --2- 172.21.15.176:0/2053832871 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c108890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:37.605 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.666+0000 7fa832c00700 1 -- 172.21.15.176:0/2053832871 >> 172.21.15.176:0/2053832871 conn(0x7fa82c1039f0 msgr2=0x7fa82c105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:37.605 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.666+0000 7fa832c00700 1 -- 172.21.15.176:0/2053832871 shutdown_connections 2024-03-13T14:37:37.605 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.666+0000 7fa832c00700 1 -- 172.21.15.176:0/2053832871 wait complete. 2024-03-13T14:37:37.605 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.667+0000 7fa832c00700 1 Processor -- start 2024-03-13T14:37:37.605 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.667+0000 7fa832c00700 1 -- start start 2024-03-13T14:37:37.605 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.667+0000 7fa832c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.667+0000 7fa832c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c19c710 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.667+0000 7fa831800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa831800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c19c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57478/0 (socket says 172.21.15.176:57478) 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa831800700 1 -- 172.21.15.176:0/3539548277 learned_addr learned my addr 172.21.15.176:0/3539548277 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa831800700 1 -- 172.21.15.176:0/3539548277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820009550 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa831800700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c19c1d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa820011670 tx=0x7fa820011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa82a000700 1 -- 172.21.15.176:0/3539548277 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8200039f0 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa82a000700 1 -- 172.21.15.176:0/3539548277 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa820003b50 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c19c910 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.668+0000 7fa82a000700 1 -- 172.21.15.176:0/3539548277 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa8200247c0 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.669+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c19cdb0 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.670+0000 7fa82a000700 1 -- 172.21.15.176:0/3539548277 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7fa82001a430 con 0x7fa82c108470 2024-03-13T14:37:37.606 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.670+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c0623c0 con 0x7fa82c108470 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.670+0000 7fa82a000700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fa814038370 0x7fa81403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.670+0000 7fa82a000700 1 -- 172.21.15.176:0/3539548277 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa82004c360 con 0x7fa82c108470 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.670+0000 7fa830e00700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fa814038370 0x7fa81403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.671+0000 7fa830e00700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fa814038370 0x7fa81403a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa818006fd0 tx=0x7fa818006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.675+0000 7fa82a000700 1 -- 172.21.15.176:0/3539548277 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa820018b40 con 0x7fa82c108470 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.888+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa82c19fcb0 con 0x7fa82c108470 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.893+0000 7fa82a000700 1 -- 172.21.15.176:0/3539548277 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa820018b40 con 0x7fa82c108470 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fa814038370 msgr2=0x7fa81403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fa814038370 0x7fa81403a830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa818006fd0 tx=0x7fa818006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 msgr2=0x7fa82c19c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c19c1d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa820011670 tx=0x7fa820011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 shutdown_connections 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7fa814038370 0x7fa81403a830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:37.607 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 --2- 172.21.15.176:0/3539548277 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa82c108470 0x7fa82c19c1d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:37.608 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 >> 172.21.15.176:0/3539548277 conn(0x7fa82c1039f0 msgr2=0x7fa82c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:37.608 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 shutdown_connections 2024-03-13T14:37:37.608 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:36.897+0000 7fa832c00700 1 -- 172.21.15.176:0/3539548277 wait complete. 2024-03-13T14:37:37.608 INFO:teuthology.orchestra.run.smithi176.stdout:Enabling the dashboard module... 2024-03-13T14:37:38.180 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:37 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3539548277' entity='client.admin' 2024-03-13T14:37:39.271 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:38 smithi176 ceph-mon[29064]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:37:39.272 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:38 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2152127959' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-13T14:37:39.575 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.186+0000 7f577f200700 1 Processor -- start 2024-03-13T14:37:39.575 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.186+0000 7f577f200700 1 -- start start 2024-03-13T14:37:39.575 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.186+0000 7f577f200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578007a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:39.575 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.187+0000 7f577f200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578007a900 con 0x7f578007bed0 2024-03-13T14:37:39.575 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.187+0000 7f577de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578007a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:39.575 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.187+0000 7f577de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578007a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57486/0 (socket says 172.21.15.176:57486) 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.187+0000 7f577de00700 1 -- 172.21.15.176:0/2207824877 learned_addr learned my addr 172.21.15.176:0/2207824877 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.187+0000 7f577de00700 1 -- 172.21.15.176:0/2207824877 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578007aa40 con 0x7f578007bed0 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.188+0000 7f577de00700 1 --2- 172.21.15.176:0/2207824877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578007a330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57700098a0 tx=0x7f5770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d20d884da6d30b server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.188+0000 7f577ca00700 1 -- 172.21.15.176:0/2207824877 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5770004180 con 0x7f578007bed0 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.188+0000 7f577ca00700 1 -- 172.21.15.176:0/2207824877 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f57700042e0 con 0x7f578007bed0 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.188+0000 7f577ca00700 1 -- 172.21.15.176:0/2207824877 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5770011af0 con 0x7f578007bed0 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.189+0000 7f577f200700 1 -- 172.21.15.176:0/2207824877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 msgr2=0x7f578007a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.189+0000 7f577f200700 1 --2- 172.21.15.176:0/2207824877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578007a330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57700098a0 tx=0x7f5770009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.189+0000 7f577f200700 1 -- 172.21.15.176:0/2207824877 shutdown_connections 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.189+0000 7f577f200700 1 --2- 172.21.15.176:0/2207824877 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578007a330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.189+0000 7f577f200700 1 -- 172.21.15.176:0/2207824877 >> 172.21.15.176:0/2207824877 conn(0x7f57801017e0 msgr2=0x7f5780103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.189+0000 7f577f200700 1 -- 172.21.15.176:0/2207824877 shutdown_connections 2024-03-13T14:37:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.189+0000 7f577f200700 1 -- 172.21.15.176:0/2207824877 wait complete. 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.190+0000 7f577f200700 1 Processor -- start 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.190+0000 7f577f200700 1 -- start start 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.191+0000 7f577f200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578019c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.191+0000 7f577f200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578019c6a0 con 0x7f578007bed0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.191+0000 7f577de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578019c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.191+0000 7f577de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578019c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57498/0 (socket says 172.21.15.176:57498) 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.191+0000 7f577de00700 1 -- 172.21.15.176:0/2152127959 learned_addr learned my addr 172.21.15.176:0/2152127959 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.192+0000 7f577de00700 1 -- 172.21.15.176:0/2152127959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5770009550 con 0x7f578007bed0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.192+0000 7f577de00700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578019c160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5770011670 tx=0x7f5770011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.192+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57700039f0 con 0x7f578007bed0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.192+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5770003b50 con 0x7f578007bed0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.192+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f578019c8a0 con 0x7f578007bed0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.192+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f578019cd40 con 0x7f578007bed0 2024-03-13T14:37:39.577 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.193+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f57700247c0 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.193+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 8) v1 ==== 45283+0+0 (secure 0 0 0) 0x7f577002d3f0 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.194+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f578010c600 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.194+0000 7f5776a00700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5764038370 0x7f576403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.194+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f577004c380 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.194+0000 7f577d400700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5764038370 0x7f576403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.195+0000 7f577d400700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5764038370 0x7f576403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f576c006fd0 tx=0x7f576c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.199+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5770010970 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.470+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f578019fb20 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.901+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f5770019460 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.903+0000 7f5776a00700 1 -- 172.21.15.176:0/2152127959 <== mon.0 v2:172.21.15.176:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f577002d6a0 con 0x7f578007bed0 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.906+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5764038370 msgr2=0x7f576403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.906+0000 7f577f200700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5764038370 0x7f576403a830 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f576c006fd0 tx=0x7f576c006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:39.578 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 msgr2=0x7f578019c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:39.579 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578019c160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5770011670 tx=0x7f5770011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:39.579 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 shutdown_connections 2024-03-13T14:37:39.579 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f5764038370 0x7f576403a830 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:39.579 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 --2- 172.21.15.176:0/2152127959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f578007bed0 0x7f578019c160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:39.579 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 >> 172.21.15.176:0/2152127959 conn(0x7f57801017e0 msgr2=0x7f5780102eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:39.579 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 shutdown_connections 2024-03-13T14:37:39.579 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:38.907+0000 7f577f200700 1 -- 172.21.15.176:0/2152127959 wait complete. 2024-03-13T14:37:40.089 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:39 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2152127959' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-13T14:37:40.089 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:39 smithi176 ceph-mon[29064]: mgrmap e9: smithi176.tzfdwd(active, since 33s) 2024-03-13T14:37:40.959 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:40 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3499243330' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-13T14:37:41.180 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "active_name": "smithi176.tzfdwd", 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.162+0000 7f4507600700 1 Processor -- start 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.162+0000 7f4507600700 1 -- start start 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.163+0000 7f4507600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f4500106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.163+0000 7f4507600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4500106e20 con 0x7f4500106430 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.163+0000 7f4504c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f4500106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:41.187 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.163+0000 7f4504c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f4500106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57524/0 (socket says 172.21.15.176:57524) 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.163+0000 7f4504c00700 1 -- 172.21.15.176:0/1492461779 learned_addr learned my addr 172.21.15.176:0/1492461779 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.164+0000 7f4504c00700 1 -- 172.21.15.176:0/1492461779 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500107630 con 0x7f4500106430 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.164+0000 7f4504c00700 1 --2- 172.21.15.176:0/1492461779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f4500106850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44f40098a0 tx=0x7f44f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5ffb9b31826dee server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.165+0000 7f44ff400700 1 -- 172.21.15.176:0/1492461779 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f4004180 con 0x7f4500106430 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.165+0000 7f44ff400700 1 -- 172.21.15.176:0/1492461779 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f44f40042e0 con 0x7f4500106430 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.165+0000 7f44ff400700 1 -- 172.21.15.176:0/1492461779 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f4011af0 con 0x7f4500106430 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.166+0000 7f4507600700 1 -- 172.21.15.176:0/1492461779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 msgr2=0x7f4500106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.166+0000 7f4507600700 1 --2- 172.21.15.176:0/1492461779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f4500106850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f44f40098a0 tx=0x7f44f4009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.166+0000 7f4507600700 1 -- 172.21.15.176:0/1492461779 shutdown_connections 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.166+0000 7f4507600700 1 --2- 172.21.15.176:0/1492461779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f4500106850 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.166+0000 7f4507600700 1 -- 172.21.15.176:0/1492461779 >> 172.21.15.176:0/1492461779 conn(0x7f45001019d0 msgr2=0x7f4500103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.167+0000 7f4507600700 1 -- 172.21.15.176:0/1492461779 shutdown_connections 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.167+0000 7f4507600700 1 -- 172.21.15.176:0/1492461779 wait complete. 2024-03-13T14:37:41.188 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.168+0000 7f4507600700 1 Processor -- start 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.168+0000 7f4507600700 1 -- start start 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.168+0000 7f4507600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f450019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.168+0000 7f4507600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450019c8a0 con 0x7f4500106430 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.169+0000 7f4504c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f450019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.169+0000 7f4504c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f450019c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57536/0 (socket says 172.21.15.176:57536) 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.169+0000 7f4504c00700 1 -- 172.21.15.176:0/3499243330 learned_addr learned my addr 172.21.15.176:0/3499243330 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.169+0000 7f4504c00700 1 -- 172.21.15.176:0/3499243330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f4009550 con 0x7f4500106430 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.169+0000 7f4504c00700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f450019c360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006b20 tx=0x7f44f4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.170+0000 7f44fd600700 1 -- 172.21.15.176:0/3499243330 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f40039c0 con 0x7f4500106430 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.170+0000 7f44fd600700 1 -- 172.21.15.176:0/3499243330 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f44f4003b20 con 0x7f4500106430 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.170+0000 7f44fd600700 1 -- 172.21.15.176:0/3499243330 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44f40197f0 con 0x7f4500106430 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.170+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450019caa0 con 0x7f4500106430 2024-03-13T14:37:41.189 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.170+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450019cf40 con 0x7f4500106430 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.171+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45000623c0 con 0x7f4500106430 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.171+0000 7f44fd600700 1 -- 172.21.15.176:0/3499243330 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f44f401a430 con 0x7f4500106430 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.172+0000 7f44fd600700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 0x7f44e803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.172+0000 7f44fd600700 1 -- 172.21.15.176:0/3499243330 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f44f404c2f0 con 0x7f4500106430 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.172+0000 7f44ffe00700 1 -- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 msgr2=0x7f44e803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/922386027 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.172+0000 7f44ffe00700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 0x7f44e803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.181+0000 7f44fd600700 1 -- 172.21.15.176:0/3499243330 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f44f401ac70 con 0x7f4500106430 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.373+0000 7f44ffe00700 1 -- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 msgr2=0x7f44e803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/922386027 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.373+0000 7f44ffe00700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 0x7f44e803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.447+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f450010a860 con 0x7f4500106430 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.448+0000 7f44fd600700 1 -- 172.21.15.176:0/3499243330 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f44f4010380 con 0x7f4500106430 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 msgr2=0x7f44e803a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T14:37:41.190 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 0x7f44e803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 msgr2=0x7f450019c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f450019c360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006b20 tx=0x7f44f4011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 shutdown_connections 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f44e80383c0 0x7f44e803a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 --2- 172.21.15.176:0/3499243330 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4500106430 0x7f450019c360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.451+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 >> 172.21.15.176:0/3499243330 conn(0x7f45001019d0 msgr2=0x7f4500103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.452+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 shutdown_connections 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:40.452+0000 7f4507600700 1 -- 172.21.15.176:0/3499243330 wait complete. 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for the mgr to restart... 2024-03-13T14:37:41.191 INFO:teuthology.orchestra.run.smithi176.stdout:Waiting for mgr epoch 9... 2024-03-13T14:37:47.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: Active manager daemon smithi176.tzfdwd restarted 2024-03-13T14:37:47.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: Activating manager daemon smithi176.tzfdwd 2024-03-13T14:37:47.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: osdmap e3: 0 total, 0 up, 0 in 2024-03-13T14:37:47.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: mgrmap e10: smithi176.tzfdwd(active, starting, since 0.00485989s) 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:47.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:47 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:37:48.849 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:48 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:37:48.849 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:48 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:37:48.849 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:48 smithi176 ceph-mon[29064]: mgrmap e11: smithi176.tzfdwd(active, since 1.00974s) 2024-03-13T14:37:49.416 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout { 2024-03-13T14:37:49.417 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-13T14:37:49.417 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-13T14:37:49.422 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout } 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.755+0000 7f53c7000700 1 Processor -- start 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.755+0000 7f53c7000700 1 -- start start 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.756+0000 7f53c7000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c0106680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.756+0000 7f53c7000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0106c50 con 0x7f53c0106260 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.756+0000 7f53c5c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c0106680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.756+0000 7f53c5c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c0106680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57552/0 (socket says 172.21.15.176:57552) 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.756+0000 7f53c5c00700 1 -- 172.21.15.176:0/1500216783 learned_addr learned my addr 172.21.15.176:0/1500216783 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.757+0000 7f53c5c00700 1 -- 172.21.15.176:0/1500216783 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0107460 con 0x7f53c0106260 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.757+0000 7f53c5c00700 1 --2- 172.21.15.176:0/1500216783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c0106680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f53b40098a0 tx=0x7f53b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a960a2aae92f415c server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.757+0000 7f53bfe00700 1 -- 172.21.15.176:0/1500216783 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53b4004180 con 0x7f53c0106260 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.757+0000 7f53bfe00700 1 -- 172.21.15.176:0/1500216783 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f53b40042e0 con 0x7f53c0106260 2024-03-13T14:37:49.423 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.757+0000 7f53bfe00700 1 -- 172.21.15.176:0/1500216783 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53b4011af0 con 0x7f53c0106260 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.758+0000 7f53c7000700 1 -- 172.21.15.176:0/1500216783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 msgr2=0x7f53c0106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.758+0000 7f53c7000700 1 --2- 172.21.15.176:0/1500216783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c0106680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f53b40098a0 tx=0x7f53b4009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.758+0000 7f53c7000700 1 -- 172.21.15.176:0/1500216783 shutdown_connections 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.758+0000 7f53c7000700 1 --2- 172.21.15.176:0/1500216783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c0106680 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.758+0000 7f53c7000700 1 -- 172.21.15.176:0/1500216783 >> 172.21.15.176:0/1500216783 conn(0x7f53c0101800 msgr2=0x7f53c0103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.758+0000 7f53c7000700 1 -- 172.21.15.176:0/1500216783 shutdown_connections 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.758+0000 7f53c7000700 1 -- 172.21.15.176:0/1500216783 wait complete. 2024-03-13T14:37:49.424 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.759+0000 7f53c7000700 1 Processor -- start 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.760+0000 7f53c7000700 1 -- start start 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.760+0000 7f53c7000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.760+0000 7f53c7000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c019c710 con 0x7f53c0106260 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.760+0000 7f53c5c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.760+0000 7f53c5c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c019c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57554/0 (socket says 172.21.15.176:57554) 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.760+0000 7f53c5c00700 1 -- 172.21.15.176:0/4134992488 learned_addr learned my addr 172.21.15.176:0/4134992488 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.761+0000 7f53c5c00700 1 -- 172.21.15.176:0/4134992488 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4009550 con 0x7f53c0106260 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.761+0000 7f53c5c00700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c019c1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53b4011670 tx=0x7f53b4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.761+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53b40039f0 con 0x7f53c0106260 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.761+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f53b4003b50 con 0x7f53c0106260 2024-03-13T14:37:49.430 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.761+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f53b40247c0 con 0x7f53c0106260 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.761+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c019c910 con 0x7f53c0106260 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.761+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c019cdb0 con 0x7f53c0106260 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.762+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 9) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f53b401ac00 con 0x7f53c0106260 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.763+0000 7f53be000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.763+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 --> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53a803af90 con 0x7f53a80383c0 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.763+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f53b404c440 con 0x7f53c0106260 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.763+0000 7f53c5200700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 msgr2=0x7f53a803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/922386027 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.763+0000 7f53c5200700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.964+0000 7f53c5200700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 msgr2=0x7f53a803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/922386027 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:41.964+0000 7f53c5200700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:42.365+0000 7f53c5200700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 msgr2=0x7f53a803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/922386027 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:42.365+0000 7f53c5200700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-13T14:37:49.431 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:43.166+0000 7f53c5200700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 msgr2=0x7f53a803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/922386027 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:43.166+0000 7f53c5200700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:44.767+0000 7f53c5200700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 msgr2=0x7f53a803a880 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/922386027 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:44.767+0000 7f53c5200700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:47.498+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mgrmap(e 10) v1 ==== 45063+0+0 (secure 0 0 0) 0x7f53b402d770 con 0x7f53c0106260 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:47.498+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 msgr2=0x7f53a803a880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:47.498+0000 7f53be000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.500+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 11) v1 ==== 45190+0+0 (secure 0 0 0) 0x7f53b401a430 con 0x7f53c0106260 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.500+0000 7f53be000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f53a803be30 0x7f53a803e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.500+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f53a803af90 con 0x7f53a803be30 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.501+0000 7f53c5200700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f53a803be30 0x7f53a803e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.503+0000 7f53c5200700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f53a803be30 0x7f53a803e220 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53ac003a10 tx=0x7f53ac009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.504+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mgr.14164 v2:172.21.15.176:6800/456564682 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f53a803af90 con 0x7f53a803be30 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.511+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f53c019d510 con 0x7f53a803be30 2024-03-13T14:37:49.432 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.511+0000 7f53be000700 1 -- 172.21.15.176:0/4134992488 <== mgr.14164 v2:172.21.15.176:6800/456564682 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f53c019d510 con 0x7f53a803be30 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f53a803be30 msgr2=0x7f53a803e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f53a803be30 0x7f53a803e220 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f53ac003a10 tx=0x7f53ac009270 comp rx=0 tx=0).stop 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 msgr2=0x7f53c019c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c019c1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53b4011670 tx=0x7f53b4011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 shutdown_connections 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f53a803be30 0x7f53a803e220 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53c0106260 0x7f53c019c1d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 --2- 172.21.15.176:0/4134992488 >> [v2:172.21.15.176:6800/922386027,v1:172.21.15.176:6801/922386027] conn(0x7f53a80383c0 0x7f53a803a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 >> 172.21.15.176:0/4134992488 conn(0x7f53c0101800 msgr2=0x7f53c0102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 shutdown_connections 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:48.512+0000 7f53c7000700 1 -- 172.21.15.176:0/4134992488 wait complete. 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:mgr epoch 9 is available 2024-03-13T14:37:49.433 INFO:teuthology.orchestra.run.smithi176.stdout:Generating a dashboard self-signed certificate... 2024-03-13T14:37:49.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:49 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:48] ENGINE Bus STARTING 2024-03-13T14:37:49.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:49 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:48] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:37:49.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:49 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:48] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:37:49.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:49 smithi176 ceph-mon[29064]: [13/Mar/2024:14:37:48] ENGINE Bus STARTED 2024-03-13T14:37:49.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:49 smithi176 ceph-mon[29064]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-13T14:37:49.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:49 smithi176 ceph-mon[29064]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-13T14:37:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:50 smithi176 ceph-mon[29064]: mgrmap e12: smithi176.tzfdwd(active, since 2s) 2024-03-13T14:37:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:50 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:51.369 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-13T14:37:51.376 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.363+0000 7f9332c00700 1 Processor -- start 2024-03-13T14:37:51.376 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.363+0000 7f9332c00700 1 -- start start 2024-03-13T14:37:51.376 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.363+0000 7f9332c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c109320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:51.376 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.363+0000 7f9332c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c074600 con 0x7f932c106f30 2024-03-13T14:37:51.376 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.364+0000 7f932be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:51.376 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.364+0000 7f932be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58192/0 (socket says 172.21.15.176:58192) 2024-03-13T14:37:51.376 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.364+0000 7f932be00700 1 -- 172.21.15.176:0/3799952310 learned_addr learned my addr 172.21.15.176:0/3799952310 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.364+0000 7f932be00700 1 -- 172.21.15.176:0/3799952310 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c109860 con 0x7f932c106f30 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.365+0000 7f932be00700 1 --2- 172.21.15.176:0/3799952310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c109320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c585c3088b92ee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.365+0000 7f932aa00700 1 -- 172.21.15.176:0/3799952310 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f931c004180 con 0x7f932c106f30 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.365+0000 7f932aa00700 1 -- 172.21.15.176:0/3799952310 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f931c0042e0 con 0x7f932c106f30 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.365+0000 7f932aa00700 1 -- 172.21.15.176:0/3799952310 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f931c011af0 con 0x7f932c106f30 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.366+0000 7f9332c00700 1 -- 172.21.15.176:0/3799952310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 msgr2=0x7f932c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.366+0000 7f9332c00700 1 --2- 172.21.15.176:0/3799952310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c109320 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f931c0098a0 tx=0x7f931c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.366+0000 7f9332c00700 1 -- 172.21.15.176:0/3799952310 shutdown_connections 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.366+0000 7f9332c00700 1 --2- 172.21.15.176:0/3799952310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c109320 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.366+0000 7f9332c00700 1 -- 172.21.15.176:0/3799952310 >> 172.21.15.176:0/3799952310 conn(0x7f932c1008c0 msgr2=0x7f932c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:51.377 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.366+0000 7f9332c00700 1 -- 172.21.15.176:0/3799952310 shutdown_connections 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.366+0000 7f9332c00700 1 -- 172.21.15.176:0/3799952310 wait complete. 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.367+0000 7f9332c00700 1 Processor -- start 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.367+0000 7f9332c00700 1 -- start start 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.368+0000 7f9332c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.368+0000 7f9332c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932c19c830 con 0x7f932c106f30 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.368+0000 7f932be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.368+0000 7f932be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58198/0 (socket says 172.21.15.176:58198) 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.368+0000 7f932be00700 1 -- 172.21.15.176:0/550818424 learned_addr learned my addr 172.21.15.176:0/550818424 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.369+0000 7f932be00700 1 -- 172.21.15.176:0/550818424 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c009550 con 0x7f932c106f30 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.369+0000 7f932be00700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c19c2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f931c011670 tx=0x7f931c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.369+0000 7f9328c00700 1 -- 172.21.15.176:0/550818424 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f931c003a40 con 0x7f932c106f30 2024-03-13T14:37:51.378 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.369+0000 7f9328c00700 1 -- 172.21.15.176:0/550818424 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f931c003ba0 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.369+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932c19ca30 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.369+0000 7f9328c00700 1 -- 172.21.15.176:0/550818424 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f931c024790 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.369+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932c19ced0 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.371+0000 7f9328c00700 1 -- 172.21.15.176:0/550818424 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f931c02d3f0 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.371+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c005320 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.371+0000 7f9328c00700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9310038370 0x7f931003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.371+0000 7f9328c00700 1 -- 172.21.15.176:0/550818424 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f931c01ac50 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.372+0000 7f932b400700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9310038370 0x7f931003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.372+0000 7f932b400700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9310038370 0x7f931003a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9318006fd0 tx=0x7f9318006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.377+0000 7f9328c00700 1 -- 172.21.15.176:0/550818424 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f931c010bd0 con 0x7f932c106f30 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.601+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f930c000bf0 con 0x7f9310038370 2024-03-13T14:37:51.379 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.705+0000 7f9328c00700 1 -- 172.21.15.176:0/550818424 <== mgr.14164 v2:172.21.15.176:6800/456564682 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f930c000bf0 con 0x7f9310038370 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9310038370 msgr2=0x7f931003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9310038370 0x7f931003a830 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9318006fd0 tx=0x7f9318006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 msgr2=0x7f932c19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c19c2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f931c011670 tx=0x7f931c011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 shutdown_connections 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9310038370 0x7f931003a830 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 --2- 172.21.15.176:0/550818424 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f932c106f30 0x7f932c19c2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 >> 172.21.15.176:0/550818424 conn(0x7f932c1008c0 msgr2=0x7f932c101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 shutdown_connections 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:50.708+0000 7f9332c00700 1 -- 172.21.15.176:0/550818424 wait complete. 2024-03-13T14:37:51.380 INFO:teuthology.orchestra.run.smithi176.stdout:Creating initial admin user... 2024-03-13T14:37:51.960 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:51 smithi176 ceph-mon[29064]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:51.960 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:51 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:51.960 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:51 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:53.055 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:52 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:53.055 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:52 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:37:53.495 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$SP/6Fd41Xz4sCDCX.cwqJ.FrSF3CsX/ta28wv/uyEHDeLIu/leu0m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1710340672, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-13T14:37:53.495 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.011+0000 7f5c38200700 1 Processor -- start 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.011+0000 7f5c38200700 1 -- start start 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.012+0000 7f5c38200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c300791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.012+0000 7f5c38200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30079790 con 0x7f5c3007ad60 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.012+0000 7f5c35800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c300791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.012+0000 7f5c35800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c300791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54824/0 (socket says 172.21.15.176:54824) 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.012+0000 7f5c35800700 1 -- 172.21.15.176:0/3264567026 learned_addr learned my addr 172.21.15.176:0/3264567026 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.013+0000 7f5c35800700 1 -- 172.21.15.176:0/3264567026 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300798d0 con 0x7f5c3007ad60 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.013+0000 7f5c35800700 1 --2- 172.21.15.176:0/3264567026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c300791c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c240098a0 tx=0x7f5c24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18adfa5a299bdc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.013+0000 7f5c2fe00700 1 -- 172.21.15.176:0/3264567026 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c24004180 con 0x7f5c3007ad60 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.013+0000 7f5c2fe00700 1 -- 172.21.15.176:0/3264567026 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5c240042e0 con 0x7f5c3007ad60 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.013+0000 7f5c2fe00700 1 -- 172.21.15.176:0/3264567026 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c24011af0 con 0x7f5c3007ad60 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.014+0000 7f5c38200700 1 -- 172.21.15.176:0/3264567026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 msgr2=0x7f5c300791c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:53.496 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.014+0000 7f5c38200700 1 --2- 172.21.15.176:0/3264567026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c300791c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5c240098a0 tx=0x7f5c24009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:53.497 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.014+0000 7f5c38200700 1 -- 172.21.15.176:0/3264567026 shutdown_connections 2024-03-13T14:37:53.497 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.014+0000 7f5c38200700 1 --2- 172.21.15.176:0/3264567026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c300791c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:53.497 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.014+0000 7f5c38200700 1 -- 172.21.15.176:0/3264567026 >> 172.21.15.176:0/3264567026 conn(0x7f5c301019b0 msgr2=0x7f5c30103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:53.497 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.014+0000 7f5c38200700 1 -- 172.21.15.176:0/3264567026 shutdown_connections 2024-03-13T14:37:53.500 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.014+0000 7f5c38200700 1 -- 172.21.15.176:0/3264567026 wait complete. 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.015+0000 7f5c38200700 1 Processor -- start 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.016+0000 7f5c38200700 1 -- start start 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.016+0000 7f5c38200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c3019c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.016+0000 7f5c38200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3019c8c0 con 0x7f5c3007ad60 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.016+0000 7f5c35800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c3019c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.016+0000 7f5c35800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c3019c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54840/0 (socket says 172.21.15.176:54840) 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.016+0000 7f5c35800700 1 -- 172.21.15.176:0/517963282 learned_addr learned my addr 172.21.15.176:0/517963282 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.017+0000 7f5c35800700 1 -- 172.21.15.176:0/517963282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24009550 con 0x7f5c3007ad60 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.017+0000 7f5c35800700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c3019c380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5c24011670 tx=0x7f5c24011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.017+0000 7f5c2e000700 1 -- 172.21.15.176:0/517963282 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c24003a40 con 0x7f5c3007ad60 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.017+0000 7f5c2e000700 1 -- 172.21.15.176:0/517963282 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5c24003ba0 con 0x7f5c3007ad60 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.017+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3019cac0 con 0x7f5c3007ad60 2024-03-13T14:37:53.501 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.017+0000 7f5c2e000700 1 -- 172.21.15.176:0/517963282 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5c24024790 con 0x7f5c3007ad60 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.018+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3019cf60 con 0x7f5c3007ad60 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.019+0000 7f5c2e000700 1 -- 172.21.15.176:0/517963282 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 12) v1 ==== 45296+0+0 (secure 0 0 0) 0x7f5c2401a430 con 0x7f5c3007ad60 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.019+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3010c7d0 con 0x7f5c3007ad60 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.019+0000 7f5c2e000700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5c180383c0 0x7f5c1803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.020+0000 7f5c2e000700 1 -- 172.21.15.176:0/517963282 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c2404c630 con 0x7f5c3007ad60 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.020+0000 7f5c34e00700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5c180383c0 0x7f5c1803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.020+0000 7f5c34e00700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5c180383c0 0x7f5c1803a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006fd0 tx=0x7f5c1c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.024+0000 7f5c2e000700 1 -- 172.21.15.176:0/517963282 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5c2401ac50 con 0x7f5c3007ad60 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.255+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5c300621e0 con 0x7f5c180383c0 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.492+0000 7f5c2e000700 1 -- 172.21.15.176:0/517963282 <== mgr.14164 v2:172.21.15.176:6800/456564682 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f5c300621e0 con 0x7f5c180383c0 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5c180383c0 msgr2=0x7f5c1803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5c180383c0 0x7f5c1803a880 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c006fd0 tx=0x7f5c1c006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:53.502 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 msgr2=0x7f5c3019c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c3019c380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5c24011670 tx=0x7f5c24011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 shutdown_connections 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5c180383c0 0x7f5c1803a880 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 --2- 172.21.15.176:0/517963282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c3007ad60 0x7f5c3019c380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 >> 172.21.15.176:0/517963282 conn(0x7f5c301019b0 msgr2=0x7f5c30103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 shutdown_connections 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:52.494+0000 7f5c38200700 1 -- 172.21.15.176:0/517963282 wait complete. 2024-03-13T14:37:53.503 INFO:teuthology.orchestra.run.smithi176.stdout:Fetching dashboard port number... 2024-03-13T14:37:54.083 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:53 smithi176 ceph-mon[29064]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:37:55.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:54 smithi176 ceph-mon[29064]: mgrmap e13: smithi176.tzfdwd(active, since 6s) 2024-03-13T14:37:55.751 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stdout 8443 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.939+0000 7f2bd6600700 1 Processor -- start 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.939+0000 7f2bd6600700 1 -- start start 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.940+0000 7f2bd6600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd0106840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.940+0000 7f2bd6600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd0106e10 con 0x7f2bd0106420 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.940+0000 7f2bcf400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd0106840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.940+0000 7f2bcf400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd0106840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54848/0 (socket says 172.21.15.176:54848) 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.940+0000 7f2bcf400700 1 -- 172.21.15.176:0/3226406032 learned_addr learned my addr 172.21.15.176:0/3226406032 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.941+0000 7f2bcf400700 1 -- 172.21.15.176:0/3226406032 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0107620 con 0x7f2bd0106420 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.941+0000 7f2bcf400700 1 --2- 172.21.15.176:0/3226406032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd0106840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00098a0 tx=0x7f2bc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce0ac5451237a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.941+0000 7f2bce000700 1 -- 172.21.15.176:0/3226406032 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bc0004180 con 0x7f2bd0106420 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.941+0000 7f2bce000700 1 -- 172.21.15.176:0/3226406032 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2bc00042e0 con 0x7f2bd0106420 2024-03-13T14:37:55.752 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.941+0000 7f2bce000700 1 -- 172.21.15.176:0/3226406032 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bc0011af0 con 0x7f2bd0106420 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.942+0000 7f2bd6600700 1 -- 172.21.15.176:0/3226406032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 msgr2=0x7f2bd0106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.942+0000 7f2bd6600700 1 --2- 172.21.15.176:0/3226406032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd0106840 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2bc00098a0 tx=0x7f2bc0009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.942+0000 7f2bd6600700 1 -- 172.21.15.176:0/3226406032 shutdown_connections 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.942+0000 7f2bd6600700 1 --2- 172.21.15.176:0/3226406032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd0106840 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.942+0000 7f2bd6600700 1 -- 172.21.15.176:0/3226406032 >> 172.21.15.176:0/3226406032 conn(0x7f2bd0101a00 msgr2=0x7f2bd0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.942+0000 7f2bd6600700 1 -- 172.21.15.176:0/3226406032 shutdown_connections 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.942+0000 7f2bd6600700 1 -- 172.21.15.176:0/3226406032 wait complete. 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.943+0000 7f2bd6600700 1 Processor -- start 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.943+0000 7f2bd6600700 1 -- start start 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.944+0000 7f2bd6600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.944+0000 7f2bd6600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019a620 con 0x7f2bd0106420 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.944+0000 7f2bcf400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.944+0000 7f2bcf400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54864/0 (socket says 172.21.15.176:54864) 2024-03-13T14:37:55.753 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.944+0000 7f2bcf400700 1 -- 172.21.15.176:0/2259413571 learned_addr learned my addr 172.21.15.176:0/2259413571 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.944+0000 7f2bcf400700 1 -- 172.21.15.176:0/2259413571 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc0009550 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.945+0000 7f2bcf400700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd019a0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0006ba0 tx=0x7f2bc0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.945+0000 7f2bc7e00700 1 -- 172.21.15.176:0/2259413571 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bc0003b00 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.945+0000 7f2bc7e00700 1 -- 172.21.15.176:0/2259413571 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2bc001a8e0 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.945+0000 7f2bc7e00700 1 -- 172.21.15.176:0/2259413571 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bc0019990 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.945+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd019a820 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.945+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd019acc0 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.946+0000 7f2bc7e00700 1 -- 172.21.15.176:0/2259413571 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f2bc0019c00 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.946+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd00623c0 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.947+0000 7f2bc7e00700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f2bb4038410 0x7f2bb403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.947+0000 7f2bc7e00700 1 -- 172.21.15.176:0/2259413571 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2bc004c7e0 con 0x7f2bd0106420 2024-03-13T14:37:55.754 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.947+0000 7f2bcea00700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f2bb4038410 0x7f2bb403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.948+0000 7f2bcea00700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f2bb4038410 0x7f2bb403a8d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc006fd0 tx=0x7f2bbc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:54.952+0000 7f2bc7e00700 1 -- 172.21.15.176:0/2259413571 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f2bc0018710 con 0x7f2bd0106420 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.166+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f2bd019dbc0 con 0x7f2bd0106420 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.167+0000 7f2bc7e00700 1 -- 172.21.15.176:0/2259413571 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f2bc0018350 con 0x7f2bd0106420 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.169+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f2bb4038410 msgr2=0x7f2bb403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.169+0000 7f2bd6600700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f2bb4038410 0x7f2bb403a8d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc006fd0 tx=0x7f2bbc006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.169+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 msgr2=0x7f2bd019a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.169+0000 7f2bd6600700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd019a0e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0006ba0 tx=0x7f2bc0011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.170+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 shutdown_connections 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.170+0000 7f2bd6600700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f2bb4038410 0x7f2bb403a8d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.170+0000 7f2bd6600700 1 --2- 172.21.15.176:0/2259413571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2bd0106420 0x7f2bd019a0e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.170+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 >> 172.21.15.176:0/2259413571 conn(0x7f2bd0101a00 msgr2=0x7f2bd01027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.170+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 shutdown_connections 2024-03-13T14:37:55.755 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:55.170+0000 7f2bd6600700 1 -- 172.21.15.176:0/2259413571 wait complete. 2024-03-13T14:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-13T14:37:55.781 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout disabled 2024-03-13T14:37:55.806 INFO:teuthology.orchestra.run.smithi176.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-13T14:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout:systemctl: stdout inactive 2024-03-13T14:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout:firewalld.service is not enabled 2024-03-13T14:37:55.807 INFO:teuthology.orchestra.run.smithi176.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-13T14:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout:Ceph Dashboard is now available at: 2024-03-13T14:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: URL: https://smithi176.front.sepia.ceph.com:8443/ 2024-03-13T14:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: User: admin 2024-03-13T14:37:55.814 INFO:teuthology.orchestra.run.smithi176.stdout: Password: g83qfvm5ug 2024-03-13T14:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:55.815 INFO:teuthology.orchestra.run.smithi176.stdout:Saving cluster configuration to /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config directory 2024-03-13T14:37:55.817 INFO:teuthology.orchestra.run.smithi176.stdout:Enabling autotune for osd_memory_target 2024-03-13T14:37:56.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:55 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2259413571' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-13T14:37:57.344 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.405+0000 7f7a82200700 1 Processor -- start 2024-03-13T14:37:57.350 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.406+0000 7f7a82200700 1 -- start start 2024-03-13T14:37:57.350 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.406+0000 7f7a82200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c106850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.406+0000 7f7a82200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c106e20 con 0x7f7a7c106430 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.406+0000 7f7a7b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c106850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.406+0000 7f7a7b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c106850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54876/0 (socket says 172.21.15.176:54876) 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.406+0000 7f7a7b400700 1 -- 172.21.15.176:0/1292670117 learned_addr learned my addr 172.21.15.176:0/1292670117 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.407+0000 7f7a7b400700 1 -- 172.21.15.176:0/1292670117 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c107630 con 0x7f7a7c106430 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.407+0000 7f7a7b400700 1 --2- 172.21.15.176:0/1292670117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c106850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0098a0 tx=0x7f7a6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9368c8073e6597 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.408+0000 7f7a7aa00700 1 -- 172.21.15.176:0/1292670117 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a6c004180 con 0x7f7a7c106430 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.408+0000 7f7a7aa00700 1 -- 172.21.15.176:0/1292670117 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7a6c0042e0 con 0x7f7a7c106430 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.408+0000 7f7a7aa00700 1 -- 172.21.15.176:0/1292670117 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a6c011af0 con 0x7f7a7c106430 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.408+0000 7f7a82200700 1 -- 172.21.15.176:0/1292670117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 msgr2=0x7f7a7c106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.408+0000 7f7a82200700 1 --2- 172.21.15.176:0/1292670117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c106850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0098a0 tx=0x7f7a6c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:57.351 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.409+0000 7f7a82200700 1 -- 172.21.15.176:0/1292670117 shutdown_connections 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.409+0000 7f7a82200700 1 --2- 172.21.15.176:0/1292670117 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c106850 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.409+0000 7f7a82200700 1 -- 172.21.15.176:0/1292670117 >> 172.21.15.176:0/1292670117 conn(0x7f7a7c1019d0 msgr2=0x7f7a7c103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.409+0000 7f7a82200700 1 -- 172.21.15.176:0/1292670117 shutdown_connections 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.409+0000 7f7a82200700 1 -- 172.21.15.176:0/1292670117 wait complete. 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.410+0000 7f7a82200700 1 Processor -- start 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.410+0000 7f7a82200700 1 -- start start 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.410+0000 7f7a82200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.410+0000 7f7a82200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7c19a710 con 0x7f7a7c106430 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.411+0000 7f7a7b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.411+0000 7f7a7b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54880/0 (socket says 172.21.15.176:54880) 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.411+0000 7f7a7b400700 1 -- 172.21.15.176:0/4157566475 learned_addr learned my addr 172.21.15.176:0/4157566475 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:57.352 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.411+0000 7f7a7b400700 1 -- 172.21.15.176:0/4157566475 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c009550 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.411+0000 7f7a7b400700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c19a1d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c011670 tx=0x7f7a6c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.411+0000 7f7a78c00700 1 -- 172.21.15.176:0/4157566475 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a6c0039f0 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.411+0000 7f7a78c00700 1 -- 172.21.15.176:0/4157566475 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7a6c003b50 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.412+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7c19a910 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.412+0000 7f7a78c00700 1 -- 172.21.15.176:0/4157566475 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7a6c0247c0 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.412+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7c19adb0 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.413+0000 7f7a78c00700 1 -- 172.21.15.176:0/4157566475 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f7a6c01a430 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.413+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7c0623c0 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.413+0000 7f7a78c00700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7a60038460 0x7f7a6003a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.413+0000 7f7a78c00700 1 -- 172.21.15.176:0/4157566475 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7a6c04c530 con 0x7f7a7c106430 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.413+0000 7f7a73e00700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7a60038460 0x7f7a6003a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:57.353 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.414+0000 7f7a73e00700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7a60038460 0x7f7a6003a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006fd0 tx=0x7f7a64006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.418+0000 7f7a78c00700 1 -- 172.21.15.176:0/4157566475 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f7a6c018b40 con 0x7f7a7c106430 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.631+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f7a7c19dcb0 con 0x7f7a7c106430 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.632+0000 7f7a78c00700 1 -- 172.21.15.176:0/4157566475 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7a6c018b40 con 0x7f7a7c106430 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7a60038460 msgr2=0x7f7a6003a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7a60038460 0x7f7a6003a920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a64006fd0 tx=0x7f7a64006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 msgr2=0x7f7a7c19a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c19a1d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c011670 tx=0x7f7a6c011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 shutdown_connections 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7a60038460 0x7f7a6003a920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 --2- 172.21.15.176:0/4157566475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7a7c106430 0x7f7a7c19a1d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 >> 172.21.15.176:0/4157566475 conn(0x7f7a7c1019d0 msgr2=0x7f7a7c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 shutdown_connections 2024-03-13T14:37:57.354 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:56.635+0000 7f7a82200700 1 -- 172.21.15.176:0/4157566475 wait complete. 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.915+0000 7f9671800700 1 Processor -- start 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.915+0000 7f9671800700 1 -- start start 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.916+0000 7f9671800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c109360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.916+0000 7f9671800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c0745c0 con 0x7f966c106f70 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.916+0000 7f966aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.916+0000 7f966aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c109360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54884/0 (socket says 172.21.15.176:54884) 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.916+0000 7f966aa00700 1 -- 172.21.15.176:0/1321485046 learned_addr learned my addr 172.21.15.176:0/1321485046 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.916+0000 7f966aa00700 1 -- 172.21.15.176:0/1321485046 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f966c074700 con 0x7f966c106f70 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.917+0000 7f966aa00700 1 --2- 172.21.15.176:0/1321485046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c109360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f965c0098a0 tx=0x7f965c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2868eebb6218a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.917+0000 7f9669600700 1 -- 172.21.15.176:0/1321485046 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f965c004180 con 0x7f966c106f70 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.917+0000 7f9669600700 1 -- 172.21.15.176:0/1321485046 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f965c0042e0 con 0x7f966c106f70 2024-03-13T14:37:58.952 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.917+0000 7f9669600700 1 -- 172.21.15.176:0/1321485046 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f965c011af0 con 0x7f966c106f70 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.918+0000 7f9671800700 1 -- 172.21.15.176:0/1321485046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 msgr2=0x7f966c109360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.918+0000 7f9671800700 1 --2- 172.21.15.176:0/1321485046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c109360 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f965c0098a0 tx=0x7f965c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.918+0000 7f9671800700 1 -- 172.21.15.176:0/1321485046 shutdown_connections 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.918+0000 7f9671800700 1 --2- 172.21.15.176:0/1321485046 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c109360 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.918+0000 7f9671800700 1 -- 172.21.15.176:0/1321485046 >> 172.21.15.176:0/1321485046 conn(0x7f966c1008c0 msgr2=0x7f966c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.918+0000 7f9671800700 1 -- 172.21.15.176:0/1321485046 shutdown_connections 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.918+0000 7f9671800700 1 -- 172.21.15.176:0/1321485046 wait complete. 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.919+0000 7f9671800700 1 Processor -- start 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.919+0000 7f9671800700 1 -- start start 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.919+0000 7f9671800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c19c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.919+0000 7f9671800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966c19c860 con 0x7f966c106f70 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.920+0000 7f966aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c19c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:58.953 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.920+0000 7f966aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c19c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:54894/0 (socket says 172.21.15.176:54894) 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.920+0000 7f966aa00700 1 -- 172.21.15.176:0/3108563461 learned_addr learned my addr 172.21.15.176:0/3108563461 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.920+0000 7f966aa00700 1 -- 172.21.15.176:0/3108563461 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965c009550 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.920+0000 7f966aa00700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c19c320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f965c011670 tx=0x7f965c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.920+0000 7f9663400700 1 -- 172.21.15.176:0/3108563461 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f965c003a40 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.921+0000 7f9663400700 1 -- 172.21.15.176:0/3108563461 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f965c003ba0 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.921+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966c19ca60 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.921+0000 7f9663400700 1 -- 172.21.15.176:0/3108563461 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f965c024790 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.921+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966c19cf00 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.922+0000 7f9663400700 1 -- 172.21.15.176:0/3108563461 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f965c01a430 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.922+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f966c10a5c0 con 0x7f966c106f70 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.922+0000 7f9663400700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9650038460 0x7f965003a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:37:58.954 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.922+0000 7f9663400700 1 -- 172.21.15.176:0/3108563461 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f965c04c700 con 0x7f966c106f70 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.923+0000 7f966a000700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9650038460 0x7f965003a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.923+0000 7f966a000700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9650038460 0x7f965003a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9658006fd0 tx=0x7f9658006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:57.927+0000 7f9663400700 1 -- 172.21.15.176:0/3108563461 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f965c01ac50 con 0x7f966c106f70 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.253+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f966c10a7e0 con 0x7f966c106f70 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.257+0000 7f9663400700 1 -- 172.21.15.176:0/3108563461 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f965c018b40 con 0x7f966c106f70 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.260+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9650038460 msgr2=0x7f965003a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9650038460 0x7f965003a920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9658006fd0 tx=0x7f9658006e40 comp rx=0 tx=0).stop 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 msgr2=0x7f966c19c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c19c320 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f965c011670 tx=0x7f965c011b50 comp rx=0 tx=0).stop 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 shutdown_connections 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9650038460 0x7f965003a920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 --2- 172.21.15.176:0/3108563461 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f966c106f70 0x7f966c19c320 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:37:58.955 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 >> 172.21.15.176:0/3108563461 conn(0x7f966c1008c0 msgr2=0x7f966c1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 shutdown_connections 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr 2024-03-13T14:37:58.261+0000 7f9671800700 1 -- 172.21.15.176:0/3108563461 wait complete. 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout:Or, if you are only running a single cluster on this host: 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.956 INFO:teuthology.orchestra.run.smithi176.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout: ceph telemetry on 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout:For more information see: 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:37:58.957 INFO:teuthology.orchestra.run.smithi176.stdout:Bootstrap complete. 2024-03-13T14:37:59.039 INFO:tasks.cephadm:Fetching config... 2024-03-13T14:37:59.039 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:37:59.040 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-13T14:37:59.084 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-13T14:37:59.085 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:37:59.085 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-13T14:37:59.169 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-13T14:37:59.169 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:37:59.169 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/keyring of=/dev/stdout 2024-03-13T14:37:59.296 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-13T14:37:59.296 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:37:59.296 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-13T14:37:59.346 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-13T14:37:59.347 DEBUG:teuthology.orchestra.run.smithi176:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM6w9SmY/uZm/MhqTx30LsgAeaBlMz17hnsFOtvVM+8wiBdavRsbpozSMsAzY1gWiosu5BloSatJVSkxRA7Dwj/cc6YjyA55geevrCwz4QtCHImkxwsl81i8RPO+en9Y88RYeyzN3sZTqi07zdZnsuUTAIy6YgKotBWFTPzwMYGYEmlQmKior3ClgVYrXlU5g/DjCWNTahxEgET4Go2Vh62VMfrernf6tLz2RNmamGmOUeUyReyDrWkeoZT2MhN6IY+J9nbV26HIWl0e3pOeksHmS8dQaki9zaEbDeac5V4thPnik7uV4y/O5xV5YlIdH9Rxc1qU9uRHxoY4EVwUNNkEtkepsX73uYDIO+OvEzMwl0CV2ag94sCGa8MJct50KImvaQaT/aukWfDE/L1TiDSMRSRL+6y2budd21Tl2R8lObMq6tY7UJ4yIctkKjHsO1+rWH/pYH4g+yAETvwX0ilezVCf3PiLHu/pV+chiJ6OCtU+d+j3K5qntXwlg1oH8= ceph-f60e35da-e146-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-13T14:37:59.451 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:37:59 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3108563461' entity='client.admin' 2024-03-13T14:37:59.455 INFO:teuthology.orchestra.run.smithi176.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM6w9SmY/uZm/MhqTx30LsgAeaBlMz17hnsFOtvVM+8wiBdavRsbpozSMsAzY1gWiosu5BloSatJVSkxRA7Dwj/cc6YjyA55geevrCwz4QtCHImkxwsl81i8RPO+en9Y88RYeyzN3sZTqi07zdZnsuUTAIy6YgKotBWFTPzwMYGYEmlQmKior3ClgVYrXlU5g/DjCWNTahxEgET4Go2Vh62VMfrernf6tLz2RNmamGmOUeUyReyDrWkeoZT2MhN6IY+J9nbV26HIWl0e3pOeksHmS8dQaki9zaEbDeac5V4thPnik7uV4y/O5xV5YlIdH9Rxc1qU9uRHxoY4EVwUNNkEtkepsX73uYDIO+OvEzMwl0CV2ag94sCGa8MJct50KImvaQaT/aukWfDE/L1TiDSMRSRL+6y2budd21Tl2R8lObMq6tY7UJ4yIctkKjHsO1+rWH/pYH4g+yAETvwX0ilezVCf3PiLHu/pV+chiJ6OCtU+d+j3K5qntXwlg1oH8= ceph-f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:37:59.500 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM6w9SmY/uZm/MhqTx30LsgAeaBlMz17hnsFOtvVM+8wiBdavRsbpozSMsAzY1gWiosu5BloSatJVSkxRA7Dwj/cc6YjyA55geevrCwz4QtCHImkxwsl81i8RPO+en9Y88RYeyzN3sZTqi07zdZnsuUTAIy6YgKotBWFTPzwMYGYEmlQmKior3ClgVYrXlU5g/DjCWNTahxEgET4Go2Vh62VMfrernf6tLz2RNmamGmOUeUyReyDrWkeoZT2MhN6IY+J9nbV26HIWl0e3pOeksHmS8dQaki9zaEbDeac5V4thPnik7uV4y/O5xV5YlIdH9Rxc1qU9uRHxoY4EVwUNNkEtkepsX73uYDIO+OvEzMwl0CV2ag94sCGa8MJct50KImvaQaT/aukWfDE/L1TiDSMRSRL+6y2budd21Tl2R8lObMq6tY7UJ4yIctkKjHsO1+rWH/pYH4g+yAETvwX0ilezVCf3PiLHu/pV+chiJ6OCtU+d+j3K5qntXwlg1oH8= ceph-f60e35da-e146-11ee-95c9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-13T14:37:59.632 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM6w9SmY/uZm/MhqTx30LsgAeaBlMz17hnsFOtvVM+8wiBdavRsbpozSMsAzY1gWiosu5BloSatJVSkxRA7Dwj/cc6YjyA55geevrCwz4QtCHImkxwsl81i8RPO+en9Y88RYeyzN3sZTqi07zdZnsuUTAIy6YgKotBWFTPzwMYGYEmlQmKior3ClgVYrXlU5g/DjCWNTahxEgET4Go2Vh62VMfrernf6tLz2RNmamGmOUeUyReyDrWkeoZT2MhN6IY+J9nbV26HIWl0e3pOeksHmS8dQaki9zaEbDeac5V4thPnik7uV4y/O5xV5YlIdH9Rxc1qU9uRHxoY4EVwUNNkEtkepsX73uYDIO+OvEzMwl0CV2ag94sCGa8MJct50KImvaQaT/aukWfDE/L1TiDSMRSRL+6y2budd21Tl2R8lObMq6tY7UJ4yIctkKjHsO1+rWH/pYH4g+yAETvwX0ilezVCf3PiLHu/pV+chiJ6OCtU+d+j3K5qntXwlg1oH8= ceph-f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:37:59.697 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-13T14:38:00.062 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:38:02.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.533+0000 7f174d000700 1 -- 172.21.15.176:0/2776486111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 msgr2=0x7f1748100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:02.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.533+0000 7f174d000700 1 --2- 172.21.15.176:0/2776486111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748100cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1738009910 tx=0x7f1738009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:02.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.533+0000 7f174d000700 1 -- 172.21.15.176:0/2776486111 shutdown_connections 2024-03-13T14:38:02.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.533+0000 7f174d000700 1 --2- 172.21.15.176:0/2776486111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748100cd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:02.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.533+0000 7f174d000700 1 -- 172.21.15.176:0/2776486111 >> 172.21.15.176:0/2776486111 conn(0x7f17480fbe30 msgr2=0x7f17480fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:02.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.533+0000 7f174d000700 1 -- 172.21.15.176:0/2776486111 shutdown_connections 2024-03-13T14:38:02.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.533+0000 7f174d000700 1 -- 172.21.15.176:0/2776486111 wait complete. 2024-03-13T14:38:02.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.534+0000 7f174d000700 1 Processor -- start 2024-03-13T14:38:02.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.534+0000 7f174d000700 1 -- start start 2024-03-13T14:38:02.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.534+0000 7f174d000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:02.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.534+0000 7f174d000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748198120 con 0x7f17481008b0 2024-03-13T14:38:02.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.535+0000 7f1745e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:02.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.535+0000 7f1745e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48894/0 (socket says 172.21.15.176:48894) 2024-03-13T14:38:02.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.535+0000 7f1745e00700 1 -- 172.21.15.176:0/917721153 learned_addr learned my addr 172.21.15.176:0/917721153 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:38:02.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.535+0000 7f1745e00700 1 -- 172.21.15.176:0/917721153 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17380095f0 con 0x7f17481008b0 2024-03-13T14:38:02.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.535+0000 7f1745e00700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748197be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1738005b40 tx=0x7f1738004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:02.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.535+0000 7f173f400700 1 -- 172.21.15.176:0/917721153 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f173801d070 con 0x7f17481008b0 2024-03-13T14:38:02.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.536+0000 7f173f400700 1 -- 172.21.15.176:0/917721153 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f173800be10 con 0x7f17481008b0 2024-03-13T14:38:02.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.536+0000 7f173f400700 1 -- 172.21.15.176:0/917721153 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f173800f460 con 0x7f17481008b0 2024-03-13T14:38:02.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.536+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1748198320 con 0x7f17481008b0 2024-03-13T14:38:02.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.536+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17481987c0 con 0x7f17481008b0 2024-03-13T14:38:02.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.537+0000 7f173f400700 1 -- 172.21.15.176:0/917721153 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f17380227e0 con 0x7f17481008b0 2024-03-13T14:38:02.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.537+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17481918d0 con 0x7f17481008b0 2024-03-13T14:38:02.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.537+0000 7f173f400700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f172c038410 0x7f172c03a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:02.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.538+0000 7f173f400700 1 -- 172.21.15.176:0/917721153 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f173804c360 con 0x7f17481008b0 2024-03-13T14:38:02.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.538+0000 7f173d400700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f172c038410 0x7f172c03a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:02.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.538+0000 7f173d400700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f172c038410 0x7f172c03a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1730006fd0 tx=0x7f1730006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:02.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.543+0000 7f173f400700 1 -- 172.21.15.176:0/917721153 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f173802ba00 con 0x7f17481008b0 2024-03-13T14:38:02.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.769+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f174802cc70 con 0x7f17481008b0 2024-03-13T14:38:02.769 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.776+0000 7f173f400700 1 -- 172.21.15.176:0/917721153 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f1738027030 con 0x7f17481008b0 2024-03-13T14:38:02.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f172c038410 msgr2=0x7f172c03a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:02.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f172c038410 0x7f172c03a8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1730006fd0 tx=0x7f1730006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:02.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 msgr2=0x7f1748197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:02.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748197be0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1738005b40 tx=0x7f1738004740 comp rx=0 tx=0).stop 2024-03-13T14:38:02.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 shutdown_connections 2024-03-13T14:38:02.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f172c038410 0x7f172c03a8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:02.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 --2- 172.21.15.176:0/917721153 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f17481008b0 0x7f1748197be0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:02.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 >> 172.21.15.176:0/917721153 conn(0x7f17480fbe30 msgr2=0x7f17480fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:02.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 shutdown_connections 2024-03-13T14:38:02.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:02.780+0000 7f174d000700 1 -- 172.21.15.176:0/917721153 wait complete. 2024-03-13T14:38:03.435 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-13T14:38:03.435 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-13T14:38:03.796 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:38:04.075 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:03 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/917721153' entity='client.admin' 2024-03-13T14:38:06.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.056+0000 7febe5000700 1 -- 172.21.15.176:0/244975143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 msgr2=0x7febe0100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:06.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.056+0000 7febe5000700 1 --2- 172.21.15.176:0/244975143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0100cd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7febd0009910 tx=0x7febd0009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:06.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.056+0000 7febe5000700 1 -- 172.21.15.176:0/244975143 shutdown_connections 2024-03-13T14:38:06.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.056+0000 7febe5000700 1 --2- 172.21.15.176:0/244975143 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0100cd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:06.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.056+0000 7febe5000700 1 -- 172.21.15.176:0/244975143 >> 172.21.15.176:0/244975143 conn(0x7febe00fbe30 msgr2=0x7febe00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:06.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.056+0000 7febe5000700 1 -- 172.21.15.176:0/244975143 shutdown_connections 2024-03-13T14:38:06.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.056+0000 7febe5000700 1 -- 172.21.15.176:0/244975143 wait complete. 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.057+0000 7febe5000700 1 Processor -- start 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.057+0000 7febe5000700 1 -- start start 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.058+0000 7febe5000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.058+0000 7febe5000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe0198170 con 0x7febe01008b0 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.058+0000 7febdde00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.058+0000 7febdde00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48922/0 (socket says 172.21.15.176:48922) 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.058+0000 7febdde00700 1 -- 172.21.15.176:0/2315551587 learned_addr learned my addr 172.21.15.176:0/2315551587 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.058+0000 7febdde00700 1 -- 172.21.15.176:0/2315551587 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd00095f0 con 0x7febe01008b0 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.059+0000 7febdde00700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0197c30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7febd0005950 tx=0x7febd0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.059+0000 7febd6a00700 1 -- 172.21.15.176:0/2315551587 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd001d070 con 0x7febe01008b0 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.059+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe0198370 con 0x7febe01008b0 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.059+0000 7febd6a00700 1 -- 172.21.15.176:0/2315551587 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7febd000be10 con 0x7febe01008b0 2024-03-13T14:38:06.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.059+0000 7febd6a00700 1 -- 172.21.15.176:0/2315551587 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7febd000f460 con 0x7febe01008b0 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.059+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe0198810 con 0x7febe01008b0 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.060+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe01918d0 con 0x7febe01008b0 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.061+0000 7febd6a00700 1 -- 172.21.15.176:0/2315551587 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7febd00227c0 con 0x7febe01008b0 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.061+0000 7febd6a00700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7febc40383c0 0x7febc403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.061+0000 7febd6a00700 1 -- 172.21.15.176:0/2315551587 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7febd004c4e0 con 0x7febe01008b0 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.062+0000 7febdd400700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7febc40383c0 0x7febc403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.062+0000 7febdd400700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7febc40383c0 0x7febc403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febcc006fd0 tx=0x7febcc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:06.065 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.067+0000 7febd6a00700 1 -- 172.21.15.176:0/2315551587 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7febd002bba0 con 0x7febe01008b0 2024-03-13T14:38:06.288 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.294+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7febe0061260 con 0x7febc40383c0 2024-03-13T14:38:06.292 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.299+0000 7febd6a00700 1 -- 172.21.15.176:0/2315551587 <== mgr.14164 v2:172.21.15.176:6800/456564682 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7febe0061260 con 0x7febc40383c0 2024-03-13T14:38:06.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7febc40383c0 msgr2=0x7febc403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:06.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7febc40383c0 0x7febc403a880 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febcc006fd0 tx=0x7febcc006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:06.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 msgr2=0x7febe0197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:06.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0197c30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7febd0005950 tx=0x7febd0009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:06.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 shutdown_connections 2024-03-13T14:38:06.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7febc40383c0 0x7febc403a880 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:06.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 --2- 172.21.15.176:0/2315551587 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7febe01008b0 0x7febe0197c30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:06.296 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 >> 172.21.15.176:0/2315551587 conn(0x7febe00fbe30 msgr2=0x7febe00fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:06.296 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 shutdown_connections 2024-03-13T14:38:06.296 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:06.302+0000 7febe5000700 1 -- 172.21.15.176:0/2315551587 wait complete. 2024-03-13T14:38:06.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:06.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:06.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:38:06.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:06.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:38:06.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-13T14:38:06.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:38:06.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: Deploying daemon ceph-exporter.smithi176 on smithi176 2024-03-13T14:38:06.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:06 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:07.178 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-03-13T14:38:07.178 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:38:07.178 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-03-13T14:38:07.249 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:38:07.249 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:38:07.355 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-03-13T14:38:07.355 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch host add smithi196 2024-03-13T14:38:07.818 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:07 smithi176 ceph-mon[29064]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:38:08.237 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:38:08.802 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:08 smithi176 ceph-mon[29064]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.490+0000 7f595ec00700 1 -- 172.21.15.176:0/874316564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 msgr2=0x7f5958100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.490+0000 7f595ec00700 1 --2- 172.21.15.176:0/874316564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f5958100cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5948009910 tx=0x7f5948009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.490+0000 7f595ec00700 1 -- 172.21.15.176:0/874316564 shutdown_connections 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.490+0000 7f595ec00700 1 --2- 172.21.15.176:0/874316564 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f5958100cd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.490+0000 7f595ec00700 1 -- 172.21.15.176:0/874316564 >> 172.21.15.176:0/874316564 conn(0x7f59580fbe30 msgr2=0x7f59580fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.491+0000 7f595ec00700 1 -- 172.21.15.176:0/874316564 shutdown_connections 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.491+0000 7f595ec00700 1 -- 172.21.15.176:0/874316564 wait complete. 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.492+0000 7f595ec00700 1 Processor -- start 2024-03-13T14:38:10.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.492+0000 7f595ec00700 1 -- start start 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.492+0000 7f595ec00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f59581959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.492+0000 7f595ec00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5958195f20 con 0x7f59581008b0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.492+0000 7f5957e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f59581959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.492+0000 7f5957e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f59581959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:48946/0 (socket says 172.21.15.176:48946) 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.492+0000 7f5957e00700 1 -- 172.21.15.176:0/2358597776 learned_addr learned my addr 172.21.15.176:0/2358597776 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.493+0000 7f5957e00700 1 -- 172.21.15.176:0/2358597776 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480095f0 con 0x7f59581008b0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.493+0000 7f5957e00700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f59581959e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5948005950 tx=0x7f5948009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.493+0000 7f5954c00700 1 -- 172.21.15.176:0/2358597776 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f594801d070 con 0x7f59581008b0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.493+0000 7f5954c00700 1 -- 172.21.15.176:0/2358597776 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f594800be10 con 0x7f59581008b0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.493+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5958196120 con 0x7f59581008b0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.493+0000 7f5954c00700 1 -- 172.21.15.176:0/2358597776 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f594800f460 con 0x7f59581008b0 2024-03-13T14:38:10.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.494+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581965c0 con 0x7f59581008b0 2024-03-13T14:38:10.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.495+0000 7f5954c00700 1 -- 172.21.15.176:0/2358597776 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f594800f690 con 0x7f59581008b0 2024-03-13T14:38:10.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.495+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595818f4d0 con 0x7f59581008b0 2024-03-13T14:38:10.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.495+0000 7f5954c00700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f593c0383c0 0x7f593c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:10.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.495+0000 7f5954c00700 1 -- 172.21.15.176:0/2358597776 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f594804c600 con 0x7f59581008b0 2024-03-13T14:38:10.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.496+0000 7f5957400700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f593c0383c0 0x7f593c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:10.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.496+0000 7f5957400700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f593c0383c0 0x7f593c03a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5944006fd0 tx=0x7f5944006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:10.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.501+0000 7f5954c00700 1 -- 172.21.15.176:0/2358597776 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f594802a360 con 0x7f59581008b0 2024-03-13T14:38:10.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:10.733+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}) v1 -- 0x7f59580611d0 con 0x7f593c0383c0 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-13T14:38:11.096 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:10 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:38:12.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:11 smithi176 ceph-mon[29064]: Deploying daemon crash.smithi176 on smithi176 2024-03-13T14:38:12.058 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:11 smithi176 ceph-mon[29064]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:38:12.807 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:12 smithi176 ceph-mon[29064]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:12.807 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:12 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:14.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:13 smithi176 ceph-mon[29064]: Deploying cephadm binary to smithi196 2024-03-13T14:38:14.995 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:14 smithi176 ceph-mon[29064]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:14.996 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:14 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:14.996 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:14 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:14.996 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:14 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:14.996 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:14 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:15.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.371+0000 7f5954c00700 1 -- 172.21.15.176:0/2358597776 <== mgr.14164 v2:172.21.15.176:6800/456564682 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f59580611d0 con 0x7f593c0383c0 2024-03-13T14:38:15.367 INFO:teuthology.orchestra.run.smithi176.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-03-13T14:38:15.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f593c0383c0 msgr2=0x7f593c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:15.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f593c0383c0 0x7f593c03a880 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5944006fd0 tx=0x7f5944006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:15.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 msgr2=0x7f59581959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:15.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f59581959e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5948005950 tx=0x7f5948009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:15.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 shutdown_connections 2024-03-13T14:38:15.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f593c0383c0 0x7f593c03a880 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:15.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 --2- 172.21.15.176:0/2358597776 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59581008b0 0x7f59581959e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:15.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 >> 172.21.15.176:0/2358597776 conn(0x7f59580fbe30 msgr2=0x7f59580fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:15.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 shutdown_connections 2024-03-13T14:38:15.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:15.375+0000 7f595ec00700 1 -- 172.21.15.176:0/2358597776 wait complete. 2024-03-13T14:38:16.108 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:15 smithi176 ceph-mon[29064]: Deploying daemon node-exporter.smithi176 on smithi176 2024-03-13T14:38:16.109 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:15 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:16.163 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch host ls --format=json 2024-03-13T14:38:16.541 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:38:16.946 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:16 smithi176 ceph-mon[29064]: Added host smithi196 2024-03-13T14:38:16.946 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:16 smithi176 ceph-mon[29064]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:18.850 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:18 smithi176 ceph-mon[29064]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:18.851 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:18 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:19.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.412+0000 7fec77000700 1 -- 172.21.15.176:0/2322673310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 msgr2=0x7fec70102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:19.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.412+0000 7fec77000700 1 --2- 172.21.15.176:0/2322673310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70102eb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec60009910 tx=0x7fec60009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:19.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.412+0000 7fec77000700 1 -- 172.21.15.176:0/2322673310 shutdown_connections 2024-03-13T14:38:19.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.412+0000 7fec77000700 1 --2- 172.21.15.176:0/2322673310 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70102eb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:19.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.412+0000 7fec77000700 1 -- 172.21.15.176:0/2322673310 >> 172.21.15.176:0/2322673310 conn(0x7fec700fe010 msgr2=0x7fec70100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:19.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.412+0000 7fec77000700 1 -- 172.21.15.176:0/2322673310 shutdown_connections 2024-03-13T14:38:19.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.412+0000 7fec77000700 1 -- 172.21.15.176:0/2322673310 wait complete. 2024-03-13T14:38:19.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.414+0000 7fec77000700 1 Processor -- start 2024-03-13T14:38:19.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.414+0000 7fec77000700 1 -- start start 2024-03-13T14:38:19.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.414+0000 7fec77000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:19.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.414+0000 7fec77000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec70198120 con 0x7fec70102a90 2024-03-13T14:38:19.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.414+0000 7fec6fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:19.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.414+0000 7fec6fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70197be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:33244/0 (socket says 172.21.15.176:33244) 2024-03-13T14:38:19.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.414+0000 7fec6fe00700 1 -- 172.21.15.176:0/906827499 learned_addr learned my addr 172.21.15.176:0/906827499 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:38:19.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.415+0000 7fec6fe00700 1 -- 172.21.15.176:0/906827499 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600095f0 con 0x7fec70102a90 2024-03-13T14:38:19.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.415+0000 7fec6fe00700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70197be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec60005950 tx=0x7fec60009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:19.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.415+0000 7fec6cc00700 1 -- 172.21.15.176:0/906827499 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec6001d070 con 0x7fec70102a90 2024-03-13T14:38:19.409 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.415+0000 7fec6cc00700 1 -- 172.21.15.176:0/906827499 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec6000be10 con 0x7fec70102a90 2024-03-13T14:38:19.409 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.415+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec70198320 con 0x7fec70102a90 2024-03-13T14:38:19.409 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.415+0000 7fec6cc00700 1 -- 172.21.15.176:0/906827499 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fec6000f460 con 0x7fec70102a90 2024-03-13T14:38:19.409 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.416+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec701987c0 con 0x7fec70102a90 2024-03-13T14:38:19.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.417+0000 7fec6cc00700 1 -- 172.21.15.176:0/906827499 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fec6000b960 con 0x7fec70102a90 2024-03-13T14:38:19.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.417+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec701916b0 con 0x7fec70102a90 2024-03-13T14:38:19.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.418+0000 7fec6cc00700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fec540383c0 0x7fec5403a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:19.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.418+0000 7fec6cc00700 1 -- 172.21.15.176:0/906827499 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fec6004c5e0 con 0x7fec70102a90 2024-03-13T14:38:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.418+0000 7fec6f400700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fec540383c0 0x7fec5403a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.418+0000 7fec6f400700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fec540383c0 0x7fec5403a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006fd0 tx=0x7fec5c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:19.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.423+0000 7fec6cc00700 1 -- 172.21.15.176:0/906827499 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fec6002bd00 con 0x7fec70102a90 2024-03-13T14:38:19.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.646+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec700611d0 con 0x7fec540383c0 2024-03-13T14:38:19.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.649+0000 7fec6cc00700 1 -- 172.21.15.176:0/906827499 <== mgr.14164 v2:172.21.15.176:6800/456564682 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fec700611d0 con 0x7fec540383c0 2024-03-13T14:38:19.642 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:38:19.642 INFO:teuthology.orchestra.run.smithi176.stdout:[{"addr": "172.21.15.176", "hostname": "smithi176", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fec540383c0 msgr2=0x7fec5403a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fec540383c0 0x7fec5403a880 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fec5c006fd0 tx=0x7fec5c006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 msgr2=0x7fec70197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70197be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec60005950 tx=0x7fec60009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 shutdown_connections 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fec540383c0 0x7fec5403a880 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 --2- 172.21.15.176:0/906827499 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec70102a90 0x7fec70197be0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:19.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 >> 172.21.15.176:0/906827499 conn(0x7fec700fe010 msgr2=0x7fec700fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:19.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 shutdown_connections 2024-03-13T14:38:19.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:19.652+0000 7fec77000700 1 -- 172.21.15.176:0/906827499 wait complete. 2024-03-13T14:38:20.826 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-13T14:38:20.826 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd crush tunables default 2024-03-13T14:38:20.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:20 smithi176 ceph-mon[29064]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:20.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:20 smithi176 ceph-mon[29064]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-13T14:38:21.242 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:38:22.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:22 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:22.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:22 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:22.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:22 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:22.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:22 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:22.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:22 smithi176 ceph-mon[29064]: Deploying daemon alertmanager.smithi176 on smithi176 2024-03-13T14:38:22.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:22 smithi176 ceph-mon[29064]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:23.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.197+0000 7f6464600700 1 -- 172.21.15.176:0/2057522907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 msgr2=0x7f645c1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:23.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.197+0000 7f6464600700 1 --2- 172.21.15.176:0/2057522907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c1073c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6450009910 tx=0x7f6450009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:23.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.197+0000 7f6464600700 1 -- 172.21.15.176:0/2057522907 shutdown_connections 2024-03-13T14:38:23.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.197+0000 7f6464600700 1 --2- 172.21.15.176:0/2057522907 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c1073c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:23.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.197+0000 7f6464600700 1 -- 172.21.15.176:0/2057522907 >> 172.21.15.176:0/2057522907 conn(0x7f645c075f00 msgr2=0x7f645c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:23.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.197+0000 7f6464600700 1 -- 172.21.15.176:0/2057522907 shutdown_connections 2024-03-13T14:38:23.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.197+0000 7f6464600700 1 -- 172.21.15.176:0/2057522907 wait complete. 2024-03-13T14:38:23.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.198+0000 7f6464600700 1 Processor -- start 2024-03-13T14:38:23.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.199+0000 7f6464600700 1 -- start start 2024-03-13T14:38:23.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.199+0000 7f6464600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:23.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.199+0000 7f6464600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c19c490 con 0x7f645c106fa0 2024-03-13T14:38:23.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.199+0000 7f6461c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:23.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.199+0000 7f6461c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:49216/0 (socket says 172.21.15.176:49216) 2024-03-13T14:38:23.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.199+0000 7f6461c00700 1 -- 172.21.15.176:0/3341883443 learned_addr learned my addr 172.21.15.176:0/3341883443 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:38:23.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.200+0000 7f6461c00700 1 -- 172.21.15.176:0/3341883443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64500095f0 con 0x7f645c106fa0 2024-03-13T14:38:23.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.200+0000 7f6461c00700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c19bf50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64500092e0 tx=0x7f6450009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:23.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.200+0000 7f645a000700 1 -- 172.21.15.176:0/3341883443 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f645001d070 con 0x7f645c106fa0 2024-03-13T14:38:23.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.200+0000 7f645a000700 1 -- 172.21.15.176:0/3341883443 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f645000be10 con 0x7f645c106fa0 2024-03-13T14:38:23.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.200+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c19c690 con 0x7f645c106fa0 2024-03-13T14:38:23.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.200+0000 7f645a000700 1 -- 172.21.15.176:0/3341883443 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f645000f460 con 0x7f645c106fa0 2024-03-13T14:38:23.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.200+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c19cb30 con 0x7f645c106fa0 2024-03-13T14:38:23.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.201+0000 7f645a000700 1 -- 172.21.15.176:0/3341883443 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f645000b960 con 0x7f645c106fa0 2024-03-13T14:38:23.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.201+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c195a20 con 0x7f645c106fa0 2024-03-13T14:38:23.195 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.202+0000 7f645a000700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f6444038410 0x7f644403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:23.195 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.202+0000 7f645a000700 1 -- 172.21.15.176:0/3341883443 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f645004c5b0 con 0x7f645c106fa0 2024-03-13T14:38:23.195 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.202+0000 7f6461200700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f6444038410 0x7f644403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:23.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.203+0000 7f6461200700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f6444038410 0x7f644403a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6448006fd0 tx=0x7f6448006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:23.200 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.207+0000 7f645a000700 1 -- 172.21.15.176:0/3341883443 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f645000f5c0 con 0x7f645c106fa0 2024-03-13T14:38:23.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.427+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f645c02cc70 con 0x7f645c106fa0 2024-03-13T14:38:23.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.585+0000 7f645a000700 1 -- 172.21.15.176:0/3341883443 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6450027070 con 0x7f645c106fa0 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.588+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f6444038410 msgr2=0x7f644403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.588+0000 7f6464600700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f6444038410 0x7f644403a8d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6448006fd0 tx=0x7f6448006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.588+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 msgr2=0x7f645c19bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.588+0000 7f6464600700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c19bf50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64500092e0 tx=0x7f6450009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.589+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 shutdown_connections 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.589+0000 7f6464600700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f6444038410 0x7f644403a8d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.589+0000 7f6464600700 1 --2- 172.21.15.176:0/3341883443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f645c106fa0 0x7f645c19bf50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.589+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 >> 172.21.15.176:0/3341883443 conn(0x7f645c075f00 msgr2=0x7f645c076be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:23.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.589+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 shutdown_connections 2024-03-13T14:38:23.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:38:23.589+0000 7f6464600700 1 -- 172.21.15.176:0/3341883443 wait complete. 2024-03-13T14:38:23.583 INFO:teuthology.orchestra.run.smithi176.stderr:adjusted tunables profile to default 2024-03-13T14:38:23.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:23 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:23.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:23 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3341883443' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-13T14:38:24.564 INFO:tasks.cephadm:Adding mon.smithi176 on smithi176 2024-03-13T14:38:24.564 INFO:tasks.cephadm:Adding mon.smithi196 on smithi196 2024-03-13T14:38:24.564 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch apply mon '2;smithi176:172.21.15.176=smithi176;smithi196:172.21.15.196=smithi196' 2024-03-13T14:38:24.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:24 smithi176 ceph-mon[29064]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:24.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:24 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3341883443' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-13T14:38:24.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:24 smithi176 ceph-mon[29064]: osdmap e4: 0 total, 0 up, 0 in 2024-03-13T14:38:24.940 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:25.037 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:27.121 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:26 smithi176 ceph-mon[29064]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:27.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.165+0000 7f5123400700 1 -- 172.21.15.196:0/2611454433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 msgr2=0x7f511c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:27.158 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.165+0000 7f5123400700 1 --2- 172.21.15.196:0/2611454433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c102eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5110009910 tx=0x7f5110009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:27.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.166+0000 7f5123400700 1 -- 172.21.15.196:0/2611454433 shutdown_connections 2024-03-13T14:38:27.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.166+0000 7f5123400700 1 --2- 172.21.15.196:0/2611454433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c102eb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:27.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.166+0000 7f5123400700 1 -- 172.21.15.196:0/2611454433 >> 172.21.15.196:0/2611454433 conn(0x7f511c0fe010 msgr2=0x7f511c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:27.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.166+0000 7f5123400700 1 -- 172.21.15.196:0/2611454433 shutdown_connections 2024-03-13T14:38:27.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.166+0000 7f5123400700 1 -- 172.21.15.196:0/2611454433 wait complete. 2024-03-13T14:38:27.159 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.167+0000 7f5123400700 1 Processor -- start 2024-03-13T14:38:27.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.167+0000 7f5123400700 1 -- start start 2024-03-13T14:38:27.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.168+0000 7f5123400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:27.160 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.168+0000 7f5123400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1980f0 con 0x7f511c102a90 2024-03-13T14:38:27.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.168+0000 7f5120a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:27.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.169+0000 7f5120a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:45794/0 (socket says 172.21.15.196:45794) 2024-03-13T14:38:27.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.169+0000 7f5120a00700 1 -- 172.21.15.196:0/429003321 learned_addr learned my addr 172.21.15.196:0/429003321 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:27.161 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.169+0000 7f5120a00700 1 -- 172.21.15.196:0/429003321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100095f0 con 0x7f511c102a90 2024-03-13T14:38:27.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.170+0000 7f5120a00700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c197bb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5110005950 tx=0x7f5110009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:27.162 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.170+0000 7f5119600700 1 -- 172.21.15.196:0/429003321 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f511001d070 con 0x7f511c102a90 2024-03-13T14:38:27.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.170+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c1982f0 con 0x7f511c102a90 2024-03-13T14:38:27.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.170+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c198790 con 0x7f511c102a90 2024-03-13T14:38:27.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.170+0000 7f5119600700 1 -- 172.21.15.196:0/429003321 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5110005c00 con 0x7f511c102a90 2024-03-13T14:38:27.163 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.170+0000 7f5119600700 1 -- 172.21.15.196:0/429003321 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f511000fbb0 con 0x7f511c102a90 2024-03-13T14:38:27.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.171+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc005320 con 0x7f511c102a90 2024-03-13T14:38:27.164 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.172+0000 7f5119600700 1 -- 172.21.15.196:0/429003321 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f511000fd10 con 0x7f511c102a90 2024-03-13T14:38:27.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.172+0000 7f5119600700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5104038410 0x7f510403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:27.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.172+0000 7f5119600700 1 -- 172.21.15.196:0/429003321 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f511004c240 con 0x7f511c102a90 2024-03-13T14:38:27.165 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.173+0000 7f511be00700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5104038410 0x7f510403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:27.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.174+0000 7f511be00700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5104038410 0x7f510403a8d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5108006fd0 tx=0x7f5108006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:27.180 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.187+0000 7f5119600700 1 -- 172.21.15.196:0/429003321 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f511000b960 con 0x7f511c102a90 2024-03-13T14:38:27.403 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.411+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 --> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi176:172.21.15.176=smithi176;smithi196:172.21.15.196=smithi196", "target": ["mon-mgr", ""]}) v1 -- 0x7f50fc000ca0 con 0x7f5104038410 2024-03-13T14:38:27.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.426+0000 7f5119600700 1 -- 172.21.15.196:0/429003321 <== mgr.14164 v2:172.21.15.176:6800/456564682 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f50fc000ca0 con 0x7f5104038410 2024-03-13T14:38:27.419 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5104038410 msgr2=0x7f510403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5104038410 0x7f510403a8d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5108006fd0 tx=0x7f5108006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 msgr2=0x7f511c197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c197bb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5110005950 tx=0x7f5110009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 shutdown_connections 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f5104038410 0x7f510403a8d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 --2- 172.21.15.196:0/429003321 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f511c102a90 0x7f511c197bb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:27.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 >> 172.21.15.196:0/429003321 conn(0x7f511c0fe010 msgr2=0x7f511c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:27.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.429+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 shutdown_connections 2024-03-13T14:38:27.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:27.430+0000 7f5123400700 1 -- 172.21.15.196:0/429003321 wait complete. 2024-03-13T14:38:28.083 DEBUG:teuthology.orchestra.run.smithi196:mon.smithi196> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi196.service 2024-03-13T14:38:28.088 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:28.088 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:28.470 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:28.570 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:28.603 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:28 smithi176 ceph-mon[29064]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi176:172.21.15.176=smithi176;smithi196:172.21.15.196=smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:38:28.603 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:28 smithi176 ceph-mon[29064]: Saving service mon spec with placement smithi176:172.21.15.176=smithi176;smithi196:172.21.15.196=smithi196;count:2 2024-03-13T14:38:28.603 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:28 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:28.603 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:28 smithi176 ceph-mon[29064]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:30.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.326+0000 7f3946000700 1 -- 172.21.15.196:0/694316896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 msgr2=0x7f3940073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:30.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.326+0000 7f3946000700 1 --2- 172.21.15.196:0/694316896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f3940073220 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3934009910 tx=0x7f3934009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:30.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.327+0000 7f3946000700 1 -- 172.21.15.196:0/694316896 shutdown_connections 2024-03-13T14:38:30.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.327+0000 7f3946000700 1 --2- 172.21.15.196:0/694316896 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f3940073220 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:30.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.327+0000 7f3946000700 1 -- 172.21.15.196:0/694316896 >> 172.21.15.196:0/694316896 conn(0x7f39400fbc70 msgr2=0x7f39400fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:30.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.327+0000 7f3946000700 1 -- 172.21.15.196:0/694316896 shutdown_connections 2024-03-13T14:38:30.320 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.327+0000 7f3946000700 1 -- 172.21.15.196:0/694316896 wait complete. 2024-03-13T14:38:30.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.328+0000 7f3946000700 1 Processor -- start 2024-03-13T14:38:30.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.328+0000 7f3946000700 1 -- start start 2024-03-13T14:38:30.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.329+0000 7f3946000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f39401979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:30.321 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.329+0000 7f3946000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3940197f30 con 0x7f3940074dc0 2024-03-13T14:38:30.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.329+0000 7f3944c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f39401979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:30.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.329+0000 7f3944c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f39401979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:45810/0 (socket says 172.21.15.196:45810) 2024-03-13T14:38:30.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.329+0000 7f3944c00700 1 -- 172.21.15.196:0/2329302904 learned_addr learned my addr 172.21.15.196:0/2329302904 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:30.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.329+0000 7f3944c00700 1 -- 172.21.15.196:0/2329302904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39340095f0 con 0x7f3940074dc0 2024-03-13T14:38:30.322 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.330+0000 7f3944c00700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f39401979f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3934005950 tx=0x7f3934009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:30.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.330+0000 7f393d600700 1 -- 172.21.15.196:0/2329302904 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f393401d070 con 0x7f3940074dc0 2024-03-13T14:38:30.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.330+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3940198130 con 0x7f3940074dc0 2024-03-13T14:38:30.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.330+0000 7f393d600700 1 -- 172.21.15.196:0/2329302904 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f393400be10 con 0x7f3940074dc0 2024-03-13T14:38:30.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.330+0000 7f393d600700 1 -- 172.21.15.196:0/2329302904 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f393400f460 con 0x7f3940074dc0 2024-03-13T14:38:30.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.330+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39401985d0 con 0x7f3940074dc0 2024-03-13T14:38:30.323 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.331+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39401914e0 con 0x7f3940074dc0 2024-03-13T14:38:30.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.331+0000 7f393d600700 1 -- 172.21.15.196:0/2329302904 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f39340227c0 con 0x7f3940074dc0 2024-03-13T14:38:30.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.332+0000 7f393d600700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f3928038410 0x7f392803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:30.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.332+0000 7f393d600700 1 -- 172.21.15.196:0/2329302904 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f393404c470 con 0x7f3940074dc0 2024-03-13T14:38:30.324 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.332+0000 7f393fe00700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f3928038410 0x7f392803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:30.325 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.333+0000 7f393fe00700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f3928038410 0x7f392803a8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f392c006fd0 tx=0x7f392c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:30.332 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.340+0000 7f393d600700 1 -- 172.21.15.196:0/2329302904 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f393402ba00 con 0x7f3940074dc0 2024-03-13T14:38:30.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.635+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f394002cc70 con 0x7f3940074dc0 2024-03-13T14:38:30.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.636+0000 7f393d600700 1 -- 172.21.15.196:0/2329302904 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3934027030 con 0x7f3940074dc0 2024-03-13T14:38:30.628 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:38:30.628 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:38:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f3928038410 msgr2=0x7f392803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f3928038410 0x7f392803a8d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f392c006fd0 tx=0x7f392c006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:30.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 msgr2=0x7f39401979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f39401979f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3934005950 tx=0x7f3934009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 shutdown_connections 2024-03-13T14:38:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f3928038410 0x7f392803a8d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 --2- 172.21.15.196:0/2329302904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3940074dc0 0x7f39401979f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 >> 172.21.15.196:0/2329302904 conn(0x7f39400fbc70 msgr2=0x7f39400fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.638+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 shutdown_connections 2024-03-13T14:38:30.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:30.639+0000 7f3946000700 1 -- 172.21.15.196:0/2329302904 wait complete. 2024-03-13T14:38:30.632 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:38:30.994 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:30 smithi176 ceph-mon[29064]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:30.994 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:30 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2329302904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:38:32.566 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:32.566 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:32.938 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:33.029 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.135 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-13T14:38:33.135 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.135 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:32 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:38:33.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:33 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-13T14:38:33.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:33 smithi176 ceph-mon[29064]: Deploying daemon grafana.smithi176 on smithi176 2024-03-13T14:38:35.062 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:34 smithi176 ceph-mon[29064]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:35.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.192+0000 7f9c23400700 1 -- 172.21.15.196:0/1468734103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 msgr2=0x7f9c1c0fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:35.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.192+0000 7f9c23400700 1 --2- 172.21.15.196:0/1468734103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c0fef80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009910 tx=0x7f9c10009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:35.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.192+0000 7f9c23400700 1 -- 172.21.15.196:0/1468734103 shutdown_connections 2024-03-13T14:38:35.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.192+0000 7f9c23400700 1 --2- 172.21.15.196:0/1468734103 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c0fef80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:35.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.192+0000 7f9c23400700 1 -- 172.21.15.196:0/1468734103 >> 172.21.15.196:0/1468734103 conn(0x7f9c1c0fa4a0 msgr2=0x7f9c1c0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:35.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.192+0000 7f9c23400700 1 -- 172.21.15.196:0/1468734103 shutdown_connections 2024-03-13T14:38:35.185 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.192+0000 7f9c23400700 1 -- 172.21.15.196:0/1468734103 wait complete. 2024-03-13T14:38:35.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.193+0000 7f9c23400700 1 Processor -- start 2024-03-13T14:38:35.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.194+0000 7f9c23400700 1 -- start start 2024-03-13T14:38:35.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.194+0000 7f9c23400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:35.186 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.194+0000 7f9c23400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c198110 con 0x7f9c1c0feb60 2024-03-13T14:38:35.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.194+0000 7f9c20a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:35.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.195+0000 7f9c20a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:43814/0 (socket says 172.21.15.196:43814) 2024-03-13T14:38:35.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.195+0000 7f9c20a00700 1 -- 172.21.15.196:0/774259769 learned_addr learned my addr 172.21.15.196:0/774259769 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:35.187 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.195+0000 7f9c20a00700 1 -- 172.21.15.196:0/774259769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c100095f0 con 0x7f9c1c0feb60 2024-03-13T14:38:35.188 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.196+0000 7f9c20a00700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c197bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005950 tx=0x7f9c10009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:35.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.196+0000 7f9c19600700 1 -- 172.21.15.196:0/774259769 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c1001d070 con 0x7f9c1c0feb60 2024-03-13T14:38:35.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.196+0000 7f9c19600700 1 -- 172.21.15.196:0/774259769 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9c1000be10 con 0x7f9c1c0feb60 2024-03-13T14:38:35.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.196+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1c198310 con 0x7f9c1c0feb60 2024-03-13T14:38:35.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.196+0000 7f9c19600700 1 -- 172.21.15.196:0/774259769 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9c1000f460 con 0x7f9c1c0feb60 2024-03-13T14:38:35.189 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.197+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c1987b0 con 0x7f9c1c0feb60 2024-03-13T14:38:35.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.198+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1c1918d0 con 0x7f9c1c0feb60 2024-03-13T14:38:35.190 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.198+0000 7f9c19600700 1 -- 172.21.15.196:0/774259769 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f9c100227c0 con 0x7f9c1c0feb60 2024-03-13T14:38:35.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.199+0000 7f9c19600700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9c04038410 0x7f9c0403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:35.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.199+0000 7f9c19600700 1 -- 172.21.15.196:0/774259769 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9c1004c220 con 0x7f9c1c0feb60 2024-03-13T14:38:35.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.199+0000 7f9c1be00700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9c04038410 0x7f9c0403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:35.192 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.200+0000 7f9c1be00700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9c04038410 0x7f9c0403a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006fd0 tx=0x7f9c08006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:35.205 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.213+0000 7f9c19600700 1 -- 172.21.15.196:0/774259769 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9c1002ba00 con 0x7f9c1c0feb60 2024-03-13T14:38:35.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.485+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9c1c0ff690 con 0x7f9c1c0feb60 2024-03-13T14:38:35.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.486+0000 7f9c19600700 1 -- 172.21.15.196:0/774259769 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9c1c0ff690 con 0x7f9c1c0feb60 2024-03-13T14:38:35.479 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:38:35.479 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:38:35.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9c04038410 msgr2=0x7f9c0403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9c04038410 0x7f9c0403a8d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006fd0 tx=0x7f9c08006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 msgr2=0x7f9c1c197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c197bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9c10005950 tx=0x7f9c10009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 shutdown_connections 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9c04038410 0x7f9c0403a8d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 --2- 172.21.15.196:0/774259769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9c1c0feb60 0x7f9c1c197bd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 >> 172.21.15.196:0/774259769 conn(0x7f9c1c0fa4a0 msgr2=0x7f9c1c0fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.489+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 shutdown_connections 2024-03-13T14:38:35.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:35.490+0000 7f9c23400700 1 -- 172.21.15.196:0/774259769 wait complete. 2024-03-13T14:38:35.486 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:38:36.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:35 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/774259769' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:38:37.035 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:36 smithi176 ceph-mon[29064]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:37.516 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:37.516 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:37.877 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:37.974 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:38.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:38 smithi176 ceph-mon[29064]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:39.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.949+0000 7fd747a00700 1 -- 172.21.15.196:0/4289807775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 msgr2=0x7fd7401073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:39.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.949+0000 7fd747a00700 1 --2- 172.21.15.196:0/4289807775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd7401073c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd734009910 tx=0x7fd734009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:39.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.949+0000 7fd747a00700 1 -- 172.21.15.196:0/4289807775 shutdown_connections 2024-03-13T14:38:39.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.949+0000 7fd747a00700 1 --2- 172.21.15.196:0/4289807775 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd7401073c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:39.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.949+0000 7fd747a00700 1 -- 172.21.15.196:0/4289807775 >> 172.21.15.196:0/4289807775 conn(0x7fd740075ff0 msgr2=0x7fd740078450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:39.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.949+0000 7fd747a00700 1 -- 172.21.15.196:0/4289807775 shutdown_connections 2024-03-13T14:38:39.942 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.950+0000 7fd747a00700 1 -- 172.21.15.196:0/4289807775 wait complete. 2024-03-13T14:38:39.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.951+0000 7fd747a00700 1 Processor -- start 2024-03-13T14:38:39.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.951+0000 7fd747a00700 1 -- start start 2024-03-13T14:38:39.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.951+0000 7fd747a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd74019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:39.943 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.951+0000 7fd747a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74019c4e0 con 0x7fd740106fa0 2024-03-13T14:38:39.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.952+0000 7fd745000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd74019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:39.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.952+0000 7fd745000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd74019bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:43844/0 (socket says 172.21.15.196:43844) 2024-03-13T14:38:39.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.952+0000 7fd745000700 1 -- 172.21.15.196:0/3542389757 learned_addr learned my addr 172.21.15.196:0/3542389757 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:39.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.952+0000 7fd745000700 1 -- 172.21.15.196:0/3542389757 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340095f0 con 0x7fd740106fa0 2024-03-13T14:38:39.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.953+0000 7fd745000700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd74019bfa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd734005b40 tx=0x7fd734004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:39.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.953+0000 7fd73d600700 1 -- 172.21.15.196:0/3542389757 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd73401d070 con 0x7fd740106fa0 2024-03-13T14:38:39.945 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.953+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74019c6e0 con 0x7fd740106fa0 2024-03-13T14:38:39.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.953+0000 7fd73d600700 1 -- 172.21.15.196:0/3542389757 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd73400be10 con 0x7fd740106fa0 2024-03-13T14:38:39.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.953+0000 7fd73d600700 1 -- 172.21.15.196:0/3542389757 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd73400f460 con 0x7fd740106fa0 2024-03-13T14:38:39.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.953+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74019cb80 con 0x7fd740106fa0 2024-03-13T14:38:39.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.954+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd740195a20 con 0x7fd740106fa0 2024-03-13T14:38:39.947 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.955+0000 7fd73d600700 1 -- 172.21.15.196:0/3542389757 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fd7340227e0 con 0x7fd740106fa0 2024-03-13T14:38:39.947 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.955+0000 7fd73d600700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd728038410 0x7fd72803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:39.947 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.955+0000 7fd73d600700 1 -- 172.21.15.196:0/3542389757 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd73404c390 con 0x7fd740106fa0 2024-03-13T14:38:39.948 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.956+0000 7fd73fe00700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd728038410 0x7fd72803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:39.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.957+0000 7fd73fe00700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd728038410 0x7fd72803a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd72c006fd0 tx=0x7fd72c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:39.963 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:39.971+0000 7fd73d600700 1 -- 172.21.15.196:0/3542389757 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd73402ba00 con 0x7fd740106fa0 2024-03-13T14:38:40.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.267+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd74002cc70 con 0x7fd740106fa0 2024-03-13T14:38:40.260 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.268+0000 7fd73d600700 1 -- 172.21.15.196:0/3542389757 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd734027020 con 0x7fd740106fa0 2024-03-13T14:38:40.261 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:38:40.261 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:38:40.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd728038410 msgr2=0x7fd72803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:40.263 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd728038410 0x7fd72803a8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd72c006fd0 tx=0x7fd72c006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 msgr2=0x7fd74019bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd74019bfa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd734005b40 tx=0x7fd734004740 comp rx=0 tx=0).stop 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 shutdown_connections 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd728038410 0x7fd72803a8d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 --2- 172.21.15.196:0/3542389757 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd740106fa0 0x7fd74019bfa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 >> 172.21.15.196:0/3542389757 conn(0x7fd740075ff0 msgr2=0x7fd740076cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 shutdown_connections 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:40.271+0000 7fd747a00700 1 -- 172.21.15.196:0/3542389757 wait complete. 2024-03-13T14:38:40.264 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:38:41.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:40 smithi176 ceph-mon[29064]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:41.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:40 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/3542389757' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:38:41.912 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:41.912 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:42.297 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:42.394 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:43.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:42 smithi176 ceph-mon[29064]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:44.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.228+0000 7fca5f000700 1 -- 172.21.15.196:0/3120033697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 msgr2=0x7fca58107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:44.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.228+0000 7fca5f000700 1 --2- 172.21.15.196:0/3120033697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca58107450 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fca4c009910 tx=0x7fca4c009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:44.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.228+0000 7fca5f000700 1 -- 172.21.15.196:0/3120033697 shutdown_connections 2024-03-13T14:38:44.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.228+0000 7fca5f000700 1 --2- 172.21.15.196:0/3120033697 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca58107450 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:44.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.228+0000 7fca5f000700 1 -- 172.21.15.196:0/3120033697 >> 172.21.15.196:0/3120033697 conn(0x7fca58075d80 msgr2=0x7fca580781c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:44.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.229+0000 7fca5f000700 1 -- 172.21.15.196:0/3120033697 shutdown_connections 2024-03-13T14:38:44.222 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.229+0000 7fca5f000700 1 -- 172.21.15.196:0/3120033697 wait complete. 2024-03-13T14:38:44.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.230+0000 7fca5f000700 1 Processor -- start 2024-03-13T14:38:44.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.230+0000 7fca5f000700 1 -- start start 2024-03-13T14:38:44.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.231+0000 7fca5f000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca5819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:44.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.231+0000 7fca5f000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca5819c320 con 0x7fca58107030 2024-03-13T14:38:44.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.231+0000 7fca5dc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca5819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:44.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.231+0000 7fca5dc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca5819bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:45920/0 (socket says 172.21.15.196:45920) 2024-03-13T14:38:44.223 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.231+0000 7fca5dc00700 1 -- 172.21.15.196:0/1319048255 learned_addr learned my addr 172.21.15.196:0/1319048255 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:44.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.232+0000 7fca5dc00700 1 -- 172.21.15.196:0/1319048255 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0095f0 con 0x7fca58107030 2024-03-13T14:38:44.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.232+0000 7fca5dc00700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca5819bde0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fca4c005b40 tx=0x7fca4c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:44.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.232+0000 7fca56000700 1 -- 172.21.15.196:0/1319048255 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca4c01d070 con 0x7fca58107030 2024-03-13T14:38:44.224 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.233+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca5819c520 con 0x7fca58107030 2024-03-13T14:38:44.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.233+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5819c9c0 con 0x7fca58107030 2024-03-13T14:38:44.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.233+0000 7fca56000700 1 -- 172.21.15.196:0/1319048255 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fca4c00be10 con 0x7fca58107030 2024-03-13T14:38:44.225 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.233+0000 7fca56000700 1 -- 172.21.15.196:0/1319048255 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fca4c00f460 con 0x7fca58107030 2024-03-13T14:38:44.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.234+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca581959a0 con 0x7fca58107030 2024-03-13T14:38:44.226 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.234+0000 7fca56000700 1 -- 172.21.15.196:0/1319048255 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fca4c0227c0 con 0x7fca58107030 2024-03-13T14:38:44.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.235+0000 7fca56000700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fca400383c0 0x7fca4003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:44.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.235+0000 7fca56000700 1 -- 172.21.15.196:0/1319048255 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fca4c04c390 con 0x7fca58107030 2024-03-13T14:38:44.227 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.235+0000 7fca5d200700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fca400383c0 0x7fca4003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:44.228 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.236+0000 7fca5d200700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fca400383c0 0x7fca4003a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fca44006fd0 tx=0x7fca44006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.261+0000 7fca56000700 1 -- 172.21.15.196:0/1319048255 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fca4c00b960 con 0x7fca58107030 2024-03-13T14:38:44.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.562+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fca5802cc70 con 0x7fca58107030 2024-03-13T14:38:44.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.563+0000 7fca56000700 1 -- 172.21.15.196:0/1319048255 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fca4c027070 con 0x7fca58107030 2024-03-13T14:38:44.555 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:38:44.555 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:38:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fca400383c0 msgr2=0x7fca4003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fca400383c0 0x7fca4003a880 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fca44006fd0 tx=0x7fca44006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 msgr2=0x7fca5819bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca5819bde0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fca4c005b40 tx=0x7fca4c004740 comp rx=0 tx=0).stop 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 shutdown_connections 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fca400383c0 0x7fca4003a880 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 --2- 172.21.15.196:0/1319048255 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fca58107030 0x7fca5819bde0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 >> 172.21.15.196:0/1319048255 conn(0x7fca58075d80 msgr2=0x7fca58076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 shutdown_connections 2024-03-13T14:38:44.559 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:44.566+0000 7fca5f000700 1 -- 172.21.15.196:0/1319048255 wait complete. 2024-03-13T14:38:44.560 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:38:45.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:44 smithi176 ceph-mon[29064]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:45.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:44 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1319048255' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:38:46.247 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:46.247 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:46.619 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:46.719 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:47.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:46 smithi176 ceph-mon[29064]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:48.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.632+0000 7f746b000700 1 -- 172.21.15.196:0/2520091145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 msgr2=0x7f74641073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:48.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.632+0000 7f746b000700 1 --2- 172.21.15.196:0/2520091145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f74641073c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7454009910 tx=0x7f7454009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:48.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.632+0000 7f746b000700 1 -- 172.21.15.196:0/2520091145 shutdown_connections 2024-03-13T14:38:48.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.632+0000 7f746b000700 1 --2- 172.21.15.196:0/2520091145 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f74641073c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:48.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.632+0000 7f746b000700 1 -- 172.21.15.196:0/2520091145 >> 172.21.15.196:0/2520091145 conn(0x7f7464075ff0 msgr2=0x7f7464078450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:48.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.632+0000 7f746b000700 1 -- 172.21.15.196:0/2520091145 shutdown_connections 2024-03-13T14:38:48.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.632+0000 7f746b000700 1 -- 172.21.15.196:0/2520091145 wait complete. 2024-03-13T14:38:48.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.633+0000 7f746b000700 1 Processor -- start 2024-03-13T14:38:48.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.633+0000 7f746b000700 1 -- start start 2024-03-13T14:38:48.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.634+0000 7f746b000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f746419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:48.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.634+0000 7f746b000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746419c4e0 con 0x7f7464106fa0 2024-03-13T14:38:48.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.634+0000 7f7463e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f746419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:48.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.634+0000 7f7463e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f746419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:45936/0 (socket says 172.21.15.196:45936) 2024-03-13T14:38:48.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.634+0000 7f7463e00700 1 -- 172.21.15.196:0/2672533450 learned_addr learned my addr 172.21.15.196:0/2672533450 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:48.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.635+0000 7f7463e00700 1 -- 172.21.15.196:0/2672533450 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74540095f0 con 0x7f7464106fa0 2024-03-13T14:38:48.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.635+0000 7f7463e00700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f746419bfa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7454005b40 tx=0x7f7454004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:48.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.636+0000 7f7460c00700 1 -- 172.21.15.196:0/2672533450 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f745401d070 con 0x7f7464106fa0 2024-03-13T14:38:48.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.636+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746419c6e0 con 0x7f7464106fa0 2024-03-13T14:38:48.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.636+0000 7f7460c00700 1 -- 172.21.15.196:0/2672533450 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f745400be10 con 0x7f7464106fa0 2024-03-13T14:38:48.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.636+0000 7f7460c00700 1 -- 172.21.15.196:0/2672533450 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f745400f460 con 0x7f7464106fa0 2024-03-13T14:38:48.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.636+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746419cb80 con 0x7f7464106fa0 2024-03-13T14:38:48.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.637+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7464195a20 con 0x7f7464106fa0 2024-03-13T14:38:48.630 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.637+0000 7f7460c00700 1 -- 172.21.15.196:0/2672533450 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f74540227e0 con 0x7f7464106fa0 2024-03-13T14:38:48.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.638+0000 7f7460c00700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7448038410 0x7f744803a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:48.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.638+0000 7f7460c00700 1 -- 172.21.15.196:0/2672533450 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f745404c450 con 0x7f7464106fa0 2024-03-13T14:38:48.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.638+0000 7f7463400700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7448038410 0x7f744803a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:48.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.639+0000 7f7463400700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7448038410 0x7f744803a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7450006fd0 tx=0x7f7450006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:48.643 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.650+0000 7f7460c00700 1 -- 172.21.15.196:0/2672533450 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f745402ba00 con 0x7f7464106fa0 2024-03-13T14:38:48.944 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.951+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f746402cc70 con 0x7f7464106fa0 2024-03-13T14:38:48.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.953+0000 7f7460c00700 1 -- 172.21.15.196:0/2672533450 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f7454027030 con 0x7f7464106fa0 2024-03-13T14:38:48.946 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:38:48.946 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.955+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7448038410 msgr2=0x7f744803a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.955+0000 7f746b000700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7448038410 0x7f744803a8d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7450006fd0 tx=0x7f7450006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.955+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 msgr2=0x7f746419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.955+0000 7f746b000700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f746419bfa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7454005b40 tx=0x7f7454004740 comp rx=0 tx=0).stop 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.956+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 shutdown_connections 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.956+0000 7f746b000700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f7448038410 0x7f744803a8d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.956+0000 7f746b000700 1 --2- 172.21.15.196:0/2672533450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7464106fa0 0x7f746419bfa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.956+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 >> 172.21.15.196:0/2672533450 conn(0x7f7464075ff0 msgr2=0x7f7464076cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.956+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 shutdown_connections 2024-03-13T14:38:48.949 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:48.956+0000 7f746b000700 1 -- 172.21.15.196:0/2672533450 wait complete. 2024-03-13T14:38:48.950 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:38:49.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:48 smithi176 ceph-mon[29064]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:50.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:49 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2672533450' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:38:50.621 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:50.621 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:51.000 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:51.105 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:51.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:50 smithi176 ceph-mon[29064]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:53.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.037+0000 7f00cae00700 1 -- 172.21.15.196:0/51553320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 msgr2=0x7f00c4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:53.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.037+0000 7f00cae00700 1 --2- 172.21.15.196:0/51553320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4102eb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009910 tx=0x7f00b4009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:53.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.037+0000 7f00cae00700 1 -- 172.21.15.196:0/51553320 shutdown_connections 2024-03-13T14:38:53.031 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.037+0000 7f00cae00700 1 --2- 172.21.15.196:0/51553320 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4102eb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:53.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.037+0000 7f00cae00700 1 -- 172.21.15.196:0/51553320 >> 172.21.15.196:0/51553320 conn(0x7f00c40fe010 msgr2=0x7f00c4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:53.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.038+0000 7f00cae00700 1 -- 172.21.15.196:0/51553320 shutdown_connections 2024-03-13T14:38:53.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.038+0000 7f00cae00700 1 -- 172.21.15.196:0/51553320 wait complete. 2024-03-13T14:38:53.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.039+0000 7f00cae00700 1 Processor -- start 2024-03-13T14:38:53.032 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.039+0000 7f00cae00700 1 -- start start 2024-03-13T14:38:53.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.039+0000 7f00cae00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:53.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.039+0000 7f00cae00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c41980f0 con 0x7f00c4102a90 2024-03-13T14:38:53.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.040+0000 7f00c3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:53.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.040+0000 7f00c3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:33636/0 (socket says 172.21.15.196:33636) 2024-03-13T14:38:53.033 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.040+0000 7f00c3e00700 1 -- 172.21.15.196:0/3033901400 learned_addr learned my addr 172.21.15.196:0/3033901400 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:53.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.041+0000 7f00c3e00700 1 -- 172.21.15.196:0/3033901400 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b40095f0 con 0x7f00c4102a90 2024-03-13T14:38:53.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.041+0000 7f00c3e00700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4197bb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f00b4005950 tx=0x7f00b4009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:53.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.041+0000 7f00c0c00700 1 -- 172.21.15.196:0/3033901400 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00b401d070 con 0x7f00c4102a90 2024-03-13T14:38:53.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.042+0000 7f00c0c00700 1 -- 172.21.15.196:0/3033901400 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f00b400be10 con 0x7f00c4102a90 2024-03-13T14:38:53.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.042+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c41982f0 con 0x7f00c4102a90 2024-03-13T14:38:53.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.042+0000 7f00c0c00700 1 -- 172.21.15.196:0/3033901400 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00b400f460 con 0x7f00c4102a90 2024-03-13T14:38:53.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.042+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c4198790 con 0x7f00c4102a90 2024-03-13T14:38:53.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.043+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c41916b0 con 0x7f00c4102a90 2024-03-13T14:38:53.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.043+0000 7f00c0c00700 1 -- 172.21.15.196:0/3033901400 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7f00b40227c0 con 0x7f00c4102a90 2024-03-13T14:38:53.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.044+0000 7f00c0c00700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f00a80383c0 0x7f00a803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:53.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.044+0000 7f00c0c00700 1 -- 172.21.15.196:0/3033901400 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f00b404c4e0 con 0x7f00c4102a90 2024-03-13T14:38:53.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.044+0000 7f00c3400700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f00a80383c0 0x7f00a803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:53.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.045+0000 7f00c3400700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f00a80383c0 0x7f00a803a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006fd0 tx=0x7f00b0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:53.049 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.056+0000 7f00c0c00700 1 -- 172.21.15.196:0/3033901400 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f00b402bba0 con 0x7f00c4102a90 2024-03-13T14:38:53.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:52 smithi176 ceph-mon[29064]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:53.339 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.346+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f00c402cc70 con 0x7f00c4102a90 2024-03-13T14:38:53.340 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.347+0000 7f00c0c00700 1 -- 172.21.15.196:0/3033901400 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f00b4027030 con 0x7f00c4102a90 2024-03-13T14:38:53.341 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:38:53.341 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f00a80383c0 msgr2=0x7f00a803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f00a80383c0 0x7f00a803a880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006fd0 tx=0x7f00b0006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 msgr2=0x7f00c4197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4197bb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f00b4005950 tx=0x7f00b4009e60 comp rx=0 tx=0).stop 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 shutdown_connections 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f00a80383c0 0x7f00a803a880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 --2- 172.21.15.196:0/3033901400 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f00c4102a90 0x7f00c4197bb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 >> 172.21.15.196:0/3033901400 conn(0x7f00c40fe010 msgr2=0x7f00c40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 shutdown_connections 2024-03-13T14:38:53.344 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:53.350+0000 7f00cae00700 1 -- 172.21.15.196:0/3033901400 wait complete. 2024-03-13T14:38:53.345 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:38:54.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:53 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/3033901400' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:38:54.979 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:54.979 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:55.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:54 smithi176 ceph-mon[29064]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:55.360 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:55.455 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:57.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:56 smithi176 ceph-mon[29064]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:57.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.466+0000 7fef79c00700 1 -- 172.21.15.196:0/4127642961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 msgr2=0x7fef74100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:57.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.466+0000 7fef79c00700 1 --2- 172.21.15.196:0/4127642961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef74100cd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fef64009910 tx=0x7fef64009c20 comp rx=0 tx=0).stop 2024-03-13T14:38:57.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.466+0000 7fef79c00700 1 -- 172.21.15.196:0/4127642961 shutdown_connections 2024-03-13T14:38:57.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.466+0000 7fef79c00700 1 --2- 172.21.15.196:0/4127642961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef74100cd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:57.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.466+0000 7fef79c00700 1 -- 172.21.15.196:0/4127642961 >> 172.21.15.196:0/4127642961 conn(0x7fef740fbe30 msgr2=0x7fef740fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:57.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.466+0000 7fef79c00700 1 -- 172.21.15.196:0/4127642961 shutdown_connections 2024-03-13T14:38:57.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.466+0000 7fef79c00700 1 -- 172.21.15.196:0/4127642961 wait complete. 2024-03-13T14:38:57.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.467+0000 7fef79c00700 1 Processor -- start 2024-03-13T14:38:57.460 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.468+0000 7fef79c00700 1 -- start start 2024-03-13T14:38:57.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.468+0000 7fef79c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef741959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:57.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.468+0000 7fef79c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef74195f00 con 0x7fef741008b0 2024-03-13T14:38:57.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.469+0000 7fef72a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef741959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:57.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.469+0000 7fef72a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef741959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:33658/0 (socket says 172.21.15.196:33658) 2024-03-13T14:38:57.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.469+0000 7fef72a00700 1 -- 172.21.15.196:0/875364578 learned_addr learned my addr 172.21.15.196:0/875364578 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:38:57.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.469+0000 7fef72a00700 1 -- 172.21.15.196:0/875364578 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640095f0 con 0x7fef741008b0 2024-03-13T14:38:57.462 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.470+0000 7fef72a00700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef741959c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fef64005b40 tx=0x7fef64004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:57.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.470+0000 7fef6be00700 1 -- 172.21.15.196:0/875364578 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef6401d070 con 0x7fef741008b0 2024-03-13T14:38:57.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.470+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef74196100 con 0x7fef741008b0 2024-03-13T14:38:57.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.470+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef741965a0 con 0x7fef741008b0 2024-03-13T14:38:57.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.470+0000 7fef6be00700 1 -- 172.21.15.196:0/875364578 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef6400be10 con 0x7fef741008b0 2024-03-13T14:38:57.463 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.470+0000 7fef6be00700 1 -- 172.21.15.196:0/875364578 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fef6400f460 con 0x7fef741008b0 2024-03-13T14:38:57.464 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.471+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef50005320 con 0x7fef741008b0 2024-03-13T14:38:57.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.472+0000 7fef6be00700 1 -- 172.21.15.196:0/875364578 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fef640227c0 con 0x7fef741008b0 2024-03-13T14:38:57.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.472+0000 7fef6be00700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fef580383c0 0x7fef5803a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:38:57.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.472+0000 7fef6be00700 1 -- 172.21.15.196:0/875364578 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fef6404c2d0 con 0x7fef741008b0 2024-03-13T14:38:57.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.473+0000 7fef6a000700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fef580383c0 0x7fef5803a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:38:57.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.474+0000 7fef6a000700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fef580383c0 0x7fef5803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:38:57.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.488+0000 7fef6be00700 1 -- 172.21.15.196:0/875364578 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef6400b960 con 0x7fef741008b0 2024-03-13T14:38:57.780 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.787+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fef50005190 con 0x7fef741008b0 2024-03-13T14:38:57.782 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.789+0000 7fef6be00700 1 -- 172.21.15.196:0/875364578 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fef64027030 con 0x7fef741008b0 2024-03-13T14:38:57.782 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:38:57.782 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fef580383c0 msgr2=0x7fef5803a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fef580383c0 0x7fef5803a880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fef5c006fd0 tx=0x7fef5c006e40 comp rx=0 tx=0).stop 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 msgr2=0x7fef741959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef741959c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fef64005b40 tx=0x7fef64004740 comp rx=0 tx=0).stop 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 shutdown_connections 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fef580383c0 0x7fef5803a880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 --2- 172.21.15.196:0/875364578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef741008b0 0x7fef741959c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:38:57.786 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 >> 172.21.15.196:0/875364578 conn(0x7fef740fbe30 msgr2=0x7fef740fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:38:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 shutdown_connections 2024-03-13T14:38:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:38:57.793+0000 7fef79c00700 1 -- 172.21.15.196:0/875364578 wait complete. 2024-03-13T14:38:57.787 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:38:59.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:58 smithi176 ceph-mon[29064]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:38:59.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:38:58 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/875364578' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:38:59.472 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:38:59.472 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:38:59.835 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:38:59.927 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:01.078 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:00 smithi176 ceph-mon[29064]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:01.872 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.878+0000 7fcd14c00700 1 -- 172.21.15.196:0/1926560375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 msgr2=0x7fcd10102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:01.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.878+0000 7fcd14c00700 1 --2- 172.21.15.196:0/1926560375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10102eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcd00009910 tx=0x7fcd00009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:01.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.879+0000 7fcd14c00700 1 -- 172.21.15.196:0/1926560375 shutdown_connections 2024-03-13T14:39:01.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.879+0000 7fcd14c00700 1 --2- 172.21.15.196:0/1926560375 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10102eb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:01.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.879+0000 7fcd14c00700 1 -- 172.21.15.196:0/1926560375 >> 172.21.15.196:0/1926560375 conn(0x7fcd100fe010 msgr2=0x7fcd10100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:01.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.879+0000 7fcd14c00700 1 -- 172.21.15.196:0/1926560375 shutdown_connections 2024-03-13T14:39:01.873 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.879+0000 7fcd14c00700 1 -- 172.21.15.196:0/1926560375 wait complete. 2024-03-13T14:39:01.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.880+0000 7fcd14c00700 1 Processor -- start 2024-03-13T14:39:01.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.880+0000 7fcd14c00700 1 -- start start 2024-03-13T14:39:01.877 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.881+0000 7fcd14c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:01.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.881+0000 7fcd14c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd101980f0 con 0x7fcd10102a90 2024-03-13T14:39:01.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.881+0000 7fcd0de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:01.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.881+0000 7fcd0de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:33684/0 (socket says 172.21.15.196:33684) 2024-03-13T14:39:01.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.881+0000 7fcd0de00700 1 -- 172.21.15.196:0/1073525665 learned_addr learned my addr 172.21.15.196:0/1073525665 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:01.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.882+0000 7fcd0de00700 1 -- 172.21.15.196:0/1073525665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000095f0 con 0x7fcd10102a90 2024-03-13T14:39:01.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.882+0000 7fcd0de00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10197bb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd00005950 tx=0x7fcd00009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:01.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.882+0000 7fcd06a00700 1 -- 172.21.15.196:0/1073525665 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd0001d070 con 0x7fcd10102a90 2024-03-13T14:39:01.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.882+0000 7fcd06a00700 1 -- 172.21.15.196:0/1073525665 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcd0000be10 con 0x7fcd10102a90 2024-03-13T14:39:01.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.882+0000 7fcd06a00700 1 -- 172.21.15.196:0/1073525665 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcd0000f460 con 0x7fcd10102a90 2024-03-13T14:39:01.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.882+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd101982f0 con 0x7fcd10102a90 2024-03-13T14:39:01.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.882+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10198790 con 0x7fcd10102a90 2024-03-13T14:39:01.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.883+0000 7fcd06a00700 1 -- 172.21.15.196:0/1073525665 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fcd000227c0 con 0x7fcd10102a90 2024-03-13T14:39:01.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.883+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd101916b0 con 0x7fcd10102a90 2024-03-13T14:39:01.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.884+0000 7fcd06a00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fccf4038410 0x7fccf403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:01.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.884+0000 7fcd06a00700 1 -- 172.21.15.196:0/1073525665 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcd0004c560 con 0x7fcd10102a90 2024-03-13T14:39:01.880 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.884+0000 7fcd07e00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fccf4038410 0x7fccf403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:01.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.885+0000 7fcd07e00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fccf4038410 0x7fccf403a8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fccf8006fd0 tx=0x7fccf8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:01.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:01.889+0000 7fcd06a00700 1 -- 172.21.15.196:0/1073525665 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fcd0002ba00 con 0x7fcd10102a90 2024-03-13T14:39:02.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.180+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcd1002cc70 con 0x7fcd10102a90 2024-03-13T14:39:02.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.182+0000 7fcd06a00700 1 -- 172.21.15.196:0/1073525665 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fcd00027030 con 0x7fcd10102a90 2024-03-13T14:39:02.175 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:02.175 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:02.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fccf4038410 msgr2=0x7fccf403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:02.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fccf4038410 0x7fccf403a8d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fccf8006fd0 tx=0x7fccf8006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:02.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 msgr2=0x7fcd10197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:02.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10197bb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd00005950 tx=0x7fcd00009e60 comp rx=0 tx=0).stop 2024-03-13T14:39:02.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 shutdown_connections 2024-03-13T14:39:02.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fccf4038410 0x7fccf403a8d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:02.178 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 --2- 172.21.15.196:0/1073525665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcd10102a90 0x7fcd10197bb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:02.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 >> 172.21.15.196:0/1073525665 conn(0x7fcd100fe010 msgr2=0x7fcd100fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:02.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 shutdown_connections 2024-03-13T14:39:02.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:02.185+0000 7fcd14c00700 1 -- 172.21.15.196:0/1073525665 wait complete. 2024-03-13T14:39:02.179 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:02.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: Deploying daemon prometheus.smithi176 on smithi176 2024-03-13T14:39:02.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:02 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1073525665' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:03.813 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:03.813 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:03.868 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:03 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:04.178 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:04.276 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:04.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:04 smithi176 ceph-mon[29064]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:06.061 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.067+0000 7faac6e00700 1 -- 172.21.15.196:0/521265092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 msgr2=0x7faac00731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.067+0000 7faac6e00700 1 --2- 172.21.15.196:0/521265092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac00731c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faab0009910 tx=0x7faab0009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.067+0000 7faac6e00700 1 -- 172.21.15.196:0/521265092 shutdown_connections 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.067+0000 7faac6e00700 1 --2- 172.21.15.196:0/521265092 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac00731c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.067+0000 7faac6e00700 1 -- 172.21.15.196:0/521265092 >> 172.21.15.196:0/521265092 conn(0x7faac00fbe60 msgr2=0x7faac00fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.067+0000 7faac6e00700 1 -- 172.21.15.196:0/521265092 shutdown_connections 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.068+0000 7faac6e00700 1 -- 172.21.15.196:0/521265092 wait complete. 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.068+0000 7faac6e00700 1 Processor -- start 2024-03-13T14:39:06.062 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.069+0000 7faac6e00700 1 -- start start 2024-03-13T14:39:06.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.069+0000 7faac6e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:06.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.069+0000 7faac6e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01980e0 con 0x7faac0074d60 2024-03-13T14:39:06.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.069+0000 7faabfe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:06.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.070+0000 7faabfe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:47906/0 (socket says 172.21.15.196:47906) 2024-03-13T14:39:06.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.070+0000 7faabfe00700 1 -- 172.21.15.196:0/501717506 learned_addr learned my addr 172.21.15.196:0/501717506 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:06.063 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.070+0000 7faabfe00700 1 -- 172.21.15.196:0/501717506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00095f0 con 0x7faac0074d60 2024-03-13T14:39:06.064 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.071+0000 7faabfe00700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac0197ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faab0005b40 tx=0x7faab0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:06.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.071+0000 7faabcc00700 1 -- 172.21.15.196:0/501717506 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faab001d070 con 0x7faac0074d60 2024-03-13T14:39:06.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.071+0000 7faabcc00700 1 -- 172.21.15.196:0/501717506 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faab000be10 con 0x7faac0074d60 2024-03-13T14:39:06.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.071+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac01982e0 con 0x7faac0074d60 2024-03-13T14:39:06.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.071+0000 7faabcc00700 1 -- 172.21.15.196:0/501717506 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7faab000f460 con 0x7faac0074d60 2024-03-13T14:39:06.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.072+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac0198780 con 0x7faac0074d60 2024-03-13T14:39:06.065 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.073+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac01916c0 con 0x7faac0074d60 2024-03-13T14:39:06.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.073+0000 7faabcc00700 1 -- 172.21.15.196:0/501717506 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7faab00227e0 con 0x7faac0074d60 2024-03-13T14:39:06.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.074+0000 7faabcc00700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7faaa4038460 0x7faaa403a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:06.066 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.074+0000 7faabcc00700 1 -- 172.21.15.196:0/501717506 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faab004cf10 con 0x7faac0074d60 2024-03-13T14:39:06.067 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.074+0000 7faabf400700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7faaa4038460 0x7faaa403a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:06.069 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.076+0000 7faabf400700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7faaa4038460 0x7faaa403a920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faaac006fd0 tx=0x7faaac006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:06.082 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.090+0000 7faabcc00700 1 -- 172.21.15.196:0/501717506 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faab003da20 con 0x7faac0074d60 2024-03-13T14:39:06.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.384+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faac002cc70 con 0x7faac0074d60 2024-03-13T14:39:06.378 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.385+0000 7faabcc00700 1 -- 172.21.15.196:0/501717506 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7faab0027030 con 0x7faac0074d60 2024-03-13T14:39:06.378 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:06.378 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:06.380 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7faaa4038460 msgr2=0x7faaa403a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:06.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7faaa4038460 0x7faaa403a920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faaac006fd0 tx=0x7faaac006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:06.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 msgr2=0x7faac0197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:06.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac0197ba0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faab0005b40 tx=0x7faab0004740 comp rx=0 tx=0).stop 2024-03-13T14:39:06.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 shutdown_connections 2024-03-13T14:39:06.381 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7faaa4038460 0x7faaa403a920 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:06.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 --2- 172.21.15.196:0/501717506 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faac0074d60 0x7faac0197ba0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7faab0005b40 tx=0x7faab0004740 comp rx=0 tx=0).stop 2024-03-13T14:39:06.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.387+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 >> 172.21.15.196:0/501717506 conn(0x7faac00fbe60 msgr2=0x7faac00fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:06.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.388+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 shutdown_connections 2024-03-13T14:39:06.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:06.388+0000 7faac6e00700 1 -- 172.21.15.196:0/501717506 wait complete. 2024-03-13T14:39:06.383 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:07.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:06 smithi176 ceph-mon[29064]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:07.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:06 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/501717506' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:08.149 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:08.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:08.502 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:08.597 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:08.829 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:08 smithi176 ceph-mon[29064]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:10.418 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.426+0000 7ff410000700 1 -- 172.21.15.196:0/3233953115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 msgr2=0x7ff408102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.426+0000 7ff410000700 1 --2- 172.21.15.196:0/3233953115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408102eb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009910 tx=0x7ff3fc009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.426+0000 7ff410000700 1 -- 172.21.15.196:0/3233953115 shutdown_connections 2024-03-13T14:39:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.426+0000 7ff410000700 1 --2- 172.21.15.196:0/3233953115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408102eb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.426+0000 7ff410000700 1 -- 172.21.15.196:0/3233953115 >> 172.21.15.196:0/3233953115 conn(0x7ff4080fe010 msgr2=0x7ff408100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.426+0000 7ff410000700 1 -- 172.21.15.196:0/3233953115 shutdown_connections 2024-03-13T14:39:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.426+0000 7ff410000700 1 -- 172.21.15.196:0/3233953115 wait complete. 2024-03-13T14:39:10.419 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.427+0000 7ff410000700 1 Processor -- start 2024-03-13T14:39:10.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.427+0000 7ff410000700 1 -- start start 2024-03-13T14:39:10.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.428+0000 7ff410000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:10.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.428+0000 7ff410000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4081980f0 con 0x7ff408102a90 2024-03-13T14:39:10.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.428+0000 7ff40d600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:10.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.428+0000 7ff40d600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:47920/0 (socket says 172.21.15.196:47920) 2024-03-13T14:39:10.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.428+0000 7ff40d600700 1 -- 172.21.15.196:0/2960013657 learned_addr learned my addr 172.21.15.196:0/2960013657 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:10.420 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.429+0000 7ff40d600700 1 -- 172.21.15.196:0/2960013657 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc0095f0 con 0x7ff408102a90 2024-03-13T14:39:10.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.429+0000 7ff40d600700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408197bb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc005950 tx=0x7ff3fc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:10.421 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.430+0000 7ff406000700 1 -- 172.21.15.196:0/2960013657 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3fc01d070 con 0x7ff408102a90 2024-03-13T14:39:10.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.430+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4081982f0 con 0x7ff408102a90 2024-03-13T14:39:10.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.430+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff408198790 con 0x7ff408102a90 2024-03-13T14:39:10.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.430+0000 7ff406000700 1 -- 172.21.15.196:0/2960013657 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff3fc005c00 con 0x7ff408102a90 2024-03-13T14:39:10.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.430+0000 7ff406000700 1 -- 172.21.15.196:0/2960013657 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3fc00fbb0 con 0x7ff408102a90 2024-03-13T14:39:10.422 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.431+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e8005320 con 0x7ff408102a90 2024-03-13T14:39:10.423 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.431+0000 7ff406000700 1 -- 172.21.15.196:0/2960013657 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7ff3fc00fd10 con 0x7ff408102a90 2024-03-13T14:39:10.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.432+0000 7ff406000700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7ff3f00383c0 0x7ff3f003a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:10.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.432+0000 7ff406000700 1 -- 172.21.15.196:0/2960013657 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff3fc04c1b0 con 0x7ff408102a90 2024-03-13T14:39:10.424 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.432+0000 7ff40cc00700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7ff3f00383c0 0x7ff3f003a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:10.425 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.433+0000 7ff40cc00700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7ff3f00383c0 0x7ff3f003a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4006fd0 tx=0x7ff3f4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:10.443 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.451+0000 7ff406000700 1 -- 172.21.15.196:0/2960013657 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff3fc00b960 con 0x7ff408102a90 2024-03-13T14:39:10.747 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.755+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff3e8005190 con 0x7ff408102a90 2024-03-13T14:39:10.748 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.756+0000 7ff406000700 1 -- 172.21.15.196:0/2960013657 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ff3fc027070 con 0x7ff408102a90 2024-03-13T14:39:10.748 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:10.748 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.758+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7ff3f00383c0 msgr2=0x7ff3f003a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.758+0000 7ff410000700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7ff3f00383c0 0x7ff3f003a880 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4006fd0 tx=0x7ff3f4006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 msgr2=0x7ff408197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408197bb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc005950 tx=0x7ff3fc009e60 comp rx=0 tx=0).stop 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 shutdown_connections 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7ff3f00383c0 0x7ff3f003a880 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 --2- 172.21.15.196:0/2960013657 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff408102a90 0x7ff408197bb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 >> 172.21.15.196:0/2960013657 conn(0x7ff4080fe010 msgr2=0x7ff4080fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 shutdown_connections 2024-03-13T14:39:10.751 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:10.759+0000 7ff410000700 1 -- 172.21.15.196:0/2960013657 wait complete. 2024-03-13T14:39:10.752 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:11.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:10 smithi176 ceph-mon[29064]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:11.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:10 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2960013657' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:12.585 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:12.585 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:12.961 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:13.059 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:13.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:12 smithi176 ceph-mon[29064]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:14.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.854+0000 7fee1f200700 1 -- 172.21.15.196:0/985902239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 msgr2=0x7fee20100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:14.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.854+0000 7fee1f200700 1 --2- 172.21.15.196:0/985902239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee20100b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fee10009910 tx=0x7fee10009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:14.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.855+0000 7fee1f200700 1 -- 172.21.15.196:0/985902239 shutdown_connections 2024-03-13T14:39:14.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.855+0000 7fee1f200700 1 --2- 172.21.15.196:0/985902239 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee20100b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:14.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.855+0000 7fee1f200700 1 -- 172.21.15.196:0/985902239 >> 172.21.15.196:0/985902239 conn(0x7fee200fbca0 msgr2=0x7fee200fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:14.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.855+0000 7fee1f200700 1 -- 172.21.15.196:0/985902239 shutdown_connections 2024-03-13T14:39:14.847 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.855+0000 7fee1f200700 1 -- 172.21.15.196:0/985902239 wait complete. 2024-03-13T14:39:14.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.856+0000 7fee1f200700 1 Processor -- start 2024-03-13T14:39:14.848 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.857+0000 7fee1f200700 1 -- start start 2024-03-13T14:39:14.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.857+0000 7fee1f200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee201957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:14.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.857+0000 7fee1f200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee20195d30 con 0x7fee201006e0 2024-03-13T14:39:14.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.858+0000 7fee1de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee201957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:14.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.858+0000 7fee1de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee201957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:48516/0 (socket says 172.21.15.196:48516) 2024-03-13T14:39:14.849 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.858+0000 7fee1de00700 1 -- 172.21.15.196:0/2950481783 learned_addr learned my addr 172.21.15.196:0/2950481783 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.858+0000 7fee1de00700 1 -- 172.21.15.196:0/2950481783 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100095f0 con 0x7fee201006e0 2024-03-13T14:39:14.850 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.859+0000 7fee1de00700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee201957f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fee10005b40 tx=0x7fee10004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:14.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.859+0000 7fee16a00700 1 -- 172.21.15.196:0/2950481783 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee1001d070 con 0x7fee201006e0 2024-03-13T14:39:14.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.859+0000 7fee16a00700 1 -- 172.21.15.196:0/2950481783 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee1000be10 con 0x7fee201006e0 2024-03-13T14:39:14.851 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.859+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee20195f30 con 0x7fee201006e0 2024-03-13T14:39:14.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.860+0000 7fee16a00700 1 -- 172.21.15.196:0/2950481783 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee1000f460 con 0x7fee201006e0 2024-03-13T14:39:14.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.860+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee201963d0 con 0x7fee201006e0 2024-03-13T14:39:14.852 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.861+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2018f300 con 0x7fee201006e0 2024-03-13T14:39:14.853 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.861+0000 7fee16a00700 1 -- 172.21.15.196:0/2950481783 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fee100227e0 con 0x7fee201006e0 2024-03-13T14:39:14.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.862+0000 7fee16a00700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fee04038410 0x7fee0403a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:14.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.862+0000 7fee16a00700 1 -- 172.21.15.196:0/2950481783 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee1004ce70 con 0x7fee201006e0 2024-03-13T14:39:14.854 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.862+0000 7fee1d400700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fee04038410 0x7fee0403a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:14.855 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.863+0000 7fee1d400700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fee04038410 0x7fee0403a8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee0c006fd0 tx=0x7fee0c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:14.868 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:14.876+0000 7fee16a00700 1 -- 172.21.15.196:0/2950481783 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fee1003da20 con 0x7fee201006e0 2024-03-13T14:39:15.062 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:14 smithi176 ceph-mon[29064]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:15.166 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.174+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fee2002cc70 con 0x7fee201006e0 2024-03-13T14:39:15.167 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.175+0000 7fee16a00700 1 -- 172.21.15.196:0/2950481783 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fee10027030 con 0x7fee201006e0 2024-03-13T14:39:15.167 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:15.167 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fee04038410 msgr2=0x7fee0403a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fee04038410 0x7fee0403a8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee0c006fd0 tx=0x7fee0c006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 msgr2=0x7fee201957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee201957f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fee10005b40 tx=0x7fee10004740 comp rx=0 tx=0).stop 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 shutdown_connections 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fee04038410 0x7fee0403a8d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 --2- 172.21.15.196:0/2950481783 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee201006e0 0x7fee201957f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 >> 172.21.15.196:0/2950481783 conn(0x7fee200fbca0 msgr2=0x7fee200fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 shutdown_connections 2024-03-13T14:39:15.170 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:15.178+0000 7fee1f200700 1 -- 172.21.15.196:0/2950481783 wait complete. 2024-03-13T14:39:15.171 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:16.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:15 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2950481783' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:16.833 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:16.833 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:17.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:16 smithi176 ceph-mon[29064]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:17.204 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:17.302 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:19.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:18 smithi176 ceph-mon[29064]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:19.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:18 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:19.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:18 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:19.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:18 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:19.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:18 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-13T14:39:19.172 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.180+0000 7fd56c000700 1 -- 172.21.15.196:0/899939887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 msgr2=0x7fd564073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:19.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.180+0000 7fd56c000700 1 --2- 172.21.15.196:0/899939887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd564073220 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd558009910 tx=0x7fd558009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:19.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.180+0000 7fd56c000700 1 -- 172.21.15.196:0/899939887 shutdown_connections 2024-03-13T14:39:19.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.180+0000 7fd56c000700 1 --2- 172.21.15.196:0/899939887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd564073220 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:19.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.180+0000 7fd56c000700 1 -- 172.21.15.196:0/899939887 >> 172.21.15.196:0/899939887 conn(0x7fd5640fbe30 msgr2=0x7fd5640fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:19.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.180+0000 7fd56c000700 1 -- 172.21.15.196:0/899939887 shutdown_connections 2024-03-13T14:39:19.173 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.180+0000 7fd56c000700 1 -- 172.21.15.196:0/899939887 wait complete. 2024-03-13T14:39:19.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.181+0000 7fd56c000700 1 Processor -- start 2024-03-13T14:39:19.174 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.182+0000 7fd56c000700 1 -- start start 2024-03-13T14:39:19.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.182+0000 7fd56c000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd56419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:19.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.182+0000 7fd56c000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56419c690 con 0x7fd564074dc0 2024-03-13T14:39:19.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.182+0000 7fd569600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd56419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:19.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.183+0000 7fd569600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd56419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:48526/0 (socket says 172.21.15.196:48526) 2024-03-13T14:39:19.175 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.183+0000 7fd569600700 1 -- 172.21.15.196:0/3258143891 learned_addr learned my addr 172.21.15.196:0/3258143891 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:19.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.183+0000 7fd569600700 1 -- 172.21.15.196:0/3258143891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5580095f0 con 0x7fd564074dc0 2024-03-13T14:39:19.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.184+0000 7fd569600700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd56419c150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd558005b40 tx=0x7fd558004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:19.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.184+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd55801d070 con 0x7fd564074dc0 2024-03-13T14:39:19.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.184+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5641044f0 con 0x7fd564074dc0 2024-03-13T14:39:19.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.184+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd55800be10 con 0x7fd564074dc0 2024-03-13T14:39:19.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.184+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56419cba0 con 0x7fd564074dc0 2024-03-13T14:39:19.177 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.184+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd55800f460 con 0x7fd564074dc0 2024-03-13T14:39:19.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.185+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd564195c70 con 0x7fd564074dc0 2024-03-13T14:39:19.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.185+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 13) v1 ==== 45340+0+0 (secure 0 0 0) 0x7fd5580227e0 con 0x7fd564074dc0 2024-03-13T14:39:19.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.186+0000 7fd562000700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 0x7fd54c03a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:19.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.186+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd55804c310 con 0x7fd564074dc0 2024-03-13T14:39:19.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.186+0000 7fd568c00700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 0x7fd54c03a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:19.179 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.187+0000 7fd568c00700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 0x7fd54c03a880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd550006fd0 tx=0x7fd550006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:19.191 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.200+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd558022470 con 0x7fd564074dc0 2024-03-13T14:39:19.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.416+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7fd55800baa0 con 0x7fd564074dc0 2024-03-13T14:39:19.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.488+0000 7fd568c00700 1 -- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 msgr2=0x7fd54c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-03-13T14:39:19.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.488+0000 7fd568c00700 1 -- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 msgr2=0x7fd54c03a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-13T14:39:19.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.488+0000 7fd568c00700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 0x7fd54c03a880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd550006fd0 tx=0x7fd550006e40 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T14:39:19.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.488+0000 7fd568c00700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 0x7fd54c03a880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd550006fd0 tx=0x7fd550006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:19.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.488+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 msgr2=0x7fd54c03a880 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-03-13T14:39:19.484 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.488+0000 7fd562000700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 0x7fd54c03a880 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:19.500 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:19.500 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:19.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.503+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd5400051a0 con 0x7fd564074dc0 2024-03-13T14:39:19.501 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.507+0000 7fd562000700 1 -- 172.21.15.196:0/3258143891 <== mon.0 v2:172.21.15.176:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fd558027070 con 0x7fd564074dc0 2024-03-13T14:39:19.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.511+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 msgr2=0x7fd56419c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:19.503 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.511+0000 7fd56c000700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd56419c150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd558005b40 tx=0x7fd558004740 comp rx=0 tx=0).stop 2024-03-13T14:39:19.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.512+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 shutdown_connections 2024-03-13T14:39:19.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.512+0000 7fd56c000700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7fd54c0383c0 0x7fd54c03a880 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:19.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.512+0000 7fd56c000700 1 --2- 172.21.15.196:0/3258143891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd564074dc0 0x7fd56419c150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:19.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.512+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 >> 172.21.15.196:0/3258143891 conn(0x7fd5640fbe30 msgr2=0x7fd5640fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:19.504 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.512+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 shutdown_connections 2024-03-13T14:39:19.506 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:19.512+0000 7fd56c000700 1 -- 172.21.15.196:0/3258143891 wait complete. 2024-03-13T14:39:19.508 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:20.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:20 smithi176 ceph-mon[29064]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-13T14:39:20.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:20 smithi176 ceph-mon[29064]: mgrmap e14: smithi176.tzfdwd(active, since 91s) 2024-03-13T14:39:20.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:20 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/3258143891' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:21.183 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:21.184 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:21.566 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:21.674 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:23.616 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.624+0000 7f9eb0a00700 1 -- 172.21.15.196:0/4019483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 msgr2=0x7f9eac102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.624+0000 7f9eb0a00700 1 --2- 172.21.15.196:0/4019483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac102eb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009910 tx=0x7f9e9c009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.624+0000 7f9eb0a00700 1 -- 172.21.15.196:0/4019483596 shutdown_connections 2024-03-13T14:39:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.624+0000 7f9eb0a00700 1 --2- 172.21.15.196:0/4019483596 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac102eb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.624+0000 7f9eb0a00700 1 -- 172.21.15.196:0/4019483596 >> 172.21.15.196:0/4019483596 conn(0x7f9eac0fe010 msgr2=0x7f9eac100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.624+0000 7f9eb0a00700 1 -- 172.21.15.196:0/4019483596 shutdown_connections 2024-03-13T14:39:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.624+0000 7f9eb0a00700 1 -- 172.21.15.196:0/4019483596 wait complete. 2024-03-13T14:39:23.617 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.625+0000 7f9eb0a00700 1 Processor -- start 2024-03-13T14:39:23.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.626+0000 7f9eb0a00700 1 -- start start 2024-03-13T14:39:23.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.626+0000 7f9eb0a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:23.618 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.626+0000 7f9eb0a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1980f0 con 0x7f9eac102a90 2024-03-13T14:39:23.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.626+0000 7f9ea9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:23.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.627+0000 7f9ea9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:37704/0 (socket says 172.21.15.196:37704) 2024-03-13T14:39:23.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.627+0000 7f9ea9e00700 1 -- 172.21.15.196:0/1268753705 learned_addr learned my addr 172.21.15.196:0/1268753705 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:23.619 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.627+0000 7f9ea9e00700 1 -- 172.21.15.196:0/1268753705 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9c0095f0 con 0x7f9eac102a90 2024-03-13T14:39:23.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.628+0000 7f9ea9e00700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac197bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005950 tx=0x7f9e9c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:23.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.628+0000 7f9ea2a00700 1 -- 172.21.15.196:0/1268753705 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e9c01d070 con 0x7f9eac102a90 2024-03-13T14:39:23.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.628+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac1982f0 con 0x7f9eac102a90 2024-03-13T14:39:23.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.628+0000 7f9ea2a00700 1 -- 172.21.15.196:0/1268753705 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c00be10 con 0x7f9eac102a90 2024-03-13T14:39:23.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.628+0000 7f9ea2a00700 1 -- 172.21.15.196:0/1268753705 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e9c00f460 con 0x7f9eac102a90 2024-03-13T14:39:23.620 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.628+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac198790 con 0x7f9eac102a90 2024-03-13T14:39:23.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.629+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac1916b0 con 0x7f9eac102a90 2024-03-13T14:39:23.621 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.630+0000 7f9ea2a00700 1 -- 172.21.15.196:0/1268753705 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 14) v1 ==== 45354+0+0 (secure 0 0 0) 0x7f9e9c0227c0 con 0x7f9eac102a90 2024-03-13T14:39:23.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.630+0000 7f9ea2a00700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 0x7f9e9003a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:23.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.630+0000 7f9ea2a00700 1 -- 172.21.15.196:0/1268753705 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9e9c04c490 con 0x7f9eac102a90 2024-03-13T14:39:23.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.630+0000 7f9ea9400700 1 -- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 msgr2=0x7f9e9003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/456564682 2024-03-13T14:39:23.622 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.630+0000 7f9ea9400700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 0x7f9e9003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T14:39:23.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.640+0000 7f9ea2a00700 1 -- 172.21.15.196:0/1268753705 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9e9c02bba0 con 0x7f9eac102a90 2024-03-13T14:39:23.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.831+0000 7f9ea9400700 1 -- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 msgr2=0x7f9e9003a8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.176:6800/456564682 2024-03-13T14:39:23.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.831+0000 7f9ea9400700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 0x7f9e9003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-13T14:39:23.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.933+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9eac02cc70 con 0x7f9eac102a90 2024-03-13T14:39:23.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.934+0000 7f9ea2a00700 1 -- 172.21.15.196:0/1268753705 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9e9c027070 con 0x7f9eac102a90 2024-03-13T14:39:23.926 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:23.926 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 msgr2=0x7f9e9003a8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T14:39:23.928 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 0x7f9e9003a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 msgr2=0x7f9eac197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac197bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c005950 tx=0x7f9e9c009e60 comp rx=0 tx=0).stop 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 shutdown_connections 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:6800/456564682,v1:172.21.15.176:6801/456564682] conn(0x7f9e90038410 0x7f9e9003a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 --2- 172.21.15.196:0/1268753705 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9eac102a90 0x7f9eac197bb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.936+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 >> 172.21.15.196:0/1268753705 conn(0x7f9eac0fe010 msgr2=0x7f9eac0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.937+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 shutdown_connections 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:23.937+0000 7f9eb0a00700 1 -- 172.21.15.196:0/1268753705 wait complete. 2024-03-13T14:39:23.929 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:24.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:23 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1268753705' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:25.603 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:25.604 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:25.972 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:26.072 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.098+0000 7fb797e00700 1 -- 172.21.15.196:0/3307933242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 msgr2=0x7fb7980ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.098+0000 7fb797e00700 1 --2- 172.21.15.196:0/3307933242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7980ff9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb788009910 tx=0x7fb788009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.098+0000 7fb797e00700 1 -- 172.21.15.196:0/3307933242 shutdown_connections 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.098+0000 7fb797e00700 1 --2- 172.21.15.196:0/3307933242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7980ff9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.098+0000 7fb797e00700 1 -- 172.21.15.196:0/3307933242 >> 172.21.15.196:0/3307933242 conn(0x7fb7980fab20 msgr2=0x7fb7980fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.098+0000 7fb797e00700 1 -- 172.21.15.196:0/3307933242 shutdown_connections 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.098+0000 7fb797e00700 1 -- 172.21.15.196:0/3307933242 wait complete. 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.100+0000 7fb797e00700 1 Processor -- start 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.100+0000 7fb797e00700 1 -- start start 2024-03-13T14:39:28.095 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.100+0000 7fb797e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7981979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.100+0000 7fb797e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb798197f00 con 0x7fb7980ff5a0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.101+0000 7fb796a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7981979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.101+0000 7fb796a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7981979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:37728/0 (socket says 172.21.15.196:37728) 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.101+0000 7fb796a00700 1 -- 172.21.15.196:0/2359331904 learned_addr learned my addr 172.21.15.196:0/2359331904 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.101+0000 7fb796a00700 1 -- 172.21.15.196:0/2359331904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7880095f0 con 0x7fb7980ff5a0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.102+0000 7fb796a00700 1 --2- 172.21.15.196:0/2359331904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7981979c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb788005b40 tx=0x7fb788004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.102+0000 7fb79ca00700 1 -- 172.21.15.196:0/2359331904 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb78801d070 con 0x7fb7980ff5a0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.102+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb798198100 con 0x7fb7980ff5a0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.102+0000 7fb79ca00700 1 -- 172.21.15.196:0/2359331904 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb78800be10 con 0x7fb7980ff5a0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.102+0000 7fb79ca00700 1 -- 172.21.15.196:0/2359331904 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb78800f460 con 0x7fb7980ff5a0 2024-03-13T14:39:28.096 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.102+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7981985a0 con 0x7fb7980ff5a0 2024-03-13T14:39:28.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.103+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7981915d0 con 0x7fb7980ff5a0 2024-03-13T14:39:28.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.104+0000 7fb79ca00700 1 -- 172.21.15.196:0/2359331904 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 15) v1 ==== 45077+0+0 (secure 0 0 0) 0x7fb78800b960 con 0x7fb7980ff5a0 2024-03-13T14:39:28.097 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.104+0000 7fb79ca00700 1 -- 172.21.15.196:0/2359331904 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb78804c270 con 0x7fb7980ff5a0 2024-03-13T14:39:28.105 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.113+0000 7fb79ca00700 1 -- 172.21.15.196:0/2359331904 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb788005c00 con 0x7fb7980ff5a0 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: Active manager daemon smithi176.tzfdwd restarted 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: Activating manager daemon smithi176.tzfdwd 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: osdmap e5: 0 total, 0 up, 0 in 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: mgrmap e15: smithi176.tzfdwd(active, starting, since 0.0090215s) 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:39:28.220 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:39:28.221 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:39:28.221 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:28 smithi176 ceph-mon[29064]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:39:28.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.415+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb79802cc70 con 0x7fb7980ff5a0 2024-03-13T14:39:28.408 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.415+0000 7fb79ca00700 1 -- 172.21.15.196:0/2359331904 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb788027030 con 0x7fb7980ff5a0 2024-03-13T14:39:28.408 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:28.408 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:28.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.418+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 msgr2=0x7fb7981979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:28.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.418+0000 7fb797e00700 1 --2- 172.21.15.196:0/2359331904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7981979c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb788005b40 tx=0x7fb788004740 comp rx=0 tx=0).stop 2024-03-13T14:39:28.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.418+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 shutdown_connections 2024-03-13T14:39:28.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.418+0000 7fb797e00700 1 --2- 172.21.15.196:0/2359331904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb7980ff5a0 0x7fb7981979c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:28.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.418+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 >> 172.21.15.196:0/2359331904 conn(0x7fb7980fab20 msgr2=0x7fb798068830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:28.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.418+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 shutdown_connections 2024-03-13T14:39:28.410 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:28.418+0000 7fb797e00700 1 -- 172.21.15.196:0/2359331904 wait complete. 2024-03-13T14:39:28.411 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2359331904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:29.345 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:29 smithi176 ceph-mon[29064]: mgrmap e16: smithi176.tzfdwd(active, since 1.01092s) 2024-03-13T14:39:30.114 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:30.114 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:30.450 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:30.555 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:30.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:39:28] ENGINE Bus STARTING 2024-03-13T14:39:30.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:39:28] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:39:30.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:39:28] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:39:30.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:39:28] ENGINE Bus STARTED 2024-03-13T14:39:30.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:30.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:31.852 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:31 smithi176 ceph-mon[29064]: mgrmap e17: smithi176.tzfdwd(active, since 2s) 2024-03-13T14:39:31.852 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:31 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.362+0000 7f35d2c00700 1 -- 172.21.15.196:0/3637765218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 msgr2=0x7f35cc100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.362+0000 7f35d2c00700 1 --2- 172.21.15.196:0/3637765218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc100cd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009910 tx=0x7f35bc009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.363+0000 7f35d2c00700 1 -- 172.21.15.196:0/3637765218 shutdown_connections 2024-03-13T14:39:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.363+0000 7f35d2c00700 1 --2- 172.21.15.196:0/3637765218 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc100cd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.363+0000 7f35d2c00700 1 -- 172.21.15.196:0/3637765218 >> 172.21.15.196:0/3637765218 conn(0x7f35cc0fbe30 msgr2=0x7f35cc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.363+0000 7f35d2c00700 1 -- 172.21.15.196:0/3637765218 shutdown_connections 2024-03-13T14:39:32.356 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.363+0000 7f35d2c00700 1 -- 172.21.15.196:0/3637765218 wait complete. 2024-03-13T14:39:32.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.364+0000 7f35d2c00700 1 Processor -- start 2024-03-13T14:39:32.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.364+0000 7f35d2c00700 1 -- start start 2024-03-13T14:39:32.357 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.365+0000 7f35d2c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:32.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.365+0000 7f35d2c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc075070 con 0x7f35cc1008b0 2024-03-13T14:39:32.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.365+0000 7f35cbe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:32.358 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.365+0000 7f35cbe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc074b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:37752/0 (socket says 172.21.15.196:37752) 2024-03-13T14:39:32.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.365+0000 7f35cbe00700 1 -- 172.21.15.196:0/3905342592 learned_addr learned my addr 172.21.15.196:0/3905342592 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:32.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.366+0000 7f35cbe00700 1 -- 172.21.15.196:0/3905342592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc0095f0 con 0x7f35cc1008b0 2024-03-13T14:39:32.359 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.366+0000 7f35cbe00700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc074b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f35bc005950 tx=0x7f35bc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:32.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.367+0000 7f35c8c00700 1 -- 172.21.15.196:0/3905342592 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35bc01d070 con 0x7f35cc1008b0 2024-03-13T14:39:32.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.367+0000 7f35c8c00700 1 -- 172.21.15.196:0/3905342592 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f35bc00be10 con 0x7f35cc1008b0 2024-03-13T14:39:32.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.367+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc073180 con 0x7f35cc1008b0 2024-03-13T14:39:32.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.367+0000 7f35c8c00700 1 -- 172.21.15.196:0/3905342592 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35bc00f460 con 0x7f35cc1008b0 2024-03-13T14:39:32.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.367+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc073620 con 0x7f35cc1008b0 2024-03-13T14:39:32.360 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.368+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc0623c0 con 0x7f35cc1008b0 2024-03-13T14:39:32.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.368+0000 7f35c8c00700 1 -- 172.21.15.196:0/3905342592 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f35bc0227c0 con 0x7f35cc1008b0 2024-03-13T14:39:32.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.369+0000 7f35c8c00700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f35b00384b0 0x7f35b003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:32.361 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.369+0000 7f35c8c00700 1 -- 172.21.15.196:0/3905342592 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f35bc04c630 con 0x7f35cc1008b0 2024-03-13T14:39:32.362 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.369+0000 7f35cb400700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f35b00384b0 0x7f35b003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:32.363 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.370+0000 7f35cb400700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f35b00384b0 0x7f35b003a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8006e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:32.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.385+0000 7f35c8c00700 1 -- 172.21.15.196:0/3905342592 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f35bc02bba0 con 0x7f35cc1008b0 2024-03-13T14:39:32.684 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.691+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f35cc18eeb0 con 0x7f35cc1008b0 2024-03-13T14:39:32.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.692+0000 7f35c8c00700 1 -- 172.21.15.196:0/3905342592 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f35bc027030 con 0x7f35cc1008b0 2024-03-13T14:39:32.686 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:32.686 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:32.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f35b00384b0 msgr2=0x7f35b003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:32.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f35b00384b0 0x7f35b003a970 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f35b8006fd0 tx=0x7f35b8006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:32.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 msgr2=0x7f35cc074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:32.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc074b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f35bc005950 tx=0x7f35bc009e60 comp rx=0 tx=0).stop 2024-03-13T14:39:32.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 shutdown_connections 2024-03-13T14:39:32.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f35b00384b0 0x7f35b003a970 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:32.689 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 --2- 172.21.15.196:0/3905342592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f35cc1008b0 0x7f35cc074b30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:32.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.695+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 >> 172.21.15.196:0/3905342592 conn(0x7f35cc0fbe30 msgr2=0x7f35cc0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:32.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.696+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 shutdown_connections 2024-03-13T14:39:32.690 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:32.696+0000 7f35d2c00700 1 -- 172.21.15.196:0/3905342592 wait complete. 2024-03-13T14:39:32.690 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:33.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:32 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/3905342592' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:35.245 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:34 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:35.245 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:34 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:35.414 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:35.415 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:35.779 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:35.886 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /etc/ceph/ceph.conf 2024-03-13T14:39:36.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:36 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:36.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:36 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:36.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:36 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:39:38.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:39:38.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:37 smithi176 ceph-mon[29064]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:39:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.576+0000 7fdc41400700 1 -- 172.21.15.196:0/418914263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 msgr2=0x7fdc3c100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.576+0000 7fdc41400700 1 --2- 172.21.15.196:0/418914263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c100cd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b0a0 tx=0x7fdc2c00b3b0 comp rx=0 tx=0).stop 2024-03-13T14:39:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.577+0000 7fdc41400700 1 -- 172.21.15.196:0/418914263 shutdown_connections 2024-03-13T14:39:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.577+0000 7fdc41400700 1 --2- 172.21.15.196:0/418914263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c100cd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.577+0000 7fdc41400700 1 -- 172.21.15.196:0/418914263 >> 172.21.15.196:0/418914263 conn(0x7fdc3c0fbe30 msgr2=0x7fdc3c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.577+0000 7fdc41400700 1 -- 172.21.15.196:0/418914263 shutdown_connections 2024-03-13T14:39:38.570 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.577+0000 7fdc41400700 1 -- 172.21.15.196:0/418914263 wait complete. 2024-03-13T14:39:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.578+0000 7fdc41400700 1 Processor -- start 2024-03-13T14:39:38.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.578+0000 7fdc41400700 1 -- start start 2024-03-13T14:39:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.578+0000 7fdc41400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.578+0000 7fdc41400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c198170 con 0x7fdc3c1008b0 2024-03-13T14:39:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.579+0000 7fdc3a800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.579+0000 7fdc3a800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:56676/0 (socket says 172.21.15.196:56676) 2024-03-13T14:39:38.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.579+0000 7fdc3a800700 1 -- 172.21.15.196:0/2603004718 learned_addr learned my addr 172.21.15.196:0/2603004718 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.579+0000 7fdc3a800700 1 -- 172.21.15.196:0/2603004718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c009d00 con 0x7fdc3c1008b0 2024-03-13T14:39:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.580+0000 7fdc3a800700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c197c30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009fd0 tx=0x7fdc2c00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.580+0000 7fdc33400700 1 -- 172.21.15.196:0/2603004718 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc2c00e040 con 0x7fdc3c1008b0 2024-03-13T14:39:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.580+0000 7fdc33400700 1 -- 172.21.15.196:0/2603004718 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc2c00bdc0 con 0x7fdc3c1008b0 2024-03-13T14:39:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.580+0000 7fdc33400700 1 -- 172.21.15.196:0/2603004718 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc2c01da40 con 0x7fdc3c1008b0 2024-03-13T14:39:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.580+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c198370 con 0x7fdc3c1008b0 2024-03-13T14:39:38.573 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.580+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c198790 con 0x7fdc3c1008b0 2024-03-13T14:39:38.574 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.581+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c1918d0 con 0x7fdc3c1008b0 2024-03-13T14:39:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.581+0000 7fdc33400700 1 -- 172.21.15.196:0/2603004718 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fdc2c01d480 con 0x7fdc3c1008b0 2024-03-13T14:39:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.582+0000 7fdc33400700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc20038550 0x7fdc2003aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.582+0000 7fdc33400700 1 -- 172.21.15.196:0/2603004718 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fdc2c04ca90 con 0x7fdc3c1008b0 2024-03-13T14:39:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.582+0000 7fdc33e00700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc20038550 0x7fdc2003aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:38.575 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.583+0000 7fdc33e00700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc20038550 0x7fdc2003aa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc24006fd0 tx=0x7fdc24006e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:38.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.589+0000 7fdc33400700 1 -- 172.21.15.196:0/2603004718 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdc2c01c9b0 con 0x7fdc3c1008b0 2024-03-13T14:39:38.881 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.889+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdc3c02cc70 con 0x7fdc3c1008b0 2024-03-13T14:39:38.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.890+0000 7fdc33400700 1 -- 172.21.15.196:0/2603004718 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fdc2c01c3e0 con 0x7fdc3c1008b0 2024-03-13T14:39:38.883 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:38.883 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.892+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc20038550 msgr2=0x7fdc2003aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.892+0000 7fdc41400700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc20038550 0x7fdc2003aa10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdc24006fd0 tx=0x7fdc24006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:38.885 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.892+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 msgr2=0x7fdc3c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.892+0000 7fdc41400700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c197c30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009fd0 tx=0x7fdc2c00bb10 comp rx=0 tx=0).stop 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.893+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 shutdown_connections 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.893+0000 7fdc41400700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc20038550 0x7fdc2003aa10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.893+0000 7fdc41400700 1 --2- 172.21.15.196:0/2603004718 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc3c1008b0 0x7fdc3c197c30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.893+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 >> 172.21.15.196:0/2603004718 conn(0x7fdc3c0fbe30 msgr2=0x7fdc3c0fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.893+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 shutdown_connections 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:38.893+0000 7fdc41400700 1 -- 172.21.15.196:0/2603004718 wait complete. 2024-03-13T14:39:38.886 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:39.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:38 smithi176 ceph-mon[29064]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:39:39.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:38 smithi176 ceph-mon[29064]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:39:39.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:38 smithi176 ceph-mon[29064]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:39:39.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:38 smithi176 ceph-mon[29064]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:39:40.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:39:40.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:39:40.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2603004718' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:40.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:40.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:40.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:40.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:40.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:40.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:39:40.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-13T14:39:40.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:39:40.644 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:40.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:41.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:40 smithi176 ceph-mon[29064]: Deploying daemon ceph-exporter.smithi196 on smithi196 2024-03-13T14:39:41.447 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:39:43.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:39:43.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.629+0000 7fc64dc00700 1 -- 172.21.15.196:0/595416071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 msgr2=0x7fc648073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:43.623 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.629+0000 7fc64dc00700 1 --2- 172.21.15.196:0/595416071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648073490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc638009910 tx=0x7fc638009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.630+0000 7fc64dc00700 1 -- 172.21.15.196:0/595416071 shutdown_connections 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.630+0000 7fc64dc00700 1 --2- 172.21.15.196:0/595416071 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648073490 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.630+0000 7fc64dc00700 1 -- 172.21.15.196:0/595416071 >> 172.21.15.196:0/595416071 conn(0x7fc6480fc160 msgr2=0x7fc6480fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.630+0000 7fc64dc00700 1 -- 172.21.15.196:0/595416071 shutdown_connections 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.630+0000 7fc64dc00700 1 -- 172.21.15.196:0/595416071 wait complete. 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.631+0000 7fc64dc00700 1 Processor -- start 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.631+0000 7fc64dc00700 1 -- start start 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.632+0000 7fc64dc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:43.624 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.632+0000 7fc64dc00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc648198340 con 0x7fc6480730b0 2024-03-13T14:39:43.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.632+0000 7fc647e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:43.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.632+0000 7fc647e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:39544/0 (socket says 172.21.15.196:39544) 2024-03-13T14:39:43.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.632+0000 7fc647e00700 1 -- 172.21.15.196:0/1274836106 learned_addr learned my addr 172.21.15.196:0/1274836106 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:43.625 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.633+0000 7fc647e00700 1 -- 172.21.15.196:0/1274836106 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380095f0 con 0x7fc6480730b0 2024-03-13T14:39:43.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.633+0000 7fc647e00700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648197e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc6380092e0 tx=0x7fc638004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:43.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.634+0000 7fc644c00700 1 -- 172.21.15.196:0/1274836106 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc63801d070 con 0x7fc6480730b0 2024-03-13T14:39:43.626 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.634+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc648198540 con 0x7fc6480730b0 2024-03-13T14:39:43.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.634+0000 7fc644c00700 1 -- 172.21.15.196:0/1274836106 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc63800be10 con 0x7fc6480730b0 2024-03-13T14:39:43.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.634+0000 7fc644c00700 1 -- 172.21.15.196:0/1274836106 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc638022e10 con 0x7fc6480730b0 2024-03-13T14:39:43.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.634+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc648198960 con 0x7fc6480730b0 2024-03-13T14:39:43.627 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.635+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6480623c0 con 0x7fc6480730b0 2024-03-13T14:39:43.628 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.636+0000 7fc644c00700 1 -- 172.21.15.196:0/1274836106 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fc638022470 con 0x7fc6480730b0 2024-03-13T14:39:43.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.636+0000 7fc644c00700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc62c038460 0x7fc62c03a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:43.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.636+0000 7fc644c00700 1 -- 172.21.15.196:0/1274836106 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fc63804c560 con 0x7fc6480730b0 2024-03-13T14:39:43.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.637+0000 7fc647400700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc62c038460 0x7fc62c03a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:43.633 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.638+0000 7fc647400700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc62c038460 0x7fc62c03a920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc634006fd0 tx=0x7fc634006e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:43.645 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.652+0000 7fc644c00700 1 -- 172.21.15.196:0/1274836106 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc63802a540 con 0x7fc6480730b0 2024-03-13T14:39:43.946 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.953+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc648109b80 con 0x7fc6480730b0 2024-03-13T14:39:43.947 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.955+0000 7fc644c00700 1 -- 172.21.15.196:0/1274836106 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fc638027030 con 0x7fc6480730b0 2024-03-13T14:39:43.948 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:43.948 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc62c038460 msgr2=0x7fc62c03a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc62c038460 0x7fc62c03a920 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc634006fd0 tx=0x7fc634006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 msgr2=0x7fc648197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648197e00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc6380092e0 tx=0x7fc638004c80 comp rx=0 tx=0).stop 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 shutdown_connections 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc62c038460 0x7fc62c03a920 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 --2- 172.21.15.196:0/1274836106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc6480730b0 0x7fc648197e00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 >> 172.21.15.196:0/1274836106 conn(0x7fc6480fc160 msgr2=0x7fc648107210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 shutdown_connections 2024-03-13T14:39:43.951 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:43.958+0000 7fc64dc00700 1 -- 172.21.15.196:0/1274836106 wait complete. 2024-03-13T14:39:43.952 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:44.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:44 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1274836106' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:45.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:45.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:45.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:45.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:45.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:39:45.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-13T14:39:45.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:39:45.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:45 smithi176 ceph-mon[29064]: Deploying daemon crash.smithi196 on smithi196 2024-03-13T14:39:45.929 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:45.929 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:46.508 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:39:48.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.472+0000 7f49c0a00700 1 -- 172.21.15.196:0/3987708331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 msgr2=0x7f49bc108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:48.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.472+0000 7f49c0a00700 1 --2- 172.21.15.196:0/3987708331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc108c20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009910 tx=0x7f49ac009c20 comp rx=0 tx=0).stop 2024-03-13T14:39:48.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.472+0000 7f49c0a00700 1 -- 172.21.15.196:0/3987708331 shutdown_connections 2024-03-13T14:39:48.465 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.472+0000 7f49c0a00700 1 --2- 172.21.15.196:0/3987708331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc108c20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:48.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.472+0000 7f49c0a00700 1 -- 172.21.15.196:0/3987708331 >> 172.21.15.196:0/3987708331 conn(0x7f49bc0fe2f0 msgr2=0x7f49bc100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:48.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.472+0000 7f49c0a00700 1 -- 172.21.15.196:0/3987708331 shutdown_connections 2024-03-13T14:39:48.466 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.472+0000 7f49c0a00700 1 -- 172.21.15.196:0/3987708331 wait complete. 2024-03-13T14:39:48.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.473+0000 7f49c0a00700 1 Processor -- start 2024-03-13T14:39:48.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.473+0000 7f49c0a00700 1 -- start start 2024-03-13T14:39:48.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.474+0000 7f49c0a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:48.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.474+0000 7f49c0a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc1982d0 con 0x7f49bc108840 2024-03-13T14:39:48.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.474+0000 7f49bb400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:48.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.474+0000 7f49bb400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:39564/0 (socket says 172.21.15.196:39564) 2024-03-13T14:39:48.467 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.474+0000 7f49bb400700 1 -- 172.21.15.196:0/1825842543 learned_addr learned my addr 172.21.15.196:0/1825842543 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:48.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.475+0000 7f49bb400700 1 -- 172.21.15.196:0/1825842543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0095f0 con 0x7f49bc108840 2024-03-13T14:39:48.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.475+0000 7f49bb400700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc197d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0092e0 tx=0x7f49ac004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:48.468 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.476+0000 7f49b3e00700 1 -- 172.21.15.196:0/1825842543 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49ac01d070 con 0x7f49bc108840 2024-03-13T14:39:48.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.476+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc198530 con 0x7f49bc108840 2024-03-13T14:39:48.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.476+0000 7f49b3e00700 1 -- 172.21.15.196:0/1825842543 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f49ac00be10 con 0x7f49bc108840 2024-03-13T14:39:48.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.476+0000 7f49b3e00700 1 -- 172.21.15.196:0/1825842543 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49ac022e10 con 0x7f49bc108840 2024-03-13T14:39:48.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.476+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc1989b0 con 0x7f49bc108840 2024-03-13T14:39:48.469 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.477+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc0623c0 con 0x7f49bc108840 2024-03-13T14:39:48.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.477+0000 7f49b3e00700 1 -- 172.21.15.196:0/1825842543 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f49ac022470 con 0x7f49bc108840 2024-03-13T14:39:48.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.478+0000 7f49b3e00700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f49a0038460 0x7f49a003a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:48.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.478+0000 7f49b3e00700 1 -- 172.21.15.196:0/1825842543 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f49ac04c4f0 con 0x7f49bc108840 2024-03-13T14:39:48.471 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.478+0000 7f49baa00700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f49a0038460 0x7f49a003a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:48.472 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.480+0000 7f49baa00700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f49a0038460 0x7f49a003a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49a8006fd0 tx=0x7f49a8006e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:48.485 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.492+0000 7f49b3e00700 1 -- 172.21.15.196:0/1825842543 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f49ac00f460 con 0x7f49bc108840 2024-03-13T14:39:48.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.800+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f49bc106650 con 0x7f49bc108840 2024-03-13T14:39:48.793 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.801+0000 7f49b3e00700 1 -- 172.21.15.196:0/1825842543 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f49ac027030 con 0x7f49bc108840 2024-03-13T14:39:48.794 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:48.794 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:48.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f49a0038460 msgr2=0x7f49a003a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:48.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f49a0038460 0x7f49a003a920 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f49a8006fd0 tx=0x7f49a8006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:48.796 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 msgr2=0x7f49bc197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc197d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0092e0 tx=0x7f49ac004c80 comp rx=0 tx=0).stop 2024-03-13T14:39:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 shutdown_connections 2024-03-13T14:39:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f49a0038460 0x7f49a003a920 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 --2- 172.21.15.196:0/1825842543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f49bc108840 0x7f49bc197d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 >> 172.21.15.196:0/1825842543 conn(0x7f49bc0fe2f0 msgr2=0x7f49bc10abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 shutdown_connections 2024-03-13T14:39:48.797 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:48.804+0000 7f49c0a00700 1 -- 172.21.15.196:0/1825842543 wait complete. 2024-03-13T14:39:48.798 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:49.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:49 smithi176 ceph-mon[29064]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:49.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:49.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:49 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1825842543' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:49.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:49.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:49.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:49.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:50.297 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:50 smithi176 ceph-mon[29064]: Deploying daemon node-exporter.smithi196 on smithi196 2024-03-13T14:39:50.484 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:50.485 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:50.843 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:39:51.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:51 smithi176 ceph-mon[29064]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:53.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:53 smithi176 ceph-mon[29064]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.556+0000 7fe41c000700 1 -- 172.21.15.196:0/3734007244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 msgr2=0x7fe414106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.556+0000 7fe41c000700 1 --2- 172.21.15.196:0/3734007244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414106bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe40800b0a0 tx=0x7fe40800b3b0 comp rx=0 tx=0).stop 2024-03-13T14:39:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.556+0000 7fe41c000700 1 -- 172.21.15.196:0/3734007244 shutdown_connections 2024-03-13T14:39:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.556+0000 7fe41c000700 1 --2- 172.21.15.196:0/3734007244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414106bf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.556+0000 7fe41c000700 1 -- 172.21.15.196:0/3734007244 >> 172.21.15.196:0/3734007244 conn(0x7fe4140fc320 msgr2=0x7fe4140fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:53.548 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.556+0000 7fe41c000700 1 -- 172.21.15.196:0/3734007244 shutdown_connections 2024-03-13T14:39:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.556+0000 7fe41c000700 1 -- 172.21.15.196:0/3734007244 wait complete. 2024-03-13T14:39:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.557+0000 7fe41c000700 1 Processor -- start 2024-03-13T14:39:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.557+0000 7fe41c000700 1 -- start start 2024-03-13T14:39:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.558+0000 7fe41c000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:53.549 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.558+0000 7fe41c000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414198440 con 0x7fe414106810 2024-03-13T14:39:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.558+0000 7fe419600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.559+0000 7fe419600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:44264/0 (socket says 172.21.15.196:44264) 2024-03-13T14:39:53.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.559+0000 7fe419600700 1 -- 172.21.15.196:0/2989234527 learned_addr learned my addr 172.21.15.196:0/2989234527 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:39:53.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.559+0000 7fe419600700 1 -- 172.21.15.196:0/2989234527 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009d00 con 0x7fe414106810 2024-03-13T14:39:53.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.559+0000 7fe419600700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414197f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe40800d010 tx=0x7fe408003ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:53.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.560+0000 7fe412000700 1 -- 172.21.15.196:0/2989234527 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe40800e040 con 0x7fe414106810 2024-03-13T14:39:53.551 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.560+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe414198640 con 0x7fe414106810 2024-03-13T14:39:53.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.560+0000 7fe412000700 1 -- 172.21.15.196:0/2989234527 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe4080044f0 con 0x7fe414106810 2024-03-13T14:39:53.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.560+0000 7fe412000700 1 -- 172.21.15.196:0/2989234527 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe40801d900 con 0x7fe414106810 2024-03-13T14:39:53.552 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.560+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe414198ac0 con 0x7fe414106810 2024-03-13T14:39:53.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.561+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4140623c0 con 0x7fe414106810 2024-03-13T14:39:53.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.562+0000 7fe412000700 1 -- 172.21.15.196:0/2989234527 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7fe408004660 con 0x7fe414106810 2024-03-13T14:39:53.553 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.562+0000 7fe412000700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe3fc038500 0x7fe3fc03a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:39:53.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.562+0000 7fe412000700 1 -- 172.21.15.196:0/2989234527 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe40804c420 con 0x7fe414106810 2024-03-13T14:39:53.554 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.563+0000 7fe418c00700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe3fc038500 0x7fe3fc03a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:39:53.555 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.564+0000 7fe418c00700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe3fc038500 0x7fe3fc03a9c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe400006fd0 tx=0x7fe400006e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:39:53.568 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.577+0000 7fe412000700 1 -- 172.21.15.196:0/2989234527 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe408025540 con 0x7fe414106810 2024-03-13T14:39:53.878 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.886+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe41419c9f0 con 0x7fe414106810 2024-03-13T14:39:53.879 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.887+0000 7fe412000700 1 -- 172.21.15.196:0/2989234527 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe408025360 con 0x7fe414106810 2024-03-13T14:39:53.879 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:39:53.879 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe3fc038500 msgr2=0x7fe3fc03a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe3fc038500 0x7fe3fc03a9c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe400006fd0 tx=0x7fe400006e40 comp rx=0 tx=0).stop 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 msgr2=0x7fe414197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414197f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe40800d010 tx=0x7fe408003ad0 comp rx=0 tx=0).stop 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 shutdown_connections 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe3fc038500 0x7fe3fc03a9c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 --2- 172.21.15.196:0/2989234527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe414106810 0x7fe414197f00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 >> 172.21.15.196:0/2989234527 conn(0x7fe4140fc320 msgr2=0x7fe414108bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:39:53.882 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 shutdown_connections 2024-03-13T14:39:53.883 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:39:53.890+0000 7fe41c000700 1 -- 172.21.15.196:0/2989234527 wait complete. 2024-03-13T14:39:53.883 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:39:54.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:54 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:54.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:54 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2989234527' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:39:55.314 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:55 smithi176 ceph-mon[29064]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:56.119 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:39:56.119 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:39:57.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:57.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:57.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:39:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:39:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-13T14:39:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:39:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:39:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: Deploying daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:39:57.134 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:56 smithi176 ceph-mon[29064]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:57.164 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:39:59.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:59 smithi176 ceph-mon[29064]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:39:59.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:39:59.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:39:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.268+0000 7f79a2a00700 1 -- 172.21.15.196:0/1854535189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 msgr2=0x7f799c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.268+0000 7f79a2a00700 1 --2- 172.21.15.196:0/1854535189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c108e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f798c009910 tx=0x7f798c009c20 comp rx=0 tx=0).stop 2024-03-13T14:40:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.268+0000 7f79a2a00700 1 -- 172.21.15.196:0/1854535189 shutdown_connections 2024-03-13T14:40:00.426 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.268+0000 7f79a2a00700 1 --2- 172.21.15.196:0/1854535189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c108e10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.268+0000 7f79a2a00700 1 -- 172.21.15.196:0/1854535189 >> 172.21.15.196:0/1854535189 conn(0x7f799c0fe580 msgr2=0x7f799c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.269+0000 7f79a2a00700 1 -- 172.21.15.196:0/1854535189 shutdown_connections 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.269+0000 7f79a2a00700 1 -- 172.21.15.196:0/1854535189 wait complete. 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.270+0000 7f79a2a00700 1 Processor -- start 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.270+0000 7f79a2a00700 1 -- start start 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.270+0000 7f79a2a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.270+0000 7f79a2a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c198480 con 0x7f799c108a30 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.271+0000 7f799be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.271+0000 7f799be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.196:44302/0 (socket says 172.21.15.196:44302) 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.271+0000 7f799be00700 1 -- 172.21.15.196:0/1015201235 learned_addr learned my addr 172.21.15.196:0/1015201235 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:40:00.427 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.271+0000 7f799be00700 1 -- 172.21.15.196:0/1015201235 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798c0095f0 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.272+0000 7f799be00700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c197f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f798c0092e0 tx=0x7f798c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.272+0000 7f7998c00700 1 -- 172.21.15.196:0/1015201235 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f798c01d070 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.272+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c1986e0 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.272+0000 7f7998c00700 1 -- 172.21.15.196:0/1015201235 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f798c00be10 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.272+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c198b60 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.272+0000 7f7998c00700 1 -- 172.21.15.196:0/1015201235 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f798c00fdf0 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.273+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c0623c0 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.274+0000 7f7998c00700 1 -- 172.21.15.196:0/1015201235 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 17) v1 ==== 45398+0+0 (secure 0 0 0) 0x7f798c00f6b0 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.274+0000 7f7998c00700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f79800384b0 0x7f798003a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.274+0000 7f7998c00700 1 -- 172.21.15.196:0/1015201235 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f798c04cbf0 con 0x7f799c108a30 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.275+0000 7f799b400700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f79800384b0 0x7f798003a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.276+0000 7f799b400700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f79800384b0 0x7f798003a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7988006fd0 tx=0x7f7988006e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:00.428 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.289+0000 7f7998c00700 1 -- 172.21.15.196:0/1015201235 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f798c03c5a0 con 0x7f799c108a30 2024-03-13T14:40:00.571 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.579+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f799c1068b0 con 0x7f799c108a30 2024-03-13T14:40:00.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.581+0000 7f7998c00700 1 -- 172.21.15.196:0/1015201235 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f798c027070 con 0x7f799c108a30 2024-03-13T14:40:00.572 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:40:00.572 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:36:19.561944Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-13T14:40:00.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f79800384b0 msgr2=0x7f798003a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:00.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f79800384b0 0x7f798003a970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7988006fd0 tx=0x7f7988006e40 comp rx=0 tx=0).stop 2024-03-13T14:40:00.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 msgr2=0x7f799c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:00.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c197f40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f798c0092e0 tx=0x7f798c004c80 comp rx=0 tx=0).stop 2024-03-13T14:40:00.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 shutdown_connections 2024-03-13T14:40:00.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f79800384b0 0x7f798003a970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:00.578 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 --2- 172.21.15.196:0/1015201235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f799c108a30 0x7f799c197f40 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f798c0092e0 tx=0x7f798c004c80 comp rx=0 tx=0).stop 2024-03-13T14:40:00.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.584+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 >> 172.21.15.196:0/1015201235 conn(0x7f799c0fe580 msgr2=0x7f799c10ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:40:00.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.585+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 shutdown_connections 2024-03-13T14:40:00.579 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:00.585+0000 7f79a2a00700 1 -- 172.21.15.196:0/1015201235 wait complete. 2024-03-13T14:40:00.579 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: Deploying daemon mon.smithi196 on smithi196 2024-03-13T14:40:01.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:01 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1015201235' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:02.391 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-03-13T14:40:02.391 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mon dump -f json 2024-03-13T14:40:03.420 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:40:03.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:03 smithi176 ceph-mon[29064]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:03.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:03 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:05.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:05 smithi176 ceph-mon[29064]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:05.906 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:05 smithi196 podman[37445]: 2024-03-13 14:40:05.715359609 +0000 UTC m=+0.203591209 container create 5403fcfbde7282ff12b09239365ff198d264b965e97fc08ce52e69f99fd0f02c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 podman[37445]: 2024-03-13 14:40:06.002803752 +0000 UTC m=+0.491035379 container init 5403fcfbde7282ff12b09239365ff198d264b965e97fc08ce52e69f99fd0f02c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 podman[37445]: 2024-03-13 14:40:06.011571788 +0000 UTC m=+0.499803428 container start 5403fcfbde7282ff12b09239365ff198d264b965e97fc08ce52e69f99fd0f02c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: set uid:gid to 167:167 (ceph:ceph) 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable), process ceph-mon, pid 2 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pidfile_write: ignore empty --pid-file 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: load: jerasure load: lrc 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: RocksDB version: 7.9.2 2024-03-13T14:40:06.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Git sha 0 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Compile date 2023-12-11 22:07:34 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: DB SUMMARY 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: DB Session ID: WN5YTPIQFG5CRDXQGO9W 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: CURRENT file: CURRENT 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: IDENTITY file: IDENTITY 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi196/store.db dir, Total Num: 0, files: 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi196/store.db: 000004.log size: 511 ; 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.error_if_exists: 0 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.create_if_missing: 0 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.paranoid_checks: 1 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-13T14:40:06.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.env: 0x5628f14bc720 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.fs: PosixFileSystem 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.info_log: 0x5628f4a612e0 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_file_opening_threads: 16 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.statistics: (nil) 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.use_fsync: 0 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_log_file_size: 0 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.keep_log_file_num: 1000 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.recycle_log_file_num: 0 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.allow_fallocate: 1 2024-03-13T14:40:06.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.allow_mmap_reads: 0 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.allow_mmap_writes: 0 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.use_direct_reads: 0 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.create_missing_column_families: 0 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.db_log_dir: 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.wal_dir: 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.advise_random_on_open: 1 2024-03-13T14:40:06.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.db_write_buffer_size: 0 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.write_buffer_manager: 0x5628f3cd2140 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.rate_limiter: (nil) 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.wal_recovery_mode: 2 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.enable_thread_tracking: 0 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.enable_pipelined_write: 0 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.unordered_write: 0 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-13T14:40:06.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-13T14:40:06.188 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.row_cache: None 2024-03-13T14:40:06.188 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.wal_filter: None 2024-03-13T14:40:06.188 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-13T14:40:06.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.allow_ingest_behind: 0 2024-03-13T14:40:06.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.two_write_queues: 0 2024-03-13T14:40:06.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.manual_wal_flush: 0 2024-03-13T14:40:06.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.wal_compression: 0 2024-03-13T14:40:06.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.atomic_flush: 0 2024-03-13T14:40:06.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-13T14:40:06.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.log_readahead_size: 0 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.best_efforts_recovery: 0 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.allow_data_in_errors: 0 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.db_host_id: __hostname__ 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_background_jobs: 2 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_background_compactions: -1 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_subcompactions: 1 2024-03-13T14:40:06.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_total_wal_size: 0 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_open_files: -1 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bytes_per_sync: 0 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_readahead_size: 0 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_background_flushes: -1 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Compression algorithms supported: 2024-03-13T14:40:06.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kZSTD supported: 0 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kXpressCompression supported: 0 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kLZ4HCCompression supported: 1 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kZlibCompression supported: 1 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kSnappyCompression supported: 1 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kLZ4Compression supported: 1 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: kBZip2Compression supported: 0 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi196/store.db/MANIFEST-000005 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.merge_operator: 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_filter: None 2024-03-13T14:40:06.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_filter_factory: None 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.sst_partitioner_factory: None 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5628f4a613e0) 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: index_type: 0 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: data_block_index_type: 0 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: index_shortening: 1 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: checksum: 4 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: no_block_cache: 0 2024-03-13T14:40:06.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache: 0x5628f3d73350 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache_options: 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: capacity : 536870912 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: num_shard_bits : 4 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: strict_capacity_limit : 0 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache_compressed: (nil) 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: persistent_cache: (nil) 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_size: 4096 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_size_deviation: 10 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_restart_interval: 16 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: index_block_restart_interval: 1 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: metadata_block_size: 4096 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: partition_filters: 0 2024-03-13T14:40:06.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: use_delta_encoding: 1 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: filter_policy: bloomfilter 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: whole_key_filtering: 1 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: verify_compression: 0 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: format_version: 5 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: enable_index_compression: 1 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_align: 0 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: max_auto_readahead_size: 262144 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: prepopulate_block_cache: 0 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: initial_auto_readahead_size: 8192 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.write_buffer_size: 33554432 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_write_buffer_number: 2 2024-03-13T14:40:06.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression: NoCompression 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression: Disabled 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.prefix_extractor: nullptr 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.num_levels: 7 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-13T14:40:06.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.level: 32767 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.strategy: 0 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.enabled: false 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-13T14:40:06.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.target_file_size_base: 67108864 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-13T14:40:06.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.arena_block_size: 1048576 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.disable_auto_compactions: 0 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-13T14:40:06.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.table_properties_collectors: 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.inplace_update_support: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.bloom_locality: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.max_successive_merges: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.paranoid_file_checks: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.force_consistency_checks: 1 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.report_bg_io_stats: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.ttl: 2592000 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-13T14:40:06.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.enable_blob_files: false 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.min_blob_size: 0 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.blob_file_size: 268435456 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.blob_file_starting_level: 0 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi196/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9d11ef0e-c5f8-4649-b9e1-4277fbe3bcb9 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710340806063070, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-13T14:40:06.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710340806063863, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710340806, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9d11ef0e-c5f8-4649-b9e1-4277fbe3bcb9", "db_session_id": "WN5YTPIQFG5CRDXQGO9W", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710340806064087, "job": 1, "event": "recovery_finished"} 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi196/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5628f3e10000 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: DB pointer 0x5628f3dfc000 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** DB Stats ** 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:40:06.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:40:06.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Block cache BinnedLRUCache@0x5628f3d73350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-13T14:40:06.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196 does not exist in monmap, will attempt to join an existing cluster 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: using public_addr v2:172.21.15.196:0/0 -> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: starting mon.smithi196 rank -1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi196 fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(???) e0 preinit fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).mds e1 new map 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).mds e1 print_map 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: e1 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: legacy client fscid: -1 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: No filesystems configured 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-13T14:40:06.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.206 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.207 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.207 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.207 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-13T14:40:06.207 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14164 172.21.15.176:0/2812990986' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-13T14:40:06.207 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mgrmap e14: smithi176.tzfdwd(active, since 91s) 2024-03-13T14:40:06.207 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/3258143891' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/1268753705' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Active manager daemon smithi176.tzfdwd restarted 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Activating manager daemon smithi176.tzfdwd 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: osdmap e5: 0 total, 0 up, 0 in 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mgrmap e15: smithi176.tzfdwd(active, starting, since 0.0090215s) 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.208 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/2359331904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mgrmap e16: smithi176.tzfdwd(active, since 1.01092s) 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: [13/Mar/2024:14:39:28] ENGINE Bus STARTING 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: [13/Mar/2024:14:39:28] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: [13/Mar/2024:14:39:28] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: [13/Mar/2024:14:39:28] ENGINE Bus STARTED 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mgrmap e17: smithi176.tzfdwd(active, since 2s) 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/3905342592' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.209 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi176", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:40:06.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/2603004718' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:06.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Deploying daemon ceph-exporter.smithi196 on smithi196 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/1274836106' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Deploying daemon crash.smithi196 on smithi196 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/1825842543' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.212 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Deploying daemon node-exporter.smithi196 on smithi196 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/2989234527' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:40:06.213 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Deploying daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:06.214 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: Deploying daemon mon.smithi196 on smithi196 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/1015201235' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: mon.smithi196@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expand map: {default=false} 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta from 'false' to 'false' 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expanded map: {default=false} 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expand map: {default=info} 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta from 'info' to 'info' 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expanded map: {default=info} 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expand map: {default=daemon} 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta from 'daemon' to 'daemon' 2024-03-13T14:40:06.215 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expanded map: {default=daemon} 2024-03-13T14:40:06.216 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expand map: {default=debug} 2024-03-13T14:40:06.216 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta from 'debug' to 'debug' 2024-03-13T14:40:06.216 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 ceph-mon[37486]: expand_channel_meta expanded map: {default=debug} 2024-03-13T14:40:06.216 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 bash[37445]: 5403fcfbde7282ff12b09239365ff198d264b965e97fc08ce52e69f99fd0f02c 2024-03-13T14:40:06.216 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:06 smithi196 systemd[1]: Started Ceph mon.smithi196 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:40:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.244+0000 7fe727c00700 1 -- 172.21.15.196:0/4034538488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7080035c0 msgr2=0x7fe708005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.244+0000 7fe727c00700 1 --2- 172.21.15.196:0/4034538488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7080035c0 0x7fe708005a50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe720100760 tx=0x7fe71400fe60 comp rx=0 tx=0).stop 2024-03-13T14:40:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.244+0000 7fe727c00700 1 -- 172.21.15.196:0/4034538488 shutdown_connections 2024-03-13T14:40:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.244+0000 7fe727c00700 1 --2- 172.21.15.196:0/4034538488 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7080035c0 0x7fe708005a50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.244+0000 7fe727c00700 1 -- 172.21.15.196:0/4034538488 >> 172.21.15.196:0/4034538488 conn(0x7fe7200fc380 msgr2=0x7fe7200fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:40:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.244+0000 7fe727c00700 1 -- 172.21.15.196:0/4034538488 shutdown_connections 2024-03-13T14:40:11.237 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.245+0000 7fe727c00700 1 -- 172.21.15.196:0/4034538488 wait complete. 2024-03-13T14:40:11.239 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.248+0000 7fe727c00700 1 Processor -- start 2024-03-13T14:40:11.240 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.249+0000 7fe727c00700 1 -- start start 2024-03-13T14:40:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe727c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 0x7fe720195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe727c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe720196370 0x7fe72019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe727c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720196990 con 0x7fe720196370 2024-03-13T14:40:11.241 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe727c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720196b00 con 0x7fe7080035c0 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe725200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 0x7fe720195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe725200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 0x7fe720195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:48576/0 (socket says 172.21.15.196:48576) 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe725200700 1 -- 172.21.15.196:0/3912379457 learned_addr learned my addr 172.21.15.196:0/3912379457 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.250+0000 7fe71fe00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe720196370 0x7fe72019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.251+0000 7fe725200700 1 -- 172.21.15.196:0/3912379457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 msgr2=0x7fe720195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.251+0000 7fe725200700 1 -- 172.21.15.196:0/3912379457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 msgr2=0x7fe720195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.251+0000 7fe725200700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 0x7fe720195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T14:40:11.242 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.251+0000 7fe725200700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 0x7fe720195e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T14:40:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.251+0000 7fe71fe00700 1 -- 172.21.15.196:0/3912379457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 msgr2=0x7fe720195e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T14:40:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.251+0000 7fe71fe00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 0x7fe720195e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.251+0000 7fe71fe00700 1 -- 172.21.15.196:0/3912379457 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71400f8b0 con 0x7fe720196370 2024-03-13T14:40:11.243 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.252+0000 7fe71fe00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe720196370 0x7fe72019a7e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00b4d0 tx=0x7fe70c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.253+0000 7fe71d600700 1 -- 172.21.15.196:0/3912379457 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70c011800 con 0x7fe720196370 2024-03-13T14:40:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.253+0000 7fe71d600700 1 -- 172.21.15.196:0/3912379457 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe70c011e10 con 0x7fe720196370 2024-03-13T14:40:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.253+0000 7fe71d600700 1 -- 172.21.15.196:0/3912379457 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70c00f2e0 con 0x7fe720196370 2024-03-13T14:40:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.254+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72019ade0 con 0x7fe720196370 2024-03-13T14:40:11.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.254+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72019b330 con 0x7fe720196370 2024-03-13T14:40:11.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.256+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7200623c0 con 0x7fe720196370 2024-03-13T14:40:11.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.258+0000 7fe71d600700 1 -- 172.21.15.196:0/3912379457 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fe70c010380 con 0x7fe720196370 2024-03-13T14:40:11.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.259+0000 7fe71d600700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe70006c5c0 0x7fe70006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:11.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.259+0000 7fe71d600700 1 -- 172.21.15.196:0/3912379457 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fe70c08a380 con 0x7fe720196370 2024-03-13T14:40:11.252 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.261+0000 7fe725200700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe70006c5c0 0x7fe70006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:11.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.262+0000 7fe725200700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe70006c5c0 0x7fe70006ea80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe714005fd0 tx=0x7fe714001e70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:11.268 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.277+0000 7fe71d600700 1 -- 172.21.15.196:0/3912379457 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe70c055ca0 con 0x7fe720196370 2024-03-13T14:40:11.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.586+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe7201a0a80 con 0x7fe720196370 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: mon.smithi176 calling monitor election 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: mon.smithi196 calling monitor election 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:11.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: mon.smithi176 is new leader, mons smithi176,smithi196 in quorum (ranks 0,1) 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: monmap e2: 2 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],smithi196=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: fsmap 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: osdmap e5: 0 total, 0 up, 0 in 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: mgrmap e17: smithi176.tzfdwd(active, since 43s) 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: overall HEALTH_OK 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:40:11.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:11.580 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.587+0000 7fe71d600700 1 -- 172.21.15.196:0/3912379457 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7fe70c015090 con 0x7fe720196370 2024-03-13T14:40:11.580 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-03-13T14:40:11.580 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","modified":"2024-03-13T14:40:06.116610Z","created":"2024-03-13T14:36:19.561944Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi176","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:3300","nonce":0},{"type":"v1","addr":"172.21.15.176:6789","nonce":0}]},"addr":"172.21.15.176:6789/0","public_addr":"172.21.15.176:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi196","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:3300","nonce":0},{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.589+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe70006c5c0 msgr2=0x7fe70006ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.589+0000 7fe727c00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe70006c5c0 0x7fe70006ea80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe714005fd0 tx=0x7fe714001e70 comp rx=0 tx=0).stop 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.589+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe720196370 msgr2=0x7fe72019a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.589+0000 7fe727c00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe720196370 0x7fe72019a7e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00b4d0 tx=0x7fe70c00b890 comp rx=0 tx=0).stop 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.590+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 shutdown_connections 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.590+0000 7fe727c00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe70006c5c0 0x7fe70006ea80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.590+0000 7fe727c00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7080035c0 0x7fe720195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.590+0000 7fe727c00700 1 --2- 172.21.15.196:0/3912379457 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe720196370 0x7fe72019a7e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:11.581 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.590+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 >> 172.21.15.196:0/3912379457 conn(0x7fe7200fc380 msgr2=0x7fe72018e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:40:11.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.590+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 shutdown_connections 2024-03-13T14:40:11.582 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:40:11.590+0000 7fe727c00700 1 -- 172.21.15.196:0/3912379457 wait complete. 2024-03-13T14:40:11.582 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: mon.smithi176 calling monitor election 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: mon.smithi196 calling monitor election 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:11.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: mon.smithi176 is new leader, mons smithi176,smithi196 in quorum (ranks 0,1) 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: monmap e2: 2 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],smithi196=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: fsmap 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: osdmap e5: 0 total, 0 up, 0 in 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: mgrmap e17: smithi176.tzfdwd(active, since 43s) 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: overall HEALTH_OK 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/966174155' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:40:11.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:12.345 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-13T14:40:12.345 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph config generate-minimal-conf 2024-03-13T14:40:12.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: mgrmap e18: smithi176.tzfdwd(active, since 43s), standbys: smithi196.vdktbk 2024-03-13T14:40:12.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:40:12.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:12.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:12.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:12.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:40:12.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/3912379457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:12.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:12.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: mgrmap e18: smithi176.tzfdwd(active, since 43s), standbys: smithi196.vdktbk 2024-03-13T14:40:12.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:40:12.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:12.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:12.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:12.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:40:12.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/3912379457' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-13T14:40:12.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:40:12.741 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:40:13.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:13 smithi176 ceph-mon[29064]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:13.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:13.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:13.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:13 smithi196 ceph-mon[37486]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:13.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:13.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:14.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.714+0000 7fb4b4000700 1 -- 172.21.15.176:0/2693604529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac105780 msgr2=0x7fb4ac105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:14.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.714+0000 7fb4b4000700 1 --2- 172.21.15.176:0/2693604529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac105780 0x7fb4ac105b60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009910 tx=0x7fb4a0009c20 comp rx=0 tx=0).stop 2024-03-13T14:40:14.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.714+0000 7fb4b4000700 1 -- 172.21.15.176:0/2693604529 shutdown_connections 2024-03-13T14:40:14.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.714+0000 7fb4b4000700 1 --2- 172.21.15.176:0/2693604529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac105780 0x7fb4ac105b60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:14.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.714+0000 7fb4b4000700 1 -- 172.21.15.176:0/2693604529 >> 172.21.15.176:0/2693604529 conn(0x7fb4ac0752b0 msgr2=0x7fb4ac0756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:40:14.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.714+0000 7fb4b4000700 1 -- 172.21.15.176:0/2693604529 shutdown_connections 2024-03-13T14:40:14.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.714+0000 7fb4b4000700 1 -- 172.21.15.176:0/2693604529 wait complete. 2024-03-13T14:40:14.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.715+0000 7fb4b4000700 1 Processor -- start 2024-03-13T14:40:14.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b4000700 1 -- start start 2024-03-13T14:40:14.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b4000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 0x7fb4ac196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:14.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b4000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac197290 0x7fb4ac1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b4000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19fe50 con 0x7fb4ac197290 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b4000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac19ff90 con 0x7fb4ac105780 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b0c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac197290 0x7fb4ac1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b0c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac197290 0x7fb4ac1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59886/0 (socket says 172.21.15.176:59886) 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b0c00700 1 -- 172.21.15.176:0/955614127 learned_addr learned my addr 172.21.15.176:0/955614127 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.716+0000 7fb4b1600700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 0x7fb4ac196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b1600700 1 -- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 msgr2=0x7fb4ac196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b1600700 1 -- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 msgr2=0x7fb4ac196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b1600700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 0x7fb4ac196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b1600700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 0x7fb4ac196d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b0c00700 1 -- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 msgr2=0x7fb4ac196d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b0c00700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 0x7fb4ac196d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:14.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b0c00700 1 -- 172.21.15.176:0/955614127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a00095f0 con 0x7fb4ac197290 2024-03-13T14:40:14.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4b0c00700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac197290 0x7fb4ac1976f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b730 tx=0x7fb49800baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:14.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4aa000700 1 -- 172.21.15.176:0/955614127 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb498004200 con 0x7fb4ac197290 2024-03-13T14:40:14.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.717+0000 7fb4aa000700 1 -- 172.21.15.176:0/955614127 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb498004360 con 0x7fb4ac197290 2024-03-13T14:40:14.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.718+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4ac1a01f0 con 0x7fb4ac197290 2024-03-13T14:40:14.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.718+0000 7fb4aa000700 1 -- 172.21.15.176:0/955614127 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb498011be0 con 0x7fb4ac197290 2024-03-13T14:40:14.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.718+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4ac1a0740 con 0x7fb4ac197290 2024-03-13T14:40:14.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.719+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4ac0623c0 con 0x7fb4ac197290 2024-03-13T14:40:14.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.720+0000 7fb4aa000700 1 -- 172.21.15.176:0/955614127 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fb4980036a0 con 0x7fb4ac197290 2024-03-13T14:40:14.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.720+0000 7fb4aa000700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb49406c5c0 0x7fb49406ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:14.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.721+0000 7fb4aa000700 1 -- 172.21.15.176:0/955614127 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb49808a370 con 0x7fb4ac197290 2024-03-13T14:40:14.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.721+0000 7fb4b1600700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb49406c5c0 0x7fb49406ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:14.713 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.722+0000 7fb4b1600700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb49406c5c0 0x7fb49406ea80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006010 tx=0x7fb4a000b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:14.717 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.726+0000 7fb4aa000700 1 -- 172.21.15.176:0/955614127 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb498055b90 con 0x7fb4ac197290 2024-03-13T14:40:14.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.991+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb4ac1a0af0 con 0x7fb4ac197290 2024-03-13T14:40:14.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.991+0000 7fb4aa000700 1 -- 172.21.15.176:0/955614127 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fb4980591b0 con 0x7fb4ac197290 2024-03-13T14:40:14.983 INFO:teuthology.orchestra.run.smithi176.stdout:# minimal ceph.conf for f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:40:14.983 INFO:teuthology.orchestra.run.smithi176.stdout:[global] 2024-03-13T14:40:14.983 INFO:teuthology.orchestra.run.smithi176.stdout: fsid = f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:40:14.983 INFO:teuthology.orchestra.run.smithi176.stdout: mon_host = [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb49406c5c0 msgr2=0x7fb49406ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb49406c5c0 0x7fb49406ea80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0006010 tx=0x7fb4a000b540 comp rx=0 tx=0).stop 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac197290 msgr2=0x7fb4ac1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac197290 0x7fb4ac1976f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb49800b730 tx=0x7fb49800baf0 comp rx=0 tx=0).stop 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 shutdown_connections 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb49406c5c0 0x7fb49406ea80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4ac105780 0x7fb4ac196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 --2- 172.21.15.176:0/955614127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb4ac197290 0x7fb4ac1976f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 >> 172.21.15.176:0/955614127 conn(0x7fb4ac0752b0 msgr2=0x7fb4ac101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 shutdown_connections 2024-03-13T14:40:14.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:14.994+0000 7fb4b4000700 1 -- 172.21.15.176:0/955614127 wait complete. 2024-03-13T14:40:15.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:15 smithi176 ceph-mon[29064]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:15.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:15.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:15.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:15.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:40:15.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:15 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/955614127' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:15.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:15 smithi196 ceph-mon[37486]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:15.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:15.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:15.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:15.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:40:15.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:15 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/955614127' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:15.804 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-13T14:40:15.805 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:40:15.805 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.conf 2024-03-13T14:40:15.893 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:40:15.894 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:40:15.982 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:40:15.982 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-03-13T14:40:16.069 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:40:16.069 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:40:16.198 INFO:tasks.cephadm:Deploying OSDs... 2024-03-13T14:40:16.198 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:40:16.198 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/scratch_devs of=/dev/stdout 2024-03-13T14:40:16.241 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-13T14:40:16.241 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_1 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-03-13 14:39:17.361341666 +0000 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-03-13 14:32:08.763301326 +0000 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-03-13 14:32:08.763301326 +0000 2024-03-13T14:40:16.339 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-03-13 14:32:08.763301326 +0000 2024-03-13T14:40:16.339 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-13T14:40:16.414 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-03-13T14:40:16.414 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-03-13T14:40:16.414 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000311226 s, 1.6 MB/s 2024-03-13T14:40:16.416 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-13T14:40:16.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.456 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_2 2024-03-13T14:40:16.536 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-13T14:40:16.536 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:16.536 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-13T14:40:16.537 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:16.537 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:16.537 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-03-13 14:39:17.362341656 +0000 2024-03-13T14:40:16.537 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-03-13 14:32:09.330296031 +0000 2024-03-13T14:40:16.537 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-03-13 14:32:09.330296031 +0000 2024-03-13T14:40:16.537 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-03-13 14:32:09.330296031 +0000 2024-03-13T14:40:16.537 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-13T14:40:16.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:40:16.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:40:16.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:40:16.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:40:16.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:16.665 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-03-13T14:40:16.665 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-03-13T14:40:16.665 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000347379 s, 1.5 MB/s 2024-03-13T14:40:16.670 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-13T14:40:16.722 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_3 2024-03-13T14:40:16.767 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-03-13 14:39:17.362341656 +0000 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-03-13 14:32:09.909290624 +0000 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-03-13 14:32:09.909290624 +0000 2024-03-13T14:40:16.768 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-03-13 14:32:09.909290624 +0000 2024-03-13T14:40:16.768 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-13T14:40:16.877 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-03-13T14:40:16.877 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-03-13T14:40:16.877 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000300411 s, 1.7 MB/s 2024-03-13T14:40:16.881 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-13T14:40:16.922 DEBUG:teuthology.orchestra.run.smithi176:> stat /dev/vg_nvme/lv_4 2024-03-13T14:40:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-13T14:40:17.011 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:17.011 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-13T14:40:17.011 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:17.011 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:17.012 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 2024-03-13 14:39:17.363341647 +0000 2024-03-13T14:40:17.012 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-03-13 14:32:10.466285446 +0000 2024-03-13T14:40:17.012 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-03-13 14:32:10.466285446 +0000 2024-03-13T14:40:17.012 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: 2024-03-13 14:32:10.466285446 +0000 2024-03-13T14:40:17.012 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-13T14:40:17.140 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records in 2024-03-13T14:40:17.140 INFO:teuthology.orchestra.run.smithi176.stderr:1+0 records out 2024-03-13T14:40:17.140 INFO:teuthology.orchestra.run.smithi176.stderr:512 bytes copied, 0.000312994 s, 1.6 MB/s 2024-03-13T14:40:17.144 DEBUG:teuthology.orchestra.run.smithi176:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-13T14:40:17.198 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:40:17.198 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-03-13T14:40:17.256 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-13T14:40:17.257 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-03-13T14:40:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-13T14:40:17.308 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:17.308 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-13T14:40:17.308 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:17.308 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:17.308 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-13 14:40:04.709240603 +0000 2024-03-13T14:40:17.309 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-13 14:32:08.769732044 +0000 2024-03-13T14:40:17.309 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-13 14:32:08.769732044 +0000 2024-03-13T14:40:17.309 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-13 14:32:08.769732044 +0000 2024-03-13T14:40:17.309 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-13T14:40:17.387 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-13T14:40:17.388 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-13T14:40:17.388 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000322541 s, 1.6 MB/s 2024-03-13T14:40:17.391 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-13T14:40:17.443 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-03-13T14:40:17.491 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-13T14:40:17.491 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:17.491 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 653 Links: 1 2024-03-13T14:40:17.491 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:17.491 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:17.491 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-13 14:40:04.710240581 +0000 2024-03-13T14:40:17.492 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-13 14:32:09.341719209 +0000 2024-03-13T14:40:17.492 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-13 14:32:09.341719209 +0000 2024-03-13T14:40:17.492 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-13 14:32:09.341719209 +0000 2024-03-13T14:40:17.492 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-13T14:40:17.566 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:17 smithi196 ceph-mon[37486]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:17.566 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:17 smithi196 ceph-mon[37486]: Reconfiguring mon.smithi176 (unknown last config time)... 2024-03-13T14:40:17.566 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:17 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:40:17.566 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:17 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:40:17.567 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:17 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:17.567 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:17 smithi196 ceph-mon[37486]: Reconfiguring daemon mon.smithi176 on smithi176 2024-03-13T14:40:17.574 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-13T14:40:17.574 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-13T14:40:17.575 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000339543 s, 1.5 MB/s 2024-03-13T14:40:17.578 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-13T14:40:17.632 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-03-13T14:40:17.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:17 smithi176 ceph-mon[29064]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:17.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:17 smithi176 ceph-mon[29064]: Reconfiguring mon.smithi176 (unknown last config time)... 2024-03-13T14:40:17.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:17 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:40:17.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:17 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:40:17.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:17 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:17.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:17 smithi176 ceph-mon[29064]: Reconfiguring daemon mon.smithi176 on smithi176 2024-03-13T14:40:17.681 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-13T14:40:17.681 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:17.681 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-13T14:40:17.682 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:17.682 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:17.682 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-13 14:40:04.710240581 +0000 2024-03-13T14:40:17.682 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-13 14:32:09.943705702 +0000 2024-03-13T14:40:17.682 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-13 14:32:09.943705702 +0000 2024-03-13T14:40:17.682 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-13 14:32:09.943705702 +0000 2024-03-13T14:40:17.682 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-13T14:40:17.769 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-13T14:40:17.770 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-13T14:40:17.770 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000326094 s, 1.6 MB/s 2024-03-13T14:40:17.774 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-13T14:40:17.831 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-03-13T14:40:17.880 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-03-13 14:40:04.711240558 +0000 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-13 14:32:10.524692665 +0000 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-13 14:32:10.524692665 +0000 2024-03-13T14:40:17.881 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-03-13 14:32:10.524692665 +0000 2024-03-13T14:40:17.881 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-13T14:40:17.967 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-03-13T14:40:17.967 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-03-13T14:40:17.967 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.00034409 s, 1.5 MB/s 2024-03-13T14:40:17.970 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-13T14:40:18.025 INFO:tasks.cephadm:Deploying osd.0 on smithi176 with /dev/vg_nvme/lv_4... 2024-03-13T14:40:18.025 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-13T14:40:18.441 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:40:19.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:19.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:19.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:19.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: Reconfiguring mgr.smithi176.tzfdwd (unknown last config time)... 2024-03-13T14:40:19.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:40:19.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:40:19.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:19.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:19 smithi176 ceph-mon[29064]: Reconfiguring daemon mgr.smithi176.tzfdwd on smithi176 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: Reconfiguring mgr.smithi176.tzfdwd (unknown last config time)... 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:19.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:19 smithi196 ceph-mon[37486]: Reconfiguring daemon mgr.smithi176.tzfdwd on smithi176 2024-03-13T14:40:20.872 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:20 smithi176 ceph-mon[29064]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:21.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:20 smithi196 ceph-mon[37486]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:21.270 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-13T14:40:21.270 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-13T14:40:21.270 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-03-13T14:40:21.271 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-03-13T14:40:21.271 INFO:teuthology.orchestra.run.smithi176.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0337032 s, 311 MB/s 2024-03-13T14:40:21.271 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-03-13T14:40:22.053 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi176:vg_nvme/lv_4 2024-03-13T14:40:22.636 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:22.637 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:22.637 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:22 smithi176 ceph-mon[29064]: Reconfiguring ceph-exporter.smithi176 (monmap changed)... 2024-03-13T14:40:22.637 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:40:22.637 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:22.637 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:22 smithi176 ceph-mon[29064]: Reconfiguring daemon ceph-exporter.smithi176 on smithi176 2024-03-13T14:40:22.638 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:22 smithi176 ceph-mon[29064]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:22.934 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:22.935 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:22.935 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:22 smithi196 ceph-mon[37486]: Reconfiguring ceph-exporter.smithi176 (monmap changed)... 2024-03-13T14:40:22.935 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:40:22.935 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:22.935 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:22 smithi196 ceph-mon[37486]: Reconfiguring daemon ceph-exporter.smithi176 on smithi176 2024-03-13T14:40:22.935 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:22 smithi196 ceph-mon[37486]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:23.374 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:40:24.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:24.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:24.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:24 smithi176 ceph-mon[29064]: Reconfiguring crash.smithi176 (monmap changed)... 2024-03-13T14:40:24.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:40:24.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:24.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:24 smithi176 ceph-mon[29064]: Reconfiguring daemon crash.smithi176 on smithi176 2024-03-13T14:40:24.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:24.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:24.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:24 smithi196 ceph-mon[37486]: Reconfiguring crash.smithi176 (monmap changed)... 2024-03-13T14:40:24.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:40:24.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:24.683 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:24 smithi196 ceph-mon[37486]: Reconfiguring daemon crash.smithi176 on smithi176 2024-03-13T14:40:25.822 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:25 smithi176 ceph-mon[29064]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:25.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:25 smithi196 ceph-mon[37486]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:25.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 -- 172.21.15.176:0/2678121799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c104010 msgr2=0x7fa45c106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:25.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 --2- 172.21.15.176:0/2678121799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c104010 0x7fa45c106400 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa448009910 tx=0x7fa448009c20 comp rx=0 tx=0).stop 2024-03-13T14:40:25.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 -- 172.21.15.176:0/2678121799 shutdown_connections 2024-03-13T14:40:25.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 --2- 172.21.15.176:0/2678121799 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c104010 0x7fa45c106400 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:25.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 --2- 172.21.15.176:0/2678121799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c1016e0 0x7fa45c103ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:25.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 -- 172.21.15.176:0/2678121799 >> 172.21.15.176:0/2678121799 conn(0x7fa45c0fb030 msgr2=0x7fa45c0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:40:25.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 -- 172.21.15.176:0/2678121799 shutdown_connections 2024-03-13T14:40:25.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.998+0000 7fa461000700 1 -- 172.21.15.176:0/2678121799 wait complete. 2024-03-13T14:40:25.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.999+0000 7fa461000700 1 Processor -- start 2024-03-13T14:40:25.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.999+0000 7fa461000700 1 -- start start 2024-03-13T14:40:25.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.999+0000 7fa461000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c1016e0 0x7fa45c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.999+0000 7fa461000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c104010 0x7fa45c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.999+0000 7fa461000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c199150 con 0x7fa45c1016e0 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:25.999+0000 7fa461000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c199290 con 0x7fa45c104010 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.000+0000 7fa459e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c1016e0 0x7fa45c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.000+0000 7fa459e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c1016e0 0x7fa45c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43438/0 (socket says 172.21.15.176:43438) 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.000+0000 7fa459e00700 1 -- 172.21.15.176:0/1120341714 learned_addr learned my addr 172.21.15.176:0/1120341714 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.000+0000 7fa459400700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c104010 0x7fa45c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.000+0000 7fa459e00700 1 -- 172.21.15.176:0/1120341714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c104010 msgr2=0x7fa45c198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.000+0000 7fa459e00700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c104010 0x7fa45c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:40:25.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.000+0000 7fa459e00700 1 -- 172.21.15.176:0/1120341714 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4480095f0 con 0x7fa45c1016e0 2024-03-13T14:40:25.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.001+0000 7fa459e00700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c1016e0 0x7fa45c1985f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00b4b0 tx=0x7fa44c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:25.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.001+0000 7fa452a00700 1 -- 172.21.15.176:0/1120341714 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44c011800 con 0x7fa45c1016e0 2024-03-13T14:40:25.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.001+0000 7fa452a00700 1 -- 172.21.15.176:0/1120341714 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa44c011e10 con 0x7fa45c1016e0 2024-03-13T14:40:25.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.001+0000 7fa452a00700 1 -- 172.21.15.176:0/1120341714 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44c00f2e0 con 0x7fa45c1016e0 2024-03-13T14:40:25.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.001+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c19dd40 con 0x7fa45c1016e0 2024-03-13T14:40:25.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.001+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c19e2b0 con 0x7fa45c1016e0 2024-03-13T14:40:25.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.002+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45c0fcc20 con 0x7fa45c1016e0 2024-03-13T14:40:25.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.003+0000 7fa452a00700 1 -- 172.21.15.176:0/1120341714 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fa44c010380 con 0x7fa45c1016e0 2024-03-13T14:40:25.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.004+0000 7fa452a00700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa44006c470 0x7fa44006e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:40:25.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.004+0000 7fa452a00700 1 -- 172.21.15.176:0/1120341714 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa44c08a320 con 0x7fa45c1016e0 2024-03-13T14:40:25.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.004+0000 7fa459400700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa44006c470 0x7fa44006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:40:25.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.005+0000 7fa459400700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa44006c470 0x7fa44006e930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa448000c00 tx=0x7fa448005fb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:40:26.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.009+0000 7fa452a00700 1 -- 172.21.15.176:0/1120341714 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa44c059650 con 0x7fa45c1016e0 2024-03-13T14:40:26.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:40:26.241+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa45c0611d0 con 0x7fa44006c470 2024-03-13T14:40:26.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:26 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:40:26.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:26 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:40:26.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:26 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:26 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:40:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:26 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:40:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:26 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:27.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:27 smithi176 ceph-mon[29064]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:27.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:27 smithi176 ceph-mon[29064]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:40:27.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:27 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:27.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:27 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:27.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:27 smithi196 ceph-mon[37486]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:27.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:27 smithi196 ceph-mon[37486]: from='client.14260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:40:27.933 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:27 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:27.933 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:27 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:28.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:28 smithi176 ceph-mon[29064]: Reconfiguring alertmanager.smithi176 (dependencies changed)... 2024-03-13T14:40:28.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:28 smithi176 ceph-mon[29064]: Reconfiguring daemon alertmanager.smithi176 on smithi176 2024-03-13T14:40:28.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:28.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:28 smithi196 ceph-mon[37486]: Reconfiguring alertmanager.smithi176 (dependencies changed)... 2024-03-13T14:40:28.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:28 smithi196 ceph-mon[37486]: Reconfiguring daemon alertmanager.smithi176 on smithi176 2024-03-13T14:40:28.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:29.766 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:29 smithi176 ceph-mon[29064]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:29.834 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:29 smithi196 ceph-mon[37486]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:31.068 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:30 smithi176 ceph-mon[29064]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:31.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:30 smithi196 ceph-mon[37486]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:33.331 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:33 smithi176 ceph-mon[29064]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:33.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:33 smithi196 ceph-mon[37486]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:35.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:34 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:35.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:34 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:35.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:34 smithi196 ceph-mon[37486]: Reconfiguring grafana.smithi176 (dependencies changed)... 2024-03-13T14:40:35.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:34 smithi196 ceph-mon[37486]: Reconfiguring daemon grafana.smithi176 on smithi176 2024-03-13T14:40:35.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:34 smithi196 ceph-mon[37486]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:35.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:34 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:35.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:34 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:35.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:34 smithi176 ceph-mon[29064]: Reconfiguring grafana.smithi176 (dependencies changed)... 2024-03-13T14:40:35.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:34 smithi176 ceph-mon[29064]: Reconfiguring daemon grafana.smithi176 on smithi176 2024-03-13T14:40:35.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:34 smithi176 ceph-mon[29064]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:36.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:35 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/606194703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c570fcfa-ceda-4e39-b200-5c8b0b7fab19"}]: dispatch 2024-03-13T14:40:36.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:35 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/606194703' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c570fcfa-ceda-4e39-b200-5c8b0b7fab19"}]': finished 2024-03-13T14:40:36.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:35 smithi176 ceph-mon[29064]: osdmap e6: 1 total, 0 up, 1 in 2024-03-13T14:40:36.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:35 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:40:36.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:35 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/606194703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c570fcfa-ceda-4e39-b200-5c8b0b7fab19"}]: dispatch 2024-03-13T14:40:36.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:35 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/606194703' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c570fcfa-ceda-4e39-b200-5c8b0b7fab19"}]': finished 2024-03-13T14:40:36.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:35 smithi196 ceph-mon[37486]: osdmap e6: 1 total, 0 up, 1 in 2024-03-13T14:40:36.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:35 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:40:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:36 smithi176 ceph-mon[29064]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:36 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/4128471314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:40:37.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:36 smithi196 ceph-mon[37486]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:37.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:36 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/4128471314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:40:39.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:39 smithi176 ceph-mon[29064]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:39.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:39 smithi196 ceph-mon[37486]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:41.050 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:40 smithi176 ceph-mon[29064]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:41.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:40 smithi196 ceph-mon[37486]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:43.215 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:43 smithi176 ceph-mon[29064]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:43.215 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:43.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:43 smithi196 ceph-mon[37486]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:43.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:45.259 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:45 smithi176 ceph-mon[29064]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:45.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:45 smithi196 ceph-mon[37486]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:47.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:47 smithi176 ceph-mon[29064]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:47.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:47.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:47.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:47 smithi196 ceph-mon[37486]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:47.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:47.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:48.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:48 smithi176 ceph-mon[29064]: Reconfiguring prometheus.smithi176 (dependencies changed)... 2024-03-13T14:40:48.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:48 smithi176 ceph-mon[29064]: Reconfiguring daemon prometheus.smithi176 on smithi176 2024-03-13T14:40:48.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:48 smithi196 ceph-mon[37486]: Reconfiguring prometheus.smithi176 (dependencies changed)... 2024-03-13T14:40:48.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:48 smithi196 ceph-mon[37486]: Reconfiguring daemon prometheus.smithi176 on smithi176 2024-03-13T14:40:49.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:49 smithi176 ceph-mon[29064]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:49.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-13T14:40:49.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:49.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:49 smithi176 ceph-mon[29064]: Deploying daemon osd.0 on smithi176 2024-03-13T14:40:49.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:49 smithi196 ceph-mon[37486]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:49.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:49 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-13T14:40:49.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:49 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:49.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:49 smithi196 ceph-mon[37486]: Deploying daemon osd.0 on smithi176 2024-03-13T14:40:51.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:50 smithi176 ceph-mon[29064]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:51.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:50 smithi196 ceph-mon[37486]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:53.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:53 smithi176 ceph-mon[29064]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:53.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:53 smithi196 ceph-mon[37486]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:55.347 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:55 smithi196 ceph-mon[37486]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:55.347 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:55 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:55.347 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:55 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:55.347 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:55 smithi196 ceph-mon[37486]: Reconfiguring ceph-exporter.smithi196 (monmap changed)... 2024-03-13T14:40:55.347 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:55 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:40:55.347 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:55 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:55.348 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:55 smithi196 ceph-mon[37486]: Reconfiguring daemon ceph-exporter.smithi196 on smithi196 2024-03-13T14:40:55.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:55 smithi176 ceph-mon[29064]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:55.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:55 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:55.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:55 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:55.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:55 smithi176 ceph-mon[29064]: Reconfiguring ceph-exporter.smithi196 (monmap changed)... 2024-03-13T14:40:55.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:55 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:40:55.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:55 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:55.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:55 smithi176 ceph-mon[29064]: Reconfiguring daemon ceph-exporter.smithi196 on smithi196 2024-03-13T14:40:57.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:57 smithi196 ceph-mon[37486]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:57.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:57.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:57.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:57 smithi196 ceph-mon[37486]: Reconfiguring crash.smithi196 (monmap changed)... 2024-03-13T14:40:57.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:40:57.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:57.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:57 smithi196 ceph-mon[37486]: Reconfiguring daemon crash.smithi196 on smithi196 2024-03-13T14:40:57.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:57 smithi176 ceph-mon[29064]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:57.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:57.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:57.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:57 smithi176 ceph-mon[29064]: Reconfiguring crash.smithi196 (monmap changed)... 2024-03-13T14:40:57.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:40:57.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:57.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:57 smithi176 ceph-mon[29064]: Reconfiguring daemon crash.smithi196 on smithi196 2024-03-13T14:40:58.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:58.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: Reconfiguring mgr.smithi196.vdktbk (monmap changed)... 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:59.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:40:59 smithi196 ceph-mon[37486]: Reconfiguring daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:40:59.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:40:59.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:59.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:40:59.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: Reconfiguring mgr.smithi196.vdktbk (monmap changed)... 2024-03-13T14:40:59.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:40:59.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:40:59.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:40:59.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:40:59 smithi176 ceph-mon[29064]: Reconfiguring daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:41:01.034 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:00 smithi176 ceph-mon[29064]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:01.034 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:00 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:01.034 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:00 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:01.034 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:00 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:41:01.034 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:00 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:41:01.034 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:00 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:01.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:00 smithi196 ceph-mon[37486]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:01.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:00 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:01.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:00 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:01.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:00 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:41:01.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:00 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:41:01.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:00 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:01.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:01 smithi196 ceph-mon[37486]: Reconfiguring mon.smithi196 (monmap changed)... 2024-03-13T14:41:01.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:01 smithi196 ceph-mon[37486]: Reconfiguring daemon mon.smithi196 on smithi196 2024-03-13T14:41:01.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:01 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:01.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:01 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:01 smithi176 ceph-mon[29064]: Reconfiguring mon.smithi196 (monmap changed)... 2024-03-13T14:41:02.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:01 smithi176 ceph-mon[29064]: Reconfiguring daemon mon.smithi196 on smithi196 2024-03-13T14:41:02.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi176.front.sepia.ceph.com:9093"}]: dispatch 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi176.front.sepia.ceph.com:3000"}]: dispatch 2024-03-13T14:41:02.955 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.956 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:41:02.956 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi176.front.sepia.ceph.com:9095"}]: dispatch 2024-03-13T14:41:02.956 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:02.956 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi176.front.sepia.ceph.com:9093"}]: dispatch 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi176.front.sepia.ceph.com:3000"}]: dispatch 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:03.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:41:03.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi176.front.sepia.ceph.com:9095"}]: dispatch 2024-03-13T14:41:03.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:03.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:04.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:03 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:41:04.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:03 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi176.front.sepia.ceph.com:9093"}]: dispatch 2024-03-13T14:41:04.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:03 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:41:04.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:03 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi176.front.sepia.ceph.com:3000"}]: dispatch 2024-03-13T14:41:04.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:03 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:41:04.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:03 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi176.front.sepia.ceph.com:9095"}]: dispatch 2024-03-13T14:41:04.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:03 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:41:04.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:03 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi176.front.sepia.ceph.com:9093"}]: dispatch 2024-03-13T14:41:04.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:03 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:41:04.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:03 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi176.front.sepia.ceph.com:3000"}]: dispatch 2024-03-13T14:41:04.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:03 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:41:04.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:03 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi176.front.sepia.ceph.com:9095"}]: dispatch 2024-03-13T14:41:04.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:04 smithi176 ceph-mon[29064]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:04.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:04 smithi196 ceph-mon[37486]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:05.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.781+0000 7fa452a00700 1 -- 172.21.15.176:0/1120341714 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa45c0611d0 con 0x7fa44006c470 2024-03-13T14:41:05.774 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 0 on host 'smithi176' 2024-03-13T14:41:05.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.784+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa44006c470 msgr2=0x7fa44006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:05.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.784+0000 7fa461000700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa44006c470 0x7fa44006e930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa448000c00 tx=0x7fa448005fb0 comp rx=0 tx=0).stop 2024-03-13T14:41:05.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.784+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c1016e0 msgr2=0x7fa45c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:05.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.784+0000 7fa461000700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c1016e0 0x7fa45c1985f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00b4b0 tx=0x7fa44c00b870 comp rx=0 tx=0).stop 2024-03-13T14:41:05.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.785+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 shutdown_connections 2024-03-13T14:41:05.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.785+0000 7fa461000700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa44006c470 0x7fa44006e930 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:05.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.785+0000 7fa461000700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa45c1016e0 0x7fa45c1985f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:05.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.785+0000 7fa461000700 1 --2- 172.21.15.176:0/1120341714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa45c104010 0x7fa45c198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:05.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.785+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 >> 172.21.15.176:0/1120341714 conn(0x7fa45c0fb030 msgr2=0x7fa45c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:41:05.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.785+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 shutdown_connections 2024-03-13T14:41:05.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:05.785+0000 7fa461000700 1 -- 172.21.15.176:0/1120341714 wait complete. 2024-03-13T14:41:06.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:05 smithi176 ceph-mon[29064]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-13T14:41:06.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:05 smithi196 ceph-mon[37486]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-13T14:41:06.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:06.611 DEBUG:teuthology.orchestra.run.smithi176:osd.0> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.0.service 2024-03-13T14:41:06.616 INFO:tasks.cephadm:Deploying osd.1 on smithi176 with /dev/vg_nvme/lv_3... 2024-03-13T14:41:06.616 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-13T14:41:07.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:06 smithi176 ceph-mon[29064]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-13T14:41:07.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:06 smithi176 ceph-mon[29064]: osdmap e7: 1 total, 0 up, 1 in 2024-03-13T14:41:07.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:07.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:06 smithi176 ceph-mon[29064]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:41:07.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:06 smithi176 ceph-mon[29064]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:07.133 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:41:06 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[47463]: 2024-03-13T14:41:06.844+0000 7f08a7000700 -1 osd.0 0 waiting for initial osdmap 2024-03-13T14:41:07.133 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:41:06 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[47463]: 2024-03-13T14:41:06.851+0000 7f08a2800700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T14:41:07.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:06 smithi196 ceph-mon[37486]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-13T14:41:07.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:06 smithi196 ceph-mon[37486]: osdmap e7: 1 total, 0 up, 1 in 2024-03-13T14:41:07.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:07.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:06 smithi196 ceph-mon[37486]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:41:07.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:06 smithi196 ceph-mon[37486]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:07.426 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:41:08.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:07 smithi176 ceph-mon[29064]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-03-13T14:41:08.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:07 smithi176 ceph-mon[29064]: osdmap e8: 1 total, 0 up, 1 in 2024-03-13T14:41:08.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:08.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:08.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:07 smithi196 ceph-mon[37486]: from='osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-03-13T14:41:08.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:07 smithi196 ceph-mon[37486]: osdmap e8: 1 total, 0 up, 1 in 2024-03-13T14:41:08.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:08.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:09.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:08 smithi176 ceph-mon[29064]: purged_snaps scrub starts 2024-03-13T14:41:09.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:08 smithi176 ceph-mon[29064]: purged_snaps scrub ok 2024-03-13T14:41:09.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:08 smithi176 ceph-mon[29064]: osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] boot 2024-03-13T14:41:09.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:08 smithi176 ceph-mon[29064]: osdmap e9: 1 total, 1 up, 1 in 2024-03-13T14:41:09.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:08 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:09.133 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:08 smithi176 ceph-mon[29064]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:09.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:08 smithi196 ceph-mon[37486]: purged_snaps scrub starts 2024-03-13T14:41:09.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:08 smithi196 ceph-mon[37486]: purged_snaps scrub ok 2024-03-13T14:41:09.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:08 smithi196 ceph-mon[37486]: osd.0 [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] boot 2024-03-13T14:41:09.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:08 smithi196 ceph-mon[37486]: osdmap e9: 1 total, 1 up, 1 in 2024-03-13T14:41:09.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:08 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:41:09.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:08 smithi196 ceph-mon[37486]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-13T14:41:09.994 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:09 smithi176 ceph-mon[29064]: osdmap e10: 1 total, 1 up, 1 in 2024-03-13T14:41:10.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:09 smithi196 ceph-mon[37486]: osdmap e10: 1 total, 1 up, 1 in 2024-03-13T14:41:10.972 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-13T14:41:10.972 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-13T14:41:10.972 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-03-13T14:41:10.972 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-03-13T14:41:10.972 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0328575 s, 319 MB/s 2024-03-13T14:41:10.972 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-03-13T14:41:11.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:10 smithi176 ceph-mon[29064]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:11.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:10 smithi196 ceph-mon[37486]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:12.294 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi176:vg_nvme/lv_3 2024-03-13T14:41:12.681 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:41:13.119 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:13.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:13.120 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:12 smithi176 ceph-mon[29064]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:13.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:13.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:13.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:12 smithi196 ceph-mon[37486]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:14.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:14 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:14.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:14 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:14.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 -- 172.21.15.176:0/311602460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 msgr2=0x7f9a340ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:14.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 --2- 172.21.15.176:0/311602460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a340ff700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009910 tx=0x7f9a24009c20 comp rx=0 tx=0).stop 2024-03-13T14:41:14.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 -- 172.21.15.176:0/311602460 shutdown_connections 2024-03-13T14:41:14.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 --2- 172.21.15.176:0/311602460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a340ffc40 0x7f9a341000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:14.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 --2- 172.21.15.176:0/311602460 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a340ff700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:14.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 -- 172.21.15.176:0/311602460 >> 172.21.15.176:0/311602460 conn(0x7f9a340fae40 msgr2=0x7f9a340fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 -- 172.21.15.176:0/311602460 shutdown_connections 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.918+0000 7f9a38a00700 1 -- 172.21.15.176:0/311602460 wait complete. 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.920+0000 7f9a38a00700 1 Processor -- start 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.920+0000 7f9a38a00700 1 -- start start 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a38a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a341983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a38a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a340ffc40 0x7f9a341988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a38a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34198f10 con 0x7f9a340ff2e0 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a38a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a34199050 con 0x7f9a340ffc40 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a32800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a341983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a32800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a341983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59470/0 (socket says 172.21.15.176:59470) 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a32800700 1 -- 172.21.15.176:0/3021591845 learned_addr learned my addr 172.21.15.176:0/3021591845 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a32800700 1 -- 172.21.15.176:0/3021591845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a340ffc40 msgr2=0x7f9a341988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:14.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a31e00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a340ffc40 0x7f9a341988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a32800700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a340ffc40 0x7f9a341988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.921+0000 7f9a32800700 1 -- 172.21.15.176:0/3021591845 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a240095f0 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.922+0000 7f9a32800700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a341983b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400bd70 tx=0x7f9a2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.922+0000 7f9a2b400700 1 -- 172.21.15.176:0/3021591845 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2401d070 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.922+0000 7f9a2b400700 1 -- 172.21.15.176:0/3021591845 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9a24022470 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.922+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3419daa0 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.922+0000 7f9a2b400700 1 -- 172.21.15.176:0/3021591845 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a2400f670 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.922+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3419df90 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.923+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3410b740 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.924+0000 7f9a2b400700 1 -- 172.21.15.176:0/3021591845 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7f9a24022a50 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.925+0000 7f9a2b400700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a1806c460 0x7f9a1806e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.925+0000 7f9a2b400700 1 -- 172.21.15.176:0/3021591845 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f9a2408bff0 con 0x7f9a340ff2e0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.925+0000 7f9a31e00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a1806c460 0x7f9a1806e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.926+0000 7f9a31e00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a1806c460 0x7f9a1806e920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a20005950 tx=0x7f9a20005cf0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:41:14.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:14.930+0000 7f9a2b400700 1 -- 172.21.15.176:0/3021591845 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f9a240bf0e0 con 0x7f9a340ff2e0 2024-03-13T14:41:15.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:15.158+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a3419e6e0 con 0x7f9a1806c460 2024-03-13T14:41:15.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:15 smithi176 ceph-mon[29064]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:15.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:15 smithi196 ceph-mon[37486]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:16.224 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:16 smithi176 ceph-mon[29064]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:41:16.224 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:41:16.224 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:41:16.224 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:16.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:16 smithi196 ceph-mon[37486]: from='client.14278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:41:16.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:41:16.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:41:16.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:17.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:17 smithi176 ceph-mon[29064]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:17.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:17 smithi196 ceph-mon[37486]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:19.339 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:19 smithi176 ceph-mon[29064]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:19.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:19 smithi196 ceph-mon[37486]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:20.827 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: Detected new or changed devices on smithi176 2024-03-13T14:41:20.827 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.827 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.827 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:41:20.827 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi176 to 11843M 2024-03-13T14:41:20.827 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.827 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:20.828 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:41:20.828 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.828 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi176 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi176 to 11843M 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:20.855 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:41:20.856 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:20.856 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:21.769 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:21 smithi196 ceph-mon[37486]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:21.769 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:21 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3455998827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a973902-7ecc-4a05-b917-a74e108df4db"}]: dispatch 2024-03-13T14:41:21.769 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:21 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3455998827' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a973902-7ecc-4a05-b917-a74e108df4db"}]': finished 2024-03-13T14:41:21.769 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:21 smithi196 ceph-mon[37486]: osdmap e11: 2 total, 1 up, 2 in 2024-03-13T14:41:21.769 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:21 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:21.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:21 smithi176 ceph-mon[29064]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:21.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:21 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3455998827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a973902-7ecc-4a05-b917-a74e108df4db"}]: dispatch 2024-03-13T14:41:21.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:21 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3455998827' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a973902-7ecc-4a05-b917-a74e108df4db"}]': finished 2024-03-13T14:41:21.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:21 smithi176 ceph-mon[29064]: osdmap e11: 2 total, 1 up, 2 in 2024-03-13T14:41:21.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:21 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:22.797 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:22 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3310098409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:41:22.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:22 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3310098409' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:41:23.853 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:23 smithi176 ceph-mon[29064]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:23.853 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:23 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:23.853 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:23 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:23.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:23 smithi196 ceph-mon[37486]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:23.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:23 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:23.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:23 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:25.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:25 smithi176 ceph-mon[29064]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:25.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:25 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:25.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:25 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:25.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:25 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:25.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:25 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:41:25.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:25 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:25.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:25 smithi196 ceph-mon[37486]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:25.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:25 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:25.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:25 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:25.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:25 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:25.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:25 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:41:25.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:25 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:27.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:27 smithi176 ceph-mon[29064]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:27.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:27 smithi196 ceph-mon[37486]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:28.622 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:28.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:29.833 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:29 smithi196 ceph-mon[37486]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:29.857 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:29 smithi176 ceph-mon[29064]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:31.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:30 smithi176 ceph-mon[29064]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:31.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-13T14:41:31.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:31.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:30 smithi176 ceph-mon[29064]: Deploying daemon osd.1 on smithi176 2024-03-13T14:41:31.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:30 smithi196 ceph-mon[37486]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:31.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-13T14:41:31.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:31.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:30 smithi196 ceph-mon[37486]: Deploying daemon osd.1 on smithi176 2024-03-13T14:41:33.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:33 smithi176 ceph-mon[29064]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:33.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:33 smithi196 ceph-mon[37486]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:35.250 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:35 smithi176 ceph-mon[29064]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:35.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:35 smithi196 ceph-mon[37486]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:37 smithi176 ceph-mon[29064]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:37.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:37 smithi196 ceph-mon[37486]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:39.525 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:39 smithi176 ceph-mon[29064]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:39.525 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:39.526 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:39.526 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:39.749 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:39 smithi196 ceph-mon[37486]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:39.750 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:39.750 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:39.750 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:40.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:40 smithi196 ceph-mon[37486]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:41.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:40 smithi176 ceph-mon[29064]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:42.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:42 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:42.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:42 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:42.470 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:42 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:42.470 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:42 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:43.400 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 1 on host 'smithi176' 2024-03-13T14:41:43.400 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.407+0000 7f9a2b400700 1 -- 172.21.15.176:0/3021591845 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9a3419e6e0 con 0x7f9a1806c460 2024-03-13T14:41:43.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a1806c460 msgr2=0x7f9a1806e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:43.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a1806c460 0x7f9a1806e920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a20005950 tx=0x7f9a20005cf0 comp rx=0 tx=0).stop 2024-03-13T14:41:43.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 msgr2=0x7f9a341983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:43.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a341983b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9a2400bd70 tx=0x7f9a2400be50 comp rx=0 tx=0).stop 2024-03-13T14:41:43.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 shutdown_connections 2024-03-13T14:41:43.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a1806c460 0x7f9a1806e920 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:43.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a340ff2e0 0x7f9a341983b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:43.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 --2- 172.21.15.176:0/3021591845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a340ffc40 0x7f9a341988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:43.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 >> 172.21.15.176:0/3021591845 conn(0x7f9a340fae40 msgr2=0x7f9a34107910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:41:43.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 shutdown_connections 2024-03-13T14:41:43.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:43.410+0000 7f9a38a00700 1 -- 172.21.15.176:0/3021591845 wait complete. 2024-03-13T14:41:43.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:43 smithi176 ceph-mon[29064]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:43.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:43 smithi176 ceph-mon[29064]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-13T14:41:43.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:43.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:43 smithi196 ceph-mon[37486]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:43.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:43 smithi196 ceph-mon[37486]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-13T14:41:43.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:44.184 DEBUG:teuthology.orchestra.run.smithi176:osd.1> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.1.service 2024-03-13T14:41:44.191 INFO:tasks.cephadm:Deploying osd.2 on smithi176 with /dev/vg_nvme/lv_2... 2024-03-13T14:41:44.192 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-13T14:41:44.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:44 smithi176 ceph-mon[29064]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-13T14:41:44.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:44 smithi176 ceph-mon[29064]: osdmap e12: 2 total, 1 up, 2 in 2024-03-13T14:41:44.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:44 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:44.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:44 smithi176 ceph-mon[29064]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:41:44.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:44 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:44.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:44 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:44.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:44 smithi196 ceph-mon[37486]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-13T14:41:44.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:44 smithi196 ceph-mon[37486]: osdmap e12: 2 total, 1 up, 2 in 2024-03-13T14:41:44.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:44 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:44.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:44 smithi196 ceph-mon[37486]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:41:44.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:44 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:44.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:44 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:44.856 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:41:45.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:45 smithi176 ceph-mon[29064]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:45.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:45 smithi176 ceph-mon[29064]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-03-13T14:41:45.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:45 smithi176 ceph-mon[29064]: osdmap e13: 2 total, 1 up, 2 in 2024-03-13T14:41:45.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:45.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:45 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:45.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:45 smithi196 ceph-mon[37486]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:45.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:45 smithi196 ceph-mon[37486]: from='osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-03-13T14:41:45.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:45 smithi196 ceph-mon[37486]: osdmap e13: 2 total, 1 up, 2 in 2024-03-13T14:41:45.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:45 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:45.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:45 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:46.592 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:46 smithi176 ceph-mon[29064]: purged_snaps scrub starts 2024-03-13T14:41:46.592 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:46 smithi176 ceph-mon[29064]: purged_snaps scrub ok 2024-03-13T14:41:46.592 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:46 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:46.592 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:46 smithi176 ceph-mon[29064]: osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] boot 2024-03-13T14:41:46.592 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:46 smithi176 ceph-mon[29064]: osdmap e14: 2 total, 2 up, 2 in 2024-03-13T14:41:46.592 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:46 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:46.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:46 smithi196 ceph-mon[37486]: purged_snaps scrub starts 2024-03-13T14:41:46.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:46 smithi196 ceph-mon[37486]: purged_snaps scrub ok 2024-03-13T14:41:46.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:46 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:46.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:46 smithi196 ceph-mon[37486]: osd.1 [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] boot 2024-03-13T14:41:46.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:46 smithi196 ceph-mon[37486]: osdmap e14: 2 total, 2 up, 2 in 2024-03-13T14:41:46.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:46 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:41:47.475 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:47 smithi176 ceph-mon[29064]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:47.476 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:47 smithi176 ceph-mon[29064]: osdmap e15: 2 total, 2 up, 2 in 2024-03-13T14:41:47.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:47 smithi196 ceph-mon[37486]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-13T14:41:47.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:47 smithi196 ceph-mon[37486]: osdmap e15: 2 total, 2 up, 2 in 2024-03-13T14:41:47.843 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-13T14:41:47.844 INFO:teuthology.orchestra.run.smithi176.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-13T14:41:47.844 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10+0 records in 2024-03-13T14:41:47.844 INFO:teuthology.orchestra.run.smithi176.stderr:10+0 records out 2024-03-13T14:41:47.844 INFO:teuthology.orchestra.run.smithi176.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0363738 s, 288 MB/s 2024-03-13T14:41:47.844 INFO:teuthology.orchestra.run.smithi176.stderr:--> Zapping successful for: 2024-03-13T14:41:48.650 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi176:vg_nvme/lv_2 2024-03-13T14:41:49.070 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:41:49.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:49 smithi176 ceph-mon[29064]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:49.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:49.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:49.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:49 smithi196 ceph-mon[37486]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:49.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:49 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:49.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:49 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:51.066 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:50 smithi176 ceph-mon[29064]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:51.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:50 smithi196 ceph-mon[37486]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:52.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.330+0000 7fef8b000700 1 -- 172.21.15.176:0/2974990956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef84074dc0 msgr2=0x7fef84073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:52.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.330+0000 7fef8b000700 1 --2- 172.21.15.176:0/2974990956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef84074dc0 0x7fef84073220 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fef74009910 tx=0x7fef74009c20 comp rx=0 tx=0).stop 2024-03-13T14:41:52.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.330+0000 7fef8b000700 1 -- 172.21.15.176:0/2974990956 shutdown_connections 2024-03-13T14:41:52.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.330+0000 7fef8b000700 1 --2- 172.21.15.176:0/2974990956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef840737f0 0x7fef84073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:52.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.330+0000 7fef8b000700 1 --2- 172.21.15.176:0/2974990956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef84074dc0 0x7fef84073220 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:52.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.330+0000 7fef8b000700 1 -- 172.21.15.176:0/2974990956 >> 172.21.15.176:0/2974990956 conn(0x7fef840fc110 msgr2=0x7fef840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:41:52.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.330+0000 7fef8b000700 1 -- 172.21.15.176:0/2974990956 shutdown_connections 2024-03-13T14:41:52.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.331+0000 7fef8b000700 1 -- 172.21.15.176:0/2974990956 wait complete. 2024-03-13T14:41:52.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.332+0000 7fef8b000700 1 Processor -- start 2024-03-13T14:41:52.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef8b000700 1 -- start start 2024-03-13T14:41:52.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef8b000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef840737f0 0x7fef84198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:41:52.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef8b000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef84074dc0 0x7fef84198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:41:52.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef8b000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef841990d0 con 0x7fef840737f0 2024-03-13T14:41:52.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef8b000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef84199210 con 0x7fef84074dc0 2024-03-13T14:41:52.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef83e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef840737f0 0x7fef84198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:41:52.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef83e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef840737f0 0x7fef84198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36922/0 (socket says 172.21.15.176:36922) 2024-03-13T14:41:52.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef83e00700 1 -- 172.21.15.176:0/3893964743 learned_addr learned my addr 172.21.15.176:0/3893964743 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:41:52.325 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.333+0000 7fef83400700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef84074dc0 0x7fef84198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef83e00700 1 -- 172.21.15.176:0/3893964743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef84074dc0 msgr2=0x7fef84198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef83e00700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef84074dc0 0x7fef84198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef83e00700 1 -- 172.21.15.176:0/3893964743 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef740095f0 con 0x7fef840737f0 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef83e00700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef840737f0 0x7fef84198570 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fef7400bd70 tx=0x7fef7400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef80c00700 1 -- 172.21.15.176:0/3893964743 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef7401d070 con 0x7fef840737f0 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef80c00700 1 -- 172.21.15.176:0/3893964743 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fef7400f460 con 0x7fef840737f0 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef8419dc60 con 0x7fef840737f0 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.334+0000 7fef80c00700 1 -- 172.21.15.176:0/3893964743 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef74021620 con 0x7fef840737f0 2024-03-13T14:41:52.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.335+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8419e150 con 0x7fef840737f0 2024-03-13T14:41:52.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.336+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef8410b920 con 0x7fef840737f0 2024-03-13T14:41:52.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.337+0000 7fef80c00700 1 -- 172.21.15.176:0/3893964743 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7fef7400fa40 con 0x7fef840737f0 2024-03-13T14:41:52.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.338+0000 7fef80c00700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fef6806c470 0x7fef6806e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:41:52.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.338+0000 7fef80c00700 1 -- 172.21.15.176:0/3893964743 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7fef7408be90 con 0x7fef840737f0 2024-03-13T14:41:52.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.338+0000 7fef83400700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fef6806c470 0x7fef6806e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:41:52.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.339+0000 7fef83400700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fef6806c470 0x7fef6806e930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fef70005950 tx=0x7fef700058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:41:52.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.343+0000 7fef80c00700 1 -- 172.21.15.176:0/3893964743 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fef7405add0 con 0x7fef840737f0 2024-03-13T14:41:52.561 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:41:52.570+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fef841080b0 con 0x7fef6806c470 2024-03-13T14:41:53.339 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:53 smithi176 ceph-mon[29064]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:53.340 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:53 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:41:53.340 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:53 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:41:53.340 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:53 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:53.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:53 smithi196 ceph-mon[37486]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:53.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:53 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:41:53.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:53 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:41:53.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:53 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:54 smithi176 ceph-mon[29064]: from='client.14296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:41:54.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:54 smithi196 ceph-mon[37486]: from='client.14296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi176:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:41:55.339 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:55 smithi176 ceph-mon[29064]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:55.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:55 smithi196 ceph-mon[37486]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:56.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: Detected new or changed devices on smithi176 2024-03-13T14:41:56.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:41:56.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:41:56.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi176 to 5921M 2024-03-13T14:41:56.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:56.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:41:56.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi176 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi176 to 5921M 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:56.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:41:57.526 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:57 smithi196 ceph-mon[37486]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:57.527 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:57 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1740375700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f72757d1-b949-45fb-abcd-c8649cec14f1"}]: dispatch 2024-03-13T14:41:57.527 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:57 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1740375700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f72757d1-b949-45fb-abcd-c8649cec14f1"}]': finished 2024-03-13T14:41:57.527 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:57 smithi196 ceph-mon[37486]: osdmap e16: 3 total, 2 up, 3 in 2024-03-13T14:41:57.527 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:41:57.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:57 smithi176 ceph-mon[29064]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:57.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:57 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1740375700' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f72757d1-b949-45fb-abcd-c8649cec14f1"}]: dispatch 2024-03-13T14:41:57.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:57 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1740375700' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f72757d1-b949-45fb-abcd-c8649cec14f1"}]': finished 2024-03-13T14:41:57.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:57 smithi176 ceph-mon[29064]: osdmap e16: 3 total, 2 up, 3 in 2024-03-13T14:41:57.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:41:58.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:58 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/4045164311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:41:58.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:58.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:58 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/4045164311' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:41:58.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:41:59.584 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:59 smithi196 ceph-mon[37486]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:59.584 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:59.584 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:41:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:59.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:59 smithi176 ceph-mon[29064]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:41:59.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:41:59.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:41:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:01.093 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:00 smithi176 ceph-mon[29064]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:01.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:00 smithi196 ceph-mon[37486]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:02.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:02.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:02.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:02.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:42:02.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:02 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:02.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:02.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:02.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:02.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:42:02.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:02 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:03.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:03 smithi176 ceph-mon[29064]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:03.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:03 smithi196 ceph-mon[37486]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:05.597 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:05 smithi176 ceph-mon[29064]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:05.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:05 smithi196 ceph-mon[37486]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:06.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-13T14:42:06.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:06.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-13T14:42:06.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:07 smithi176 ceph-mon[29064]: Deploying daemon osd.2 on smithi176 2024-03-13T14:42:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:07 smithi176 ceph-mon[29064]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:07.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:07 smithi196 ceph-mon[37486]: Deploying daemon osd.2 on smithi176 2024-03-13T14:42:07.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:07 smithi196 ceph-mon[37486]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:09.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:09 smithi176 ceph-mon[29064]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:09.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:09 smithi196 ceph-mon[37486]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:11.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:10 smithi176 ceph-mon[29064]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:11.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:10 smithi196 ceph-mon[37486]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:13.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:13 smithi176 ceph-mon[29064]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:13.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:13.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:13 smithi196 ceph-mon[37486]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:13.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:15.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:15 smithi176 ceph-mon[29064]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:15.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:15.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:42:15.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:15.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:15 smithi196 ceph-mon[37486]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:15.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:15.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:42:15.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:17.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:17 smithi176 ceph-mon[29064]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:17.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:17 smithi196 ceph-mon[37486]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:18.855 INFO:teuthology.orchestra.run.smithi176.stdout:Created osd(s) 2 on host 'smithi176' 2024-03-13T14:42:18.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.863+0000 7fef80c00700 1 -- 172.21.15.176:0/3893964743 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fef841080b0 con 0x7fef6806c470 2024-03-13T14:42:18.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.866+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fef6806c470 msgr2=0x7fef6806e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:42:18.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fef6806c470 0x7fef6806e930 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fef70005950 tx=0x7fef700058e0 comp rx=0 tx=0).stop 2024-03-13T14:42:18.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef840737f0 msgr2=0x7fef84198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:42:18.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef840737f0 0x7fef84198570 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fef7400bd70 tx=0x7fef7400be50 comp rx=0 tx=0).stop 2024-03-13T14:42:18.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 shutdown_connections 2024-03-13T14:42:18.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fef6806c470 0x7fef6806e930 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:18.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fef840737f0 0x7fef84198570 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:18.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 --2- 172.21.15.176:0/3893964743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef84074dc0 0x7fef84198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:18.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 >> 172.21.15.176:0/3893964743 conn(0x7fef840fc110 msgr2=0x7fef84106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:42:18.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 shutdown_connections 2024-03-13T14:42:18.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:42:18.867+0000 7fef8b000700 1 -- 172.21.15.176:0/3893964743 wait complete. 2024-03-13T14:42:19.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:19 smithi176 ceph-mon[29064]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:19.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:19 smithi176 ceph-mon[29064]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-13T14:42:19.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:19.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:19.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:19 smithi196 ceph-mon[37486]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:19.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:19 smithi196 ceph-mon[37486]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-13T14:42:19.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:19.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:20.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:20 smithi196 ceph-mon[37486]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-13T14:42:20.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:20 smithi196 ceph-mon[37486]: osdmap e17: 3 total, 2 up, 3 in 2024-03-13T14:42:20.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:20 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:20.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:20 smithi196 ceph-mon[37486]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:42:20.211 DEBUG:teuthology.orchestra.run.smithi176:osd.2> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.2.service 2024-03-13T14:42:20.222 INFO:tasks.cephadm:Deploying osd.3 on smithi196 with /dev/vg_nvme/lv_4... 2024-03-13T14:42:20.222 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-13T14:42:20.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:20 smithi176 ceph-mon[29064]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-13T14:42:20.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:20 smithi176 ceph-mon[29064]: osdmap e17: 3 total, 2 up, 3 in 2024-03-13T14:42:20.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:20 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:20.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:20 smithi176 ceph-mon[29064]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:42:20.620 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi196/config 2024-03-13T14:42:21.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:21 smithi176 ceph-mon[29064]: purged_snaps scrub starts 2024-03-13T14:42:21.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:21 smithi176 ceph-mon[29064]: purged_snaps scrub ok 2024-03-13T14:42:21.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:21 smithi176 ceph-mon[29064]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:21.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:21 smithi176 ceph-mon[29064]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-03-13T14:42:21.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:21 smithi176 ceph-mon[29064]: osdmap e18: 3 total, 2 up, 3 in 2024-03-13T14:42:21.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:21 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:21.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:21 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:21.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:21 smithi196 ceph-mon[37486]: purged_snaps scrub starts 2024-03-13T14:42:21.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:21 smithi196 ceph-mon[37486]: purged_snaps scrub ok 2024-03-13T14:42:21.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:21 smithi196 ceph-mon[37486]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-13T14:42:21.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:21 smithi196 ceph-mon[37486]: from='osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]': finished 2024-03-13T14:42:21.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:21 smithi196 ceph-mon[37486]: osdmap e18: 3 total, 2 up, 3 in 2024-03-13T14:42:21.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:21 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:21.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:21 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:22.361 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:22 smithi196 ceph-mon[37486]: osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] boot 2024-03-13T14:42:22.361 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:22 smithi196 ceph-mon[37486]: osdmap e19: 3 total, 3 up, 3 in 2024-03-13T14:42:22.361 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:22.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:22 smithi176 ceph-mon[29064]: osd.2 [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] boot 2024-03-13T14:42:22.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:22 smithi176 ceph-mon[29064]: osdmap e19: 3 total, 3 up, 3 in 2024-03-13T14:42:22.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:42:23.185 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-13T14:42:23.185 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-13T14:42:23.185 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-13T14:42:23.185 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-13T14:42:23.186 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.035171 s, 298 MB/s 2024-03-13T14:42:23.186 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-13T14:42:23.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:23 smithi176 ceph-mon[29064]: pgmap v94: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:23.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:23 smithi176 ceph-mon[29064]: osdmap e20: 3 total, 3 up, 3 in 2024-03-13T14:42:23.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:23 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-13T14:42:23.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:23 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:23.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:23 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:23.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:23 smithi196 ceph-mon[37486]: pgmap v94: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:23.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:23 smithi196 ceph-mon[37486]: osdmap e20: 3 total, 3 up, 3 in 2024-03-13T14:42:23.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:23 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-13T14:42:23.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:23 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:23.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:23 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:23.925 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-03-13T14:42:24.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-13T14:42:24.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:24 smithi196 ceph-mon[37486]: osdmap e21: 3 total, 3 up, 3 in 2024-03-13T14:42:24.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-13T14:42:24.297 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi196/config 2024-03-13T14:42:24.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-13T14:42:24.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:24 smithi176 ceph-mon[29064]: osdmap e21: 3 total, 3 up, 3 in 2024-03-13T14:42:24.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-13T14:42:25.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:25 smithi176 ceph-mon[29064]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:25.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:25 smithi176 ceph-mon[29064]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-13T14:42:25.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:25 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-13T14:42:25.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:25 smithi176 ceph-mon[29064]: osdmap e22: 3 total, 3 up, 3 in 2024-03-13T14:42:25.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:25 smithi196 ceph-mon[37486]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:25.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:25 smithi196 ceph-mon[37486]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-13T14:42:25.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:25 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-13T14:42:25.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:25 smithi196 ceph-mon[37486]: osdmap e22: 3 total, 3 up, 3 in 2024-03-13T14:42:25.881 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:25 smithi176 sudo[64527]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-13T14:42:25.881 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:25 smithi176 sudo[64527]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:25.881 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:25 smithi176 sudo[64527]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:25.881 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:25 smithi176 sudo[64527]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.444+0000 7ff227400700 1 -- 172.21.15.196:0/226097835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 msgr2=0x7ff220073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.444+0000 7ff227400700 1 --2- 172.21.15.196:0/226097835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220073c10 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff208009910 tx=0x7ff208009c20 comp rx=0 tx=0).stop 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.445+0000 7ff227400700 1 -- 172.21.15.196:0/226097835 shutdown_connections 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.445+0000 7ff227400700 1 --2- 172.21.15.196:0/226097835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220073c10 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.445+0000 7ff227400700 1 --2- 172.21.15.196:0/226097835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff220074d60 0x7ff2200731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.445+0000 7ff227400700 1 -- 172.21.15.196:0/226097835 >> 172.21.15.196:0/226097835 conn(0x7ff2200fc140 msgr2=0x7ff2200fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.445+0000 7ff227400700 1 -- 172.21.15.196:0/226097835 shutdown_connections 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.445+0000 7ff227400700 1 -- 172.21.15.196:0/226097835 wait complete. 2024-03-13T14:42:26.436 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.446+0000 7ff227400700 1 Processor -- start 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.446+0000 7ff227400700 1 -- start start 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.446+0000 7ff227400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.446+0000 7ff227400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff220074d60 0x7ff220198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.446+0000 7ff227400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2201990c0 con 0x7ff220074d60 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.446+0000 7ff227400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff220199200 con 0x7ff220073790 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff224a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff224a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:59208/0 (socket says 172.21.15.196:59208) 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff224a00700 1 -- 172.21.15.196:0/3823405616 learned_addr learned my addr 172.21.15.196:0/3823405616 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff217e00700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff220074d60 0x7ff220198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:42:26.438 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff224a00700 1 -- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff220074d60 msgr2=0x7ff220198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff224a00700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff220074d60 0x7ff220198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff224a00700 1 -- 172.21.15.196:0/3823405616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2080095f0 con 0x7ff220073790 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.447+0000 7ff224a00700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220198560 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff21000d8d0 tx=0x7ff21000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.448+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2100098a0 con 0x7ff220073790 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.448+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff210010460 con 0x7ff220073790 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.448+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff21000f5f0 con 0x7ff220073790 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.448+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22019dcb0 con 0x7ff220073790 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.448+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22019e1b0 con 0x7ff220073790 2024-03-13T14:42:26.439 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.449+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22010b910 con 0x7ff220073790 2024-03-13T14:42:26.440 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.450+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 18) v1 ==== 90258+0+0 (secure 0 0 0) 0x7ff210009a00 con 0x7ff220073790 2024-03-13T14:42:26.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.451+0000 7ff21e000700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff20406c470 0x7ff20406e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:42:26.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.451+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3232+0+0 (secure 0 0 0) 0x7ff21008ace0 con 0x7ff220073790 2024-03-13T14:42:26.441 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.451+0000 7ff217e00700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff20406c470 0x7ff20406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:42:26.442 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.452+0000 7ff217e00700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff20406c470 0x7ff20406e930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff208006010 tx=0x7ff20800b560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:42:26.446 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64556]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-13T14:42:26.446 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64556]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:26.446 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64556]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:26.446 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64556]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:26.447 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.457+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff2100598f0 con 0x7ff220073790 2024-03-13T14:42:26.671 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.681+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff220108120 con 0x7ff20406c470 2024-03-13T14:42:26.835 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:26.845+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff210056230 con 0x7ff220073790 2024-03-13T14:42:26.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:26 smithi196 ceph-mon[37486]: osdmap e23: 3 total, 3 up, 3 in 2024-03-13T14:42:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:26 smithi196 ceph-mon[37486]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:26 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:26 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:42:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:26 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:42:26.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:26 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:27.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:26 smithi176 ceph-mon[29064]: osdmap e23: 3 total, 3 up, 3 in 2024-03-13T14:42:27.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:26 smithi176 ceph-mon[29064]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:27.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:26 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:27.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:26 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:42:27.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:26 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:42:27.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:26 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:27.132 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64580]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-13T14:42:27.132 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64580]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:27.132 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64580]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:27.132 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:26 smithi176 sudo[64580]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:27.785 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:27 smithi176 sudo[67021]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-13T14:42:27.785 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:27 smithi176 sudo[67021]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:27.785 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:27 smithi176 sudo[67021]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:27.785 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:42:27 smithi176 sudo[67021]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:28.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:27 smithi176 ceph-mon[29064]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:42:28.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:27 smithi176 ceph-mon[29064]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-13T14:42:28.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:27 smithi176 ceph-mon[29064]: Cluster is now healthy 2024-03-13T14:42:28.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:27 smithi176 ceph-mon[29064]: mgrmap e19: smithi176.tzfdwd(active, since 2m), standbys: smithi196.vdktbk 2024-03-13T14:42:28.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:27 smithi196 ceph-mon[37486]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:42:28.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:27 smithi196 ceph-mon[37486]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-13T14:42:28.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:27 smithi196 ceph-mon[37486]: Cluster is now healthy 2024-03-13T14:42:28.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:27 smithi196 ceph-mon[37486]: mgrmap e19: smithi176.tzfdwd(active, since 2m), standbys: smithi196.vdktbk 2024-03-13T14:42:28.631 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67026]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-13T14:42:28.632 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67026]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:28.632 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67026]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:28.632 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67026]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:29.131 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67061]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-13T14:42:29.131 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67061]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:29.131 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67061]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:29.131 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:42:28 smithi176 sudo[67061]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-13T14:42:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67167]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67167]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:30.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67223]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-13T14:42:30.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67223]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:30.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67223]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:30.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:29 smithi176 sudo[67223]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:30.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-13T14:42:30.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:30.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41322]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:30.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41322]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:31.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: pgmap v101: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:31.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:31.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: Detected new or changed devices on smithi176 2024-03-13T14:42:31.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi176 to 3947M 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-13T14:42:31.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:30 smithi176 ceph-mon[29064]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-13T14:42:31.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41325]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-13T14:42:31.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-13T14:42:31.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41325]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 sudo[41325]: pam_unix(sudo:session): session closed for user root 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: pgmap v101: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi176 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi176 to 3947M 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:42:31.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:31.434 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:42:31.434 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-13T14:42:31.434 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:30 smithi196 ceph-mon[37486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-13T14:42:32.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:42:32.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:42:32.046 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: pgmap v102: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/636563931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1c42f2a-0efa-4c44-a965-9e5904aeace0"}]: dispatch 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1c42f2a-0efa-4c44-a965-9e5904aeace0"}]: dispatch 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1c42f2a-0efa-4c44-a965-9e5904aeace0"}]': finished 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: osdmap e24: 4 total, 3 up, 4 in 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:32.047 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:31 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/2525678163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: pgmap v102: 1 pgs: 1 active+clean; 449 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/636563931' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1c42f2a-0efa-4c44-a965-9e5904aeace0"}]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e1c42f2a-0efa-4c44-a965-9e5904aeace0"}]: dispatch 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e1c42f2a-0efa-4c44-a965-9e5904aeace0"}]': finished 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: osdmap e24: 4 total, 3 up, 4 in 2024-03-13T14:42:32.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:32.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:32.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:31 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/2525678163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:42:33.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:32 smithi176 ceph-mon[29064]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-03-13T14:42:33.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:32 smithi196 ceph-mon[37486]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 6.7 KiB/s rd, 326 KiB/s wr, 22 op/s 2024-03-13T14:42:35.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:35 smithi176 ceph-mon[29064]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-03-13T14:42:35.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:35 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:35.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:35 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:35.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:35 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:35.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:35 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:42:35.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:35 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:35.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:35 smithi196 ceph-mon[37486]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 218 KiB/s wr, 14 op/s 2024-03-13T14:42:35.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:35 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:35.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:35 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:35.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:35 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:35.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:35 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:42:35.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:35 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:37.582 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:37 smithi196 ceph-mon[37486]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-03-13T14:42:37.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:37 smithi176 ceph-mon[29064]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 163 KiB/s wr, 11 op/s 2024-03-13T14:42:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:39 smithi176 ceph-mon[29064]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-13T14:42:39.902 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:39 smithi196 ceph-mon[37486]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-13T14:42:41.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:40 smithi196 ceph-mon[37486]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-13T14:42:41.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:40 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-13T14:42:41.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:40 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:41.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:40 smithi196 ceph-mon[37486]: Deploying daemon osd.3 on smithi196 2024-03-13T14:42:41.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:40 smithi176 ceph-mon[29064]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-03-13T14:42:41.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:40 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-13T14:42:41.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:40 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:42:41.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:40 smithi176 ceph-mon[29064]: Deploying daemon osd.3 on smithi196 2024-03-13T14:42:43.285 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:43 smithi196 ceph-mon[37486]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 118 KiB/s wr, 8 op/s 2024-03-13T14:42:43.285 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:43 smithi176 ceph-mon[29064]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.4 KiB/s rd, 118 KiB/s wr, 8 op/s 2024-03-13T14:42:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:45.381 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:45 smithi196 ceph-mon[37486]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:45.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:45 smithi176 ceph-mon[29064]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:47.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:47 smithi176 ceph-mon[29064]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:47.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:47 smithi196 ceph-mon[37486]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:49.353 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:49 smithi196 ceph-mon[37486]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:49.353 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:49 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:42:49.353 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:49 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:49.354 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:49 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:49.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:49 smithi176 ceph-mon[29064]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:49.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:42:49.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:49.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:49 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:51.144 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:50 smithi196 ceph-mon[37486]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:51.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:50 smithi176 ceph-mon[29064]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:53.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:53 smithi176 ceph-mon[29064]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:53.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:53 smithi176 ceph-mon[29064]: from='osd.3 [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:42:53.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:53 smithi176 ceph-mon[29064]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:42:53.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:53 smithi196 ceph-mon[37486]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:53.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:53 smithi196 ceph-mon[37486]: from='osd.3 [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:42:53.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:53 smithi196 ceph-mon[37486]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:42:53.572 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.582+0000 7ff21e000700 1 -- 172.21.15.196:0/3823405616 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff220108120 con 0x7ff20406c470 2024-03-13T14:42:53.573 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 3 on host 'smithi196' 2024-03-13T14:42:53.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff20406c470 msgr2=0x7ff20406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:42:53.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff20406c470 0x7ff20406e930 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff208006010 tx=0x7ff20800b560 comp rx=0 tx=0).stop 2024-03-13T14:42:53.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 msgr2=0x7ff220198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:42:53.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220198560 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff21000d8d0 tx=0x7ff21000dbe0 comp rx=0 tx=0).stop 2024-03-13T14:42:53.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 shutdown_connections 2024-03-13T14:42:53.576 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff20406c470 0x7ff20406e930 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:53.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff220073790 0x7ff220198560 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:53.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 --2- 172.21.15.196:0/3823405616 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff220074d60 0x7ff220198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:42:53.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.586+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 >> 172.21.15.196:0/3823405616 conn(0x7ff2200fc140 msgr2=0x7ff220106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:42:53.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.587+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 shutdown_connections 2024-03-13T14:42:53.577 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:42:53.587+0000 7ff227400700 1 -- 172.21.15.196:0/3823405616 wait complete. 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: osdmap e25: 4 total, 3 up, 4 in 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='osd.3 [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:54.295 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:54 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:54.357 DEBUG:teuthology.orchestra.run.smithi196:osd.3> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.3.service 2024-03-13T14:42:54.362 INFO:tasks.cephadm:Deploying osd.4 on smithi196 with /dev/vg_nvme/lv_3... 2024-03-13T14:42:54.362 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-13T14:42:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-13T14:42:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: osdmap e25: 4 total, 3 up, 4 in 2024-03-13T14:42:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='osd.3 [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:42:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:42:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:54.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:54.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:54.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:54 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:55.373 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:55.374 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-13T14:42:55.374 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: osdmap e26: 4 total, 3 up, 4 in 2024-03-13T14:42:55.374 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:55.374 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:55.374 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: osd.3 [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] boot 2024-03-13T14:42:55.374 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: osdmap e27: 4 total, 4 up, 4 in 2024-03-13T14:42:55.374 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:55 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: osdmap e26: 4 total, 3 up, 4 in 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: osd.3 [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] boot 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: osdmap e27: 4 total, 4 up, 4 in 2024-03-13T14:42:55.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:55 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:42:55.703 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi196/config 2024-03-13T14:42:56.302 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:56 smithi196 ceph-mon[37486]: purged_snaps scrub starts 2024-03-13T14:42:56.302 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:56 smithi196 ceph-mon[37486]: purged_snaps scrub ok 2024-03-13T14:42:56.302 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:56 smithi196 ceph-mon[37486]: osdmap e28: 4 total, 4 up, 4 in 2024-03-13T14:42:56.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:56 smithi176 ceph-mon[29064]: purged_snaps scrub starts 2024-03-13T14:42:56.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:56 smithi176 ceph-mon[29064]: purged_snaps scrub ok 2024-03-13T14:42:56.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:56 smithi176 ceph-mon[29064]: osdmap e28: 4 total, 4 up, 4 in 2024-03-13T14:42:57.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:57 smithi176 ceph-mon[29064]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:42:57.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:57 smithi176 ceph-mon[29064]: osdmap e29: 4 total, 4 up, 4 in 2024-03-13T14:42:57.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:57 smithi196 ceph-mon[37486]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:42:57.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:57 smithi196 ceph-mon[37486]: osdmap e29: 4 total, 4 up, 4 in 2024-03-13T14:42:59.158 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-13T14:42:59.224 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-13T14:42:59.224 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-13T14:42:59.224 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-13T14:42:59.224 INFO:teuthology.orchestra.run.smithi196.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0327257 s, 320 MB/s 2024-03-13T14:42:59.224 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 2024-03-13T14:42:59.224 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-13T14:42:59.224 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:58 smithi196 ceph-mon[37486]: osdmap e30: 4 total, 4 up, 4 in 2024-03-13T14:42:59.224 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:58 smithi196 ceph-mon[37486]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:42:59.224 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:59.225 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:59.225 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:59.225 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:42:59.321 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:58 smithi176 ceph-mon[29064]: osdmap e30: 4 total, 4 up, 4 in 2024-03-13T14:42:59.321 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:58 smithi176 ceph-mon[29064]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:42:59.321 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:42:59.322 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:59.322 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:42:59.322 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:43:00.060 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi196 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi196 to 14915M 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: osdmap e31: 4 total, 4 up, 4 in 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:43:00.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:00.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:00.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:42:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:00.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: Detected new or changed devices on smithi196 2024-03-13T14:43:00.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi196 to 14915M 2024-03-13T14:43:00.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: osdmap e31: 4 total, 4 up, 4 in 2024-03-13T14:43:00.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:00.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:00.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:00.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:00.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:43:00.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:00.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:00.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:42:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:00.463 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi196/config 2024-03-13T14:43:01.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:00 smithi196 ceph-mon[37486]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:01.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:00 smithi176 ceph-mon[29064]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:02.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.498+0000 7fd726200700 1 -- 172.21.15.196:0/303393563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720101700 msgr2=0x7fd720103af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:43:02.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.498+0000 7fd726200700 1 --2- 172.21.15.196:0/303393563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720101700 0x7fd720103af0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd710009910 tx=0x7fd710009c20 comp rx=0 tx=0).stop 2024-03-13T14:43:02.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.498+0000 7fd726200700 1 -- 172.21.15.196:0/303393563 shutdown_connections 2024-03-13T14:43:02.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.498+0000 7fd726200700 1 --2- 172.21.15.196:0/303393563 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720104030 0x7fd720106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:02.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.498+0000 7fd726200700 1 --2- 172.21.15.196:0/303393563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720101700 0x7fd720103af0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:02.489 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.498+0000 7fd726200700 1 -- 172.21.15.196:0/303393563 >> 172.21.15.196:0/303393563 conn(0x7fd7200fb030 msgr2=0x7fd7200fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:43:02.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.499+0000 7fd726200700 1 -- 172.21.15.196:0/303393563 shutdown_connections 2024-03-13T14:43:02.490 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.499+0000 7fd726200700 1 -- 172.21.15.196:0/303393563 wait complete. 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd726200700 1 Processor -- start 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd726200700 1 -- start start 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd726200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720101700 0x7fd720198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd726200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720104030 0x7fd720198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd726200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7201990c0 con 0x7fd720101700 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd726200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd720199200 con 0x7fd720104030 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd71ea00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720104030 0x7fd720198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd71ea00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720104030 0x7fd720198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:44076/0 (socket says 172.21.15.196:44076) 2024-03-13T14:43:02.492 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.500+0000 7fd71ea00700 1 -- 172.21.15.196:0/1316922415 learned_addr learned my addr 172.21.15.196:0/1316922415 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.501+0000 7fd71f400700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720101700 0x7fd720198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.501+0000 7fd71ea00700 1 -- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720101700 msgr2=0x7fd720198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.501+0000 7fd71ea00700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720101700 0x7fd720198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.501+0000 7fd71ea00700 1 -- 172.21.15.196:0/1316922415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100095f0 con 0x7fd720104030 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.501+0000 7fd71ea00700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720104030 0x7fd720198aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00d8d0 tx=0x7fd70c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.501+0000 7fd717e00700 1 -- 172.21.15.196:0/1316922415 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd70c009940 con 0x7fd720104030 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.502+0000 7fd717e00700 1 -- 172.21.15.196:0/1316922415 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd70c010460 con 0x7fd720104030 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.502+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72019dcb0 con 0x7fd720104030 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.502+0000 7fd717e00700 1 -- 172.21.15.196:0/1316922415 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd70c00f5f0 con 0x7fd720104030 2024-03-13T14:43:02.493 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.502+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72019e200 con 0x7fd720104030 2024-03-13T14:43:02.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.503+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7200fcc20 con 0x7fd720104030 2024-03-13T14:43:02.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.504+0000 7fd717e00700 1 -- 172.21.15.196:0/1316922415 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd70c0105d0 con 0x7fd720104030 2024-03-13T14:43:02.494 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.505+0000 7fd717e00700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd70406c4e0 0x7fd70406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:43:02.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.505+0000 7fd717e00700 1 -- 172.21.15.196:0/1316922415 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7fd70c08b230 con 0x7fd720104030 2024-03-13T14:43:02.495 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.505+0000 7fd71f400700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd70406c4e0 0x7fd70406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:43:02.496 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.506+0000 7fd71f400700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd70406c4e0 0x7fd70406e9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd710006010 tx=0x7fd71000b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:43:02.500 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.510+0000 7fd717e00700 1 -- 172.21.15.196:0/1316922415 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd70c059aa0 con 0x7fd720104030 2024-03-13T14:43:02.726 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:02.736+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7200611d0 con 0x7fd70406c4e0 2024-03-13T14:43:03.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:03 smithi176 ceph-mon[29064]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-13T14:43:03.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:03 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:43:03.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:03 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:43:03.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:03 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:03.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:03 smithi196 ceph-mon[37486]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-13T14:43:03.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:03 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:43:03.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:03 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:43:03.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:03 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:04.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:04 smithi176 ceph-mon[29064]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:43:04.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:04 smithi196 ceph-mon[37486]: from='client.24145 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:43:05.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:05 smithi196 ceph-mon[37486]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-13T14:43:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:05 smithi176 ceph-mon[29064]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-03-13T14:43:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:07 smithi176 ceph-mon[29064]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-13T14:43:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:07 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1607411826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4c7c89d-a349-4b39-b64a-e1080d213acd"}]: dispatch 2024-03-13T14:43:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:07 smithi176 ceph-mon[29064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4c7c89d-a349-4b39-b64a-e1080d213acd"}]: dispatch 2024-03-13T14:43:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:07 smithi176 ceph-mon[29064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4c7c89d-a349-4b39-b64a-e1080d213acd"}]': finished 2024-03-13T14:43:07.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:07 smithi176 ceph-mon[29064]: osdmap e32: 5 total, 4 up, 5 in 2024-03-13T14:43:07.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:07 smithi196 ceph-mon[37486]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-03-13T14:43:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:07 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/1607411826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4c7c89d-a349-4b39-b64a-e1080d213acd"}]: dispatch 2024-03-13T14:43:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:07 smithi196 ceph-mon[37486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b4c7c89d-a349-4b39-b64a-e1080d213acd"}]: dispatch 2024-03-13T14:43:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:07 smithi196 ceph-mon[37486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b4c7c89d-a349-4b39-b64a-e1080d213acd"}]': finished 2024-03-13T14:43:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:07 smithi196 ceph-mon[37486]: osdmap e32: 5 total, 4 up, 5 in 2024-03-13T14:43:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:08 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/1715329739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:43:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:08 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/1715329739' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:43:09.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:09 smithi176 ceph-mon[29064]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-13T14:43:09.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:09 smithi196 ceph-mon[37486]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-13T14:43:11.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:10 smithi196 ceph-mon[37486]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-13T14:43:11.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:10 smithi176 ceph-mon[29064]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-13T14:43:13.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:13 smithi196 ceph-mon[37486]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:13.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:13.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:13 smithi176 ceph-mon[29064]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:13.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:15.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:15 smithi176 ceph-mon[29064]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:15.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:15 smithi196 ceph-mon[37486]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:16.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-13T14:43:16.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:16.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-13T14:43:16.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:17.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:17 smithi176 ceph-mon[29064]: Deploying daemon osd.4 on smithi196 2024-03-13T14:43:17.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:17 smithi176 ceph-mon[29064]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:17.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:17 smithi196 ceph-mon[37486]: Deploying daemon osd.4 on smithi196 2024-03-13T14:43:17.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:17 smithi196 ceph-mon[37486]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:19.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:19 smithi176 ceph-mon[29064]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:19.423 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:19 smithi196 ceph-mon[37486]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:21.100 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:20 smithi196 ceph-mon[37486]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:21.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:20 smithi176 ceph-mon[29064]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:23.286 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:23 smithi196 ceph-mon[37486]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:23.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:23 smithi176 ceph-mon[29064]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:24.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:43:24.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:24.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:24.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:43:24.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:24.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:25.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:25 smithi176 ceph-mon[29064]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:25.884 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:25 smithi196 ceph-mon[37486]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:27.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:27 smithi196 ceph-mon[37486]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:27.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:27 smithi196 ceph-mon[37486]: from='osd.4 [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:43:27.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:27 smithi196 ceph-mon[37486]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:43:27.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:27 smithi176 ceph-mon[29064]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:27.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:27 smithi176 ceph-mon[29064]: from='osd.4 [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:43:27.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:27 smithi176 ceph-mon[29064]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:43:28.628 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 4 on host 'smithi196' 2024-03-13T14:43:28.629 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.637+0000 7fd717e00700 1 -- 172.21.15.196:0/1316922415 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd7200611d0 con 0x7fd70406c4e0 2024-03-13T14:43:28.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.640+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd70406c4e0 msgr2=0x7fd70406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:43:28.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.640+0000 7fd726200700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd70406c4e0 0x7fd70406e9a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd710006010 tx=0x7fd71000b540 comp rx=0 tx=0).stop 2024-03-13T14:43:28.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.640+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720104030 msgr2=0x7fd720198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:43:28.631 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.640+0000 7fd726200700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720104030 0x7fd720198aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00d8d0 tx=0x7fd70c00dc90 comp rx=0 tx=0).stop 2024-03-13T14:43:28.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.641+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 shutdown_connections 2024-03-13T14:43:28.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.641+0000 7fd726200700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd70406c4e0 0x7fd70406e9a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:28.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.641+0000 7fd726200700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd720101700 0x7fd720198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:28.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.641+0000 7fd726200700 1 --2- 172.21.15.196:0/1316922415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd720104030 0x7fd720198aa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:28.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.641+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 >> 172.21.15.196:0/1316922415 conn(0x7fd7200fb030 msgr2=0x7fd7200ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:43:28.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.641+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 shutdown_connections 2024-03-13T14:43:28.632 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:28.641+0000 7fd726200700 1 -- 172.21.15.196:0/1316922415 wait complete. 2024-03-13T14:43:28.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:28 smithi176 ceph-mon[29064]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-13T14:43:28.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:28 smithi176 ceph-mon[29064]: osdmap e33: 5 total, 4 up, 5 in 2024-03-13T14:43:28.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:28 smithi176 ceph-mon[29064]: from='osd.4 [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:43:28.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:28.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:28 smithi176 ceph-mon[29064]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:43:28.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:28.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:28 smithi196 ceph-mon[37486]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-13T14:43:28.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:28 smithi196 ceph-mon[37486]: osdmap e33: 5 total, 4 up, 5 in 2024-03-13T14:43:28.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:28 smithi196 ceph-mon[37486]: from='osd.4 [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:43:28.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:28.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:28 smithi196 ceph-mon[37486]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:43:28.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:29.524 DEBUG:teuthology.orchestra.run.smithi196:osd.4> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.4.service 2024-03-13T14:43:29.530 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_2... 2024-03-13T14:43:29.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-13T14:43:29.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:29.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: osdmap e34: 5 total, 4 up, 5 in 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: osd.4 [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] boot 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: osdmap e35: 5 total, 5 up, 5 in 2024-03-13T14:43:29.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:29.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: osdmap e34: 5 total, 4 up, 5 in 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: osd.4 [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] boot 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: osdmap e35: 5 total, 5 up, 5 in 2024-03-13T14:43:29.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:43:29.943 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi196/config 2024-03-13T14:43:31.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:30 smithi196 ceph-mon[37486]: purged_snaps scrub starts 2024-03-13T14:43:31.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:30 smithi196 ceph-mon[37486]: purged_snaps scrub ok 2024-03-13T14:43:31.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:30 smithi196 ceph-mon[37486]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:31.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:30 smithi176 ceph-mon[29064]: purged_snaps scrub starts 2024-03-13T14:43:31.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:30 smithi176 ceph-mon[29064]: purged_snaps scrub ok 2024-03-13T14:43:31.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:30 smithi176 ceph-mon[29064]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:32.049 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:31 smithi196 ceph-mon[37486]: osdmap e36: 5 total, 5 up, 5 in 2024-03-13T14:43:32.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:31 smithi176 ceph-mon[29064]: osdmap e36: 5 total, 5 up, 5 in 2024-03-13T14:43:33.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:32 smithi196 ceph-mon[37486]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:33.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:32 smithi176 ceph-mon[29064]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:33.765 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-13T14:43:33.765 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-13T14:43:33.765 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-03-13T14:43:33.765 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-03-13T14:43:33.765 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0609149 s, 172 MB/s 2024-03-13T14:43:33.765 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-03-13T14:43:34.869 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-03-13T14:43:35.093 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:35 smithi196 ceph-mon[37486]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:35.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:35 smithi176 ceph-mon[29064]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:36.019 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi196/config 2024-03-13T14:43:37.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:37.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: Detected new or changed devices on smithi196 2024-03-13T14:43:37.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:43:37.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi196 to 7457M 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:37 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:37.382 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi196 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi196 to 7457M 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:37.383 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:37 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.302+0000 7fb9c9200700 1 -- 172.21.15.196:0/3869163387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 msgr2=0x7fb9c4075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.302+0000 7fb9c9200700 1 --2- 172.21.15.196:0/3869163387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c4075b60 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009910 tx=0x7fb9b4009c20 comp rx=0 tx=0).stop 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.302+0000 7fb9c9200700 1 -- 172.21.15.196:0/3869163387 shutdown_connections 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.302+0000 7fb9c9200700 1 --2- 172.21.15.196:0/3869163387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9c4076990 0x7fb9c4076e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.303+0000 7fb9c9200700 1 --2- 172.21.15.196:0/3869163387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c4075b60 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.303+0000 7fb9c9200700 1 -- 172.21.15.196:0/3869163387 >> 172.21.15.196:0/3869163387 conn(0x7fb9c40fe3a0 msgr2=0x7fb9c41007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.303+0000 7fb9c9200700 1 -- 172.21.15.196:0/3869163387 shutdown_connections 2024-03-13T14:43:38.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.303+0000 7fb9c9200700 1 -- 172.21.15.196:0/3869163387 wait complete. 2024-03-13T14:43:38.293 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:38 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:38.293 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:38 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:38.294 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:38 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:38.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.304+0000 7fb9c9200700 1 Processor -- start 2024-03-13T14:43:38.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.304+0000 7fb9c9200700 1 -- start start 2024-03-13T14:43:38.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c9200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:43:38.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c9200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9c4076990 0x7fb9c419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:43:38.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c9200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c419d4f0 con 0x7fb9c4076990 2024-03-13T14:43:38.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c9200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c419d630 con 0x7fb9c4075740 2024-03-13T14:43:38.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:43:38.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:38314/0 (socket says 172.21.15.196:38314) 2024-03-13T14:43:38.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1e00700 1 -- 172.21.15.196:0/1271519003 learned_addr learned my addr 172.21.15.196:0/1271519003 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:43:38.297 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1400700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9c4076990 0x7fb9c419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:43:38.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1e00700 1 -- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9c4076990 msgr2=0x7fb9c419ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:43:38.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1e00700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9c4076990 0x7fb9c419ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:43:38.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1e00700 1 -- 172.21.15.196:0/1271519003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b40095f0 con 0x7fb9c4075740 2024-03-13T14:43:38.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.305+0000 7fb9c1400700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9c4076990 0x7fb9c419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:43:38.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.306+0000 7fb9c1e00700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c419c990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b5c0 tx=0x7fb9b400be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:43:38.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.306+0000 7fb9baa00700 1 -- 172.21.15.196:0/1271519003 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b401d070 con 0x7fb9c4075740 2024-03-13T14:43:38.298 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.306+0000 7fb9baa00700 1 -- 172.21.15.196:0/1271519003 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb9b4022470 con 0x7fb9c4075740 2024-03-13T14:43:38.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.306+0000 7fb9baa00700 1 -- 172.21.15.196:0/1271519003 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9b400f690 con 0x7fb9c4075740 2024-03-13T14:43:38.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.306+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c41a2080 con 0x7fb9c4075740 2024-03-13T14:43:38.299 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.306+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c41a25f0 con 0x7fb9c4075740 2024-03-13T14:43:38.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.307+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c410fdb0 con 0x7fb9c4075740 2024-03-13T14:43:38.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.309+0000 7fb9baa00700 1 -- 172.21.15.196:0/1271519003 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb9b4004b70 con 0x7fb9c4075740 2024-03-13T14:43:38.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.310+0000 7fb9baa00700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb9a806c4e0 0x7fb9a806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:43:38.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.310+0000 7fb9baa00700 1 -- 172.21.15.196:0/1271519003 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fb9b408c9d0 con 0x7fb9c4075740 2024-03-13T14:43:38.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.310+0000 7fb9c1400700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb9a806c4e0 0x7fb9a806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:43:38.301 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.311+0000 7fb9c1400700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb9a806c4e0 0x7fb9a806e9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007900 tx=0x7fb9b0008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:43:38.304 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.315+0000 7fb9baa00700 1 -- 172.21.15.196:0/1271519003 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb9b405b1c0 con 0x7fb9c4075740 2024-03-13T14:43:38.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:38 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:38.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:38 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:43:38.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:38 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:43:38.526 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:43:38.536+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9c410e760 con 0x7fb9a806c4e0 2024-03-13T14:43:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:39 smithi196 ceph-mon[37486]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:39 smithi196 ceph-mon[37486]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:43:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:43:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:43:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:39 smithi176 ceph-mon[29064]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:39 smithi176 ceph-mon[29064]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:43:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-13T14:43:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-13T14:43:39.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:40.993 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:40 smithi196 ceph-mon[37486]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:41.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:40 smithi176 ceph-mon[29064]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:43 smithi176 ceph-mon[29064]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:43 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/3600582276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb96c7b-7a78-4c40-996f-f8141029c310"}]: dispatch 2024-03-13T14:43:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:43 smithi176 ceph-mon[29064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb96c7b-7a78-4c40-996f-f8141029c310"}]: dispatch 2024-03-13T14:43:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:43 smithi176 ceph-mon[29064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb96c7b-7a78-4c40-996f-f8141029c310"}]': finished 2024-03-13T14:43:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:43 smithi176 ceph-mon[29064]: osdmap e37: 6 total, 5 up, 6 in 2024-03-13T14:43:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:43:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:43 smithi196 ceph-mon[37486]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:43 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/3600582276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb96c7b-7a78-4c40-996f-f8141029c310"}]: dispatch 2024-03-13T14:43:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:43 smithi196 ceph-mon[37486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efb96c7b-7a78-4c40-996f-f8141029c310"}]: dispatch 2024-03-13T14:43:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:43 smithi196 ceph-mon[37486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efb96c7b-7a78-4c40-996f-f8141029c310"}]': finished 2024-03-13T14:43:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:43 smithi196 ceph-mon[37486]: osdmap e37: 6 total, 5 up, 6 in 2024-03-13T14:43:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:43:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:44.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:44 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/3985113508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:43:44.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:44 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/3985113508' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-13T14:43:45.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:45 smithi176 ceph-mon[29064]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:45.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:45 smithi196 ceph-mon[37486]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:47.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:47 smithi176 ceph-mon[29064]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:47.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:47 smithi196 ceph-mon[37486]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:49.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:49 smithi196 ceph-mon[37486]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:49.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:49 smithi176 ceph-mon[29064]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:51.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:50 smithi196 ceph-mon[37486]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:51.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:50 smithi176 ceph-mon[29064]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:52.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:51 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-13T14:43:52.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:51 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:52.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:51 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-13T14:43:52.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:51 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:43:53.101 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:52 smithi196 ceph-mon[37486]: Deploying daemon osd.5 on smithi196 2024-03-13T14:43:53.102 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:52 smithi196 ceph-mon[37486]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:53.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:52 smithi176 ceph-mon[29064]: Deploying daemon osd.5 on smithi196 2024-03-13T14:43:53.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:52 smithi176 ceph-mon[29064]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:55.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:55 smithi176 ceph-mon[29064]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:55.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:55 smithi196 ceph-mon[37486]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:57.368 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:57 smithi196 ceph-mon[37486]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:57.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:57 smithi176 ceph-mon[29064]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:58.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:58.400 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:43:59.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:43:59 smithi176 ceph-mon[29064]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:43:59.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:43:59 smithi196 ceph-mon[37486]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:01.329 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:01 smithi196 ceph-mon[37486]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:01.329 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:01 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:01.329 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:01 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:01.329 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:01 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:44:01.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:01 smithi176 ceph-mon[29064]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:01.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:01.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:01.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:44:03.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:03 smithi176 ceph-mon[29064]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:03 smithi196 ceph-mon[37486]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:04.265 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:04 smithi196 ceph-mon[37486]: from='osd.5 [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:44:04.265 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:04 smithi196 ceph-mon[37486]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:44:04.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:04 smithi176 ceph-mon[29064]: from='osd.5 [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:44:04.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:04 smithi176 ceph-mon[29064]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:44:05.144 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:05.144 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-13T14:44:05.144 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: osdmap e38: 6 total, 5 up, 6 in 2024-03-13T14:44:05.144 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:05.144 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='osd.5 [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:44:05.145 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:44:05.145 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:05.145 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:05.145 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:44:05.145 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:44:05.146 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:05 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-13T14:44:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: osdmap e38: 6 total, 5 up, 6 in 2024-03-13T14:44:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='osd.5 [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:44:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:44:05.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:05.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:05.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:44:05.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:44:05.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:05 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:05.790 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-03-13T14:44:05.790 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.799+0000 7fb9baa00700 1 -- 172.21.15.196:0/1271519003 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb9c410e760 con 0x7fb9a806c4e0 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb9a806c4e0 msgr2=0x7fb9a806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb9a806c4e0 0x7fb9a806e9a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007900 tx=0x7fb9b0008040 comp rx=0 tx=0).stop 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 msgr2=0x7fb9c419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c419c990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400b5c0 tx=0x7fb9b400be20 comp rx=0 tx=0).stop 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 shutdown_connections 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb9a806c4e0 0x7fb9a806e9a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9c4075740 0x7fb9c419c990 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 --2- 172.21.15.196:0/1271519003 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9c4076990 0x7fb9c419ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 >> 172.21.15.196:0/1271519003 conn(0x7fb9c40fe3a0 msgr2=0x7fb9c410d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:05.794 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.803+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 shutdown_connections 2024-03-13T14:44:05.795 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:05.804+0000 7fb9c9200700 1 -- 172.21.15.196:0/1271519003 wait complete. 2024-03-13T14:44:06.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:06 smithi176 ceph-mon[29064]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-13T14:44:06.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:06 smithi176 ceph-mon[29064]: osdmap e39: 6 total, 5 up, 6 in 2024-03-13T14:44:06.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:06.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:06.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:06.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:44:06.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:06 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:06.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:06 smithi196 ceph-mon[37486]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-03-13T14:44:06.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:06 smithi196 ceph-mon[37486]: osdmap e39: 6 total, 5 up, 6 in 2024-03-13T14:44:06.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:06.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:06.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:06.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:44:06.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:06 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:06.648 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.5.service 2024-03-13T14:44:06.653 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-03-13T14:44:06.653 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd stat -f json 2024-03-13T14:44:07.062 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:07.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:07 smithi196 ceph-mon[37486]: purged_snaps scrub starts 2024-03-13T14:44:07.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:07 smithi196 ceph-mon[37486]: purged_snaps scrub ok 2024-03-13T14:44:07.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:07 smithi196 ceph-mon[37486]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:07.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:07 smithi196 ceph-mon[37486]: osd.5 [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] boot 2024-03-13T14:44:07.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:07 smithi196 ceph-mon[37486]: osdmap e40: 6 total, 6 up, 6 in 2024-03-13T14:44:07.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:07 smithi176 ceph-mon[29064]: purged_snaps scrub starts 2024-03-13T14:44:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:07 smithi176 ceph-mon[29064]: purged_snaps scrub ok 2024-03-13T14:44:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:07 smithi176 ceph-mon[29064]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-03-13T14:44:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:07 smithi176 ceph-mon[29064]: osd.5 [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] boot 2024-03-13T14:44:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:07 smithi176 ceph-mon[29064]: osdmap e40: 6 total, 6 up, 6 in 2024-03-13T14:44:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:44:08.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:08 smithi196 ceph-mon[37486]: osdmap e41: 6 total, 6 up, 6 in 2024-03-13T14:44:08.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:08 smithi196 ceph-mon[37486]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:08.966 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:08 smithi176 ceph-mon[29064]: osdmap e41: 6 total, 6 up, 6 in 2024-03-13T14:44:08.966 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:08 smithi176 ceph-mon[29064]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 -- 172.21.15.176:0/2642334551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 msgr2=0x7f69e81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 --2- 172.21.15.176:0/2642334551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e81043f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009910 tx=0x7f69d4009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 -- 172.21.15.176:0/2642334551 shutdown_connections 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 --2- 172.21.15.176:0/2642334551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e81043f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 --2- 172.21.15.176:0/2642334551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e8102e10 0x7f69e8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 -- 172.21.15.176:0/2642334551 >> 172.21.15.176:0/2642334551 conn(0x7f69e80fe3d0 msgr2=0x7f69e81007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 -- 172.21.15.176:0/2642334551 shutdown_connections 2024-03-13T14:44:09.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.248+0000 7f69ef200700 1 -- 172.21.15.176:0/2642334551 wait complete. 2024-03-13T14:44:09.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.249+0000 7f69ef200700 1 Processor -- start 2024-03-13T14:44:09.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.249+0000 7f69ef200700 1 -- start start 2024-03-13T14:44:09.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.249+0000 7f69ef200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e8102e10 0x7f69e81985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:09.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.249+0000 7f69ef200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e8198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:09.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69ef200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e8199150 con 0x7f69e8103f70 2024-03-13T14:44:09.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69ef200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e8199290 con 0x7f69e8102e10 2024-03-13T14:44:09.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e8198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e8198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58712/0 (socket says 172.21.15.176:58712) 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7400700 1 -- 172.21.15.176:0/3577145766 learned_addr learned my addr 172.21.15.176:0/3577145766 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7e00700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e8102e10 0x7f69e81985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7400700 1 -- 172.21.15.176:0/3577145766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e8102e10 msgr2=0x7f69e81985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7400700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e8102e10 0x7f69e81985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7400700 1 -- 172.21.15.176:0/3577145766 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d40095f0 con 0x7f69e8103f70 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.250+0000 7f69e7e00700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e8102e10 0x7f69e81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.251+0000 7f69e7400700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e8198b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69d400bd70 tx=0x7f69d400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.251+0000 7f69e4c00700 1 -- 172.21.15.176:0/3577145766 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d401d070 con 0x7f69e8103f70 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.251+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e819dce0 con 0x7f69e8103f70 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.251+0000 7f69e4c00700 1 -- 172.21.15.176:0/3577145766 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f69d4022470 con 0x7f69e8103f70 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.251+0000 7f69e4c00700 1 -- 172.21.15.176:0/3577145766 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d400f670 con 0x7f69e8103f70 2024-03-13T14:44:09.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.251+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e819e250 con 0x7f69e8103f70 2024-03-13T14:44:09.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.252+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e810b970 con 0x7f69e8103f70 2024-03-13T14:44:09.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.253+0000 7f69e4c00700 1 -- 172.21.15.176:0/3577145766 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f69d4022a50 con 0x7f69e8103f70 2024-03-13T14:44:09.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.254+0000 7f69e4c00700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f69cc06c4e0 0x7f69cc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:09.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.255+0000 7f69e4c00700 1 -- 172.21.15.176:0/3577145766 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f69d408cc20 con 0x7f69e8103f70 2024-03-13T14:44:09.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.255+0000 7f69e7e00700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f69cc06c4e0 0x7f69cc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:09.245 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.255+0000 7f69e7e00700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f69cc06c4e0 0x7f69cc06e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69d8007900 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:09.250 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.260+0000 7f69e4c00700 1 -- 172.21.15.176:0/3577145766 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f69d405b1e0 con 0x7f69e8103f70 2024-03-13T14:44:09.465 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.475+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f69e810bb90 con 0x7f69e8103f70 2024-03-13T14:44:09.466 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.476+0000 7f69e4c00700 1 -- 172.21.15.176:0/3577145766 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7f69d40270a0 con 0x7f69e8103f70 2024-03-13T14:44:09.466 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f69cc06c4e0 msgr2=0x7f69cc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f69cc06c4e0 0x7f69cc06e9a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69d8007900 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 msgr2=0x7f69e8198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e8198b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f69d400bd70 tx=0x7f69d400be50 comp rx=0 tx=0).stop 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 shutdown_connections 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f69cc06c4e0 0x7f69cc06e9a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e8102e10 0x7f69e81985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 --2- 172.21.15.176:0/3577145766 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8103f70 0x7f69e8198b30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 >> 172.21.15.176:0/3577145766 conn(0x7f69e80fe3d0 msgr2=0x7f69e8107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:09.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 shutdown_connections 2024-03-13T14:44:09.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:09.479+0000 7f69ef200700 1 -- 172.21.15.176:0/3577145766 wait complete. 2024-03-13T14:44:10.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:09 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3577145766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-13T14:44:10.146 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:09 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3577145766' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-13T14:44:10.175 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1710341046,"num_in_osds":6,"osd_in_since":1710341022,"num_remapped_pgs":0} 2024-03-13T14:44:10.176 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-13T14:44:10.584 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:11.065 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:10 smithi176 ceph-mon[29064]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:11.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:10 smithi196 ceph-mon[37486]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:12.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi196 2024-03-13T14:44:12.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi196 to 4971M 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:44:12.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: Detected new or changed devices on smithi196 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi196 to 4971M 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:44:12.437 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:44:12.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.600+0000 7fa912600700 1 -- 172.21.15.176:0/696689455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c1071d0 msgr2=0x7fa90c107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:12.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.600+0000 7fa912600700 1 --2- 172.21.15.176:0/696689455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c1071d0 0x7fa90c107650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4009910 tx=0x7fa8f4009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:12.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.601+0000 7fa912600700 1 -- 172.21.15.176:0/696689455 shutdown_connections 2024-03-13T14:44:12.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.601+0000 7fa912600700 1 --2- 172.21.15.176:0/696689455 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c1071d0 0x7fa90c107650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:12.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.601+0000 7fa912600700 1 --2- 172.21.15.176:0/696689455 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90c10d280 0x7fa90c10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:12.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.601+0000 7fa912600700 1 -- 172.21.15.176:0/696689455 >> 172.21.15.176:0/696689455 conn(0x7fa90c0769e0 msgr2=0x7fa90c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:12.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.601+0000 7fa912600700 1 -- 172.21.15.176:0/696689455 shutdown_connections 2024-03-13T14:44:12.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.601+0000 7fa912600700 1 -- 172.21.15.176:0/696689455 wait complete. 2024-03-13T14:44:12.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa912600700 1 Processor -- start 2024-03-13T14:44:12.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa912600700 1 -- start start 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa912600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90c1071d0 0x7fa90c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa912600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c10d280 0x7fa90c19d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa912600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c19d980 con 0x7fa90c10d280 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa912600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c1a1710 con 0x7fa90c1071d0 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa903e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c10d280 0x7fa90c19d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa903e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c10d280 0x7fa90c19d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42962/0 (socket says 172.21.15.176:42962) 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.602+0000 7fa903e00700 1 -- 172.21.15.176:0/627683525 learned_addr learned my addr 172.21.15.176:0/627683525 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.603+0000 7fa90b400700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90c1071d0 0x7fa90c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.603+0000 7fa903e00700 1 -- 172.21.15.176:0/627683525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90c1071d0 msgr2=0x7fa90c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.603+0000 7fa903e00700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90c1071d0 0x7fa90c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.603+0000 7fa903e00700 1 -- 172.21.15.176:0/627683525 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f40095f0 con 0x7fa90c10d280 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.603+0000 7fa903e00700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c10d280 0x7fa90c19d2a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4006010 tx=0x7fa8f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.603+0000 7fa908c00700 1 -- 172.21.15.176:0/627683525 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8f401d070 con 0x7fa90c10d280 2024-03-13T14:44:12.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.603+0000 7fa908c00700 1 -- 172.21.15.176:0/627683525 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa8f400f460 con 0x7fa90c10d280 2024-03-13T14:44:12.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.604+0000 7fa908c00700 1 -- 172.21.15.176:0/627683525 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8f4017630 con 0x7fa90c10d280 2024-03-13T14:44:12.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.604+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90c1a1990 con 0x7fa90c10d280 2024-03-13T14:44:12.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.604+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90c1a1f00 con 0x7fa90c10d280 2024-03-13T14:44:12.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.605+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90c04ea90 con 0x7fa90c10d280 2024-03-13T14:44:12.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.607+0000 7fa908c00700 1 -- 172.21.15.176:0/627683525 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa8f400f5d0 con 0x7fa90c10d280 2024-03-13T14:44:12.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.608+0000 7fa908c00700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa8f006c4e0 0x7fa8f006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:12.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.608+0000 7fa908c00700 1 -- 172.21.15.176:0/627683525 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa8f408cad0 con 0x7fa90c10d280 2024-03-13T14:44:12.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.608+0000 7fa90b400700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa8f006c4e0 0x7fa8f006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:12.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.609+0000 7fa90b400700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa8f006c4e0 0x7fa8f006e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc006fd0 tx=0x7fa8fc008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:12.603 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.613+0000 7fa908c00700 1 -- 172.21.15.176:0/627683525 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa8f4057610 con 0x7fa90c10d280 2024-03-13T14:44:12.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.826+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa90c19e0c0 con 0x7fa90c10d280 2024-03-13T14:44:12.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.829+0000 7fa908c00700 1 -- 172.21.15.176:0/627683525 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11384 (secure 0 0 0) 0x7fa8f405ac30 con 0x7fa90c10d280 2024-03-13T14:44:12.819 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:44:12.819 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":41,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","created":"2024-03-13T14:36:32.120003+0000","modified":"2024-03-13T14:44:07.773938+0000","last_up_change":"2024-03-13T14:44:06.116104+0000","last_in_change":"2024-03-13T14:43:42.351790+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T14:42:22.643268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c570fcfa-ceda-4e39-b200-5c8b0b7fab19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6803","nonce":2000924858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6805","nonce":2000924858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6809","nonce":2000924858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6807","nonce":2000924858}]},"public_addr":"172.21.15.176:6803/2000924858","cluster_addr":"172.21.15.176:6805/2000924858","heartbeat_back_addr":"172.21.15.176:6809/2000924858","heartbeat_front_addr":"172.21.15.176:6807/2000924858","state":["exists","up"]},{"osd":1,"uuid":"5a973902-7ecc-4a05-b917-a74e108df4db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6811","nonce":2968433057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6813","nonce":2968433057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6817","nonce":2968433057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6815","nonce":2968433057}]},"public_addr":"172.21.15.176:6811/2968433057","cluster_addr":"172.21.15.176:6813/2968433057","heartbeat_back_addr":"172.21.15.176:6817/2968433057","heartbeat_front_addr":"172.21.15.176:6815/2968433057","state":["exists","up"]},{"osd":2,"uuid":"f72757d1-b949-45fb-abcd-c8649cec14f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6819","nonce":2465800922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6821","nonce":2465800922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6825","nonce":2465800922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6823","nonce":2465800922}]},"public_addr":"172.21.15.176:6819/2465800922","cluster_addr":"172.21.15.176:6821/2465800922","heartbeat_back_addr":"172.21.15.176:6825/2465800922","heartbeat_front_addr":"172.21.15.176:6823/2465800922","state":["exists","up"]},{"osd":3,"uuid":"e1c42f2a-0efa-4c44-a965-9e5904aeace0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6801","nonce":3087694250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6803","nonce":3087694250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6807","nonce":3087694250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6805","nonce":3087694250}]},"public_addr":"172.21.15.196:6801/3087694250","cluster_addr":"172.21.15.196:6803/3087694250","heartbeat_back_addr":"172.21.15.196:6807/3087694250","heartbeat_front_addr":"172.21.15.196:6805/3087694250","state":["exists","up"]},{"osd":4,"uuid":"b4c7c89d-a349-4b39-b64a-e1080d213acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6809","nonce":111802733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6811","nonce":111802733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6815","nonce":111802733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6813","nonce":111802733}]},"public_addr":"172.21.15.196:6809/111802733","cluster_addr":"172.21.15.196:6811/111802733","heartbeat_back_addr":"172.21.15.196:6815/111802733","heartbeat_front_addr":"172.21.15.196:6813/111802733","state":["exists","up"]},{"osd":5,"uuid":"efb96c7b-7a78-4c40-996f-f8141029c310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6817","nonce":3680704272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6819","nonce":3680704272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6823","nonce":3680704272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6821","nonce":3680704272}]},"public_addr":"172.21.15.196:6817/3680704272","cluster_addr":"172.21.15.196:6819/3680704272","heartbeat_back_addr":"172.21.15.196:6823/3680704272","heartbeat_front_addr":"172.21.15.196:6821/3680704272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:06.529418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:43.356501+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:19.060195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:53.551733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:43:28.434416+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:44:05.009478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.176:6801/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/698195875":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1825335586":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6801/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1319558216":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6800/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2108157434":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6801/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1930887076":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/248607826":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1567438135":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1477585567":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2332303582":"2024-03-14T14:37:05.490145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T14:44:12.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa8f006c4e0 msgr2=0x7fa8f006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:12.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa8f006c4e0 0x7fa8f006e9a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc006fd0 tx=0x7fa8fc008040 comp rx=0 tx=0).stop 2024-03-13T14:44:12.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c10d280 msgr2=0x7fa90c19d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:12.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c10d280 0x7fa90c19d2a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa8f4006010 tx=0x7fa8f400be20 comp rx=0 tx=0).stop 2024-03-13T14:44:12.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 shutdown_connections 2024-03-13T14:44:12.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa8f006c4e0 0x7fa8f006e9a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:12.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa90c1071d0 0x7fa90c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:12.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 --2- 172.21.15.176:0/627683525 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa90c10d280 0x7fa90c19d2a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:12.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.832+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 >> 172.21.15.176:0/627683525 conn(0x7fa90c0769e0 msgr2=0x7fa90c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:12.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.833+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 shutdown_connections 2024-03-13T14:44:12.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:12.833+0000 7fa912600700 1 -- 172.21.15.176:0/627683525 wait complete. 2024-03-13T14:44:13.548 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:13 smithi176 ceph-mon[29064]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:13.548 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:13 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/627683525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:44:13.548 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:13.607 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-13T14:42:22.643268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-13T14:44:13.607 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-13T14:44:13.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:13 smithi196 ceph-mon[37486]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:13.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:13 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/627683525' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:44:13.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:14.009 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:15.589 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:15 smithi176 ceph-mon[29064]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:15.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:15 smithi196 ceph-mon[37486]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.018+0000 7fc8c2c00700 1 -- 172.21.15.176:0/742209861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc100e10 msgr2=0x7fc8bc1011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.018+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/742209861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc100e10 0x7fc8bc1011f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009910 tx=0x7fc8ac009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.018+0000 7fc8c2c00700 1 -- 172.21.15.176:0/742209861 shutdown_connections 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.018+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/742209861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc1017c0 0x7fc8bc105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.018+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/742209861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc100e10 0x7fc8bc1011f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.018+0000 7fc8c2c00700 1 -- 172.21.15.176:0/742209861 >> 172.21.15.176:0/742209861 conn(0x7fc8bc0fc6c0 msgr2=0x7fc8bc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.019+0000 7fc8c2c00700 1 -- 172.21.15.176:0/742209861 shutdown_connections 2024-03-13T14:44:16.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.019+0000 7fc8c2c00700 1 -- 172.21.15.176:0/742209861 wait complete. 2024-03-13T14:44:16.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.020+0000 7fc8c2c00700 1 Processor -- start 2024-03-13T14:44:16.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.020+0000 7fc8c2c00700 1 -- start start 2024-03-13T14:44:16.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8c2c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc100e10 0x7fc8bc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:16.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8c2c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc1017c0 0x7fc8bc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:16.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8c2c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc073b10 con 0x7fc8bc1017c0 2024-03-13T14:44:16.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8c2c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc073c50 con 0x7fc8bc100e10 2024-03-13T14:44:16.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8bb400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc1017c0 0x7fc8bc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:16.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8bb400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc1017c0 0x7fc8bc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42974/0 (socket says 172.21.15.176:42974) 2024-03-13T14:44:16.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8bb400700 1 -- 172.21.15.176:0/3534812169 learned_addr learned my addr 172.21.15.176:0/3534812169 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8bbe00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc100e10 0x7fc8bc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.021+0000 7fc8bb400700 1 -- 172.21.15.176:0/3534812169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc100e10 msgr2=0x7fc8bc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8bb400700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc100e10 0x7fc8bc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8bb400700 1 -- 172.21.15.176:0/3534812169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac0095f0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8bb400700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc1017c0 0x7fc8bc0735d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800d900 tx=0x7fc8a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8b8c00700 1 -- 172.21.15.176:0/3534812169 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a8009900 con 0x7fc8bc1017c0 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8b8c00700 1 -- 172.21.15.176:0/3534812169 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc8a8010460 con 0x7fc8bc1017c0 2024-03-13T14:44:16.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc1a29c0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8b8c00700 1 -- 172.21.15.176:0/3534812169 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8a800f5f0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.022+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc1a2dd0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.023+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8bc1091b0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.025+0000 7fc8b8c00700 1 -- 172.21.15.176:0/3534812169 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc8a80105d0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.025+0000 7fc8b8c00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc8a006c4e0 0x7fc8a006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:16.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.026+0000 7fc8b8c00700 1 -- 172.21.15.176:0/3534812169 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc8a808b2f0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.026+0000 7fc8bbe00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc8a006c4e0 0x7fc8a006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:16.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.027+0000 7fc8bbe00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc8a006c4e0 0x7fc8a006e9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00b5c0 tx=0x7fc8ac01a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:16.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.031+0000 7fc8b8c00700 1 -- 172.21.15.176:0/3534812169 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc8a8055df0 con 0x7fc8bc1017c0 2024-03-13T14:44:16.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.244+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8bc02cd60 con 0x7fc8bc1017c0 2024-03-13T14:44:16.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.245+0000 7fc8b8c00700 1 -- 172.21.15.176:0/3534812169 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8a8059410 con 0x7fc8bc1017c0 2024-03-13T14:44:16.235 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 1 2024-03-13T14:44:16.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc8a006c4e0 msgr2=0x7fc8a006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:16.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc8a006c4e0 0x7fc8a006e9a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac00b5c0 tx=0x7fc8ac01a040 comp rx=0 tx=0).stop 2024-03-13T14:44:16.238 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc1017c0 msgr2=0x7fc8bc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:16.238 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc1017c0 0x7fc8bc0735d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800d900 tx=0x7fc8a800dcc0 comp rx=0 tx=0).stop 2024-03-13T14:44:16.238 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 shutdown_connections 2024-03-13T14:44:16.238 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc8a006c4e0 0x7fc8a006e9a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:16.238 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8bc100e10 0x7fc8bc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:16.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 --2- 172.21.15.176:0/3534812169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc8bc1017c0 0x7fc8bc0735d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:16.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 >> 172.21.15.176:0/3534812169 conn(0x7fc8bc0fc6c0 msgr2=0x7fc8bc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:16.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 shutdown_connections 2024-03-13T14:44:16.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:16.248+0000 7fc8c2c00700 1 -- 172.21.15.176:0/3534812169 wait complete. 2024-03-13T14:44:16.916 INFO:tasks.cephadm:Setting up client nodes... 2024-03-13T14:44:16.916 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-13T14:44:17.312 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:17.459 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:17 smithi176 ceph-mon[29064]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:17.460 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:17 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3534812169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T14:44:17.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:17 smithi196 ceph-mon[37486]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:17.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:17 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3534812169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T14:44:19.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.368+0000 7f5ae7800700 1 -- 172.21.15.176:0/971204093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 msgr2=0x7f5ae0103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:19.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.368+0000 7f5ae7800700 1 --2- 172.21.15.176:0/971204093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0103230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009910 tx=0x7f5ad4009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:19.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.368+0000 7f5ae7800700 1 -- 172.21.15.176:0/971204093 shutdown_connections 2024-03-13T14:44:19.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.368+0000 7f5ae7800700 1 --2- 172.21.15.176:0/971204093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ae0103f70 0x7f5ae01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:19.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.368+0000 7f5ae7800700 1 --2- 172.21.15.176:0/971204093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0103230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:19.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.368+0000 7f5ae7800700 1 -- 172.21.15.176:0/971204093 >> 172.21.15.176:0/971204093 conn(0x7f5ae00fe3d0 msgr2=0x7f5ae01007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:19.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.369+0000 7f5ae7800700 1 -- 172.21.15.176:0/971204093 shutdown_connections 2024-03-13T14:44:19.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.369+0000 7f5ae7800700 1 -- 172.21.15.176:0/971204093 wait complete. 2024-03-13T14:44:19.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae7800700 1 Processor -- start 2024-03-13T14:44:19.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae7800700 1 -- start start 2024-03-13T14:44:19.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae7800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:19.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae7800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ae0103f70 0x7f5ae0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:19.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae7800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01990e0 con 0x7f5ae0102e10 2024-03-13T14:44:19.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae7800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0199220 con 0x7f5ae0103f70 2024-03-13T14:44:19.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae4e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:19.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae4e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42994/0 (socket says 172.21.15.176:42994) 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.370+0000 7f5ae4e00700 1 -- 172.21.15.176:0/3978846311 learned_addr learned my addr 172.21.15.176:0/3978846311 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.371+0000 7f5adfe00700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ae0103f70 0x7f5ae0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.371+0000 7f5ae4e00700 1 -- 172.21.15.176:0/3978846311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ae0103f70 msgr2=0x7f5ae0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.371+0000 7f5ae4e00700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ae0103f70 0x7f5ae0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.371+0000 7f5ae4e00700 1 -- 172.21.15.176:0/3978846311 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad40095f0 con 0x7f5ae0102e10 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.371+0000 7f5ae4e00700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0198580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400bd70 tx=0x7f5ad400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.372+0000 7f5add600700 1 -- 172.21.15.176:0/3978846311 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad401d070 con 0x7f5ae0102e10 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.372+0000 7f5add600700 1 -- 172.21.15.176:0/3978846311 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ad4022470 con 0x7f5ae0102e10 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.372+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae019dc70 con 0x7f5ae0102e10 2024-03-13T14:44:19.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.372+0000 7f5add600700 1 -- 172.21.15.176:0/3978846311 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad400f670 con 0x7f5ae0102e10 2024-03-13T14:44:19.367 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.372+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae019e160 con 0x7f5ae0102e10 2024-03-13T14:44:19.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.373+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae010b8f0 con 0x7f5ae0102e10 2024-03-13T14:44:19.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.374+0000 7f5add600700 1 -- 172.21.15.176:0/3978846311 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5ad4022a50 con 0x7f5ae0102e10 2024-03-13T14:44:19.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.375+0000 7f5add600700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5ac806c4e0 0x7f5ac806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:19.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.375+0000 7f5add600700 1 -- 172.21.15.176:0/3978846311 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5ad408ca50 con 0x7f5ae0102e10 2024-03-13T14:44:19.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.375+0000 7f5adfe00700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5ac806c4e0 0x7f5ac806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:19.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.376+0000 7f5adfe00700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5ac806c4e0 0x7f5ac806e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005950 tx=0x7f5acc0058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:19.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.379+0000 7f5add600700 1 -- 172.21.15.176:0/3978846311 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5ad405b1b0 con 0x7f5ae0102e10 2024-03-13T14:44:19.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:19 smithi176 ceph-mon[29064]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:19.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.680+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f5ae010bb10 con 0x7f5ae0102e10 2024-03-13T14:44:19.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.689+0000 7f5add600700 1 -- 172.21.15.176:0/3978846311 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f5ad4027050 con 0x7f5ae0102e10 2024-03-13T14:44:19.679 INFO:teuthology.orchestra.run.smithi176.stdout:[client.0] 2024-03-13T14:44:19.679 INFO:teuthology.orchestra.run.smithi176.stdout: key = AQDDu/FlBsexKBAAJLJF5Qt55RsakJRu2REi5Q== 2024-03-13T14:44:19.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:19 smithi196 ceph-mon[37486]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:19.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5ac806c4e0 msgr2=0x7f5ac806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:19.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5ac806c4e0 0x7f5ac806e9a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005950 tx=0x7f5acc0058e0 comp rx=0 tx=0).stop 2024-03-13T14:44:19.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 msgr2=0x7f5ae0198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:19.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0198580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400bd70 tx=0x7f5ad400be50 comp rx=0 tx=0).stop 2024-03-13T14:44:19.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 shutdown_connections 2024-03-13T14:44:19.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5ac806c4e0 0x7f5ac806e9a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:19.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ae0102e10 0x7f5ae0198580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:19.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 --2- 172.21.15.176:0/3978846311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ae0103f70 0x7f5ae0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:19.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 >> 172.21.15.176:0/3978846311 conn(0x7f5ae00fe3d0 msgr2=0x7f5ae0107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:19.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 shutdown_connections 2024-03-13T14:44:19.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:19.692+0000 7f5ae7800700 1 -- 172.21.15.176:0/3978846311 wait complete. 2024-03-13T14:44:20.324 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:20 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3978846311' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-13T14:44:20.324 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:20 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3978846311' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-13T14:44:20.373 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:44:20.373 DEBUG:teuthology.orchestra.run.smithi176:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-13T14:44:20.373 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-13T14:44:20.520 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-13T14:44:20.610 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:20 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3978846311' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-13T14:44:20.610 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:20 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3978846311' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-13T14:44:20.917 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi196/config 2024-03-13T14:44:21.333 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:21 smithi196 ceph-mon[37486]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:21.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:21 smithi176 ceph-mon[29064]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 -- 172.21.15.196:0/3629007793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794100bb0 msgr2=0x7fc794100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 --2- 172.21.15.196:0/3629007793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794100bb0 0x7fc794100fd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc788009910 tx=0x7fc788009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 -- 172.21.15.196:0/3629007793 shutdown_connections 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 --2- 172.21.15.196:0/3629007793 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc794101db0 0x7fc794102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 --2- 172.21.15.196:0/3629007793 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794100bb0 0x7fc794100fd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 -- 172.21.15.196:0/3629007793 >> 172.21.15.196:0/3629007793 conn(0x7fc7940fc110 msgr2=0x7fc7940fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 -- 172.21.15.196:0/3629007793 shutdown_connections 2024-03-13T14:44:23.289 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.299+0000 7fc79c800700 1 -- 172.21.15.196:0/3629007793 wait complete. 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.300+0000 7fc79c800700 1 Processor -- start 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.300+0000 7fc79c800700 1 -- start start 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.300+0000 7fc79c800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc794100bb0 0x7fc794198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.300+0000 7fc79c800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794101db0 0x7fc794198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.300+0000 7fc79c800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7941990d0 con 0x7fc794100bb0 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.300+0000 7fc79c800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794199210 con 0x7fc794101db0 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.301+0000 7fc799400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794101db0 0x7fc794198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.301+0000 7fc799400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794101db0 0x7fc794198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:45402/0 (socket says 172.21.15.196:45402) 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.301+0000 7fc799400700 1 -- 172.21.15.196:0/883121483 learned_addr learned my addr 172.21.15.196:0/883121483 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.301+0000 7fc799e00700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc794100bb0 0x7fc794198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.301+0000 7fc799400700 1 -- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc794100bb0 msgr2=0x7fc794198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:23.291 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.301+0000 7fc799400700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc794100bb0 0x7fc794198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:23.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.301+0000 7fc799400700 1 -- 172.21.15.196:0/883121483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880095f0 con 0x7fc794101db0 2024-03-13T14:44:23.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.302+0000 7fc799400700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794101db0 0x7fc794198ab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c8a0 tx=0x7fc78400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:23.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.302+0000 7fc78ea00700 1 -- 172.21.15.196:0/883121483 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc78400cea0 con 0x7fc794101db0 2024-03-13T14:44:23.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.302+0000 7fc78ea00700 1 -- 172.21.15.196:0/883121483 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc784004830 con 0x7fc794101db0 2024-03-13T14:44:23.292 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.302+0000 7fc78ea00700 1 -- 172.21.15.196:0/883121483 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7840055b0 con 0x7fc794101db0 2024-03-13T14:44:23.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.302+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79419dcc0 con 0x7fc794101db0 2024-03-13T14:44:23.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.302+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79419e210 con 0x7fc794101db0 2024-03-13T14:44:23.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.303+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc794105e90 con 0x7fc794101db0 2024-03-13T14:44:23.293 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.304+0000 7fc78ea00700 1 -- 172.21.15.196:0/883121483 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc784007ad0 con 0x7fc794101db0 2024-03-13T14:44:23.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.306+0000 7fc78ea00700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc77c06c4e0 0x7fc77c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:23.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.306+0000 7fc78ea00700 1 -- 172.21.15.196:0/883121483 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc78408b1c0 con 0x7fc794101db0 2024-03-13T14:44:23.295 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.306+0000 7fc799e00700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc77c06c4e0 0x7fc77c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:23.296 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.307+0000 7fc799e00700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc77c06c4e0 0x7fc77c06e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc788006010 tx=0x7fc78800b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:23.300 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.311+0000 7fc78ea00700 1 -- 172.21.15.196:0/883121483 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc784059750 con 0x7fc794101db0 2024-03-13T14:44:23.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:23 smithi196 ceph-mon[37486]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:23.586 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.596+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc79402cc70 con 0x7fc794101db0 2024-03-13T14:44:23.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.608+0000 7fc78ea00700 1 -- 172.21.15.196:0/883121483 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc7840592e0 con 0x7fc794101db0 2024-03-13T14:44:23.598 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-03-13T14:44:23.598 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDHu/FlggayIxAAmRWW2ZiESJ92czzjBosMbw== 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc77c06c4e0 msgr2=0x7fc77c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc77c06c4e0 0x7fc77c06e9a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc788006010 tx=0x7fc78800b540 comp rx=0 tx=0).stop 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794101db0 msgr2=0x7fc794198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794101db0 0x7fc794198ab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c8a0 tx=0x7fc78400cc60 comp rx=0 tx=0).stop 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 shutdown_connections 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc77c06c4e0 0x7fc77c06e9a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc794100bb0 0x7fc794198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 --2- 172.21.15.196:0/883121483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794101db0 0x7fc794198ab0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.610+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 >> 172.21.15.196:0/883121483 conn(0x7fc7940fc110 msgr2=0x7fc794102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.611+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 shutdown_connections 2024-03-13T14:44:23.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:44:23.611+0000 7fc79c800700 1 -- 172.21.15.196:0/883121483 wait complete. 2024-03-13T14:44:23.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:23 smithi176 ceph-mon[29064]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:24.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:24 smithi196 ceph-mon[37486]: from='client.? 172.21.15.196:0/883121483' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-13T14:44:24.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:24 smithi196 ceph-mon[37486]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-13T14:44:24.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:24 smithi196 ceph-mon[37486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-13T14:44:24.518 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:44:24.518 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-13T14:44:24.518 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-13T14:44:24.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:24 smithi176 ceph-mon[29064]: from='client.? 172.21.15.196:0/883121483' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-13T14:44:24.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:24 smithi176 ceph-mon[29064]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-13T14:44:24.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:24 smithi176 ceph-mon[29064]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-13T14:44:24.672 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-13T14:44:24.672 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-13T14:44:24.672 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph mgr dump --format=json 2024-03-13T14:44:25.089 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:25.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:25 smithi176 ceph-mon[29064]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:25.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:25 smithi196 ceph-mon[37486]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 -- 172.21.15.176:0/1701019951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7298103000 msgr2=0x7f72981033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 --2- 172.21.15.176:0/1701019951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7298103000 0x7f72981033e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f728c009910 tx=0x7f728c009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 -- 172.21.15.176:0/1701019951 shutdown_connections 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 --2- 172.21.15.176:0/1701019951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72981039b0 0x7f7298107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 --2- 172.21.15.176:0/1701019951 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7298103000 0x7f72981033e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 -- 172.21.15.176:0/1701019951 >> 172.21.15.176:0/1701019951 conn(0x7f72980fe8d0 msgr2=0x7f7298100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 -- 172.21.15.176:0/1701019951 shutdown_connections 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.014+0000 7f729fc00700 1 -- 172.21.15.176:0/1701019951 wait complete. 2024-03-13T14:44:27.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.015+0000 7f729fc00700 1 Processor -- start 2024-03-13T14:44:27.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.015+0000 7f729fc00700 1 -- start start 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f729fc00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7298103000 0x7f729819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f729fc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f72981039b0 0x7f729819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f729fc00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729819f430 con 0x7f72981039b0 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f729fc00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72981988e0 con 0x7f7298103000 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f7297e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f72981039b0 0x7f729819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f7297e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f72981039b0 0x7f729819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:52224/0 (socket says 172.21.15.176:52224) 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f7297e00700 1 -- 172.21.15.176:0/1817409792 learned_addr learned my addr 172.21.15.176:0/1817409792 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:27.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f729d200700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7298103000 0x7f729819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f7297e00700 1 -- 172.21.15.176:0/1817409792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7298103000 msgr2=0x7f729819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f7297e00700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7298103000 0x7f729819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f7297e00700 1 -- 172.21.15.176:0/1817409792 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0095f0 con 0x7f72981039b0 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.016+0000 7f729d200700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7298103000 0x7f729819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.017+0000 7f7297e00700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f72981039b0 0x7f729819eda0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f728400b480 tx=0x7f728400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.017+0000 7f7295600700 1 -- 172.21.15.176:0/1817409792 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72840118c0 con 0x7f72981039b0 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.017+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7298198bc0 con 0x7f72981039b0 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.017+0000 7f7295600700 1 -- 172.21.15.176:0/1817409792 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7284010380 con 0x7f72981039b0 2024-03-13T14:44:27.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.017+0000 7f7295600700 1 -- 172.21.15.176:0/1817409792 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728400f4f0 con 0x7f72981039b0 2024-03-13T14:44:27.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.017+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7298199110 con 0x7f72981039b0 2024-03-13T14:44:27.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.018+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729810b3a0 con 0x7f72981039b0 2024-03-13T14:44:27.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.019+0000 7f7295600700 1 -- 172.21.15.176:0/1817409792 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f72840104f0 con 0x7f72981039b0 2024-03-13T14:44:27.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.020+0000 7f7295600700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f728006c4e0 0x7f728006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:27.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.020+0000 7f7295600700 1 -- 172.21.15.176:0/1817409792 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f728408b130 con 0x7f72981039b0 2024-03-13T14:44:27.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.020+0000 7f729d200700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f728006c4e0 0x7f728006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:27.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.021+0000 7f729d200700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f728006c4e0 0x7f728006e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f728c006010 tx=0x7f728c00b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:27.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.025+0000 7f7295600700 1 -- 172.21.15.176:0/1817409792 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f72840596e0 con 0x7f72981039b0 2024-03-13T14:44:27.286 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.297+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7298199390 con 0x7f72981039b0 2024-03-13T14:44:27.292 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.302+0000 7f7295600700 1 -- 172.21.15.176:0/1817409792 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+173046 (secure 0 0 0) 0x7f7284059270 con 0x7f72981039b0 2024-03-13T14:44:27.295 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:44:27.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f728006c4e0 msgr2=0x7f728006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:27.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f728006c4e0 0x7f728006e9a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f728c006010 tx=0x7f728c00b540 comp rx=0 tx=0).stop 2024-03-13T14:44:27.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f72981039b0 msgr2=0x7f729819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:27.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f72981039b0 0x7f729819eda0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f728400b480 tx=0x7f728400b840 comp rx=0 tx=0).stop 2024-03-13T14:44:27.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 shutdown_connections 2024-03-13T14:44:27.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f728006c4e0 0x7f728006e9a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:27.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7298103000 0x7f729819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:27.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 --2- 172.21.15.176:0/1817409792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f72981039b0 0x7f729819eda0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:27.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.306+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 >> 172.21.15.176:0/1817409792 conn(0x7f72980fe8d0 msgr2=0x7f72980ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:27.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.307+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 shutdown_connections 2024-03-13T14:44:27.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:27.307+0000 7f729fc00700 1 -- 172.21.15.176:0/1817409792 wait complete. 2024-03-13T14:44:27.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:27 smithi176 ceph-mon[29064]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:27.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:27 smithi196 ceph-mon[37486]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:28.040 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":19,"active_gid":14223,"active_name":"smithi176.tzfdwd","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6800","nonce":2698132085},{"type":"v1","addr":"172.21.15.176:6801","nonce":2698132085}]},"active_addr":"172.21.15.176:6801/2698132085","active_change":"2024-03-13T14:39:27.975144+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14247,"name":"smithi196.vdktbk","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.176:8443/","prometheus":"http://172.21.15.176:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":1665184531}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":665298320}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":1541570168}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.176:0","nonce":258177446}]}]} 2024-03-13T14:44:28.044 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-13T14:44:28.044 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-13T14:44:28.045 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-13T14:44:28.442 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:28.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:28 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1817409792' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-13T14:44:28.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:28.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:28 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1817409792' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-13T14:44:28.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:29.550 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:29 smithi176 ceph-mon[29064]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:29.582 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:29 smithi196 ceph-mon[37486]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:30.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 -- 172.21.15.176:0/2901316065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59481039b0 msgr2=0x7f5948107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:30.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 --2- 172.21.15.176:0/2901316065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59481039b0 0x7f5948107a00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5934009910 tx=0x7f5934009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:30.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 -- 172.21.15.176:0/2901316065 shutdown_connections 2024-03-13T14:44:30.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 --2- 172.21.15.176:0/2901316065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f59481039b0 0x7f5948107a00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:30.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 --2- 172.21.15.176:0/2901316065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5948103000 0x7f59481033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:30.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 -- 172.21.15.176:0/2901316065 >> 172.21.15.176:0/2901316065 conn(0x7f59480fe8d0 msgr2=0x7f5948100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:30.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 -- 172.21.15.176:0/2901316065 shutdown_connections 2024-03-13T14:44:30.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.340+0000 7f594e000700 1 -- 172.21.15.176:0/2901316065 wait complete. 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.341+0000 7f594e000700 1 Processor -- start 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.341+0000 7f594e000700 1 -- start start 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f594e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5948103000 0x7f594819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f594e000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59481039b0 0x7f594819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f594e000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594819f300 con 0x7f5948103000 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f594e000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5948198840 con 0x7f59481039b0 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f5947400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5948103000 0x7f594819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f5947400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5948103000 0x7f594819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:52242/0 (socket says 172.21.15.176:52242) 2024-03-13T14:44:30.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f5947400700 1 -- 172.21.15.176:0/1658485350 learned_addr learned my addr 172.21.15.176:0/1658485350 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:30.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.342+0000 7f5946a00700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59481039b0 0x7f594819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:30.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f5947400700 1 -- 172.21.15.176:0/1658485350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59481039b0 msgr2=0x7f594819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:30.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f5947400700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59481039b0 0x7f594819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f5947400700 1 -- 172.21.15.176:0/1658485350 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340095f0 con 0x7f5948103000 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f5947400700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5948103000 0x7f594819e7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f593800c930 tx=0x7f593800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f593fe00700 1 -- 172.21.15.176:0/1658485350 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5938007ad0 con 0x7f5948103000 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f593fe00700 1 -- 172.21.15.176:0/1658485350 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5938007c30 con 0x7f5948103000 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f593fe00700 1 -- 172.21.15.176:0/1658485350 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5938018630 con 0x7f5948103000 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5948198b20 con 0x7f5948103000 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.343+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5948199070 con 0x7f5948103000 2024-03-13T14:44:30.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.345+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594810b400 con 0x7f5948103000 2024-03-13T14:44:30.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.346+0000 7f593fe00700 1 -- 172.21.15.176:0/1658485350 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f593800f450 con 0x7f5948103000 2024-03-13T14:44:30.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.346+0000 7f593fe00700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f592c06c4e0 0x7f592c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:30.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.347+0000 7f593fe00700 1 -- 172.21.15.176:0/1658485350 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f593808b680 con 0x7f5948103000 2024-03-13T14:44:30.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.347+0000 7f5946a00700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f592c06c4e0 0x7f592c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:30.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.347+0000 7f5946a00700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f592c06c4e0 0x7f592c06e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59480ff0a0 tx=0x7f593401a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:30.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.351+0000 7f593fe00700 1 -- 172.21.15.176:0/1658485350 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5938059c10 con 0x7f5948103000 2024-03-13T14:44:30.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.567+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59481992f0 con 0x7f5948103000 2024-03-13T14:44:30.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.569+0000 7f593fe00700 1 -- 172.21.15.176:0/1658485350 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11384 (secure 0 0 0) 0x7f59380597a0 con 0x7f5948103000 2024-03-13T14:44:30.559 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:44:30.559 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":41,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","created":"2024-03-13T14:36:32.120003+0000","modified":"2024-03-13T14:44:07.773938+0000","last_up_change":"2024-03-13T14:44:06.116104+0000","last_in_change":"2024-03-13T14:43:42.351790+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T14:42:22.643268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c570fcfa-ceda-4e39-b200-5c8b0b7fab19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6803","nonce":2000924858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6805","nonce":2000924858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6809","nonce":2000924858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6807","nonce":2000924858}]},"public_addr":"172.21.15.176:6803/2000924858","cluster_addr":"172.21.15.176:6805/2000924858","heartbeat_back_addr":"172.21.15.176:6809/2000924858","heartbeat_front_addr":"172.21.15.176:6807/2000924858","state":["exists","up"]},{"osd":1,"uuid":"5a973902-7ecc-4a05-b917-a74e108df4db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6811","nonce":2968433057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6813","nonce":2968433057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6817","nonce":2968433057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6815","nonce":2968433057}]},"public_addr":"172.21.15.176:6811/2968433057","cluster_addr":"172.21.15.176:6813/2968433057","heartbeat_back_addr":"172.21.15.176:6817/2968433057","heartbeat_front_addr":"172.21.15.176:6815/2968433057","state":["exists","up"]},{"osd":2,"uuid":"f72757d1-b949-45fb-abcd-c8649cec14f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6819","nonce":2465800922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6821","nonce":2465800922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6825","nonce":2465800922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6823","nonce":2465800922}]},"public_addr":"172.21.15.176:6819/2465800922","cluster_addr":"172.21.15.176:6821/2465800922","heartbeat_back_addr":"172.21.15.176:6825/2465800922","heartbeat_front_addr":"172.21.15.176:6823/2465800922","state":["exists","up"]},{"osd":3,"uuid":"e1c42f2a-0efa-4c44-a965-9e5904aeace0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6801","nonce":3087694250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6803","nonce":3087694250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6807","nonce":3087694250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6805","nonce":3087694250}]},"public_addr":"172.21.15.196:6801/3087694250","cluster_addr":"172.21.15.196:6803/3087694250","heartbeat_back_addr":"172.21.15.196:6807/3087694250","heartbeat_front_addr":"172.21.15.196:6805/3087694250","state":["exists","up"]},{"osd":4,"uuid":"b4c7c89d-a349-4b39-b64a-e1080d213acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6809","nonce":111802733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6811","nonce":111802733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6815","nonce":111802733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6813","nonce":111802733}]},"public_addr":"172.21.15.196:6809/111802733","cluster_addr":"172.21.15.196:6811/111802733","heartbeat_back_addr":"172.21.15.196:6815/111802733","heartbeat_front_addr":"172.21.15.196:6813/111802733","state":["exists","up"]},{"osd":5,"uuid":"efb96c7b-7a78-4c40-996f-f8141029c310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6817","nonce":3680704272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6819","nonce":3680704272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6823","nonce":3680704272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6821","nonce":3680704272}]},"public_addr":"172.21.15.196:6817/3680704272","cluster_addr":"172.21.15.196:6819/3680704272","heartbeat_back_addr":"172.21.15.196:6823/3680704272","heartbeat_front_addr":"172.21.15.196:6821/3680704272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:06.529418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:43.356501+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:19.060195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:53.551733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:43:28.434416+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:44:05.009478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.176:6801/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/698195875":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1825335586":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6801/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1319558216":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6800/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2108157434":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6801/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1930887076":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/248607826":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1567438135":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1477585567":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2332303582":"2024-03-14T14:37:05.490145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T14:44:30.563 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.573+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f592c06c4e0 msgr2=0x7f592c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:30.563 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.573+0000 7f594e000700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f592c06c4e0 0x7f592c06e9a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59480ff0a0 tx=0x7f593401a040 comp rx=0 tx=0).stop 2024-03-13T14:44:30.563 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.573+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5948103000 msgr2=0x7f594819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:30.563 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.573+0000 7f594e000700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5948103000 0x7f594819e7c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f593800c930 tx=0x7f593800ccf0 comp rx=0 tx=0).stop 2024-03-13T14:44:30.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.574+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 shutdown_connections 2024-03-13T14:44:30.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.574+0000 7f594e000700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f592c06c4e0 0x7f592c06e9a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:30.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.574+0000 7f594e000700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5948103000 0x7f594819e7c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:30.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.574+0000 7f594e000700 1 --2- 172.21.15.176:0/1658485350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59481039b0 0x7f594819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:30.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.574+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 >> 172.21.15.176:0/1658485350 conn(0x7f59480fe8d0 msgr2=0x7f5948100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:30.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.574+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 shutdown_connections 2024-03-13T14:44:30.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:30.574+0000 7f594e000700 1 -- 172.21.15.176:0/1658485350 wait complete. 2024-03-13T14:44:31.232 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:30 smithi176 ceph-mon[29064]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:31.232 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:30 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1658485350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:44:31.293 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-13T14:44:31.293 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd dump --format=json 2024-03-13T14:44:31.302 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:30 smithi196 ceph-mon[37486]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:31.302 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:30 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1658485350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:44:31.700 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:33.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:33 smithi176 ceph-mon[29064]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:33.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:33 smithi196 ceph-mon[37486]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:33.618 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 -- 172.21.15.176:0/3795038939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d8103000 msgr2=0x7fc7d81033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:33.618 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 --2- 172.21.15.176:0/3795038939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d8103000 0x7fc7d81033e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009910 tx=0x7fc7c8009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:33.618 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 -- 172.21.15.176:0/3795038939 shutdown_connections 2024-03-13T14:44:33.618 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 --2- 172.21.15.176:0/3795038939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7d81039b0 0x7fc7d8107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:33.619 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 --2- 172.21.15.176:0/3795038939 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d8103000 0x7fc7d81033e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:33.619 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 -- 172.21.15.176:0/3795038939 >> 172.21.15.176:0/3795038939 conn(0x7fc7d80fe8d0 msgr2=0x7fc7d8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:33.619 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 -- 172.21.15.176:0/3795038939 shutdown_connections 2024-03-13T14:44:33.619 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.628+0000 7fc7dd400700 1 -- 172.21.15.176:0/3795038939 wait complete. 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.629+0000 7fc7dd400700 1 Processor -- start 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7dd400700 1 -- start start 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7dd400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7d8103000 0x7fc7d819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7dd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d81039b0 0x7fc7d819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7dd400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d819f3c0 con 0x7fc7d81039b0 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7dd400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d8198870 con 0x7fc7d8103000 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7d5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d81039b0 0x7fc7d819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7d5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d81039b0 0x7fc7d819ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40504/0 (socket says 172.21.15.176:40504) 2024-03-13T14:44:33.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.630+0000 7fc7d5e00700 1 -- 172.21.15.176:0/479685061 learned_addr learned my addr 172.21.15.176:0/479685061 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:33.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7d6800700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7d8103000 0x7fc7d819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:33.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7d5e00700 1 -- 172.21.15.176:0/479685061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7d8103000 msgr2=0x7fc7d819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:33.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7d5e00700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7d8103000 0x7fc7d819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:33.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7d5e00700 1 -- 172.21.15.176:0/479685061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80095f0 con 0x7fc7d81039b0 2024-03-13T14:44:33.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7d5e00700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d81039b0 0x7fc7d819ed30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c8f0 tx=0x7fc7c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:33.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7cf400700 1 -- 172.21.15.176:0/479685061 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c4007a50 con 0x7fc7d81039b0 2024-03-13T14:44:33.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7cf400700 1 -- 172.21.15.176:0/479685061 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc7c4007bb0 con 0x7fc7d81039b0 2024-03-13T14:44:33.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7cf400700 1 -- 172.21.15.176:0/479685061 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c4018630 con 0x7fc7d81039b0 2024-03-13T14:44:33.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.631+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d8198b50 con 0x7fc7d81039b0 2024-03-13T14:44:33.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.632+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d81990a0 con 0x7fc7d81039b0 2024-03-13T14:44:33.622 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.633+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d810b3a0 con 0x7fc7d81039b0 2024-03-13T14:44:33.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.634+0000 7fc7cf400700 1 -- 172.21.15.176:0/479685061 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc7c4007d20 con 0x7fc7d81039b0 2024-03-13T14:44:33.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.634+0000 7fc7cf400700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc7bc06c4e0 0x7fc7bc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:33.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.635+0000 7fc7cf400700 1 -- 172.21.15.176:0/479685061 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc7c408b4c0 con 0x7fc7d81039b0 2024-03-13T14:44:33.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.635+0000 7fc7d6800700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc7bc06c4e0 0x7fc7bc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:33.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.635+0000 7fc7d6800700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc7bc06c4e0 0x7fc7bc06e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800b5c0 tx=0x7fc7c801a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:33.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.639+0000 7fc7cf400700 1 -- 172.21.15.176:0/479685061 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc7c4059a50 con 0x7fc7d81039b0 2024-03-13T14:44:33.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.852+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7d804ea90 con 0x7fc7d81039b0 2024-03-13T14:44:33.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.854+0000 7fc7cf400700 1 -- 172.21.15.176:0/479685061 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11384 (secure 0 0 0) 0x7fc7c40595e0 con 0x7fc7d81039b0 2024-03-13T14:44:33.845 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:44:33.845 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":41,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","created":"2024-03-13T14:36:32.120003+0000","modified":"2024-03-13T14:44:07.773938+0000","last_up_change":"2024-03-13T14:44:06.116104+0000","last_in_change":"2024-03-13T14:43:42.351790+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T14:42:22.643268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c570fcfa-ceda-4e39-b200-5c8b0b7fab19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6803","nonce":2000924858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6805","nonce":2000924858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6809","nonce":2000924858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6807","nonce":2000924858}]},"public_addr":"172.21.15.176:6803/2000924858","cluster_addr":"172.21.15.176:6805/2000924858","heartbeat_back_addr":"172.21.15.176:6809/2000924858","heartbeat_front_addr":"172.21.15.176:6807/2000924858","state":["exists","up"]},{"osd":1,"uuid":"5a973902-7ecc-4a05-b917-a74e108df4db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6811","nonce":2968433057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6813","nonce":2968433057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6817","nonce":2968433057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6815","nonce":2968433057}]},"public_addr":"172.21.15.176:6811/2968433057","cluster_addr":"172.21.15.176:6813/2968433057","heartbeat_back_addr":"172.21.15.176:6817/2968433057","heartbeat_front_addr":"172.21.15.176:6815/2968433057","state":["exists","up"]},{"osd":2,"uuid":"f72757d1-b949-45fb-abcd-c8649cec14f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6819","nonce":2465800922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6821","nonce":2465800922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6825","nonce":2465800922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6823","nonce":2465800922}]},"public_addr":"172.21.15.176:6819/2465800922","cluster_addr":"172.21.15.176:6821/2465800922","heartbeat_back_addr":"172.21.15.176:6825/2465800922","heartbeat_front_addr":"172.21.15.176:6823/2465800922","state":["exists","up"]},{"osd":3,"uuid":"e1c42f2a-0efa-4c44-a965-9e5904aeace0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6801","nonce":3087694250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6803","nonce":3087694250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6807","nonce":3087694250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6805","nonce":3087694250}]},"public_addr":"172.21.15.196:6801/3087694250","cluster_addr":"172.21.15.196:6803/3087694250","heartbeat_back_addr":"172.21.15.196:6807/3087694250","heartbeat_front_addr":"172.21.15.196:6805/3087694250","state":["exists","up"]},{"osd":4,"uuid":"b4c7c89d-a349-4b39-b64a-e1080d213acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6809","nonce":111802733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6811","nonce":111802733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6815","nonce":111802733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6813","nonce":111802733}]},"public_addr":"172.21.15.196:6809/111802733","cluster_addr":"172.21.15.196:6811/111802733","heartbeat_back_addr":"172.21.15.196:6815/111802733","heartbeat_front_addr":"172.21.15.196:6813/111802733","state":["exists","up"]},{"osd":5,"uuid":"efb96c7b-7a78-4c40-996f-f8141029c310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6817","nonce":3680704272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6819","nonce":3680704272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6823","nonce":3680704272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6821","nonce":3680704272}]},"public_addr":"172.21.15.196:6817/3680704272","cluster_addr":"172.21.15.196:6819/3680704272","heartbeat_back_addr":"172.21.15.196:6823/3680704272","heartbeat_front_addr":"172.21.15.196:6821/3680704272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:06.529418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:43.356501+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:19.060195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:53.551733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:43:28.434416+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:44:05.009478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.176:6801/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/698195875":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1825335586":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6801/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1319558216":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6800/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2108157434":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6801/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1930887076":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/248607826":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1567438135":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1477585567":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2332303582":"2024-03-14T14:37:05.490145+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T14:44:33.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc7bc06c4e0 msgr2=0x7fc7bc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:33.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc7bc06c4e0 0x7fc7bc06e9a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc7c800b5c0 tx=0x7fc7c801a040 comp rx=0 tx=0).stop 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d81039b0 msgr2=0x7fc7d819ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d81039b0 0x7fc7d819ed30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c8f0 tx=0x7fc7c400ccb0 comp rx=0 tx=0).stop 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 shutdown_connections 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc7bc06c4e0 0x7fc7bc06e9a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7d8103000 0x7fc7d819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 --2- 172.21.15.176:0/479685061 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7d81039b0 0x7fc7d819ed30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.858+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 >> 172.21.15.176:0/479685061 conn(0x7fc7d80fe8d0 msgr2=0x7fc7d80ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.859+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 shutdown_connections 2024-03-13T14:44:33.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:33.859+0000 7fc7dd400700 1 -- 172.21.15.176:0/479685061 wait complete. 2024-03-13T14:44:34.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:34 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/479685061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:44:34.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:34 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/479685061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:44:34.566 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-13T14:44:34.566 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-13T14:44:34.567 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-13T14:44:34.567 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-13T14:44:34.567 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-13T14:44:34.568 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-13T14:44:35.131 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:35.132 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:35.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:35 smithi176 ceph-mon[29064]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:35.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:35 smithi196 ceph-mon[37486]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:36.191 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:36.288 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:36.290 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:36.291 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:37.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:37 smithi176 ceph-mon[29064]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:37.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:37 smithi196 ceph-mon[37486]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:39.369 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:39 smithi176 ceph-mon[29064]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:39.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:39 smithi196 ceph-mon[37486]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.532+0000 7fbfeda00700 1 -- 172.21.15.176:0/2677870870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 msgr2=0x7fbfe8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.532+0000 7fbfeda00700 1 --2- 172.21.15.176:0/2677870870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe8101180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009910 tx=0x7fbfd8009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.532+0000 7fbfeda00700 1 -- 172.21.15.176:0/2677870870 shutdown_connections 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.532+0000 7fbfeda00700 1 --2- 172.21.15.176:0/2677870870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfe8101750 0x7fbfe81057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.532+0000 7fbfeda00700 1 --2- 172.21.15.176:0/2677870870 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe8101180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.532+0000 7fbfeda00700 1 -- 172.21.15.176:0/2677870870 >> 172.21.15.176:0/2677870870 conn(0x7fbfe80fc630 msgr2=0x7fbfe80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.532+0000 7fbfeda00700 1 -- 172.21.15.176:0/2677870870 shutdown_connections 2024-03-13T14:44:40.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.533+0000 7fbfeda00700 1 -- 172.21.15.176:0/2677870870 wait complete. 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.534+0000 7fbfeda00700 1 Processor -- start 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.534+0000 7fbfeda00700 1 -- start start 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.534+0000 7fbfeda00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.534+0000 7fbfeda00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfe8101750 0x7fbfe819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.534+0000 7fbfeda00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe819f350 con 0x7fbfe8100da0 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.534+0000 7fbfeda00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8198800 con 0x7fbfe8101750 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.534+0000 7fbfe6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.535+0000 7fbfe6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40516/0 (socket says 172.21.15.176:40516) 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.535+0000 7fbfe6a00700 1 -- 172.21.15.176:0/4205031603 learned_addr learned my addr 172.21.15.176:0/4205031603 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.535+0000 7fbfe6000700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfe8101750 0x7fbfe819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:40.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.535+0000 7fbfe6a00700 1 -- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfe8101750 msgr2=0x7fbfe819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:40.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.535+0000 7fbfe6a00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfe8101750 0x7fbfe819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.535+0000 7fbfe6a00700 1 -- 172.21.15.176:0/4205031603 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80095f0 con 0x7fbfe8100da0 2024-03-13T14:44:40.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.535+0000 7fbfe6a00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe819e780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80092e0 tx=0x7fbfd800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:40.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.536+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfd801d070 con 0x7fbfe8100da0 2024-03-13T14:44:40.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.536+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbfd8004fa0 con 0x7fbfe8100da0 2024-03-13T14:44:40.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.536+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe8198a80 con 0x7fbfe8100da0 2024-03-13T14:44:40.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.536+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfd800f460 con 0x7fbfe8100da0 2024-03-13T14:44:40.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.536+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe8198f70 con 0x7fbfe8100da0 2024-03-13T14:44:40.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.537+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbfe810dfe0 con 0x7fbfe8100da0 2024-03-13T14:44:40.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.539+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fbfd8022810 con 0x7fbfe8100da0 2024-03-13T14:44:40.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.540+0000 7fbfdf400700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fbfcc06c4e0 0x7fbfcc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:40.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.540+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbfd808ca90 con 0x7fbfe8100da0 2024-03-13T14:44:40.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.540+0000 7fbfe6000700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fbfcc06c4e0 0x7fbfcc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:40.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.540+0000 7fbfdf400700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] conn(0x7fbfcc0721c0 0x7fbfcc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:40.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.540+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 --> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbfcc074c90 con 0x7fbfcc0721c0 2024-03-13T14:44:40.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.540+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd80912a0 con 0x7fbfe8100da0 2024-03-13T14:44:40.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.541+0000 7fbfe7400700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] conn(0x7fbfcc0721c0 0x7fbfcc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:40.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.541+0000 7fbfe6000700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fbfcc06c4e0 0x7fbfcc06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4005950 tx=0x7fbfd40058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:40.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.542+0000 7fbfe7400700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] conn(0x7fbfcc0721c0 0x7fbfcc0745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:40.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.544+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== osd.3 v2:172.21.15.196:6800/3087694250 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbfcc074c90 con 0x7fbfcc0721c0 2024-03-13T14:44:40.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.561+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 --> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbfe804ea90 con 0x7fbfcc0721c0 2024-03-13T14:44:40.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.562+0000 7fbfdf400700 1 -- 172.21.15.176:0/4205031603 <== osd.3 v2:172.21.15.196:6800/3087694250 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbfe804ea90 con 0x7fbfcc0721c0 2024-03-13T14:44:40.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.562+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] conn(0x7fbfcc0721c0 msgr2=0x7fbfcc0745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:40.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.562+0000 7fbfeda00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] conn(0x7fbfcc0721c0 0x7fbfcc0745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fbfcc06c4e0 msgr2=0x7fbfcc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fbfcc06c4e0 0x7fbfcc06e9a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4005950 tx=0x7fbfd40058e0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 msgr2=0x7fbfe819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe819e780 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80092e0 tx=0x7fbfd800be20 comp rx=0 tx=0).stop 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 shutdown_connections 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fbfcc06c4e0 0x7fbfcc06e9a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbfe8100da0 0x7fbfe819e780 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:6800/3087694250,v1:172.21.15.196:6801/3087694250] conn(0x7fbfcc0721c0 0x7fbfcc0745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 --2- 172.21.15.176:0/4205031603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbfe8101750 0x7fbfe819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 >> 172.21.15.176:0/4205031603 conn(0x7fbfe80fc630 msgr2=0x7fbfe80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:40.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 shutdown_connections 2024-03-13T14:44:40.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:40.563+0000 7fbfeda00700 1 -- 172.21.15.176:0/4205031603 wait complete. 2024-03-13T14:44:41.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:40 smithi176 ceph-mon[29064]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:41.286 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:40 smithi196 ceph-mon[37486]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:41.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 -- 172.21.15.176:0/891620387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 msgr2=0x7f0cdc1057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:41.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 --2- 172.21.15.176:0/891620387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc1057a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8009910 tx=0x7f0cc8009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:41.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 -- 172.21.15.176:0/891620387 shutdown_connections 2024-03-13T14:44:41.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 --2- 172.21.15.176:0/891620387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc1057a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 --2- 172.21.15.176:0/891620387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cdc100da0 0x7f0cdc101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 -- 172.21.15.176:0/891620387 >> 172.21.15.176:0/891620387 conn(0x7f0cdc0fc630 msgr2=0x7f0cdc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:41.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 -- 172.21.15.176:0/891620387 shutdown_connections 2024-03-13T14:44:41.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.964+0000 7f0ce1800700 1 -- 172.21.15.176:0/891620387 wait complete. 2024-03-13T14:44:41.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.965+0000 7f0ce1800700 1 Processor -- start 2024-03-13T14:44:41.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.965+0000 7f0ce1800700 1 -- start start 2024-03-13T14:44:41.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0ce1800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cdc100da0 0x7f0cdc073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:41.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0ce1800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:41.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0ce1800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc073b10 con 0x7f0cdc101750 2024-03-13T14:44:41.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0ce1800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cdc073c50 con 0x7f0cdc100da0 2024-03-13T14:44:41.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0cda000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:41.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0cda000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46738/0 (socket says 172.21.15.176:46738) 2024-03-13T14:44:41.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0cda000700 1 -- 172.21.15.176:0/84159665 learned_addr learned my addr 172.21.15.176:0/84159665 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:41.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.966+0000 7f0cdaa00700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cdc100da0 0x7f0cdc073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:41.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0cda000700 1 -- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cdc100da0 msgr2=0x7f0cdc073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:41.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0cda000700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cdc100da0 0x7f0cdc073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0cda000700 1 -- 172.21.15.176:0/84159665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc80095f0 con 0x7f0cdc101750 2024-03-13T14:44:41.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0cda000700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc0735d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800bda0 tx=0x7f0cc800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:41.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc801d070 con 0x7f0cdc101750 2024-03-13T14:44:41.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cdc073ed0 con 0x7f0cdc101750 2024-03-13T14:44:41.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cc8022470 con 0x7f0cdc101750 2024-03-13T14:44:41.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc800f690 con 0x7f0cdc101750 2024-03-13T14:44:41.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.967+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cdc1a2c30 con 0x7f0cdc101750 2024-03-13T14:44:41.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.968+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0cdc10e040 con 0x7f0cdc101750 2024-03-13T14:44:41.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.970+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0cc8022b00 con 0x7f0cdc101750 2024-03-13T14:44:41.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.970+0000 7f0cd3400700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0cc006c4e0 0x7f0cc006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:41.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.971+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0cc808d110 con 0x7f0cdc101750 2024-03-13T14:44:41.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.971+0000 7f0cdaa00700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0cc006c4e0 0x7f0cc006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:41.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.971+0000 7f0cd3400700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] conn(0x7f0cc00721c0 0x7f0cc00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:41.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.971+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 --> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cc0074c90 con 0x7f0cc00721c0 2024-03-13T14:44:41.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.971+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cc808d570 con 0x7f0cdc101750 2024-03-13T14:44:41.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.971+0000 7f0cdaa00700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0cc006c4e0 0x7f0cc006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc005fd0 tx=0x7f0ccc00a560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:41.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.972+0000 7f0cdb400700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] conn(0x7f0cc00721c0 0x7f0cc00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:41.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.972+0000 7f0cdb400700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] conn(0x7f0cc00721c0 0x7f0cc00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:41.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.974+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== osd.4 v2:172.21.15.196:6808/111802733 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0cc0074c90 con 0x7f0cc00721c0 2024-03-13T14:44:41.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.992+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 --> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0cdc04ea90 con 0x7f0cc00721c0 2024-03-13T14:44:41.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.992+0000 7f0cd3400700 1 -- 172.21.15.176:0/84159665 <== osd.4 v2:172.21.15.196:6808/111802733 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0cdc04ea90 con 0x7f0cc00721c0 2024-03-13T14:44:41.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] conn(0x7f0cc00721c0 msgr2=0x7f0cc00745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:41.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] conn(0x7f0cc00721c0 0x7f0cc00745e0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0cc006c4e0 msgr2=0x7f0cc006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:41.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0cc006c4e0 0x7f0cc006e9a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc005fd0 tx=0x7f0ccc00a560 comp rx=0 tx=0).stop 2024-03-13T14:44:41.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 msgr2=0x7f0cdc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:41.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc0735d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800bda0 tx=0x7f0cc800be80 comp rx=0 tx=0).stop 2024-03-13T14:44:41.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 shutdown_connections 2024-03-13T14:44:41.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:6808/111802733,v1:172.21.15.196:6809/111802733] conn(0x7f0cc00721c0 0x7f0cc00745e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0cc006c4e0 0x7f0cc006e9a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0cdc100da0 0x7f0cdc073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.993+0000 7f0ce1800700 1 --2- 172.21.15.176:0/84159665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0cdc101750 0x7f0cdc0735d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:41.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.994+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 >> 172.21.15.176:0/84159665 conn(0x7f0cdc0fc630 msgr2=0x7f0cdc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:41.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.994+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 shutdown_connections 2024-03-13T14:44:41.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:41.994+0000 7f0ce1800700 1 -- 172.21.15.176:0/84159665 wait complete. 2024-03-13T14:44:43.168 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:43 smithi176 ceph-mon[29064]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:43.168 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:43.208 INFO:teuthology.orchestra.run.smithi176.stdout:115964117016 2024-03-13T14:44:43.208 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-13T14:44:43.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:43 smithi196 ceph-mon[37486]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:43.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:44.015 INFO:teuthology.orchestra.run.smithi176.stdout:150323855376 2024-03-13T14:44:44.015 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-13T14:44:44.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 -- 172.21.15.176:0/896763912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1009a0 msgr2=0x7f2b4c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:44.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 --2- 172.21.15.176:0/896763912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1009a0 0x7f2b4c100e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009910 tx=0x7f2b38009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:44.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 -- 172.21.15.176:0/896763912 shutdown_connections 2024-03-13T14:44:44.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 --2- 172.21.15.176:0/896763912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1009a0 0x7f2b4c100e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 --2- 172.21.15.176:0/896763912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b4c1069a0 0x7f2b4c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 -- 172.21.15.176:0/896763912 >> 172.21.15.176:0/896763912 conn(0x7f2b4c0fc490 msgr2=0x7f2b4c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:44.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 -- 172.21.15.176:0/896763912 shutdown_connections 2024-03-13T14:44:44.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.832+0000 7f2b51a00700 1 -- 172.21.15.176:0/896763912 wait complete. 2024-03-13T14:44:44.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.833+0000 7f2b51a00700 1 Processor -- start 2024-03-13T14:44:44.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.833+0000 7f2b51a00700 1 -- start start 2024-03-13T14:44:44.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b51a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b4c1009a0 0x7f2b4c1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:44.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b51a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1069a0 0x7f2b4c196b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:44.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b51a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c1971f0 con 0x7f2b4c1069a0 2024-03-13T14:44:44.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b51a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4c19af80 con 0x7f2b4c1009a0 2024-03-13T14:44:44.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1069a0 0x7f2b4c196b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1069a0 0x7f2b4c196b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46752/0 (socket says 172.21.15.176:46752) 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4b400700 1 -- 172.21.15.176:0/1893246957 learned_addr learned my addr 172.21.15.176:0/1893246957 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4b400700 1 -- 172.21.15.176:0/1893246957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b4c1009a0 msgr2=0x7f2b4c1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4be00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b4c1009a0 0x7f2b4c1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4b400700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b4c1009a0 0x7f2b4c1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4b400700 1 -- 172.21.15.176:0/1893246957 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380095f0 con 0x7f2b4c1069a0 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.834+0000 7f2b4be00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b4c1009a0 0x7f2b4c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:44:44.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.835+0000 7f2b4b400700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1069a0 0x7f2b4c196b10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800b5c0 tx=0x7f2b3800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:44.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.835+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3801d070 con 0x7f2b4c1069a0 2024-03-13T14:44:44.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.835+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b38022470 con 0x7f2b4c1069a0 2024-03-13T14:44:44.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.835+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b4c19b200 con 0x7f2b4c1069a0 2024-03-13T14:44:44.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.835+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b3800f670 con 0x7f2b4c1069a0 2024-03-13T14:44:44.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.835+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b4c19b6f0 con 0x7f2b4c1069a0 2024-03-13T14:44:44.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.836+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b4c10df20 con 0x7f2b4c1069a0 2024-03-13T14:44:44.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.837+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f2b38004c30 con 0x7f2b4c1069a0 2024-03-13T14:44:44.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.838+0000 7f2b48c00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f2b3006c4e0 0x7f2b3006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:44.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.838+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f2b3808d340 con 0x7f2b4c1069a0 2024-03-13T14:44:44.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.839+0000 7f2b4be00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f2b3006c4e0 0x7f2b3006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:44.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.839+0000 7f2b48c00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] conn(0x7f2b300721c0 0x7f2b300745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:44.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.839+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 --> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b30074c90 con 0x7f2b300721c0 2024-03-13T14:44:44.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.839+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b3808d700 con 0x7f2b4c1069a0 2024-03-13T14:44:44.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.839+0000 7f2b51000700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] conn(0x7f2b300721c0 0x7f2b300745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:44.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.839+0000 7f2b4be00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f2b3006c4e0 0x7f2b3006e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c007900 tx=0x7f2b3c008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:44.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.840+0000 7f2b51000700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] conn(0x7f2b300721c0 0x7f2b300745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:44.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.840+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== osd.1 v2:172.21.15.176:6810/2968433057 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2b30074c90 con 0x7f2b300721c0 2024-03-13T14:44:44.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.857+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 --> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b4c04f2e0 con 0x7f2b300721c0 2024-03-13T14:44:44.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b48c00700 1 -- 172.21.15.176:0/1893246957 <== osd.1 v2:172.21.15.176:6810/2968433057 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2b4c04f2e0 con 0x7f2b300721c0 2024-03-13T14:44:44.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] conn(0x7f2b300721c0 msgr2=0x7f2b300745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:44.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] conn(0x7f2b300721c0 0x7f2b300745e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f2b3006c4e0 msgr2=0x7f2b3006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f2b3006c4e0 0x7f2b3006e9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c007900 tx=0x7f2b3c008040 comp rx=0 tx=0).stop 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1069a0 msgr2=0x7f2b4c196b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1069a0 0x7f2b4c196b10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2b3800b5c0 tx=0x7f2b3800be00 comp rx=0 tx=0).stop 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 shutdown_connections 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6810/2968433057,v1:172.21.15.176:6811/2968433057] conn(0x7f2b300721c0 0x7f2b300745e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f2b3006c4e0 0x7f2b3006e9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b4c1009a0 0x7f2b4c1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 --2- 172.21.15.176:0/1893246957 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2b4c1069a0 0x7f2b4c196b10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.858+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 >> 172.21.15.176:0/1893246957 conn(0x7f2b4c0fc490 msgr2=0x7f2b4c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.859+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 shutdown_connections 2024-03-13T14:44:44.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:44.859+0000 7f2b51a00700 1 -- 172.21.15.176:0/1893246957 wait complete. 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 -- 172.21.15.176:0/2927145720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 msgr2=0x7f9a64073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 --2- 172.21.15.176:0/2927145720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a64073540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9a54009910 tx=0x7f9a54009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 -- 172.21.15.176:0/2927145720 shutdown_connections 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 --2- 172.21.15.176:0/2927145720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a64073a80 0x7f9a6410cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 --2- 172.21.15.176:0/2927145720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a64073540 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 -- 172.21.15.176:0/2927145720 >> 172.21.15.176:0/2927145720 conn(0x7f9a640fc650 msgr2=0x7f9a640fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 -- 172.21.15.176:0/2927145720 shutdown_connections 2024-03-13T14:44:45.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.345+0000 7f9a68a00700 1 -- 172.21.15.176:0/2927145720 wait complete. 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.347+0000 7f9a68a00700 1 Processor -- start 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.347+0000 7f9a68a00700 1 -- start start 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.347+0000 7f9a68a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a6419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.347+0000 7f9a68a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a64073a80 0x7f9a6419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.347+0000 7f9a68a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6419f2b0 con 0x7f9a64073160 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.347+0000 7f9a68a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a64198760 con 0x7f9a64073a80 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a6419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a6419e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46772/0 (socket says 172.21.15.176:46772) 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61e00700 1 -- 172.21.15.176:0/3303544295 learned_addr learned my addr 172.21.15.176:0/3303544295 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61400700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a64073a80 0x7f9a6419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61e00700 1 -- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a64073a80 msgr2=0x7f9a6419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61e00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a64073a80 0x7f9a6419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61e00700 1 -- 172.21.15.176:0/3303544295 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a540095f0 con 0x7f9a64073160 2024-03-13T14:44:45.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61400700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a64073a80 0x7f9a6419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:44:45.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.348+0000 7f9a61e00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a6419e6e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9a54000c00 tx=0x7f9a5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:45.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.349+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a5401d070 con 0x7f9a64073160 2024-03-13T14:44:45.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.349+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a641989e0 con 0x7f9a64073160 2024-03-13T14:44:45.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.349+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9a54004fa0 con 0x7f9a64073160 2024-03-13T14:44:45.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.349+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a5400f460 con 0x7f9a64073160 2024-03-13T14:44:45.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.349+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a64198ed0 con 0x7f9a64073160 2024-03-13T14:44:45.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.350+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9a64110250 con 0x7f9a64073160 2024-03-13T14:44:45.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.351+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f9a54022870 con 0x7f9a64073160 2024-03-13T14:44:45.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.352+0000 7f9a5aa00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a4806c4e0 0x7f9a4806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.352+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9a5408ca90 con 0x7f9a64073160 2024-03-13T14:44:45.342 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.352+0000 7f9a61400700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a4806c4e0 0x7f9a4806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.342 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.352+0000 7f9a5aa00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] conn(0x7f9a480721c0 0x7f9a480745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.343 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.352+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 --> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9a48074c90 con 0x7f9a480721c0 2024-03-13T14:44:45.343 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.352+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a540912a0 con 0x7f9a64073160 2024-03-13T14:44:45.343 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.353+0000 7f9a61400700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a4806c4e0 0x7f9a4806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005fd0 tx=0x7f9a50005ee0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:45.343 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.353+0000 7f9a62800700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] conn(0x7f9a480721c0 0x7f9a480745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.343 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.354+0000 7f9a62800700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] conn(0x7f9a480721c0 0x7f9a480745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:45.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.356+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== osd.5 v2:172.21.15.196:6816/3680704272 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9a48074c90 con 0x7f9a480721c0 2024-03-13T14:44:45.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.374+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 --> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9a6404ea90 con 0x7f9a480721c0 2024-03-13T14:44:45.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.374+0000 7f9a5aa00700 1 -- 172.21.15.176:0/3303544295 <== osd.5 v2:172.21.15.196:6816/3680704272 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9a6404ea90 con 0x7f9a480721c0 2024-03-13T14:44:45.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.375+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] conn(0x7f9a480721c0 msgr2=0x7f9a480745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.375+0000 7f9a68a00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] conn(0x7f9a480721c0 0x7f9a480745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.375+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a4806c4e0 msgr2=0x7f9a4806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.375+0000 7f9a68a00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a4806c4e0 0x7f9a4806e9a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a50005fd0 tx=0x7f9a50005ee0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.375+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 msgr2=0x7f9a6419e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.375+0000 7f9a68a00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a6419e6e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9a54000c00 tx=0x7f9a5400be20 comp rx=0 tx=0).stop 2024-03-13T14:44:45.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 shutdown_connections 2024-03-13T14:44:45.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:6816/3680704272,v1:172.21.15.196:6817/3680704272] conn(0x7f9a480721c0 0x7f9a480745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f9a4806c4e0 0x7f9a4806e9a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9a64073160 0x7f9a6419e6e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 --2- 172.21.15.176:0/3303544295 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a64073a80 0x7f9a6419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 >> 172.21.15.176:0/3303544295 conn(0x7f9a640fc650 msgr2=0x7f9a641076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:45.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 shutdown_connections 2024-03-13T14:44:45.367 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:45 smithi176 ceph-mon[29064]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:45.367 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.376+0000 7f9a68a00700 1 -- 172.21.15.176:0/3303544295 wait complete. 2024-03-13T14:44:45.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:45 smithi196 ceph-mon[37486]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:45.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 -- 172.21.15.176:0/2445247421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424069020 msgr2=0x7f2424105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 --2- 172.21.15.176:0/2445247421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424069020 0x7f2424105790 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f240c009910 tx=0x7f240c009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:45.946 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 -- 172.21.15.176:0/2445247421 shutdown_connections 2024-03-13T14:44:45.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 --2- 172.21.15.176:0/2445247421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424069020 0x7f2424105790 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 --2- 172.21.15.176:0/2445247421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2424068670 0x7f2424068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 -- 172.21.15.176:0/2445247421 >> 172.21.15.176:0/2445247421 conn(0x7f24240755a0 msgr2=0x7f24240759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:45.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 -- 172.21.15.176:0/2445247421 shutdown_connections 2024-03-13T14:44:45.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.956+0000 7f242ae00700 1 -- 172.21.15.176:0/2445247421 wait complete. 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.957+0000 7f242ae00700 1 Processor -- start 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f242ae00700 1 -- start start 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f242ae00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424068670 0x7f242419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f242ae00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2424069020 0x7f242419ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f242ae00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242419f2a0 con 0x7f2424068670 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f242ae00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424198750 con 0x7f2424069020 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f2423e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424068670 0x7f242419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f2423e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424068670 0x7f242419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46798/0 (socket says 172.21.15.176:46798) 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.958+0000 7f2423e00700 1 -- 172.21.15.176:0/2414766111 learned_addr learned my addr 172.21.15.176:0/2414766111 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:45.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f241b400700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2424069020 0x7f242419ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f2423e00700 1 -- 172.21.15.176:0/2414766111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2424069020 msgr2=0x7f242419ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f2423e00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2424069020 0x7f242419ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f2423e00700 1 -- 172.21.15.176:0/2414766111 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c0095f0 con 0x7f2424068670 2024-03-13T14:44:45.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f2423e00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424068670 0x7f242419e6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f241400ada0 tx=0x7f241400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:45.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f241400f070 con 0x7f2424068670 2024-03-13T14:44:45.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2414007bf0 con 0x7f2424068670 2024-03-13T14:44:45.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2414005670 con 0x7f2424068670 2024-03-13T14:44:45.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.959+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2424198a30 con 0x7f2424068670 2024-03-13T14:44:45.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.960+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2424198f30 con 0x7f2424068670 2024-03-13T14:44:45.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.960+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f242410e060 con 0x7f2424068670 2024-03-13T14:44:45.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.962+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f24140058e0 con 0x7f2424068670 2024-03-13T14:44:45.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.962+0000 7f2421600700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f240806c4e0 0x7f240806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.963+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f241408aef0 con 0x7f2424068670 2024-03-13T14:44:45.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.963+0000 7f241b400700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f240806c4e0 0x7f240806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.963+0000 7f2421600700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] conn(0x7f24080721c0 0x7f24080745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:45.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.963+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 --> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2408074c90 con 0x7f24080721c0 2024-03-13T14:44:45.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.963+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2414090570 con 0x7f2424068670 2024-03-13T14:44:45.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.963+0000 7f2428e00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] conn(0x7f24080721c0 0x7f24080745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:45.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.964+0000 7f241b400700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f240806c4e0 0x7f240806e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2424199bc0 tx=0x7f240c01a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:45.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.964+0000 7f2428e00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] conn(0x7f24080721c0 0x7f24080745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:45.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.965+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== osd.0 v2:172.21.15.176:6802/2000924858 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2408074c90 con 0x7f24080721c0 2024-03-13T14:44:45.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.981+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 --> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f242404ea90 con 0x7f24080721c0 2024-03-13T14:44:45.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.982+0000 7f2421600700 1 -- 172.21.15.176:0/2414766111 <== osd.0 v2:172.21.15.176:6802/2000924858 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f242404ea90 con 0x7f24080721c0 2024-03-13T14:44:45.972 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.982+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] conn(0x7f24080721c0 msgr2=0x7f24080745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.972 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.982+0000 7f242ae00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] conn(0x7f24080721c0 0x7f24080745e0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f240806c4e0 msgr2=0x7f240806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f240806c4e0 0x7f240806e9a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2424199bc0 tx=0x7f240c01a040 comp rx=0 tx=0).stop 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424068670 msgr2=0x7f242419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424068670 0x7f242419e6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f241400ada0 tx=0x7f241400c330 comp rx=0 tx=0).stop 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 shutdown_connections 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f240806c4e0 0x7f240806e9a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:6802/2000924858,v1:172.21.15.176:6803/2000924858] conn(0x7f24080721c0 0x7f24080745e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2424068670 0x7f242419e6d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 --2- 172.21.15.176:0/2414766111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2424069020 0x7f242419ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:45.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 >> 172.21.15.176:0/2414766111 conn(0x7f24240755a0 msgr2=0x7f2424105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:45.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 shutdown_connections 2024-03-13T14:44:45.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:45.983+0000 7f242ae00700 1 -- 172.21.15.176:0/2414766111 wait complete. 2024-03-13T14:44:46.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.179+0000 7f4ee3e00700 1 -- 172.21.15.176:0/244818228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc0731c0 msgr2=0x7f4edc0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:46.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.179+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/244818228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc0731c0 0x7f4edc0735a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000b0a0 tx=0x7f4ed000b3b0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.179+0000 7f4ee3e00700 1 -- 172.21.15.176:0/244818228 shutdown_connections 2024-03-13T14:44:46.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.179+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/244818228 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4edc073ae0 0x7f4edc10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.179+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/244818228 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc0731c0 0x7f4edc0735a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.179+0000 7f4ee3e00700 1 -- 172.21.15.176:0/244818228 >> 172.21.15.176:0/244818228 conn(0x7f4edc0fc630 msgr2=0x7f4edc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.179+0000 7f4ee3e00700 1 -- 172.21.15.176:0/244818228 shutdown_connections 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.180+0000 7f4ee3e00700 1 -- 172.21.15.176:0/244818228 wait complete. 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee3e00700 1 Processor -- start 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee3e00700 1 -- start start 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee3e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4edc0731c0 0x7f4edc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc073ae0 0x7f4edc198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee3e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1995a0 con 0x7f4edc073ae0 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee3e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc19d360 con 0x7f4edc0731c0 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee0a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc073ae0 0x7f4edc198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee0a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc073ae0 0x7f4edc198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46814/0 (socket says 172.21.15.176:46814) 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.181+0000 7f4ee0a00700 1 -- 172.21.15.176:0/1054637569 learned_addr learned my addr 172.21.15.176:0/1054637569 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4ee1400700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4edc0731c0 0x7f4edc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:46.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4ee0a00700 1 -- 172.21.15.176:0/1054637569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4edc0731c0 msgr2=0x7f4edc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4ee0a00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4edc0731c0 0x7f4edc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4ee0a00700 1 -- 172.21.15.176:0/1054637569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed0009d00 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4ee0a00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc073ae0 0x7f4edc198f10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d8d0 tx=0x7f4ec800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec8009940 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4ec8010460 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.182+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc19d640 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.183+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec800f5f0 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.183+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc19db90 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.183+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4edc110260 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.185+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f4ec80105d0 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.185+0000 7f4eda000700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f4ec406c4e0 0x7f4ec406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.186+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4ec808b490 con 0x7f4edc073ae0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.186+0000 7f4ee1400700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f4ec406c4e0 0x7f4ec406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:46.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.186+0000 7f4eda000700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] conn(0x7f4ec40721c0 0x7f4ec40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.186+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 --> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ec4074c90 con 0x7f4ec40721c0 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.186+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ec808dac0 con 0x7f4edc073ae0 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.186+0000 7f4ee1e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] conn(0x7f4ec40721c0 0x7f4ec40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.186+0000 7f4ee1400700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f4ec406c4e0 0x7f4ec406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009480 tx=0x7f4ed000b7b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.187+0000 7f4ee1e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] conn(0x7f4ec40721c0 0x7f4ec40745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.187+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== osd.2 v2:172.21.15.176:6818/2465800922 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4ec4074c90 con 0x7f4ec40721c0 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.205+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 --> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4edc04ea90 con 0x7f4ec40721c0 2024-03-13T14:44:46.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.205+0000 7f4eda000700 1 -- 172.21.15.176:0/1054637569 <== osd.2 v2:172.21.15.176:6818/2465800922 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4edc04ea90 con 0x7f4ec40721c0 2024-03-13T14:44:46.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.205+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] conn(0x7f4ec40721c0 msgr2=0x7f4ec40745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:46.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.205+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] conn(0x7f4ec40721c0 0x7f4ec40745e0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f4ec406c4e0 msgr2=0x7f4ec406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:46.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f4ec406c4e0 0x7f4ec406e9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009480 tx=0x7f4ed000b7b0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc073ae0 msgr2=0x7f4edc198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:46.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc073ae0 0x7f4edc198f10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d8d0 tx=0x7f4ec800dc90 comp rx=0 tx=0).stop 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 shutdown_connections 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6818/2465800922,v1:172.21.15.176:6819/2465800922] conn(0x7f4ec40721c0 0x7f4ec40745e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f4ec406c4e0 0x7f4ec406e9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4edc0731c0 0x7f4edc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 --2- 172.21.15.176:0/1054637569 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4edc073ae0 0x7f4edc198f10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 >> 172.21.15.176:0/1054637569 conn(0x7f4edc0fc630 msgr2=0x7f4edc107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 shutdown_connections 2024-03-13T14:44:46.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:46.206+0000 7f4ee3e00700 1 -- 172.21.15.176:0/1054637569 wait complete. 2024-03-13T14:44:47.259 INFO:teuthology.orchestra.run.smithi176.stdout:60129542183 2024-03-13T14:44:47.259 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-13T14:44:47.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:47 smithi176 ceph-mon[29064]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:47.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:47 smithi196 ceph-mon[37486]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:47.821 INFO:teuthology.orchestra.run.smithi176.stdout:38654705710 2024-03-13T14:44:47.822 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-13T14:44:48.179 INFO:teuthology.orchestra.run.smithi176.stdout:171798691849 2024-03-13T14:44:48.179 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-13T14:44:48.565 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:48.570 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:48.608 INFO:teuthology.orchestra.run.smithi176.stdout:81604378656 2024-03-13T14:44:48.608 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-13T14:44:49.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:49 smithi176 ceph-mon[29064]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:49.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:49 smithi196 ceph-mon[37486]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:50.337 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:50.339 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:51.063 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:50 smithi176 ceph-mon[29064]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:51.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:50 smithi196 ceph-mon[37486]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:51.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.474+0000 7f73d4a00700 1 -- 172.21.15.176:0/1121508752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 msgr2=0x7f73d01079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:51.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.474+0000 7f73d4a00700 1 --2- 172.21.15.176:0/1121508752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d01079f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009910 tx=0x7f73bc009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:51.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.475+0000 7f73d4a00700 1 -- 172.21.15.176:0/1121508752 shutdown_connections 2024-03-13T14:44:51.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.475+0000 7f73d4a00700 1 --2- 172.21.15.176:0/1121508752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d01079f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.475+0000 7f73d4a00700 1 --2- 172.21.15.176:0/1121508752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102ff0 0x7f73d01033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.475+0000 7f73d4a00700 1 -- 172.21.15.176:0/1121508752 >> 172.21.15.176:0/1121508752 conn(0x7f73d00fe880 msgr2=0x7f73d0100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.475+0000 7f73d4a00700 1 -- 172.21.15.176:0/1121508752 shutdown_connections 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.475+0000 7f73d4a00700 1 -- 172.21.15.176:0/1121508752 wait complete. 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73d4a00700 1 Processor -- start 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73d4a00700 1 -- start start 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73d4a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102ff0 0x7f73d019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73d4a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73d4a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d019f3c0 con 0x7f73d01039a0 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73d4a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0198870 con 0x7f73d0102ff0 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73cd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:51.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73cd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46832/0 (socket says 172.21.15.176:46832) 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.476+0000 7f73cd400700 1 -- 172.21.15.176:0/448689307 learned_addr learned my addr 172.21.15.176:0/448689307 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73cde00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102ff0 0x7f73d019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73cd400700 1 -- 172.21.15.176:0/448689307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102ff0 msgr2=0x7f73d019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73cd400700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102ff0 0x7f73d019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73cd400700 1 -- 172.21.15.176:0/448689307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0095f0 con 0x7f73d01039a0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73cde00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102ff0 0x7f73d019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73cd400700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d019ed30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0092e0 tx=0x7f73bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73c6a00700 1 -- 172.21.15.176:0/448689307 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc01d070 con 0x7f73d01039a0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73c6a00700 1 -- 172.21.15.176:0/448689307 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f73bc022470 con 0x7f73d01039a0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73c6a00700 1 -- 172.21.15.176:0/448689307 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73bc00f690 con 0x7f73d01039a0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.477+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d0198af0 con 0x7f73d01039a0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.478+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d0198f90 con 0x7f73d01039a0 2024-03-13T14:44:51.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.479+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d010b340 con 0x7f73d01039a0 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.480+0000 7f73c6a00700 1 -- 172.21.15.176:0/448689307 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f73bc0225e0 con 0x7f73d01039a0 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.481+0000 7f73c6a00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f73b406c4e0 0x7f73b406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.481+0000 7f73c6a00700 1 -- 172.21.15.176:0/448689307 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f73bc08cad0 con 0x7f73d01039a0 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.481+0000 7f73cde00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f73b406c4e0 0x7f73b406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.482+0000 7f73cde00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f73b406c4e0 0x7f73b406e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73c0007900 tx=0x7f73c0008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.486+0000 7f73c6a00700 1 -- 172.21.15.176:0/448689307 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f73bc05b110 con 0x7f73d01039a0 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.710+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f73d004ea90 con 0x7f73d01039a0 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stdout:150323855378 2024-03-13T14:44:51.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.710+0000 7f73c6a00700 1 -- 172.21.15.176:0/448689307 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73bc027020 con 0x7f73d01039a0 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f73b406c4e0 msgr2=0x7f73b406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f73b406c4e0 0x7f73b406e9a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73c0007900 tx=0x7f73c0008040 comp rx=0 tx=0).stop 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 msgr2=0x7f73d019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d019ed30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0092e0 tx=0x7f73bc00be20 comp rx=0 tx=0).stop 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 shutdown_connections 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f73b406c4e0 0x7f73b406e9a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73d0102ff0 0x7f73d019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 --2- 172.21.15.176:0/448689307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f73d01039a0 0x7f73d019ed30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 >> 172.21.15.176:0/448689307 conn(0x7f73d00fe880 msgr2=0x7f73d00fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 shutdown_connections 2024-03-13T14:44:51.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:51.714+0000 7f73d4a00700 1 -- 172.21.15.176:0/448689307 wait complete. 2024-03-13T14:44:52.075 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:51 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/448689307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-13T14:44:52.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:51 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/448689307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-13T14:44:53.054 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:52 smithi176 ceph-mon[29064]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:53.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:52 smithi196 ceph-mon[37486]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:53.493 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:54.294 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855376 got 150323855378 for osd.4 2024-03-13T14:44:54.294 DEBUG:teuthology.parallel:result is None 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 -- 172.21.15.176:0/1555442157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 msgr2=0x7f97ac111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 --2- 172.21.15.176:0/1555442157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac111610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9794009910 tx=0x7f9794009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 -- 172.21.15.176:0/1555442157 shutdown_connections 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 --2- 172.21.15.176:0/1555442157 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac111610 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 --2- 172.21.15.176:0/1555442157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac073130 0x7f97ac073510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 -- 172.21.15.176:0/1555442157 >> 172.21.15.176:0/1555442157 conn(0x7f97ac0fc630 msgr2=0x7f97ac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 -- 172.21.15.176:0/1555442157 shutdown_connections 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.909+0000 7f97b2600700 1 -- 172.21.15.176:0/1555442157 wait complete. 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.911+0000 7f97b2600700 1 Processor -- start 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.911+0000 7f97b2600700 1 -- start start 2024-03-13T14:44:54.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97b2600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac073130 0x7f97ac19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97b2600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac19d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97b2600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac19d930 con 0x7f97ac073a50 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97b2600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1a16c0 con 0x7f97ac073130 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97a3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac19d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97a3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac19d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39780/0 (socket says 172.21.15.176:39780) 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97a3e00700 1 -- 172.21.15.176:0/2176134324 learned_addr learned my addr 172.21.15.176:0/2176134324 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.912+0000 7f97ab400700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac073130 0x7f97ac19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.913+0000 7f97a3e00700 1 -- 172.21.15.176:0/2176134324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac073130 msgr2=0x7f97ac19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.913+0000 7f97a3e00700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac073130 0x7f97ac19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.913+0000 7f97a3e00700 1 -- 172.21.15.176:0/2176134324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940095f0 con 0x7f97ac073a50 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.913+0000 7f97ab400700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac073130 0x7f97ac19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.913+0000 7f97a3e00700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac19d250 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f979400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.914+0000 7f97a8c00700 1 -- 172.21.15.176:0/2176134324 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f979401d070 con 0x7f97ac073a50 2024-03-13T14:44:54.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.914+0000 7f97a8c00700 1 -- 172.21.15.176:0/2176134324 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f979400f460 con 0x7f97ac073a50 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.914+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac1a1940 con 0x7f97ac073a50 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.914+0000 7f97a8c00700 1 -- 172.21.15.176:0/2176134324 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9794021640 con 0x7f97ac073a50 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.914+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1a1e90 con 0x7f97ac073a50 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.915+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac10ed90 con 0x7f97ac073a50 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.916+0000 7f97a8c00700 1 -- 172.21.15.176:0/2176134324 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f979400fab0 con 0x7f97ac073a50 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.917+0000 7f97a8c00700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f979006c4e0 0x7f979006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.917+0000 7f97a8c00700 1 -- 172.21.15.176:0/2176134324 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f979408c750 con 0x7f97ac073a50 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.917+0000 7f97ab400700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f979006c4e0 0x7f979006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.918+0000 7f97ab400700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f979006c4e0 0x7f979006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f979c005950 tx=0x7f979c005f50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:54.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:54.922+0000 7f97a8c00700 1 -- 172.21.15.176:0/2176134324 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f97940572f0 con 0x7f97ac073a50 2024-03-13T14:44:55.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.146+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97ac04ea90 con 0x7f97ac073a50 2024-03-13T14:44:55.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.147+0000 7f97a8c00700 1 -- 172.21.15.176:0/2176134324 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f979405a910 con 0x7f97ac073a50 2024-03-13T14:44:55.137 INFO:teuthology.orchestra.run.smithi176.stdout:38654705712 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f979006c4e0 msgr2=0x7f979006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f979006c4e0 0x7f979006e9a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f979c005950 tx=0x7f979c005f50 comp rx=0 tx=0).stop 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 msgr2=0x7f97ac19d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac19d250 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9794006010 tx=0x7f979400be20 comp rx=0 tx=0).stop 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 shutdown_connections 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f979006c4e0 0x7f979006e9a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97ac073130 0x7f97ac19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 --2- 172.21.15.176:0/2176134324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97ac073a50 0x7f97ac19d250 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.150+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 >> 172.21.15.176:0/2176134324 conn(0x7f97ac0fc630 msgr2=0x7f97ac103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:55.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.151+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 shutdown_connections 2024-03-13T14:44:55.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.151+0000 7f97b2600700 1 -- 172.21.15.176:0/2176134324 wait complete. 2024-03-13T14:44:55.141 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:55 smithi176 ceph-mon[29064]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:55.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.199+0000 7fdcb0c00700 1 -- 172.21.15.176:0/2145629024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 msgr2=0x7fdcac0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:55.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.199+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/2145629024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac0756a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009910 tx=0x7fdc9c009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:55.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.199+0000 7fdcb0c00700 1 -- 172.21.15.176:0/2145629024 shutdown_connections 2024-03-13T14:44:55.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.199+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/2145629024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcac075be0 0x7fdcac1114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.199+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/2145629024 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac0756a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.199+0000 7fdcb0c00700 1 -- 172.21.15.176:0/2145629024 >> 172.21.15.176:0/2145629024 conn(0x7fdcac0fe870 msgr2=0x7fdcac100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:55.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.200+0000 7fdcb0c00700 1 -- 172.21.15.176:0/2145629024 shutdown_connections 2024-03-13T14:44:55.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.200+0000 7fdcb0c00700 1 -- 172.21.15.176:0/2145629024 wait complete. 2024-03-13T14:44:55.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.201+0000 7fdcb0c00700 1 Processor -- start 2024-03-13T14:44:55.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.201+0000 7fdcb0c00700 1 -- start start 2024-03-13T14:44:55.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.201+0000 7fdcb0c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac1a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:55.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.201+0000 7fdcb0c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcac075be0 0x7fdcac1a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:55.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.201+0000 7fdcb0c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac1a3630 con 0x7fdcac0752c0 2024-03-13T14:44:55.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.201+0000 7fdcb0c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac19caf0 con 0x7fdcac075be0 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.201+0000 7fdca9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac1a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.202+0000 7fdca9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac1a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39788/0 (socket says 172.21.15.176:39788) 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.202+0000 7fdca9e00700 1 -- 172.21.15.176:0/1029340533 learned_addr learned my addr 172.21.15.176:0/1029340533 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.202+0000 7fdca9400700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcac075be0 0x7fdcac1a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.202+0000 7fdca9e00700 1 -- 172.21.15.176:0/1029340533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcac075be0 msgr2=0x7fdcac1a2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.202+0000 7fdca9e00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcac075be0 0x7fdcac1a2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.202+0000 7fdca9e00700 1 -- 172.21.15.176:0/1029340533 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9c0095f0 con 0x7fdcac0752c0 2024-03-13T14:44:55.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.202+0000 7fdca9e00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac1a2a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:55.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.203+0000 7fdca2a00700 1 -- 172.21.15.176:0/1029340533 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc9c01d070 con 0x7fdcac0752c0 2024-03-13T14:44:55.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.203+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac19cdd0 con 0x7fdcac0752c0 2024-03-13T14:44:55.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.203+0000 7fdca2a00700 1 -- 172.21.15.176:0/1029340533 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc9c00f460 con 0x7fdcac0752c0 2024-03-13T14:44:55.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.203+0000 7fdca2a00700 1 -- 172.21.15.176:0/1029340533 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc9c021640 con 0x7fdcac0752c0 2024-03-13T14:44:55.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.203+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac19d320 con 0x7fdcac0752c0 2024-03-13T14:44:55.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.205+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcac10ec50 con 0x7fdcac0752c0 2024-03-13T14:44:55.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.205+0000 7fdca2a00700 1 -- 172.21.15.176:0/1029340533 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fdc9c00f5d0 con 0x7fdcac0752c0 2024-03-13T14:44:55.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.206+0000 7fdca2a00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc9006c4e0 0x7fdc9006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:55.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.206+0000 7fdca2a00700 1 -- 172.21.15.176:0/1029340533 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdc9c08c780 con 0x7fdcac0752c0 2024-03-13T14:44:55.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.206+0000 7fdca9400700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc9006c4e0 0x7fdc9006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:55.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.207+0000 7fdca9400700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc9006c4e0 0x7fdc9006e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc98005950 tx=0x7fdc980058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:55.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.215+0000 7fdca2a00700 1 -- 172.21.15.176:0/1029340533 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fdc9c05ad30 con 0x7fdcac0752c0 2024-03-13T14:44:55.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.439+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdcac04ea90 con 0x7fdcac0752c0 2024-03-13T14:44:55.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:55 smithi196 ceph-mon[37486]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:55.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.441+0000 7fdca2a00700 1 -- 172.21.15.176:0/1029340533 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdc9c05a8c0 con 0x7fdcac0752c0 2024-03-13T14:44:55.431 INFO:teuthology.orchestra.run.smithi176.stdout:60129542185 2024-03-13T14:44:55.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc9006c4e0 msgr2=0x7fdc9006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc9006c4e0 0x7fdc9006e9a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc98005950 tx=0x7fdc980058e0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 msgr2=0x7fdcac1a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac1a2a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c006010 tx=0x7fdc9c0056a0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 shutdown_connections 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fdc9006c4e0 0x7fdc9006e9a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdcac0752c0 0x7fdcac1a2a10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 --2- 172.21.15.176:0/1029340533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcac075be0 0x7fdcac1a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 >> 172.21.15.176:0/1029340533 conn(0x7fdcac0fe870 msgr2=0x7fdcac0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:55.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 shutdown_connections 2024-03-13T14:44:55.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:55.444+0000 7fdcb0c00700 1 -- 172.21.15.176:0/1029340533 wait complete. 2024-03-13T14:44:56.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:56 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2176134324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-13T14:44:56.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:56 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1029340533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-13T14:44:56.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:56 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2176134324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-13T14:44:56.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:56 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1029340533' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-13T14:44:56.533 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542183 got 60129542185 for osd.1 2024-03-13T14:44:56.533 DEBUG:teuthology.parallel:result is None 2024-03-13T14:44:56.674 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705710 got 38654705712 for osd.0 2024-03-13T14:44:56.674 DEBUG:teuthology.parallel:result is None 2024-03-13T14:44:57.156 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 -- 172.21.15.176:0/3947471106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 msgr2=0x7ff84c0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 --2- 172.21.15.176:0/3947471106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c0735a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009910 tx=0x7ff83c009c20 comp rx=0 tx=0).stop 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 -- 172.21.15.176:0/3947471106 shutdown_connections 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 --2- 172.21.15.176:0/3947471106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c073ae0 0x7ff84c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 --2- 172.21.15.176:0/3947471106 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c0735a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 -- 172.21.15.176:0/3947471106 >> 172.21.15.176:0/3947471106 conn(0x7ff84c0fc630 msgr2=0x7ff84c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 -- 172.21.15.176:0/3947471106 shutdown_connections 2024-03-13T14:44:57.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.193+0000 7ff851800700 1 -- 172.21.15.176:0/3947471106 wait complete. 2024-03-13T14:44:57.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.194+0000 7ff851800700 1 Processor -- start 2024-03-13T14:44:57.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.194+0000 7ff851800700 1 -- start start 2024-03-13T14:44:57.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff851800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:57.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff851800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c073ae0 0x7ff84c198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:57.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff851800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c199520 con 0x7ff84c0731c0 2024-03-13T14:44:57.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff851800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19d330 con 0x7ff84c073ae0 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff84aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff84aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39802/0 (socket says 172.21.15.176:39802) 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff84aa00700 1 -- 172.21.15.176:0/1369875952 learned_addr learned my addr 172.21.15.176:0/1369875952 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.195+0000 7ff84a000700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c073ae0 0x7ff84c198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff84aa00700 1 -- 172.21.15.176:0/1369875952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c073ae0 msgr2=0x7ff84c198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff84aa00700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c073ae0 0x7ff84c198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff84aa00700 1 -- 172.21.15.176:0/1369875952 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c0095f0 con 0x7ff84c0731c0 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff84aa00700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c198950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0092e0 tx=0x7ff83c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff843400700 1 -- 172.21.15.176:0/1369875952 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff83c01d070 con 0x7ff84c0731c0 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff843400700 1 -- 172.21.15.176:0/1369875952 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff83c022470 con 0x7ff84c0731c0 2024-03-13T14:44:57.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff843400700 1 -- 172.21.15.176:0/1369875952 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff83c00f670 con 0x7ff84c0731c0 2024-03-13T14:44:57.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c19d5b0 con 0x7ff84c0731c0 2024-03-13T14:44:57.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.196+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c19daa0 con 0x7ff84c0731c0 2024-03-13T14:44:57.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.198+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c10a540 con 0x7ff84c0731c0 2024-03-13T14:44:57.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.199+0000 7ff843400700 1 -- 172.21.15.176:0/1369875952 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff83c004c30 con 0x7ff84c0731c0 2024-03-13T14:44:57.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.200+0000 7ff843400700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff83006c4e0 0x7ff83006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:44:57.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.200+0000 7ff843400700 1 -- 172.21.15.176:0/1369875952 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff83c08cad0 con 0x7ff84c0731c0 2024-03-13T14:44:57.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.200+0000 7ff84a000700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff83006c4e0 0x7ff83006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:44:57.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.201+0000 7ff84a000700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff83006c4e0 0x7ff83006e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff83800a560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:44:57.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.208+0000 7ff843400700 1 -- 172.21.15.176:0/1369875952 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff83c05b110 con 0x7ff84c0731c0 2024-03-13T14:44:57.263 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:57 smithi176 ceph-mon[29064]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:57.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.428+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff84c19ddd0 con 0x7ff84c0731c0 2024-03-13T14:44:57.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.429+0000 7ff843400700 1 -- 172.21.15.176:0/1369875952 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff83c05aca0 con 0x7ff84c0731c0 2024-03-13T14:44:57.419 INFO:teuthology.orchestra.run.smithi176.stdout:115964117019 2024-03-13T14:44:57.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff83006c4e0 msgr2=0x7ff83006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:57.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff83006c4e0 0x7ff83006e9a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff83800a560 comp rx=0 tx=0).stop 2024-03-13T14:44:57.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 msgr2=0x7ff84c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c198950 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0092e0 tx=0x7ff83c00be00 comp rx=0 tx=0).stop 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 shutdown_connections 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff83006c4e0 0x7ff83006e9a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff84c0731c0 0x7ff84c198950 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 --2- 172.21.15.176:0/1369875952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff84c073ae0 0x7ff84c198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.431+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 >> 172.21.15.176:0/1369875952 conn(0x7ff84c0fc630 msgr2=0x7ff84c107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.432+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 shutdown_connections 2024-03-13T14:44:57.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:44:57.432+0000 7ff851800700 1 -- 172.21.15.176:0/1369875952 wait complete. 2024-03-13T14:44:57.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:57 smithi196 ceph-mon[37486]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:58.243 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117016 got 115964117019 for osd.3 2024-03-13T14:44:58.243 DEBUG:teuthology.parallel:result is None 2024-03-13T14:44:58.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:58 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1369875952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-13T14:44:58.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:58.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:58 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1369875952' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-13T14:44:58.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:44:59.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:44:59 smithi176 ceph-mon[29064]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:44:59.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:44:59 smithi196 ceph-mon[37486]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:00.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.109+0000 7f3f73a00700 1 -- 172.21.15.176:0/2219219715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 msgr2=0x7f3f6c1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:00.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.109+0000 7f3f73a00700 1 --2- 172.21.15.176:0/2219219715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c1033a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000b0a0 tx=0x7f3f6000b3b0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.109+0000 7f3f73a00700 1 -- 172.21.15.176:0/2219219715 shutdown_connections 2024-03-13T14:45:00.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.109+0000 7f3f73a00700 1 --2- 172.21.15.176:0/2219219715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f6c103970 0x7f3f6c1079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.109+0000 7f3f73a00700 1 --2- 172.21.15.176:0/2219219715 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c1033a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.109+0000 7f3f73a00700 1 -- 172.21.15.176:0/2219219715 >> 172.21.15.176:0/2219219715 conn(0x7f3f6c0fe870 msgr2=0x7f3f6c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.110+0000 7f3f73a00700 1 -- 172.21.15.176:0/2219219715 shutdown_connections 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.110+0000 7f3f73a00700 1 -- 172.21.15.176:0/2219219715 wait complete. 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.111+0000 7f3f73a00700 1 Processor -- start 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.111+0000 7f3f73a00700 1 -- start start 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.111+0000 7f3f73a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.111+0000 7f3f73a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f6c103970 0x7f3f6c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.111+0000 7f3f73a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c19f430 con 0x7f3f6c102fc0 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.111+0000 7f3f73a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6c1988e0 con 0x7f3f6c103970 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f71000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:00.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f71000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39818/0 (socket says 172.21.15.176:39818) 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f71000700 1 -- 172.21.15.176:0/1089480403 learned_addr learned my addr 172.21.15.176:0/1089480403 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f71000700 1 -- 172.21.15.176:0/1089480403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f6c103970 msgr2=0x7f3f6c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f6be00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f6c103970 0x7f3f6c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f71000700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f6c103970 0x7f3f6c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f71000700 1 -- 172.21.15.176:0/1089480403 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f60009d00 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f71000700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c19e860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000b070 tx=0x7f3f6000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.112+0000 7f3f69600700 1 -- 172.21.15.176:0/1089480403 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6000e040 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.113+0000 7f3f69600700 1 -- 172.21.15.176:0/1089480403 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3f600044f0 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.113+0000 7f3f69600700 1 -- 172.21.15.176:0/1089480403 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6001d9a0 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.113+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6c198b60 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.113+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6c199000 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.114+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f6c10b360 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.115+0000 7f3f69600700 1 -- 172.21.15.176:0/1089480403 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3f60004660 con 0x7f3f6c102fc0 2024-03-13T14:45:00.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.116+0000 7f3f69600700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:00.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.116+0000 7f3f69600700 1 -- 172.21.15.176:0/1089480403 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3f6008cfc0 con 0x7f3f6c102fc0 2024-03-13T14:45:00.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.117+0000 7f3f6be00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:00.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.118+0000 7f3f6be00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006fd0 tx=0x7f3f58008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:00.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.122+0000 7f3f69600700 1 -- 172.21.15.176:0/1089480403 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3f600c00e0 con 0x7f3f6c102fc0 2024-03-13T14:45:00.367 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.377+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3f6c02ccc0 con 0x7f3f6c102fc0 2024-03-13T14:45:00.367 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.378+0000 7f3f69600700 1 -- 172.21.15.176:0/1089480403 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f6005b510 con 0x7f3f6c102fc0 2024-03-13T14:45:00.368 INFO:teuthology.orchestra.run.smithi176.stdout:171798691852 2024-03-13T14:45:00.370 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.380+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3f5406c4e0 msgr2=0x7f3f5406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:00.370 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.380+0000 7f3f73a00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f58006fd0 tx=0x7f3f58008040 comp rx=0 tx=0).stop 2024-03-13T14:45:00.370 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.380+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 msgr2=0x7f3f6c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.380+0000 7f3f73a00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c19e860 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000b070 tx=0x7f3f6000cdb0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.381+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 shutdown_connections 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.381+0000 7f3f73a00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3f5406c4e0 0x7f3f5406e9a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.381+0000 7f3f73a00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3f6c102fc0 0x7f3f6c19e860 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.381+0000 7f3f73a00700 1 --2- 172.21.15.176:0/1089480403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f6c103970 0x7f3f6c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.381+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 >> 172.21.15.176:0/1089480403 conn(0x7f3f6c0fe870 msgr2=0x7f3f6c0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.381+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 shutdown_connections 2024-03-13T14:45:00.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:00.381+0000 7f3f73a00700 1 -- 172.21.15.176:0/1089480403 wait complete. 2024-03-13T14:45:00.978 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:00 smithi176 ceph-mon[29064]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:00.978 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:00 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1089480403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.381+0000 7f05bb800700 1 -- 172.21.15.176:0/4080196192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b4073ae0 msgr2=0x7f05b410ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.381+0000 7f05bb800700 1 --2- 172.21.15.176:0/4080196192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b4073ae0 0x7f05b410ce40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f059c009910 tx=0x7f059c009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.382+0000 7f05bb800700 1 -- 172.21.15.176:0/4080196192 shutdown_connections 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.382+0000 7f05bb800700 1 --2- 172.21.15.176:0/4080196192 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b4073ae0 0x7f05b410ce40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.382+0000 7f05bb800700 1 --2- 172.21.15.176:0/4080196192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b40731c0 0x7f05b40735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.382+0000 7f05bb800700 1 -- 172.21.15.176:0/4080196192 >> 172.21.15.176:0/4080196192 conn(0x7f05b40fc630 msgr2=0x7f05b40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.382+0000 7f05bb800700 1 -- 172.21.15.176:0/4080196192 shutdown_connections 2024-03-13T14:45:01.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.382+0000 7f05bb800700 1 -- 172.21.15.176:0/4080196192 wait complete. 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.383+0000 7f05bb800700 1 Processor -- start 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.383+0000 7f05bb800700 1 -- start start 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05bb800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b40731c0 0x7f05b4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05bb800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b4073ae0 0x7f05b4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05bb800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b4199530 con 0x7f05b40731c0 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05bb800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b419d340 con 0x7f05b4073ae0 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05b8e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b40731c0 0x7f05b4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05b8e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b40731c0 0x7f05b4198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39834/0 (socket says 172.21.15.176:39834) 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05b8e00700 1 -- 172.21.15.176:0/2020841869 learned_addr learned my addr 172.21.15.176:0/2020841869 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:01.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05abe00700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b4073ae0 0x7f05b4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:01.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05b8e00700 1 -- 172.21.15.176:0/2020841869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b4073ae0 msgr2=0x7f05b4198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:01.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05b8e00700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b4073ae0 0x7f05b4198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:01.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05b8e00700 1 -- 172.21.15.176:0/2020841869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0095f0 con 0x7f05b40731c0 2024-03-13T14:45:01.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.384+0000 7f05abe00700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b4073ae0 0x7f05b4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:45:01.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.385+0000 7f05b8e00700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b40731c0 0x7f05b4198960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c8a0 tx=0x7f05a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:01.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.385+0000 7f05b2000700 1 -- 172.21.15.176:0/2020841869 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05a400cea0 con 0x7f05b40731c0 2024-03-13T14:45:01.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.385+0000 7f05b2000700 1 -- 172.21.15.176:0/2020841869 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05a4004830 con 0x7f05b40731c0 2024-03-13T14:45:01.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.385+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b419d620 con 0x7f05b40731c0 2024-03-13T14:45:01.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.385+0000 7f05b2000700 1 -- 172.21.15.176:0/2020841869 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05a40055b0 con 0x7f05b40731c0 2024-03-13T14:45:01.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.385+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b419db70 con 0x7f05b40731c0 2024-03-13T14:45:01.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.387+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b410a540 con 0x7f05b40731c0 2024-03-13T14:45:01.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.387+0000 7f05b2000700 1 -- 172.21.15.176:0/2020841869 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f05a4007ad0 con 0x7f05b40731c0 2024-03-13T14:45:01.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.389+0000 7f05b2000700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f059806c4e0 0x7f059806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:01.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.389+0000 7f05b2000700 1 -- 172.21.15.176:0/2020841869 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f05a408b240 con 0x7f05b40731c0 2024-03-13T14:45:01.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.389+0000 7f05abe00700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f059806c4e0 0x7f059806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:01.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.390+0000 7f05abe00700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f059806c4e0 0x7f059806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05b4199f80 tx=0x7f059c00b560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:01.383 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.393+0000 7f05b2000700 1 -- 172.21.15.176:0/2020841869 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f05a40597f0 con 0x7f05b40731c0 2024-03-13T14:45:01.409 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691852 for osd.5 2024-03-13T14:45:01.409 DEBUG:teuthology.parallel:result is None 2024-03-13T14:45:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:00 smithi196 ceph-mon[37486]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:00 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1089480403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-13T14:45:01.600 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.610+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05b404ea90 con 0x7f05b40731c0 2024-03-13T14:45:01.600 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.611+0000 7f05b2000700 1 -- 172.21.15.176:0/2020841869 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f05a4059380 con 0x7f05b40731c0 2024-03-13T14:45:01.600 INFO:teuthology.orchestra.run.smithi176.stdout:81604378658 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f059806c4e0 msgr2=0x7f059806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f059806c4e0 0x7f059806e9a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05b4199f80 tx=0x7f059c00b560 comp rx=0 tx=0).stop 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b40731c0 msgr2=0x7f05b4198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b40731c0 0x7f05b4198960 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05a400c8a0 tx=0x7f05a400cc60 comp rx=0 tx=0).stop 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 shutdown_connections 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f059806c4e0 0x7f059806e9a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f05b40731c0 0x7f05b4198960 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 --2- 172.21.15.176:0/2020841869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b4073ae0 0x7f05b4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 >> 172.21.15.176:0/2020841869 conn(0x7f05b40fc630 msgr2=0x7f05b4107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 shutdown_connections 2024-03-13T14:45:01.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:01.614+0000 7f05bb800700 1 -- 172.21.15.176:0/2020841869 wait complete. 2024-03-13T14:45:02.317 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:01 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2020841869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-13T14:45:02.361 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378656 got 81604378658 for osd.2 2024-03-13T14:45:02.361 DEBUG:teuthology.parallel:result is None 2024-03-13T14:45:02.361 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-13T14:45:02.361 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-13T14:45:02.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:01 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2020841869' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-13T14:45:02.765 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:03.195 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:02 smithi176 ceph-mon[29064]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:02 smithi196 ceph-mon[37486]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 -- 172.21.15.176:0/1211723148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 msgr2=0x7f76cc10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 --2- 172.21.15.176:0/1211723148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc10d660 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009910 tx=0x7f76bc009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 -- 172.21.15.176:0/1211723148 shutdown_connections 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 --2- 172.21.15.176:0/1211723148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76cc1071d0 0x7f76cc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 --2- 172.21.15.176:0/1211723148 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc10d660 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 -- 172.21.15.176:0/1211723148 >> 172.21.15.176:0/1211723148 conn(0x7f76cc0769e0 msgr2=0x7f76cc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 -- 172.21.15.176:0/1211723148 shutdown_connections 2024-03-13T14:45:04.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.659+0000 7f76d2200700 1 -- 172.21.15.176:0/1211723148 wait complete. 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.660+0000 7f76d2200700 1 Processor -- start 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.660+0000 7f76d2200700 1 -- start start 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76d2200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76cc1071d0 0x7f76cc1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76d2200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc1a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76d2200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc1a36a0 con 0x7f76cc10d280 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76d2200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc19cb50 con 0x7f76cc1071d0 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76caa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc1a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76caa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc1a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43602/0 (socket says 172.21.15.176:43602) 2024-03-13T14:45:04.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76caa00700 1 -- 172.21.15.176:0/2313366933 learned_addr learned my addr 172.21.15.176:0/2313366933 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:04.652 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76cb400700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76cc1071d0 0x7f76cc1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:04.652 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76caa00700 1 -- 172.21.15.176:0/2313366933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76cc1071d0 msgr2=0x7f76cc1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:04.652 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76caa00700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76cc1071d0 0x7f76cc1a2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:04.652 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.661+0000 7f76caa00700 1 -- 172.21.15.176:0/2313366933 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0095f0 con 0x7f76cc10d280 2024-03-13T14:45:04.652 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.662+0000 7f76caa00700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc1a3010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76b800d900 tx=0x7f76b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:04.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.662+0000 7f76c3e00700 1 -- 172.21.15.176:0/2313366933 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76b8004a00 con 0x7f76cc10d280 2024-03-13T14:45:04.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.662+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc19ce30 con 0x7f76cc10d280 2024-03-13T14:45:04.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.662+0000 7f76c3e00700 1 -- 172.21.15.176:0/2313366933 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f76b80054d0 con 0x7f76cc10d280 2024-03-13T14:45:04.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.662+0000 7f76c3e00700 1 -- 172.21.15.176:0/2313366933 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76b8009d60 con 0x7f76cc10d280 2024-03-13T14:45:04.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.662+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc19d380 con 0x7f76cc10d280 2024-03-13T14:45:04.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.663+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc04ea90 con 0x7f76cc10d280 2024-03-13T14:45:04.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.664+0000 7f76c3e00700 1 -- 172.21.15.176:0/2313366933 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f76b8005020 con 0x7f76cc10d280 2024-03-13T14:45:04.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.665+0000 7f76c3e00700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f76b006c4e0 0x7f76b006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:04.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.665+0000 7f76c3e00700 1 -- 172.21.15.176:0/2313366933 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f76b808b6f0 con 0x7f76cc10d280 2024-03-13T14:45:04.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.665+0000 7f76cb400700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f76b006c4e0 0x7f76b006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:04.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.666+0000 7f76cb400700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f76b006c4e0 0x7f76b006e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006010 tx=0x7f76bc00b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:04.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.670+0000 7f76c3e00700 1 -- 172.21.15.176:0/2313366933 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f76b8059c00 con 0x7f76cc10d280 2024-03-13T14:45:04.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.875+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76cc19d820 con 0x7f76b006c4e0 2024-03-13T14:45:04.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.877+0000 7f76c3e00700 1 -- 172.21.15.176:0/2313366933 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27162 (secure 0 0 0) 0x7f76cc19d820 con 0x7f76b006c4e0 2024-03-13T14:45:04.867 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f76b006c4e0 msgr2=0x7f76b006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f76b006c4e0 0x7f76b006e9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006010 tx=0x7f76bc00b540 comp rx=0 tx=0).stop 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 msgr2=0x7f76cc1a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc1a3010 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f76b800d900 tx=0x7f76b800dcc0 comp rx=0 tx=0).stop 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 shutdown_connections 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f76b006c4e0 0x7f76b006e9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76cc1071d0 0x7f76cc1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:04.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 --2- 172.21.15.176:0/2313366933 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76cc10d280 0x7f76cc1a3010 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:04.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.880+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 >> 172.21.15.176:0/2313366933 conn(0x7f76cc0769e0 msgr2=0x7f76cc0782d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:04.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.881+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 shutdown_connections 2024-03-13T14:45:04.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:04.881+0000 7f76d2200700 1 -- 172.21.15.176:0/2313366933 wait complete. 2024-03-13T14:45:04.871 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-03-13T14:45:05.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:05 smithi176 ceph-mon[29064]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:05.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:05 smithi196 ceph-mon[37486]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:05.601 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-03-13T14:45:04.073412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006394"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-13T14:44:06.125336+0000","last_change":"2024-03-13T14:43:29.511580+0000","last_active":"2024-03-13T14:44:06.125336+0000","last_peered":"2024-03-13T14:44:06.125336+0000","last_clean":"2024-03-13T14:44:06.125336+0000","last_became_active":"2024-03-13T14:43:29.510769+0000","last_became_peered":"2024-03-13T14:43:29.510769+0000","last_unstale":"2024-03-13T14:44:06.125336+0000","last_undegraded":"2024-03-13T14:44:06.125336+0000","last_fullsized":"2024-03-13T14:44:06.125336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T14:42:23.107140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T14:42:23.107140+0000","last_clean_scrub_stamp":"2024-03-13T14:42:23.107140+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:57:21.061981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":4,"up_from":35,"seq":150323855380,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":1.1220000000000001}]},{"osd":1,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.1819999999999999}]},{"osd":2,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":1.3300000000000001}]},{"osd":3,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":1.478},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":1.55}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]}]},{"osd":3,"up_from":27,"seq":115964117020,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.92900000000000005,"15min":0.91800000000000004},"min":{"1min":0.746,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.028},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":0.96599999999999997},"min":{"1min":0.66300000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.248,"5min":1.423,"15min":1.423},"last":0.90800000000000003}]},{"osd":1,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.96699999999999997,"15min":0.95399999999999996},"min":{"1min":0.73599999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.70199999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.96299999999999997}]},{"osd":2,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0269999999999999,"15min":1.03},"min":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.319,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.079,"5min":1.0600000000000001,"15min":1.0569999999999999},"min":{"1min":0.70199999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.1799999999999999}]},{"osd":4,"last update":"Wed Mar 13 14:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.3300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.85199999999999998,"15min":0.82899999999999996},"min":{"1min":0.56100000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.9649999999999999,"5min":2.9649999999999999,"15min":2.9649999999999999},"last":1.2889999999999999}]},{"osd":1,"last update":"Wed Mar 13 14:44:27 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.57999999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.028,"5min":3.028,"15min":3.028},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.96599999999999997,"15min":0.95199999999999996},"min":{"1min":0.70699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":1.075}]},{"osd":3,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1539999999999999,"15min":1.1479999999999999},"min":{"1min":0.89000000000000001,"5min":0.871,"15min":0.871},"max":{"1min":1.5720000000000001,"5min":2.8399999999999999,"15min":2.8399999999999999},"last":1.159},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":0.876,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.504,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.417}]},{"osd":4,"last update":"Wed Mar 13 14:44:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":3.1030000000000002,"5min":3.1030000000000002,"15min":3.1030000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.4710000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2569999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 13 14:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.72099999999999997,"15min":0.66700000000000004},"min":{"1min":0.51900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.73799999999999999,"15min":0.67200000000000004},"min":{"1min":0.47299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.226,"5min":1.272,"15min":1.272},"last":0.95099999999999996}]},{"osd":2,"last update":"Wed Mar 13 14:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.82699999999999996,"15min":0.80200000000000005},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.755},{"interface":"front","average":{"1min":1.135,"5min":0.98299999999999998,"15min":0.95799999999999996},"min":{"1min":0.623,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed Mar 13 14:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.0980000000000001,"15min":1.083},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.155,"15min":1.151},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.99099999999999999}]},{"osd":4,"last update":"Wed Mar 13 14:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.751,"15min":0.70499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.80400000000000005,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.97299999999999998}]},{"osd":2,"last update":"Wed Mar 13 14:44:25 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.93100000000000005,"15min":0.91300000000000003},"min":{"1min":0.72099999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.2450000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.113}]},{"osd":3,"last update":"Wed Mar 13 14:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.137,"15min":1.129},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.167,"15min":1.1619999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.0269999999999999}]},{"osd":4,"last update":"Wed Mar 13 14:44:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T14:45:05.602 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph pg dump --format=json 2024-03-13T14:45:06.017 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:06.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:06 smithi176 ceph-mon[29064]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-13T14:45:06.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:06 smithi196 ceph-mon[37486]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-13T14:45:07.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:07 smithi196 ceph-mon[37486]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:07.476 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:07 smithi176 ceph-mon[29064]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:07.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 -- 172.21.15.176:0/1109173583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 msgr2=0x7f63c0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:07.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 --2- 172.21.15.176:0/1109173583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0105790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009910 tx=0x7f63ac009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:07.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 -- 172.21.15.176:0/1109173583 shutdown_connections 2024-03-13T14:45:07.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 --2- 172.21.15.176:0/1109173583 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0105790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:07.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 --2- 172.21.15.176:0/1109173583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63c0068670 0x7f63c0068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:07.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 -- 172.21.15.176:0/1109173583 >> 172.21.15.176:0/1109173583 conn(0x7f63c00755a0 msgr2=0x7f63c00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:07.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 -- 172.21.15.176:0/1109173583 shutdown_connections 2024-03-13T14:45:07.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.880+0000 7f63c5200700 1 -- 172.21.15.176:0/1109173583 wait complete. 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.881+0000 7f63c5200700 1 Processor -- start 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.882+0000 7f63c5200700 1 -- start start 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.882+0000 7f63c5200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63c0068670 0x7f63c0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.882+0000 7f63c5200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.882+0000 7f63c5200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c01995e0 con 0x7f63c0069020 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.882+0000 7f63c5200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c019d380 con 0x7f63c0068670 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:07.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43624/0 (socket says 172.21.15.176:43624) 2024-03-13T14:45:07.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bd400700 1 -- 172.21.15.176:0/2092636747 learned_addr learned my addr 172.21.15.176:0/2092636747 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:07.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bde00700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63c0068670 0x7f63c0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bd400700 1 -- 172.21.15.176:0/2092636747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63c0068670 msgr2=0x7f63c0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bd400700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63c0068670 0x7f63c0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bd400700 1 -- 172.21.15.176:0/2092636747 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac0095f0 con 0x7f63c0069020 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.883+0000 7f63bd400700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0198f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0092e0 tx=0x7f63ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.884+0000 7f63b6a00700 1 -- 172.21.15.176:0/2092636747 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63ac01d070 con 0x7f63c0069020 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.884+0000 7f63b6a00700 1 -- 172.21.15.176:0/2092636747 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f63ac022470 con 0x7f63c0069020 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.884+0000 7f63b6a00700 1 -- 172.21.15.176:0/2092636747 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63ac00f690 con 0x7f63c0069020 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.884+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c019d600 con 0x7f63c0069020 2024-03-13T14:45:07.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.884+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c019daf0 con 0x7f63c0069020 2024-03-13T14:45:07.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.885+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c004ea90 con 0x7f63c0069020 2024-03-13T14:45:07.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.886+0000 7f63b6a00700 1 -- 172.21.15.176:0/2092636747 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f63ac0225e0 con 0x7f63c0069020 2024-03-13T14:45:07.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.887+0000 7f63b6a00700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f63a406c4e0 0x7f63a406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:07.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.887+0000 7f63b6a00700 1 -- 172.21.15.176:0/2092636747 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f63ac08ca40 con 0x7f63c0069020 2024-03-13T14:45:07.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.887+0000 7f63bde00700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f63a406c4e0 0x7f63a406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:07.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.888+0000 7f63bde00700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f63a406c4e0 0x7f63a406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005fd0 tx=0x7f63b000a560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:07.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:07.892+0000 7f63b6a00700 1 -- 172.21.15.176:0/2092636747 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f63ac05b080 con 0x7f63c0069020 2024-03-13T14:45:08.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.102+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63c019de70 con 0x7f63a406c4e0 2024-03-13T14:45:08.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.106+0000 7f63b6a00700 1 -- 172.21.15.176:0/2092636747 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27188 (secure 0 0 0) 0x7f63c019de70 con 0x7f63a406c4e0 2024-03-13T14:45:08.097 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:45:08.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.109+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f63a406c4e0 msgr2=0x7f63a406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:08.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.109+0000 7f63c5200700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f63a406c4e0 0x7f63a406e9a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005fd0 tx=0x7f63b000a560 comp rx=0 tx=0).stop 2024-03-13T14:45:08.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.109+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 msgr2=0x7f63c0198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:08.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.109+0000 7f63c5200700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0198f50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0092e0 tx=0x7f63ac00be20 comp rx=0 tx=0).stop 2024-03-13T14:45:08.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.110+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 shutdown_connections 2024-03-13T14:45:08.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.110+0000 7f63c5200700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f63a406c4e0 0x7f63a406e9a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:08.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.110+0000 7f63c5200700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63c0068670 0x7f63c0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:08.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.110+0000 7f63c5200700 1 --2- 172.21.15.176:0/2092636747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f63c0069020 0x7f63c0198f50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:08.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.110+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 >> 172.21.15.176:0/2092636747 conn(0x7f63c00755a0 msgr2=0x7f63c0102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:08.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.110+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 shutdown_connections 2024-03-13T14:45:08.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:08.110+0000 7f63c5200700 1 -- 172.21.15.176:0/2092636747 wait complete. 2024-03-13T14:45:08.100 INFO:teuthology.orchestra.run.smithi176.stderr:dumped all 2024-03-13T14:45:08.943 INFO:teuthology.orchestra.run.smithi176.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-03-13T14:45:08.074182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165636,"kb_used_data":3924,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305276,"statfs":{"total":575970213888,"available":575800602624,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006048"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-03-13T14:44:06.125336+0000","last_change":"2024-03-13T14:43:29.511580+0000","last_active":"2024-03-13T14:44:06.125336+0000","last_peered":"2024-03-13T14:44:06.125336+0000","last_clean":"2024-03-13T14:44:06.125336+0000","last_became_active":"2024-03-13T14:43:29.510769+0000","last_became_peered":"2024-03-13T14:43:29.510769+0000","last_unstale":"2024-03-13T14:44:06.125336+0000","last_undegraded":"2024-03-13T14:44:06.125336+0000","last_fullsized":"2024-03-13T14:44:06.125336+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-13T14:42:23.107140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-13T14:42:23.107140+0000","last_clean_scrub_stamp":"2024-03-13T14:42:23.107140+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-14T17:57:21.061981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]}]},{"osd":4,"up_from":35,"seq":150323855381,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.995}]},{"osd":1,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":1.2470000000000001}]},{"osd":2,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.96999999999999997}]},{"osd":3,"last update":"Wed Mar 13 14:44:34 2024","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":2.2069999999999999,"5min":2.2069999999999999,"15min":2.2069999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":3.2320000000000002,"5min":3.2320000000000002,"15min":3.2320000000000002},"last":1.1759999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":27,"seq":115964117021,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.92900000000000005,"15min":0.91800000000000004},"min":{"1min":0.746,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.4490000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.96399999999999997,"15min":0.96599999999999997},"min":{"1min":0.66300000000000003,"5min":0.62,"15min":0.62},"max":{"1min":1.248,"5min":1.423,"15min":1.423},"last":1.2010000000000001}]},{"osd":1,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.96699999999999997,"15min":0.95399999999999996},"min":{"1min":0.73599999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.94899999999999995,"15min":0.94799999999999995},"min":{"1min":0.70199999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.0509999999999999}]},{"osd":2,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0269999999999999,"15min":1.03},"min":{"1min":0.77500000000000002,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.319,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.0600000000000001,"15min":1.0569999999999999},"min":{"1min":0.70199999999999996,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":1.3560000000000001}]},{"osd":4,"last update":"Wed Mar 13 14:44:33 2024","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.258}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1539999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378660,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27316,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717836,"statfs":{"total":95995035648,"available":95967064064,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:27 2024","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.80400000000000005,"15min":0.78600000000000003},"min":{"1min":0.55200000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.9060000000000001,"5min":2.9060000000000001,"15min":2.9060000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.85199999999999998,"15min":0.82899999999999996},"min":{"1min":0.56100000000000005,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.9649999999999999,"5min":2.9649999999999999,"15min":2.9649999999999999},"last":0.95699999999999996}]},{"osd":1,"last update":"Wed Mar 13 14:44:27 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.86099999999999999,"15min":0.84599999999999997},"min":{"1min":0.57999999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":3.028,"5min":3.028,"15min":3.028},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.96599999999999997,"15min":0.95199999999999996},"min":{"1min":0.70699999999999996,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.98699999999999999}]},{"osd":3,"last update":"Wed Mar 13 14:44:56 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1539999999999999,"15min":1.1479999999999999},"min":{"1min":0.89000000000000001,"5min":0.871,"15min":0.871},"max":{"1min":1.5720000000000001,"5min":2.8399999999999999,"15min":2.8399999999999999},"last":1.425},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.246,"15min":1.2490000000000001},"min":{"1min":0.876,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.504,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":1.373}]},{"osd":4,"last update":"Wed Mar 13 14:44:31 2024","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":3.1030000000000002,"5min":3.1030000000000002,"15min":3.1030000000000002},"last":1.171},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"max":{"1min":2.7480000000000002,"5min":2.7480000000000002,"15min":2.7480000000000002},"last":1.1200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705714,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 13 14:44:52 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.72099999999999997,"15min":0.66700000000000004},"min":{"1min":0.51900000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.73799999999999999,"15min":0.67200000000000004},"min":{"1min":0.47299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.226,"5min":1.272,"15min":1.272},"last":0.95099999999999996}]},{"osd":2,"last update":"Wed Mar 13 14:44:26 2024","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.82699999999999996,"15min":0.80200000000000005},"min":{"1min":0.57499999999999996,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.755},{"interface":"front","average":{"1min":1.135,"5min":0.98299999999999998,"15min":0.95799999999999996},"min":{"1min":0.623,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.63400000000000001}]},{"osd":3,"last update":"Wed Mar 13 14:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.0980000000000001,"15min":1.083},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.155,"15min":1.151},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.99099999999999999}]},{"osd":4,"last update":"Wed Mar 13 14:44:30 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.121,"15min":1.121},"min":{"1min":0.877,"5min":0.877,"15min":0.877},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.91000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542188,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 13 14:44:57 2024","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.751,"15min":0.70499999999999996},"min":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.80400000000000005,"15min":0.76200000000000001},"min":{"1min":0.45900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":1.0129999999999999}]},{"osd":2,"last update":"Wed Mar 13 14:44:25 2024","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.93100000000000005,"15min":0.91300000000000003},"min":{"1min":0.72099999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.98199999999999998,"5min":1.0029999999999999,"15min":1.0069999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.2450000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0660000000000001}]},{"osd":3,"last update":"Wed Mar 13 14:44:57 2024","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.137,"15min":1.129},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.167,"15min":1.1619999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.2509999999999999}]},{"osd":4,"last update":"Wed Mar 13 14:44:33 2024","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.47,"5min":1.47,"15min":1.47},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":1.2150000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2869999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-13T14:45:08.944 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-13T14:45:08.945 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-13T14:45:08.945 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-13T14:45:08.945 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph health --format=json 2024-03-13T14:45:09.347 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:09.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:09 smithi196 ceph-mon[37486]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:09.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:09 smithi196 ceph-mon[37486]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-13T14:45:09.498 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:09 smithi176 ceph-mon[29064]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:09.498 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:09 smithi176 ceph-mon[29064]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-13T14:45:11.110 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:10 smithi176 ceph-mon[29064]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:11.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 -- 172.21.15.176:0/795833781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073130 msgr2=0x7f1ef0073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:11.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/795833781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073130 0x7f1ef0073510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009910 tx=0x7f1ee0009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:11.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 -- 172.21.15.176:0/795833781 shutdown_connections 2024-03-13T14:45:11.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/795833781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef0073a50 0x7f1ef0111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:11.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/795833781 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073130 0x7f1ef0073510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:11.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 -- 172.21.15.176:0/795833781 >> 172.21.15.176:0/795833781 conn(0x7f1ef00fc630 msgr2=0x7f1ef00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:11.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 -- 172.21.15.176:0/795833781 shutdown_connections 2024-03-13T14:45:11.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.280+0000 7f1ef5e00700 1 -- 172.21.15.176:0/795833781 wait complete. 2024-03-13T14:45:11.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.281+0000 7f1ef5e00700 1 Processor -- start 2024-03-13T14:45:11.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1ef5e00700 1 -- start start 2024-03-13T14:45:11.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1ef5e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef0073130 0x7f1ef019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:11.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1ef5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073a50 0x7f1ef019d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1ef5e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef019d870 con 0x7f1ef0073a50 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1ef5e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef01a0530 con 0x7f1ef0073130 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1eee000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073a50 0x7f1ef019d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1eee000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073a50 0x7f1ef019d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43646/0 (socket says 172.21.15.176:43646) 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.282+0000 7f1eee000700 1 -- 172.21.15.176:0/2040417190 learned_addr learned my addr 172.21.15.176:0/2040417190 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1eeea00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef0073130 0x7f1ef019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1eee000700 1 -- 172.21.15.176:0/2040417190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef0073130 msgr2=0x7f1ef019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1eee000700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef0073130 0x7f1ef019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1eee000700 1 -- 172.21.15.176:0/2040417190 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00095f0 con 0x7f1ef0073a50 2024-03-13T14:45:11.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1eee000700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073a50 0x7f1ef019d1e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00d8d0 tx=0x7f1edc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:11.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1ef4a00700 1 -- 172.21.15.176:0/2040417190 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc009940 con 0x7f1ef0073a50 2024-03-13T14:45:11.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1ef4a00700 1 -- 172.21.15.176:0/2040417190 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1edc010460 con 0x7f1ef0073a50 2024-03-13T14:45:11.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.283+0000 7f1ef4a00700 1 -- 172.21.15.176:0/2040417190 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc00f5f0 con 0x7f1ef0073a50 2024-03-13T14:45:11.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.284+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef01a0810 con 0x7f1ef0073a50 2024-03-13T14:45:11.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.284+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef01a0d60 con 0x7f1ef0073a50 2024-03-13T14:45:11.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.285+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef010ed90 con 0x7f1ef0073a50 2024-03-13T14:45:11.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.286+0000 7f1ef4a00700 1 -- 172.21.15.176:0/2040417190 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1edc0105d0 con 0x7f1ef0073a50 2024-03-13T14:45:11.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.287+0000 7f1ef4a00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1ed406c4e0 0x7f1ed406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:11.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.287+0000 7f1ef4a00700 1 -- 172.21.15.176:0/2040417190 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1edc08b490 con 0x7f1ef0073a50 2024-03-13T14:45:11.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.288+0000 7f1eeea00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1ed406c4e0 0x7f1ed406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:11.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.289+0000 7f1eeea00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1ed406c4e0 0x7f1ed406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006010 tx=0x7f1ee001a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:11.281 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.292+0000 7f1ef4a00700 1 -- 172.21.15.176:0/2040417190 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1edc055f90 con 0x7f1ef0073a50 2024-03-13T14:45:11.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:10 smithi196 ceph-mon[37486]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:11.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.564+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1ef004ea90 con 0x7f1ef0073a50 2024-03-13T14:45:11.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.564+0000 7f1ef4a00700 1 -- 172.21.15.176:0/2040417190 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1edc08b020 con 0x7f1ef0073a50 2024-03-13T14:45:11.554 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:45:11.554 INFO:teuthology.orchestra.run.smithi176.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-13T14:45:11.556 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1ed406c4e0 msgr2=0x7f1ed406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:11.556 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1ed406c4e0 0x7f1ed406e9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006010 tx=0x7f1ee001a040 comp rx=0 tx=0).stop 2024-03-13T14:45:11.556 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073a50 msgr2=0x7f1ef019d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:11.556 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073a50 0x7f1ef019d1e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1edc00d8d0 tx=0x7f1edc00dc90 comp rx=0 tx=0).stop 2024-03-13T14:45:11.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 shutdown_connections 2024-03-13T14:45:11.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1ed406c4e0 0x7f1ed406e9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:11.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ef0073130 0x7f1ef019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:11.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 --2- 172.21.15.176:0/2040417190 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ef0073a50 0x7f1ef019d1e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:11.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 >> 172.21.15.176:0/2040417190 conn(0x7f1ef00fc630 msgr2=0x7f1ef0103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:11.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 shutdown_connections 2024-03-13T14:45:11.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:11.567+0000 7f1ef5e00700 1 -- 172.21.15.176:0/2040417190 wait complete. 2024-03-13T14:45:12.246 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:12.246 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:11 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2040417190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-13T14:45:12.296 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-13T14:45:12.296 INFO:tasks.cephadm:Setup complete, yielding 2024-03-13T14:45:12.297 INFO:teuthology.run_tasks:Running task print... 2024-03-13T14:45:12.314 INFO:teuthology.task.print:**** done end installing v18.2.1 cephadm ... 2024-03-13T14:45:12.314 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T14:45:12.348 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:45:12.348 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-03-13T14:45:12.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:12.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:11 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2040417190' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-13T14:45:12.724 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:13.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:12 smithi176 ceph-mon[29064]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:13.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:13.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:45:13.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:13.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:12 smithi196 ceph-mon[37486]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:13.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:13.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:45:13.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:14.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:14.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.893+0000 7ffb28e00700 1 -- 172.21.15.176:0/1825336960 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24107200 msgr2=0x7ffb24107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.893+0000 7ffb28e00700 1 --2- 172.21.15.176:0/1825336960 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24107200 0x7ffb24107620 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ffb14009910 tx=0x7ffb14009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.894+0000 7ffb28e00700 1 -- 172.21.15.176:0/1825336960 shutdown_connections 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.894+0000 7ffb28e00700 1 --2- 172.21.15.176:0/1825336960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb24108400 0x7ffb24108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.894+0000 7ffb28e00700 1 --2- 172.21.15.176:0/1825336960 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24107200 0x7ffb24107620 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.894+0000 7ffb28e00700 1 -- 172.21.15.176:0/1825336960 >> 172.21.15.176:0/1825336960 conn(0x7ffb240761b0 msgr2=0x7ffb24078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.894+0000 7ffb28e00700 1 -- 172.21.15.176:0/1825336960 shutdown_connections 2024-03-13T14:45:14.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.894+0000 7ffb28e00700 1 -- 172.21.15.176:0/1825336960 wait complete. 2024-03-13T14:45:14.886 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.895+0000 7ffb28e00700 1 Processor -- start 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.895+0000 7ffb28e00700 1 -- start start 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.895+0000 7ffb28e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb24107200 0x7ffb2419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.895+0000 7ffb28e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24108400 0x7ffb2419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.895+0000 7ffb28e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2419d510 con 0x7ffb24108400 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.895+0000 7ffb28e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2419d650 con 0x7ffb24107200 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb19400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24108400 0x7ffb2419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb19400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24108400 0x7ffb2419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60376/0 (socket says 172.21.15.176:60376) 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb19400700 1 -- 172.21.15.176:0/3294280021 learned_addr learned my addr 172.21.15.176:0/3294280021 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb21e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb24107200 0x7ffb2419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb19400700 1 -- 172.21.15.176:0/3294280021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb24107200 msgr2=0x7ffb2419c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb19400700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb24107200 0x7ffb2419c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb19400700 1 -- 172.21.15.176:0/3294280021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140095f0 con 0x7ffb24108400 2024-03-13T14:45:14.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.896+0000 7ffb19400700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24108400 0x7ffb2419cef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d8d0 tx=0x7ffb0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:14.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.897+0000 7ffb1b400700 1 -- 172.21.15.176:0/3294280021 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb0c009940 con 0x7ffb24108400 2024-03-13T14:45:14.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.897+0000 7ffb1b400700 1 -- 172.21.15.176:0/3294280021 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c010460 con 0x7ffb24108400 2024-03-13T14:45:14.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.897+0000 7ffb1b400700 1 -- 172.21.15.176:0/3294280021 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb0c00f5f0 con 0x7ffb24108400 2024-03-13T14:45:14.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.897+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb241a2100 con 0x7ffb24108400 2024-03-13T14:45:14.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.897+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb241a2650 con 0x7ffb24108400 2024-03-13T14:45:14.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.898+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2410fd60 con 0x7ffb24108400 2024-03-13T14:45:14.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.899+0000 7ffb1b400700 1 -- 172.21.15.176:0/3294280021 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ffb0c0105d0 con 0x7ffb24108400 2024-03-13T14:45:14.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.900+0000 7ffb1b400700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb0406c4e0 0x7ffb0406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:14.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.900+0000 7ffb1b400700 1 -- 172.21.15.176:0/3294280021 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ffb0c08b3f0 con 0x7ffb24108400 2024-03-13T14:45:14.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.900+0000 7ffb21e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb0406c4e0 0x7ffb0406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:14.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.901+0000 7ffb21e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb0406c4e0 0x7ffb0406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb14006010 tx=0x7ffb1400b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:14.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:14.905+0000 7ffb1b400700 1 -- 172.21.15.176:0/3294280021 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffb0c059a20 con 0x7ffb24108400 2024-03-13T14:45:15.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.122+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7ffb2410ff20 con 0x7ffb24108400 2024-03-13T14:45:15.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.130+0000 7ffb1b400700 1 -- 172.21.15.176:0/3294280021 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7ffb0c08b070 con 0x7ffb24108400 2024-03-13T14:45:15.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb0406c4e0 msgr2=0x7ffb0406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:15.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb0406c4e0 0x7ffb0406e9a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb14006010 tx=0x7ffb1400b540 comp rx=0 tx=0).stop 2024-03-13T14:45:15.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24108400 msgr2=0x7ffb2419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:15.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24108400 0x7ffb2419cef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d8d0 tx=0x7ffb0c00dc90 comp rx=0 tx=0).stop 2024-03-13T14:45:15.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 shutdown_connections 2024-03-13T14:45:15.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb0406c4e0 0x7ffb0406e9a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:15.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb24107200 0x7ffb2419c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:15.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 --2- 172.21.15.176:0/3294280021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb24108400 0x7ffb2419cef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:15.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 >> 172.21.15.176:0/3294280021 conn(0x7ffb240761b0 msgr2=0x7ffb2410b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:15.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 shutdown_connections 2024-03-13T14:45:15.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:15.134+0000 7ffb28e00700 1 -- 172.21.15.176:0/3294280021 wait complete. 2024-03-13T14:45:15.248 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:14 smithi176 ceph-mon[29064]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:14 smithi196 ceph-mon[37486]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:15.984 INFO:teuthology.run_tasks:Running task print... 2024-03-13T14:45:15.993 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-03-13T14:45:15.993 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T14:45:16.059 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:45:16.060 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph orch status' 2024-03-13T14:45:16.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:16 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3294280021' entity='client.admin' 2024-03-13T14:45:16.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:16.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:16.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:45:16.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:16 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3294280021' entity='client.admin' 2024-03-13T14:45:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:45:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:16.475 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:17.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:17 smithi196 ceph-mon[37486]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:17.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:17 smithi176 ceph-mon[29064]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:18.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.538+0000 7f0e8b200700 1 -- 172.21.15.176:0/1591055001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e84074dc0 msgr2=0x7f0e84073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:18.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.538+0000 7f0e8b200700 1 --2- 172.21.15.176:0/1591055001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e84074dc0 0x7f0e84073220 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009910 tx=0x7f0e74009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:18.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.539+0000 7f0e8b200700 1 -- 172.21.15.176:0/1591055001 shutdown_connections 2024-03-13T14:45:18.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.539+0000 7f0e8b200700 1 --2- 172.21.15.176:0/1591055001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e840737f0 0x7f0e84073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:18.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.539+0000 7f0e8b200700 1 --2- 172.21.15.176:0/1591055001 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e84074dc0 0x7f0e84073220 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:18.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.539+0000 7f0e8b200700 1 -- 172.21.15.176:0/1591055001 >> 172.21.15.176:0/1591055001 conn(0x7f0e840fc110 msgr2=0x7f0e840fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:18.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.539+0000 7f0e8b200700 1 -- 172.21.15.176:0/1591055001 shutdown_connections 2024-03-13T14:45:18.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.539+0000 7f0e8b200700 1 -- 172.21.15.176:0/1591055001 wait complete. 2024-03-13T14:45:18.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.540+0000 7f0e8b200700 1 Processor -- start 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.540+0000 7f0e8b200700 1 -- start start 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.540+0000 7f0e8b200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e840737f0 0x7f0e84198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e8b200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84074dc0 0x7f0e84198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e8b200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e841990d0 con 0x7f0e840737f0 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e8b200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e84199210 con 0x7f0e84074dc0 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e83e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e840737f0 0x7f0e84198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e83e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e840737f0 0x7f0e84198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60402/0 (socket says 172.21.15.176:60402) 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e83e00700 1 -- 172.21.15.176:0/833334612 learned_addr learned my addr 172.21.15.176:0/833334612 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e83400700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84074dc0 0x7f0e84198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e83e00700 1 -- 172.21.15.176:0/833334612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84074dc0 msgr2=0x7f0e84198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e83e00700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84074dc0 0x7f0e84198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.541+0000 7f0e83e00700 1 -- 172.21.15.176:0/833334612 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e740095f0 con 0x7f0e840737f0 2024-03-13T14:45:18.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.542+0000 7f0e83e00700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e840737f0 0x7f0e84198570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400bda0 tx=0x7f0e7400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:18.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.542+0000 7f0e80c00700 1 -- 172.21.15.176:0/833334612 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e7401d070 con 0x7f0e840737f0 2024-03-13T14:45:18.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.542+0000 7f0e80c00700 1 -- 172.21.15.176:0/833334612 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e7400f460 con 0x7f0e840737f0 2024-03-13T14:45:18.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.542+0000 7f0e80c00700 1 -- 172.21.15.176:0/833334612 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e74021620 con 0x7f0e840737f0 2024-03-13T14:45:18.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.542+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8419dc60 con 0x7f0e840737f0 2024-03-13T14:45:18.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.542+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8419e150 con 0x7f0e840737f0 2024-03-13T14:45:18.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.543+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8410b920 con 0x7f0e840737f0 2024-03-13T14:45:18.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.544+0000 7f0e80c00700 1 -- 172.21.15.176:0/833334612 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0e7400f5d0 con 0x7f0e840737f0 2024-03-13T14:45:18.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.545+0000 7f0e80c00700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e6806c4e0 0x7f0e6806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:18.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.545+0000 7f0e80c00700 1 -- 172.21.15.176:0/833334612 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0e7408c8a0 con 0x7f0e840737f0 2024-03-13T14:45:18.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.545+0000 7f0e83400700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e6806c4e0 0x7f0e6806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:18.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.546+0000 7f0e83400700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e6806c4e0 0x7f0e6806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0e70005950 tx=0x7f0e700058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:18.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.550+0000 7f0e80c00700 1 -- 172.21.15.176:0/833334612 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0e7405adb0 con 0x7f0e840737f0 2024-03-13T14:45:18.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.782+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e841080b0 con 0x7f0e6806c4e0 2024-03-13T14:45:18.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.784+0000 7f0e80c00700 1 -- 172.21.15.176:0/833334612 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0e841080b0 con 0x7f0e6806c4e0 2024-03-13T14:45:18.774 INFO:teuthology.orchestra.run.smithi176.stdout:Backend: cephadm 2024-03-13T14:45:18.774 INFO:teuthology.orchestra.run.smithi176.stdout:Available: Yes 2024-03-13T14:45:18.774 INFO:teuthology.orchestra.run.smithi176.stdout:Paused: No 2024-03-13T14:45:18.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e6806c4e0 msgr2=0x7f0e6806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e6806c4e0 0x7f0e6806e9a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0e70005950 tx=0x7f0e700058e0 comp rx=0 tx=0).stop 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e840737f0 msgr2=0x7f0e84198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e840737f0 0x7f0e84198570 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400bda0 tx=0x7f0e7400be80 comp rx=0 tx=0).stop 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 shutdown_connections 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e6806c4e0 0x7f0e6806e9a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e840737f0 0x7f0e84198570 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 --2- 172.21.15.176:0/833334612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e84074dc0 0x7f0e84198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 >> 172.21.15.176:0/833334612 conn(0x7f0e840fc110 msgr2=0x7f0e84106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 shutdown_connections 2024-03-13T14:45:18.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:18.787+0000 7f0e8b200700 1 -- 172.21.15.176:0/833334612 wait complete. 2024-03-13T14:45:19.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:19 smithi196 ceph-mon[37486]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:19.623 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:19 smithi176 ceph-mon[29064]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:19.677 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-13T14:45:20.073 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:20.422 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:20 smithi176 ceph-mon[29064]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:20.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:20 smithi196 ceph-mon[37486]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:21.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:21 smithi176 ceph-mon[29064]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:21.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:21 smithi196 ceph-mon[37486]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:22.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 -- 172.21.15.176:0/1513489578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50073790 msgr2=0x7ffb50073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:22.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 --2- 172.21.15.176:0/1513489578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50073790 0x7ffb50073c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009910 tx=0x7ffb38009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:22.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 -- 172.21.15.176:0/1513489578 shutdown_connections 2024-03-13T14:45:22.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 --2- 172.21.15.176:0/1513489578 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50073790 0x7ffb50073c10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:22.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 --2- 172.21.15.176:0/1513489578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb50074d60 0x7ffb500731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:22.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 -- 172.21.15.176:0/1513489578 >> 172.21.15.176:0/1513489578 conn(0x7ffb500fc140 msgr2=0x7ffb500fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:22.185 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 -- 172.21.15.176:0/1513489578 shutdown_connections 2024-03-13T14:45:22.185 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.194+0000 7ffb54e00700 1 -- 172.21.15.176:0/1513489578 wait complete. 2024-03-13T14:45:22.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.195+0000 7ffb54e00700 1 Processor -- start 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.195+0000 7ffb54e00700 1 -- start start 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb54e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb50073790 0x7ffb50198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb54e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50074d60 0x7ffb50198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb54e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501990f0 con 0x7ffb50074d60 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb54e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50199230 con 0x7ffb50073790 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb47e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50074d60 0x7ffb50198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb47e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50074d60 0x7ffb50198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44420/0 (socket says 172.21.15.176:44420) 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb47e00700 1 -- 172.21.15.176:0/4105680204 learned_addr learned my addr 172.21.15.176:0/4105680204 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.196+0000 7ffb4de00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb50073790 0x7ffb50198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb47e00700 1 -- 172.21.15.176:0/4105680204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb50073790 msgr2=0x7ffb50198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb47e00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb50073790 0x7ffb50198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb47e00700 1 -- 172.21.15.176:0/4105680204 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380095f0 con 0x7ffb50074d60 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb4de00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb50073790 0x7ffb50198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb47e00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50074d60 0x7ffb50198ad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005b40 tx=0x7ffb3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:22.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb46a00700 1 -- 172.21.15.176:0/4105680204 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3801d070 con 0x7ffb50074d60 2024-03-13T14:45:22.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb46a00700 1 -- 172.21.15.176:0/4105680204 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb38022470 con 0x7ffb50074d60 2024-03-13T14:45:22.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5019dc80 con 0x7ffb50074d60 2024-03-13T14:45:22.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb46a00700 1 -- 172.21.15.176:0/4105680204 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3800f690 con 0x7ffb50074d60 2024-03-13T14:45:22.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.197+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5019e1f0 con 0x7ffb50074d60 2024-03-13T14:45:22.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.198+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5010b990 con 0x7ffb50074d60 2024-03-13T14:45:22.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.199+0000 7ffb46a00700 1 -- 172.21.15.176:0/4105680204 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ffb380225e0 con 0x7ffb50074d60 2024-03-13T14:45:22.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.200+0000 7ffb46a00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:22.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.200+0000 7ffb46a00700 1 -- 172.21.15.176:0/4105680204 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ffb3808cdf0 con 0x7ffb50074d60 2024-03-13T14:45:22.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.201+0000 7ffb4de00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:22.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.201+0000 7ffb4de00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb40007900 tx=0x7ffb40008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:22.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.205+0000 7ffb46a00700 1 -- 172.21.15.176:0/4105680204 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ffb3805b4a0 con 0x7ffb50074d60 2024-03-13T14:45:22.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.432+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb50108120 con 0x7ffb3406c4e0 2024-03-13T14:45:22.447 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.457+0000 7ffb46a00700 1 -- 172.21.15.176:0/4105680204 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7ffb50108120 con 0x7ffb3406c4e0 2024-03-13T14:45:22.447 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (4m) 2m ago 6m 14.1M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (7m) 2m ago 7m 7088k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (5m) 77s ago 5m 7004k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (7m) 2m ago 7m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (5m) 77s ago 5m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (4m) 2m ago 6m 73.0M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:9283,8765,8443 running (8m) 2m ago 8m 482M - 18.2.1 5be31c24972a c3fcbaf472c1 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (5m) 77s ago 5m 440M - 18.2.1 5be31c24972a 4af30d3028b6 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (8m) 2m ago 8m 45.3M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (5m) 77s ago 5m 33.0M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (7m) 2m ago 7m 7465k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (5m) 77s ago 5m 7302k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:45:22.448 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (4m) 2m ago 4m 40.0M 3947M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:45:22.449 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (3m) 2m ago 3m 39.7M 3947M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:45:22.449 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (3m) 2m ago 3m 39.8M 3947M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:45:22.449 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (2m) 77s ago 2m 41.2M 4971M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:45:22.449 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (119s) 77s ago 119s 37.7M 4971M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:45:22.449 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (82s) 77s ago 82s 11.0M 4971M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:45:22.449 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (4m) 2m ago 6m 37.6M - 2.43.0 a07b618ecd1d 4ec38b851bbe 2024-03-13T14:45:22.451 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.461+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb3406c4e0 msgr2=0x7ffb3406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:22.451 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.461+0000 7ffb54e00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffb40007900 tx=0x7ffb40008040 comp rx=0 tx=0).stop 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.461+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50074d60 msgr2=0x7ffb50198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.461+0000 7ffb54e00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50074d60 0x7ffb50198ad0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffb38005b40 tx=0x7ffb3800be20 comp rx=0 tx=0).stop 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.462+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 shutdown_connections 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.462+0000 7ffb54e00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ffb3406c4e0 0x7ffb3406e9a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.462+0000 7ffb54e00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb50073790 0x7ffb50198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.462+0000 7ffb54e00700 1 --2- 172.21.15.176:0/4105680204 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb50074d60 0x7ffb50198ad0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.462+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 >> 172.21.15.176:0/4105680204 conn(0x7ffb500fc140 msgr2=0x7ffb50106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.462+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 shutdown_connections 2024-03-13T14:45:22.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:22.462+0000 7ffb54e00700 1 -- 172.21.15.176:0/4105680204 wait complete. 2024-03-13T14:45:23.230 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-13T14:45:23.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:23 smithi196 ceph-mon[37486]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:23.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:23 smithi196 ceph-mon[37486]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:23.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:23 smithi176 ceph-mon[29064]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:23.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:23 smithi176 ceph-mon[29064]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:23.637 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:25.489 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:25 smithi176 ceph-mon[29064]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:25.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:25 smithi196 ceph-mon[37486]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:25.784 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.794+0000 7f04f7600700 1 -- 172.21.15.176:0/43367207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f0100cb0 msgr2=0x7f04f0101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:25.784 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.794+0000 7f04f7600700 1 --2- 172.21.15.176:0/43367207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f0100cb0 0x7f04f0101110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f04dc009910 tx=0x7f04dc009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:25.784 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.794+0000 7f04f7600700 1 -- 172.21.15.176:0/43367207 shutdown_connections 2024-03-13T14:45:25.784 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.794+0000 7f04f7600700 1 --2- 172.21.15.176:0/43367207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f0100cb0 0x7f04f0101110 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:25.784 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.794+0000 7f04f7600700 1 --2- 172.21.15.176:0/43367207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04f00ffab0 0x7f04f00ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:25.784 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.794+0000 7f04f7600700 1 -- 172.21.15.176:0/43367207 >> 172.21.15.176:0/43367207 conn(0x7f04f00fb030 msgr2=0x7f04f00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:25.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.794+0000 7f04f7600700 1 -- 172.21.15.176:0/43367207 shutdown_connections 2024-03-13T14:45:25.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.795+0000 7f04f7600700 1 -- 172.21.15.176:0/43367207 wait complete. 2024-03-13T14:45:25.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f7600700 1 Processor -- start 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f7600700 1 -- start start 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f7600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f00ffab0 0x7f04f0109410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f7600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04f0100cb0 0x7f04f0107a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f7600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f0107fa0 con 0x7f04f00ffab0 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f7600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04f01080e0 con 0x7f04f0100cb0 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f4c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f00ffab0 0x7f04f0109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f4c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f00ffab0 0x7f04f0109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44424/0 (socket says 172.21.15.176:44424) 2024-03-13T14:45:25.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.796+0000 7f04f4c00700 1 -- 172.21.15.176:0/1947172942 learned_addr learned my addr 172.21.15.176:0/1947172942 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04efe00700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04f0100cb0 0x7f04f0107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04f4c00700 1 -- 172.21.15.176:0/1947172942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04f0100cb0 msgr2=0x7f04f0107a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04f4c00700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04f0100cb0 0x7f04f0107a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04f4c00700 1 -- 172.21.15.176:0/1947172942 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc0095f0 con 0x7f04f00ffab0 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04efe00700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04f0100cb0 0x7f04f0107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04f4c00700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f00ffab0 0x7f04f0109410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04e400d900 tx=0x7f04e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04ed600700 1 -- 172.21.15.176:0/1947172942 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e40041f0 con 0x7f04f00ffab0 2024-03-13T14:45:25.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04f01083c0 con 0x7f04f00ffab0 2024-03-13T14:45:25.788 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04ed600700 1 -- 172.21.15.176:0/1947172942 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f04e4004350 con 0x7f04f00ffab0 2024-03-13T14:45:25.788 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04ed600700 1 -- 172.21.15.176:0/1947172942 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04e4003da0 con 0x7f04f00ffab0 2024-03-13T14:45:25.788 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.797+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04f019e3c0 con 0x7f04f00ffab0 2024-03-13T14:45:25.788 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.799+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04f004ea90 con 0x7f04f00ffab0 2024-03-13T14:45:25.788 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.799+0000 7f04ed600700 1 -- 172.21.15.176:0/1947172942 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f04e4009730 con 0x7f04f00ffab0 2024-03-13T14:45:25.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.800+0000 7f04ed600700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f04d806c4e0 0x7f04d806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:25.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.800+0000 7f04ed600700 1 -- 172.21.15.176:0/1947172942 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f04e408af90 con 0x7f04f00ffab0 2024-03-13T14:45:25.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.801+0000 7f04efe00700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f04d806c4e0 0x7f04d806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:25.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.801+0000 7f04efe00700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f04d806c4e0 0x7f04d806e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04dc006010 tx=0x7f04dc00b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:25.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:25.805+0000 7f04ed600700 1 -- 172.21.15.176:0/1947172942 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f04e4059660 con 0x7f04f00ffab0 2024-03-13T14:45:26.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.024+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f04f0105600 con 0x7f04d806c4e0 2024-03-13T14:45:26.026 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.036+0000 7f04ed600700 1 -- 172.21.15.176:0/1947172942 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7f04f0105600 con 0x7f04d806c4e0 2024-03-13T14:45:26.026 INFO:teuthology.orchestra.run.smithi176.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-13T14:45:26.026 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager ?:9093,9094 1/1 2m ago 7m count:1 2024-03-13T14:45:26.026 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter 2/2 2m ago 7m * 2024-03-13T14:45:26.026 INFO:teuthology.orchestra.run.smithi176.stdout:crash 2/2 2m ago 8m * 2024-03-13T14:45:26.026 INFO:teuthology.orchestra.run.smithi176.stdout:grafana ?:3000 1/1 2m ago 7m count:1 2024-03-13T14:45:26.027 INFO:teuthology.orchestra.run.smithi176.stdout:mgr 2/2 2m ago 8m count:2 2024-03-13T14:45:26.027 INFO:teuthology.orchestra.run.smithi176.stdout:mon 2/2 2m ago 6m smithi176:172.21.15.176=smithi176;smithi196:172.21.15.196=smithi196;count:2 2024-03-13T14:45:26.027 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter ?:9100 2/2 2m ago 7m * 2024-03-13T14:45:26.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd 6 2m ago - 2024-03-13T14:45:26.027 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus ?:9095 1/1 2m ago 7m count:1 2024-03-13T14:45:26.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.039+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f04d806c4e0 msgr2=0x7f04d806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:26.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.039+0000 7f04f7600700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f04d806c4e0 0x7f04d806e9a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f04dc006010 tx=0x7f04dc00b540 comp rx=0 tx=0).stop 2024-03-13T14:45:26.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.039+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f00ffab0 msgr2=0x7f04f0109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:26.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.039+0000 7f04f7600700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f00ffab0 0x7f04f0109410 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f04e400d900 tx=0x7f04e400dcc0 comp rx=0 tx=0).stop 2024-03-13T14:45:26.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.040+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 shutdown_connections 2024-03-13T14:45:26.030 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.040+0000 7f04f7600700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f04d806c4e0 0x7f04d806e9a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:26.030 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.040+0000 7f04f7600700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f04f00ffab0 0x7f04f0109410 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:26.030 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.040+0000 7f04f7600700 1 --2- 172.21.15.176:0/1947172942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04f0100cb0 0x7f04f0107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:26.030 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.040+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 >> 172.21.15.176:0/1947172942 conn(0x7f04f00fb030 msgr2=0x7f04f0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:26.030 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.040+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 shutdown_connections 2024-03-13T14:45:26.030 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:26.040+0000 7f04f7600700 1 -- 172.21.15.176:0/1947172942 wait complete. 2024-03-13T14:45:26.819 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-13T14:45:27.249 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:27.504 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:27 smithi196 ceph-mon[37486]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:27.504 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:27 smithi196 ceph-mon[37486]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:27.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:27 smithi176 ceph-mon[29064]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:27.538 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:27 smithi176 ceph-mon[29064]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:28.427 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:28.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:29.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 -- 172.21.15.176:0/1366169560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 msgr2=0x7f0e1c073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:29.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 --2- 172.21.15.176:0/1366169560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c073220 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009910 tx=0x7f0e0c009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:29.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 -- 172.21.15.176:0/1366169560 shutdown_connections 2024-03-13T14:45:29.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 --2- 172.21.15.176:0/1366169560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c0737f0 0x7f0e1c073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:29.370 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 --2- 172.21.15.176:0/1366169560 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c073220 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:29.370 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 -- 172.21.15.176:0/1366169560 >> 172.21.15.176:0/1366169560 conn(0x7f0e1c0fc140 msgr2=0x7f0e1c0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:29.370 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 -- 172.21.15.176:0/1366169560 shutdown_connections 2024-03-13T14:45:29.370 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.379+0000 7f0e21000700 1 -- 172.21.15.176:0/1366169560 wait complete. 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.380+0000 7f0e21000700 1 Processor -- start 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.381+0000 7f0e21000700 1 -- start start 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.381+0000 7f0e21000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c0737f0 0x7f0e1c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.381+0000 7f0e21000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.381+0000 7f0e21000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1990e0 con 0x7f0e1c074dc0 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.381+0000 7f0e21000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c199220 con 0x7f0e1c0737f0 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.381+0000 7f0e19400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.382+0000 7f0e19400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44452/0 (socket says 172.21.15.176:44452) 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.382+0000 7f0e19400700 1 -- 172.21.15.176:0/98002864 learned_addr learned my addr 172.21.15.176:0/98002864 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:29.371 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.382+0000 7f0e19e00700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c0737f0 0x7f0e1c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:29.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.382+0000 7f0e19400700 1 -- 172.21.15.176:0/98002864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c0737f0 msgr2=0x7f0e1c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:29.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.382+0000 7f0e19400700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c0737f0 0x7f0e1c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:29.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.382+0000 7f0e19400700 1 -- 172.21.15.176:0/98002864 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0095f0 con 0x7f0e1c074dc0 2024-03-13T14:45:29.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.382+0000 7f0e19400700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c198ac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d900 tx=0x7f0e0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:29.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.383+0000 7f0e12a00700 1 -- 172.21.15.176:0/98002864 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e080041f0 con 0x7f0e1c074dc0 2024-03-13T14:45:29.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.383+0000 7f0e12a00700 1 -- 172.21.15.176:0/98002864 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e08004350 con 0x7f0e1c074dc0 2024-03-13T14:45:29.373 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.383+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c19dcd0 con 0x7f0e1c074dc0 2024-03-13T14:45:29.373 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.383+0000 7f0e12a00700 1 -- 172.21.15.176:0/98002864 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e08003da0 con 0x7f0e1c074dc0 2024-03-13T14:45:29.373 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.383+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c19e220 con 0x7f0e1c074dc0 2024-03-13T14:45:29.373 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.384+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c10b930 con 0x7f0e1c074dc0 2024-03-13T14:45:29.374 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.385+0000 7f0e12a00700 1 -- 172.21.15.176:0/98002864 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f0e08009730 con 0x7f0e1c074dc0 2024-03-13T14:45:29.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.386+0000 7f0e12a00700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e0006c4e0 0x7f0e0006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:29.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.386+0000 7f0e12a00700 1 -- 172.21.15.176:0/98002864 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0e0808b5d0 con 0x7f0e1c074dc0 2024-03-13T14:45:29.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.386+0000 7f0e19e00700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e0006c4e0 0x7f0e0006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:29.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.387+0000 7f0e19e00700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e0006c4e0 0x7f0e0006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c00b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:29.380 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.391+0000 7f0e12a00700 1 -- 172.21.15.176:0/98002864 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f0e08059c00 con 0x7f0e1c074dc0 2024-03-13T14:45:29.582 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:29 smithi196 ceph-mon[37486]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:29.600 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.611+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e1c1080c0 con 0x7f0e0006c4e0 2024-03-13T14:45:29.600 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:29 smithi176 ceph-mon[29064]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:29.602 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.613+0000 7f0e12a00700 1 -- 172.21.15.176:0/98002864 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f0e1c1080c0 con 0x7f0e0006c4e0 2024-03-13T14:45:29.602 INFO:teuthology.orchestra.run.smithi176.stdout:HOST ADDR LABELS STATUS 2024-03-13T14:45:29.603 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176 172.21.15.176 2024-03-13T14:45:29.603 INFO:teuthology.orchestra.run.smithi176.stdout:smithi196 172.21.15.196 2024-03-13T14:45:29.603 INFO:teuthology.orchestra.run.smithi176.stdout:2 hosts in cluster 2024-03-13T14:45:29.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e0006c4e0 msgr2=0x7f0e0006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:29.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e0006c4e0 0x7f0e0006e9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c006010 tx=0x7f0e0c00b540 comp rx=0 tx=0).stop 2024-03-13T14:45:29.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 msgr2=0x7f0e1c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:29.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c198ac0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d900 tx=0x7f0e0800dcc0 comp rx=0 tx=0).stop 2024-03-13T14:45:29.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 shutdown_connections 2024-03-13T14:45:29.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f0e0006c4e0 0x7f0e0006e9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:29.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e1c0737f0 0x7f0e1c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:29.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 --2- 172.21.15.176:0/98002864 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e1c074dc0 0x7f0e1c198ac0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:29.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 >> 172.21.15.176:0/98002864 conn(0x7f0e1c0fc140 msgr2=0x7f0e1c1069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:29.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.616+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 shutdown_connections 2024-03-13T14:45:29.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:29.617+0000 7f0e21000700 1 -- 172.21.15.176:0/98002864 wait complete. 2024-03-13T14:45:30.399 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-13T14:45:30.483 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:30 smithi176 ceph-mon[29064]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:30.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:30 smithi196 ceph-mon[37486]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:30.811 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:31.251 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:31 smithi176 ceph-mon[29064]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:31.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:31 smithi196 ceph-mon[37486]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 -- 172.21.15.176:0/908166469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0074d40 msgr2=0x7fada00731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 --2- 172.21.15.176:0/908166469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0074d40 0x7fada00731a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fad90009910 tx=0x7fad90009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 -- 172.21.15.176:0/908166469 shutdown_connections 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 --2- 172.21.15.176:0/908166469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0073770 0x7fada0073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 --2- 172.21.15.176:0/908166469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0074d40 0x7fada00731a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 -- 172.21.15.176:0/908166469 >> 172.21.15.176:0/908166469 conn(0x7fada00fc140 msgr2=0x7fada00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 -- 172.21.15.176:0/908166469 shutdown_connections 2024-03-13T14:45:32.965 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.975+0000 7fada7200700 1 -- 172.21.15.176:0/908166469 wait complete. 2024-03-13T14:45:32.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.976+0000 7fada7200700 1 Processor -- start 2024-03-13T14:45:32.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fada7200700 1 -- start start 2024-03-13T14:45:32.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fada7200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0073770 0x7fada0198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:32.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fada7200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0074d40 0x7fada0198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:32.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fada7200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0199090 con 0x7fada0073770 2024-03-13T14:45:32.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fada7200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01991d0 con 0x7fada0074d40 2024-03-13T14:45:32.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fad9fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0073770 0x7fada0198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fad9fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0073770 0x7fada0198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53366/0 (socket says 172.21.15.176:53366) 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fad9fe00700 1 -- 172.21.15.176:0/4041217885 learned_addr learned my addr 172.21.15.176:0/4041217885 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.977+0000 7fad9f400700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0074d40 0x7fada0198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fad9fe00700 1 -- 172.21.15.176:0/4041217885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0074d40 msgr2=0x7fada0198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fad9fe00700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0074d40 0x7fada0198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fad9fe00700 1 -- 172.21.15.176:0/4041217885 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad900095f0 con 0x7fada0073770 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fad9fe00700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0073770 0x7fada0198530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b5c0 tx=0x7fad9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fad9cc00700 1 -- 172.21.15.176:0/4041217885 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad9001d070 con 0x7fada0073770 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fad9cc00700 1 -- 172.21.15.176:0/4041217885 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad9000f460 con 0x7fada0073770 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada019dc20 con 0x7fada0073770 2024-03-13T14:45:32.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.978+0000 7fad9cc00700 1 -- 172.21.15.176:0/4041217885 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad90021640 con 0x7fada0073770 2024-03-13T14:45:32.969 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.979+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada019e110 con 0x7fada0073770 2024-03-13T14:45:32.969 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.980+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada010b910 con 0x7fada0073770 2024-03-13T14:45:32.970 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.981+0000 7fad9cc00700 1 -- 172.21.15.176:0/4041217885 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fad9000f5d0 con 0x7fada0073770 2024-03-13T14:45:32.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.982+0000 7fad9cc00700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad8406c4e0 0x7fad8406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:32.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.982+0000 7fad9cc00700 1 -- 172.21.15.176:0/4041217885 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fad9008c8d0 con 0x7fada0073770 2024-03-13T14:45:32.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.982+0000 7fad9f400700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad8406c4e0 0x7fad8406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:32.972 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.983+0000 7fad9f400700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad8406c4e0 0x7fad8406e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005950 tx=0x7fad8c0058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:32.976 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:32.987+0000 7fad9cc00700 1 -- 172.21.15.176:0/4041217885 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fad9005ae60 con 0x7fada0073770 2024-03-13T14:45:33.198 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.208+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fada0108140 con 0x7fad8406c4e0 2024-03-13T14:45:33.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.211+0000 7fad9cc00700 1 -- 172.21.15.176:0/4041217885 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fada0108140 con 0x7fad8406c4e0 2024-03-13T14:45:33.201 INFO:teuthology.orchestra.run.smithi176.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-13T14:45:33.201 INFO:teuthology.orchestra.run.smithi176.stdout:smithi176 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330099400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-13T14:45:33.201 INFO:teuthology.orchestra.run.smithi176.stdout:smithi196 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6246000Z400BGN 372G No 82s ago Has a FileSystem, LVM detected 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad8406c4e0 msgr2=0x7fad8406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad8406c4e0 0x7fad8406e9a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fad8c005950 tx=0x7fad8c0058e0 comp rx=0 tx=0).stop 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0073770 msgr2=0x7fada0198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0073770 0x7fada0198530 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fad9000b5c0 tx=0x7fad9000be20 comp rx=0 tx=0).stop 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 shutdown_connections 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad8406c4e0 0x7fad8406e9a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fada0073770 0x7fada0198530 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 --2- 172.21.15.176:0/4041217885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0074d40 0x7fada0198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.214+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 >> 172.21.15.176:0/4041217885 conn(0x7fada00fc140 msgr2=0x7fada0106a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.215+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 shutdown_connections 2024-03-13T14:45:33.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:33.215+0000 7fada7200700 1 -- 172.21.15.176:0/4041217885 wait complete. 2024-03-13T14:45:33.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:33 smithi176 ceph-mon[29064]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:33.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:33 smithi196 ceph-mon[37486]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:33.950 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T14:45:33.961 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:45:33.961 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-03-13T14:45:34.354 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:34.499 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:34 smithi176 ceph-mon[29064]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:34.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:34 smithi196 ceph-mon[37486]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:35.528 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:35 smithi176 ceph-mon[29064]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:35.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:35 smithi196 ceph-mon[37486]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:36.479 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.489+0000 7feb91400700 1 -- 172.21.15.176:0/2046301043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c102e10 msgr2=0x7feb8c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:36.479 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.489+0000 7feb91400700 1 --2- 172.21.15.176:0/2046301043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c102e10 0x7feb8c103230 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009910 tx=0x7feb7c009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:36.479 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.490+0000 7feb91400700 1 -- 172.21.15.176:0/2046301043 shutdown_connections 2024-03-13T14:45:36.480 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.490+0000 7feb91400700 1 --2- 172.21.15.176:0/2046301043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8c103f70 0x7feb8c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:36.480 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.490+0000 7feb91400700 1 --2- 172.21.15.176:0/2046301043 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c102e10 0x7feb8c103230 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:36.480 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.490+0000 7feb91400700 1 -- 172.21.15.176:0/2046301043 >> 172.21.15.176:0/2046301043 conn(0x7feb8c0fe3d0 msgr2=0x7feb8c1007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:36.480 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.490+0000 7feb91400700 1 -- 172.21.15.176:0/2046301043 shutdown_connections 2024-03-13T14:45:36.480 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.490+0000 7feb91400700 1 -- 172.21.15.176:0/2046301043 wait complete. 2024-03-13T14:45:36.480 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.491+0000 7feb91400700 1 Processor -- start 2024-03-13T14:45:36.480 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.491+0000 7feb91400700 1 -- start start 2024-03-13T14:45:36.481 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb91400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8c102e10 0x7feb8c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:36.481 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb91400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c103f70 0x7feb8c198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:36.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb91400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1990e0 con 0x7feb8c103f70 2024-03-13T14:45:36.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb91400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c199220 con 0x7feb8c102e10 2024-03-13T14:45:36.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb89e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c103f70 0x7feb8c198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:36.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb89e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c103f70 0x7feb8c198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:53380/0 (socket says 172.21.15.176:53380) 2024-03-13T14:45:36.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb89e00700 1 -- 172.21.15.176:0/2998203021 learned_addr learned my addr 172.21.15.176:0/2998203021 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:36.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.492+0000 7feb8a800700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8c102e10 0x7feb8c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:36.483 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb89e00700 1 -- 172.21.15.176:0/2998203021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8c102e10 msgr2=0x7feb8c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:36.483 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb89e00700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8c102e10 0x7feb8c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:36.483 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb89e00700 1 -- 172.21.15.176:0/2998203021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c0095f0 con 0x7feb8c103f70 2024-03-13T14:45:36.484 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb89e00700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c103f70 0x7feb8c198ac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c8a0 tx=0x7feb7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:36.484 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb83400700 1 -- 172.21.15.176:0/2998203021 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb7800cea0 con 0x7feb8c103f70 2024-03-13T14:45:36.484 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c19dcd0 con 0x7feb8c103f70 2024-03-13T14:45:36.484 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb83400700 1 -- 172.21.15.176:0/2998203021 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb78004830 con 0x7feb8c103f70 2024-03-13T14:45:36.484 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb83400700 1 -- 172.21.15.176:0/2998203021 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb780055b0 con 0x7feb8c103f70 2024-03-13T14:45:36.484 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.493+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c19e220 con 0x7feb8c103f70 2024-03-13T14:45:36.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.495+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c10b8f0 con 0x7feb8c103f70 2024-03-13T14:45:36.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.496+0000 7feb83400700 1 -- 172.21.15.176:0/2998203021 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7feb78007ad0 con 0x7feb8c103f70 2024-03-13T14:45:36.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.496+0000 7feb83400700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7feb7006c4e0 0x7feb7006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:36.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.497+0000 7feb83400700 1 -- 172.21.15.176:0/2998203021 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7feb7808b210 con 0x7feb8c103f70 2024-03-13T14:45:36.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.497+0000 7feb8a800700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7feb7006c4e0 0x7feb7006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:36.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.498+0000 7feb8a800700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7feb7006c4e0 0x7feb7006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006010 tx=0x7feb7c00b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:36.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.501+0000 7feb83400700 1 -- 172.21.15.176:0/2998203021 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7feb780597a0 con 0x7feb8c103f70 2024-03-13T14:45:36.735 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:36.746+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7feb8c108950 con 0x7feb7006c4e0 2024-03-13T14:45:37.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:37 smithi176 ceph-mon[29064]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:37.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:37 smithi196 ceph-mon[37486]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:38.381 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.391+0000 7feb83400700 1 -- 172.21.15.176:0/2998203021 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feb8c108950 con 0x7feb7006c4e0 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7feb7006c4e0 msgr2=0x7feb7006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7feb7006c4e0 0x7feb7006e9a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006010 tx=0x7feb7c00b540 comp rx=0 tx=0).stop 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c103f70 msgr2=0x7feb8c198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c103f70 0x7feb8c198ac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c8a0 tx=0x7feb7800cc60 comp rx=0 tx=0).stop 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 shutdown_connections 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7feb7006c4e0 0x7feb7006e9a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8c102e10 0x7feb8c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 --2- 172.21.15.176:0/2998203021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7feb8c103f70 0x7feb8c198ac0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 >> 172.21.15.176:0/2998203021 conn(0x7feb8c0fe3d0 msgr2=0x7feb8c107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 shutdown_connections 2024-03-13T14:45:38.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:38.394+0000 7feb91400700 1 -- 172.21.15.176:0/2998203021 wait complete. 2024-03-13T14:45:38.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:38 smithi176 ceph-mon[29064]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:38.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:38 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-13T14:45:38.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:38 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-13T14:45:38.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:38 smithi176 ceph-mon[29064]: osdmap e42: 6 total, 6 up, 6 in 2024-03-13T14:45:38.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:38 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-13T14:45:38.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:38 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[29060]: 2024-03-13T14:45:38.361+0000 7fd686800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T14:45:38.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:38 smithi196 ceph-mon[37486]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:45:38.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:38 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-03-13T14:45:38.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:38 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-03-13T14:45:38.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:38 smithi196 ceph-mon[37486]: osdmap e42: 6 total, 6 up, 6 in 2024-03-13T14:45:38.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:38 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-03-13T14:45:39.196 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-13T14:45:39.522 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: pgmap v215: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: osdmap e43: 6 total, 6 up, 6 in 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: osdmap e44: 6 total, 6 up, 6 in 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: fsmap cephfs:0 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: Saving service mds.cephfs spec with placement count:4 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:39.523 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:45:39.524 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:39.524 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:39.524 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:39.524 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:39.524 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:39 smithi176 ceph-mon[29064]: Deploying daemon mds.cephfs.smithi176.ldskqt on smithi176 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: pgmap v215: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: osdmap e43: 6 total, 6 up, 6 in 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: osdmap e44: 6 total, 6 up, 6 in 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: fsmap cephfs:0 2024-03-13T14:45:39.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: Saving service mds.cephfs spec with placement count:4 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:39.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:39 smithi196 ceph-mon[37486]: Deploying daemon mds.cephfs.smithi176.ldskqt on smithi176 2024-03-13T14:45:40.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:40 smithi176 ceph-mon[29064]: osdmap e45: 6 total, 6 up, 6 in 2024-03-13T14:45:40.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:40 smithi196 ceph-mon[37486]: osdmap e45: 6 total, 6 up, 6 in 2024-03-13T14:45:40.858 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:41.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:41 smithi176 ceph-mon[29064]: pgmap v219: 65 pgs: 6 creating+peering, 10 active+clean, 49 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:41.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:41 smithi176 ceph-mon[29064]: osdmap e46: 6 total, 6 up, 6 in 2024-03-13T14:45:41.383 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:41 smithi176 ceph-mon[29064]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-13T14:45:41.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:41 smithi196 ceph-mon[37486]: pgmap v219: 65 pgs: 6 creating+peering, 10 active+clean, 49 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:41.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:41 smithi196 ceph-mon[37486]: osdmap e46: 6 total, 6 up, 6 in 2024-03-13T14:45:41.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:41 smithi196 ceph-mon[37486]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-13T14:45:43.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 -- 172.21.15.176:0/853764521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb968103e00 msgr2=0x7fb9681061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 --2- 172.21.15.176:0/853764521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb968103e00 0x7fb9681061f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb958009910 tx=0x7fb958009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 -- 172.21.15.176:0/853764521 shutdown_connections 2024-03-13T14:45:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 --2- 172.21.15.176:0/853764521 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb968103e00 0x7fb9681061f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 --2- 172.21.15.176:0/853764521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9681014d0 0x7fb9681038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 -- 172.21.15.176:0/853764521 >> 172.21.15.176:0/853764521 conn(0x7fb9680fae40 msgr2=0x7fb9680fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 -- 172.21.15.176:0/853764521 shutdown_connections 2024-03-13T14:45:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.399+0000 7fb96f200700 1 -- 172.21.15.176:0/853764521 wait complete. 2024-03-13T14:45:43.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96f200700 1 Processor -- start 2024-03-13T14:45:43.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96f200700 1 -- start start 2024-03-13T14:45:43.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96f200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9681014d0 0x7fb968198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96f200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968103e00 0x7fb9681988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96f200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb968198ec0 con 0x7fb9681014d0 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96f200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb968199000 con 0x7fb968103e00 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9681014d0 0x7fb968198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9681014d0 0x7fb968198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56404/0 (socket says 172.21.15.176:56404) 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.401+0000 7fb96de00700 1 -- 172.21.15.176:0/458032260 learned_addr learned my addr 172.21.15.176:0/458032260 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.402+0000 7fb96d400700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968103e00 0x7fb9681988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.402+0000 7fb96de00700 1 -- 172.21.15.176:0/458032260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968103e00 msgr2=0x7fb9681988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.402+0000 7fb96de00700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968103e00 0x7fb9681988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.402+0000 7fb96de00700 1 -- 172.21.15.176:0/458032260 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580095f0 con 0x7fb9681014d0 2024-03-13T14:45:43.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.402+0000 7fb96de00700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9681014d0 0x7fb968198360 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00c8a0 tx=0x7fb95c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.404+0000 7fb962a00700 1 -- 172.21.15.176:0/458032260 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95c00cea0 con 0x7fb9681014d0 2024-03-13T14:45:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.404+0000 7fb962a00700 1 -- 172.21.15.176:0/458032260 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb95c004830 con 0x7fb9681014d0 2024-03-13T14:45:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.404+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb96819dab0 con 0x7fb9681014d0 2024-03-13T14:45:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.404+0000 7fb962a00700 1 -- 172.21.15.176:0/458032260 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb95c0055b0 con 0x7fb9681014d0 2024-03-13T14:45:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.404+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb96819e000 con 0x7fb9681014d0 2024-03-13T14:45:43.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.406+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9680fca50 con 0x7fb9681014d0 2024-03-13T14:45:43.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.406+0000 7fb962a00700 1 -- 172.21.15.176:0/458032260 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fb95c007ad0 con 0x7fb9681014d0 2024-03-13T14:45:43.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.408+0000 7fb962a00700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb95006c4e0 0x7fb95006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:43.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.408+0000 7fb962a00700 1 -- 172.21.15.176:0/458032260 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fb95c08b570 con 0x7fb9681014d0 2024-03-13T14:45:43.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.408+0000 7fb96d400700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb95006c4e0 0x7fb95006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.409+0000 7fb96d400700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb95006c4e0 0x7fb95006e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb958006010 tx=0x7fb95800b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:43.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.413+0000 7fb962a00700 1 -- 172.21.15.176:0/458032260 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fb95c0597c0 con 0x7fb9681014d0 2024-03-13T14:45:43.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: pgmap v221: 65 pgs: 6 creating+peering, 46 active+clean, 13 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:43.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:43.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:43.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:43.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:43.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.680+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb968061a70 con 0x7fb9681014d0 2024-03-13T14:45:43.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.680+0000 7fb962a00700 1 -- 172.21.15.176:0/458032260 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 4 v4) v1 ==== 75+0+1266 (secure 0 0 0) 0x7fb95c01d510 con 0x7fb9681014d0 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout:e4 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:45:43.670 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 4 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:45:43.404396+0000 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 0 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:45:43.671 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=14468} 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 0 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{0:14468} state up:creating seq 1 addr [v2:172.21.15.176:6826/3592546199,v1:172.21.15.176:6827/3592546199] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:45:43.672 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:45:43.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.683+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb95006c4e0 msgr2=0x7fb95006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:43.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.683+0000 7fb96f200700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb95006c4e0 0x7fb95006e9a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb958006010 tx=0x7fb95800b540 comp rx=0 tx=0).stop 2024-03-13T14:45:43.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.683+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9681014d0 msgr2=0x7fb968198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:43.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.683+0000 7fb96f200700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9681014d0 0x7fb968198360 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb95c00c8a0 tx=0x7fb95c00cc60 comp rx=0 tx=0).stop 2024-03-13T14:45:43.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.684+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 shutdown_connections 2024-03-13T14:45:43.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.684+0000 7fb96f200700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fb95006c4e0 0x7fb95006e9a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:43.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.684+0000 7fb96f200700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb9681014d0 0x7fb968198360 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:43.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.684+0000 7fb96f200700 1 --2- 172.21.15.176:0/458032260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968103e00 0x7fb9681988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:43.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.684+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 >> 172.21.15.176:0/458032260 conn(0x7fb9680fae40 msgr2=0x7fb9680ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:43.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.684+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 shutdown_connections 2024-03-13T14:45:43.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:43.684+0000 7fb96f200700 1 -- 172.21.15.176:0/458032260 wait complete. 2024-03-13T14:45:43.674 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 4 2024-03-13T14:45:43.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: pgmap v221: 65 pgs: 6 creating+peering, 46 active+clean, 13 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:45:43.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:43.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:43.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:43.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:43.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:44.544 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T14:45:44.554 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:45:44.555 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: Deploying daemon mds.cephfs.smithi196.tcitrk on smithi196 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.176:6826/3592546199,v1:172.21.15.176:6827/3592546199] up:boot 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: daemon mds.cephfs.smithi176.ldskqt assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: Cluster is now healthy 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: fsmap cephfs:0 1 up:standby 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:creating} 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: daemon mds.cephfs.smithi176.ldskqt is now active in filesystem cephfs as rank 0 2024-03-13T14:45:44.645 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:44 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/458032260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:45:44.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: Deploying daemon mds.cephfs.smithi196.tcitrk on smithi196 2024-03-13T14:45:44.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.176:6826/3592546199,v1:172.21.15.176:6827/3592546199] up:boot 2024-03-13T14:45:44.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: daemon mds.cephfs.smithi176.ldskqt assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: Cluster is now healthy 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: fsmap cephfs:0 1 up:standby 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:creating} 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: daemon mds.cephfs.smithi176.ldskqt is now active in filesystem cephfs as rank 0 2024-03-13T14:45:44.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:44 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/458032260' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:45:44.974 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:45.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:45 smithi196 ceph-mon[37486]: pgmap v222: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-13T14:45:45.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:45 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.176:6826/3592546199,v1:172.21.15.176:6827/3592546199] up:active 2024-03-13T14:45:45.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:45 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2024-03-13T14:45:45.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:45 smithi176 ceph-mon[29064]: pgmap v222: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 1 op/s 2024-03-13T14:45:45.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:45 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.176:6826/3592546199,v1:172.21.15.176:6827/3592546199] up:active 2024-03-13T14:45:45.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:45 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2024-03-13T14:45:47.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 -- 172.21.15.176:0/2205179788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 msgr2=0x7f70b0101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:47.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2205179788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b0101010 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009910 tx=0x7f70a0009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:47.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 -- 172.21.15.176:0/2205179788 shutdown_connections 2024-03-13T14:45:47.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2205179788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b0101df0 0x7f70b0102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:47.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2205179788 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b0101010 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:47.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 -- 172.21.15.176:0/2205179788 >> 172.21.15.176:0/2205179788 conn(0x7f70b00fc190 msgr2=0x7f70b00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:47.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 -- 172.21.15.176:0/2205179788 shutdown_connections 2024-03-13T14:45:47.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.012+0000 7f70b5e00700 1 -- 172.21.15.176:0/2205179788 wait complete. 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.013+0000 7f70b5e00700 1 Processor -- start 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70b5e00700 1 -- start start 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70b5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b01927b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70b5e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b0101df0 0x7f70b0192cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70b5e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b018f2a0 con 0x7f70b0100bf0 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70b5e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b018f410 con 0x7f70b0101df0 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70aea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b01927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70aea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b01927b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56422/0 (socket says 172.21.15.176:56422) 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.014+0000 7f70aea00700 1 -- 172.21.15.176:0/2372395278 learned_addr learned my addr 172.21.15.176:0/2372395278 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70aea00700 1 -- 172.21.15.176:0/2372395278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b0101df0 msgr2=0x7f70b0192cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70aea00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b0101df0 0x7f70b0192cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:47.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70aea00700 1 -- 172.21.15.176:0/2372395278 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a00095f0 con 0x7f70b0100bf0 2024-03-13T14:45:47.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70aea00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b01927b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f70a000bd70 tx=0x7f70a000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:47.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70b4a00700 1 -- 172.21.15.176:0/2372395278 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a001d070 con 0x7f70b0100bf0 2024-03-13T14:45:47.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70b4a00700 1 -- 172.21.15.176:0/2372395278 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70a000f460 con 0x7f70b0100bf0 2024-03-13T14:45:47.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70b4a00700 1 -- 172.21.15.176:0/2372395278 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a0021640 con 0x7f70b0100bf0 2024-03-13T14:45:47.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b018f690 con 0x7f70b0100bf0 2024-03-13T14:45:47.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.015+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b018fbb0 con 0x7f70b0100bf0 2024-03-13T14:45:47.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.017+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b0105ed0 con 0x7f70b0100bf0 2024-03-13T14:45:47.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.018+0000 7f70b4a00700 1 -- 172.21.15.176:0/2372395278 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f70a000fab0 con 0x7f70b0100bf0 2024-03-13T14:45:47.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.018+0000 7f70b4a00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f709406c4e0 0x7f709406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:47.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.019+0000 7f70b4a00700 1 -- 172.21.15.176:0/2372395278 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f70a008cc70 con 0x7f70b0100bf0 2024-03-13T14:45:47.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.019+0000 7f70ae000700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f709406c4e0 0x7f709406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:47.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.019+0000 7f70ae000700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f709406c4e0 0x7f709406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f709c0097b0 tx=0x7f709c006d20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:47.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.024+0000 7f70b4a00700 1 -- 172.21.15.176:0/2372395278 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f70a0093080 con 0x7f70b0100bf0 2024-03-13T14:45:47.283 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.293+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f70b002cef0 con 0x7f70b0100bf0 2024-03-13T14:45:47.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.442+0000 7f70b4a00700 1 -- 172.21.15.176:0/2372395278 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f70a005aec0 con 0x7f70b0100bf0 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f709406c4e0 msgr2=0x7f709406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f709406c4e0 0x7f709406e9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f709c0097b0 tx=0x7f709c006d20 comp rx=0 tx=0).stop 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 msgr2=0x7f70b01927b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b01927b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f70a000bd70 tx=0x7f70a000be50 comp rx=0 tx=0).stop 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 shutdown_connections 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f709406c4e0 0x7f709406e9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f70b0100bf0 0x7f70b01927b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 --2- 172.21.15.176:0/2372395278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70b0101df0 0x7f70b0192cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.445+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 >> 172.21.15.176:0/2372395278 conn(0x7f70b00fc190 msgr2=0x7f70b0103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:47.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.446+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 shutdown_connections 2024-03-13T14:45:47.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:47.446+0000 7f70b5e00700 1 -- 172.21.15.176:0/2372395278 wait complete. 2024-03-13T14:45:47.608 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: pgmap v223: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-13T14:45:47.608 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:47.608 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:47.608 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:47.609 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:47.609 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:47.609 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:47.609 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:47 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2372395278' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-13T14:45:47.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: pgmap v223: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-03-13T14:45:47.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:47.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:47.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:47.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:47.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:47.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:47.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:47 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2372395278' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-03-13T14:45:48.348 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T14:45:48.360 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:45:48.360 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-03-13T14:45:48.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:48 smithi176 ceph-mon[29064]: Deploying daemon mds.cephfs.smithi176.hjrgov on smithi176 2024-03-13T14:45:48.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:48 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:boot 2024-03-13T14:45:48.565 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:48 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2372395278' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-13T14:45:48.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:48 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby 2024-03-13T14:45:48.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:48 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:45:48.566 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:48 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby 2024-03-13T14:45:48.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:48 smithi196 ceph-mon[37486]: Deploying daemon mds.cephfs.smithi176.hjrgov on smithi176 2024-03-13T14:45:48.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:48 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:boot 2024-03-13T14:45:48.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:48 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2372395278' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-03-13T14:45:48.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:48 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby 2024-03-13T14:45:48.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:48 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:45:48.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:48 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby 2024-03-13T14:45:49.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:49 smithi176 ceph-mon[29064]: pgmap v224: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-03-13T14:45:49.785 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:49.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:49 smithi196 ceph-mon[37486]: pgmap v224: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 5 op/s 2024-03-13T14:45:51.222 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:50 smithi176 ceph-mon[29064]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-13T14:45:51.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:50 smithi196 ceph-mon[37486]: pgmap v225: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s wr, 4 op/s 2024-03-13T14:45:52.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.274+0000 7f2055000700 1 -- 172.21.15.176:0/454710970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 msgr2=0x7f2050102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:52.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.274+0000 7f2055000700 1 --2- 172.21.15.176:0/454710970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050102220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f203c009910 tx=0x7f203c009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:52.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.275+0000 7f2055000700 1 -- 172.21.15.176:0/454710970 shutdown_connections 2024-03-13T14:45:52.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.275+0000 7f2055000700 1 --2- 172.21.15.176:0/454710970 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050102220 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:52.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.275+0000 7f2055000700 1 --2- 172.21.15.176:0/454710970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2050100bc0 0x7f2050100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:52.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.275+0000 7f2055000700 1 -- 172.21.15.176:0/454710970 >> 172.21.15.176:0/454710970 conn(0x7f20500fc140 msgr2=0x7f20500fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:52.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.275+0000 7f2055000700 1 -- 172.21.15.176:0/454710970 shutdown_connections 2024-03-13T14:45:52.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.275+0000 7f2055000700 1 -- 172.21.15.176:0/454710970 wait complete. 2024-03-13T14:45:52.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.276+0000 7f2055000700 1 Processor -- start 2024-03-13T14:45:52.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.276+0000 7f2055000700 1 -- start start 2024-03-13T14:45:52.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.276+0000 7f2055000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2050100bc0 0x7f2050198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:52.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f2055000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f2055000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050199030 con 0x7f2050101dc0 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f2055000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2050199170 con 0x7f2050100bc0 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204d400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204d400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36074/0 (socket says 172.21.15.176:36074) 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204d400700 1 -- 172.21.15.176:0/2386507895 learned_addr learned my addr 172.21.15.176:0/2386507895 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204de00700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2050100bc0 0x7f2050198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204d400700 1 -- 172.21.15.176:0/2386507895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2050100bc0 msgr2=0x7f2050198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204d400700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2050100bc0 0x7f2050198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204d400700 1 -- 172.21.15.176:0/2386507895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0095f0 con 0x7f2050101dc0 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.277+0000 7f204de00700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2050100bc0 0x7f2050198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.278+0000 7f204d400700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050198aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f203c00bd70 tx=0x7f203c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.278+0000 7f2046a00700 1 -- 172.21.15.176:0/2386507895 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203c01d070 con 0x7f2050101dc0 2024-03-13T14:45:52.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.278+0000 7f2046a00700 1 -- 172.21.15.176:0/2386507895 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f203c022470 con 0x7f2050101dc0 2024-03-13T14:45:52.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.278+0000 7f2046a00700 1 -- 172.21.15.176:0/2386507895 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203c00f670 con 0x7f2050101dc0 2024-03-13T14:45:52.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.278+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f205019dbd0 con 0x7f2050101dc0 2024-03-13T14:45:52.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.278+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f205019e090 con 0x7f2050101dc0 2024-03-13T14:45:52.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.279+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050105ea0 con 0x7f2050101dc0 2024-03-13T14:45:52.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.280+0000 7f2046a00700 1 -- 172.21.15.176:0/2386507895 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f203c022a50 con 0x7f2050101dc0 2024-03-13T14:45:52.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.281+0000 7f2046a00700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f203406c4e0 0x7f203406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:52.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.281+0000 7f2046a00700 1 -- 172.21.15.176:0/2386507895 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f203c08ce90 con 0x7f2050101dc0 2024-03-13T14:45:52.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.281+0000 7f204de00700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f203406c4e0 0x7f203406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:52.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.282+0000 7f204de00700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f203406c4e0 0x7f203406e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2040005fd0 tx=0x7f2040005e40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:52.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.286+0000 7f2046a00700 1 -- 172.21.15.176:0/2386507895 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f203c05b190 con 0x7f2050101dc0 2024-03-13T14:45:52.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.544+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f205002cc70 con 0x7f2050101dc0 2024-03-13T14:45:52.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:52.996+0000 7f2046a00700 1 -- 172.21.15.176:0/2386507895 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7f203c05ad20 con 0x7f2050101dc0 2024-03-13T14:45:52.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f203406c4e0 msgr2=0x7f203406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:52.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f203406c4e0 0x7f203406e9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2040005fd0 tx=0x7f2040005e40 comp rx=0 tx=0).stop 2024-03-13T14:45:52.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 msgr2=0x7f2050198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:52.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050198aa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f203c00bd70 tx=0x7f203c00be50 comp rx=0 tx=0).stop 2024-03-13T14:45:52.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 shutdown_connections 2024-03-13T14:45:52.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f203406c4e0 0x7f203406e9a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:52.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2050100bc0 0x7f2050198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:52.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 --2- 172.21.15.176:0/2386507895 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2050101dc0 0x7f2050198aa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:52.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 >> 172.21.15.176:0/2386507895 conn(0x7f20500fc140 msgr2=0x7f2050102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:52.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 shutdown_connections 2024-03-13T14:45:52.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:53.000+0000 7f2055000700 1 -- 172.21.15.176:0/2386507895 wait complete. 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: Deploying daemon mds.cephfs.smithi196.wabhcu on smithi196 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] up:boot 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2 up:standby 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: pgmap v226: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-03-13T14:45:53.056 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:52 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2386507895' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: Deploying daemon mds.cephfs.smithi196.wabhcu on smithi196 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] up:boot 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2 up:standby 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:45:53.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: pgmap v226: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 3 op/s 2024-03-13T14:45:53.132 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:52 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2386507895' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-13T14:45:53.830 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T14:45:53.840 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:45:53.840 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-03-13T14:45:54.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:53 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2386507895' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-13T14:45:54.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:53 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2 up:standby 2024-03-13T14:45:54.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:53 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 1 up:standby 2024-03-13T14:45:54.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:53 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:54.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:53 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2386507895' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-13T14:45:54.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:53 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2 up:standby 2024-03-13T14:45:54.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:53 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 1 up:standby 2024-03-13T14:45:54.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:53 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:54.238 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:55.298 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:55 smithi196 ceph-mon[37486]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-13T14:45:55.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:55 smithi176 ceph-mon[29064]: pgmap v227: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-13T14:45:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 -- 172.21.15.176:0/2137662698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 msgr2=0x7fe740073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 --2- 172.21.15.176:0/2137662698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740073220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe734009910 tx=0x7fe734009c20 comp rx=0 tx=0).stop 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 -- 172.21.15.176:0/2137662698 shutdown_connections 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 --2- 172.21.15.176:0/2137662698 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7400737f0 0x7fe740073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 --2- 172.21.15.176:0/2137662698 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740073220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 -- 172.21.15.176:0/2137662698 >> 172.21.15.176:0/2137662698 conn(0x7fe7400fc110 msgr2=0x7fe7400fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 -- 172.21.15.176:0/2137662698 shutdown_connections 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.283+0000 7fe748200700 1 -- 172.21.15.176:0/2137662698 wait complete. 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.284+0000 7fe748200700 1 Processor -- start 2024-03-13T14:45:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.284+0000 7fe748200700 1 -- start start 2024-03-13T14:45:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe748200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7400737f0 0x7fe740198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe748200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe748200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe740199040 con 0x7fe740074dc0 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe748200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe740199180 con 0x7fe7400737f0 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe744e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe744e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36090/0 (socket says 172.21.15.176:36090) 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe744e00700 1 -- 172.21.15.176:0/2481166892 learned_addr learned my addr 172.21.15.176:0/2481166892 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.285+0000 7fe745800700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7400737f0 0x7fe740198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe744e00700 1 -- 172.21.15.176:0/2481166892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7400737f0 msgr2=0x7fe740198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe744e00700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7400737f0 0x7fe740198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe744e00700 1 -- 172.21.15.176:0/2481166892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340095f0 con 0x7fe740074dc0 2024-03-13T14:45:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe744e00700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740198ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c910 tx=0x7fe72c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe73e000700 1 -- 172.21.15.176:0/2481166892 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72c0041f0 con 0x7fe740074dc0 2024-03-13T14:45:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe73e000700 1 -- 172.21.15.176:0/2481166892 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe72c00ce60 con 0x7fe740074dc0 2024-03-13T14:45:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74019dc40 con 0x7fe740074dc0 2024-03-13T14:45:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.286+0000 7fe73e000700 1 -- 172.21.15.176:0/2481166892 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe72c003d20 con 0x7fe740074dc0 2024-03-13T14:45:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.287+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74019e160 con 0x7fe740074dc0 2024-03-13T14:45:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.288+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74010b920 con 0x7fe740074dc0 2024-03-13T14:45:56.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.289+0000 7fe73e000700 1 -- 172.21.15.176:0/2481166892 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fe72c007900 con 0x7fe740074dc0 2024-03-13T14:45:56.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.289+0000 7fe73e000700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe72806c4e0 0x7fe72806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:45:56.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.290+0000 7fe73e000700 1 -- 172.21.15.176:0/2481166892 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fe72c08b510 con 0x7fe740074dc0 2024-03-13T14:45:56.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.290+0000 7fe745800700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe72806c4e0 0x7fe72806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:45:56.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.291+0000 7fe745800700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe72806c4e0 0x7fe72806e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe734006010 tx=0x7fe73400b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:45:56.284 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.295+0000 7fe73e000700 1 -- 172.21.15.176:0/2481166892 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fe72c059780 con 0x7fe740074dc0 2024-03-13T14:45:56.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:56.562+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7fe74019e7d0 con 0x7fe740074dc0 2024-03-13T14:45:57.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.018+0000 7fe73e000700 1 -- 172.21.15.176:0/2481166892 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe72c003e80 con 0x7fe740074dc0 2024-03-13T14:45:57.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.022+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe72806c4e0 msgr2=0x7fe72806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:57.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.022+0000 7fe748200700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe72806c4e0 0x7fe72806e9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe734006010 tx=0x7fe73400b540 comp rx=0 tx=0).stop 2024-03-13T14:45:57.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.022+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 msgr2=0x7fe740198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:45:57.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.022+0000 7fe748200700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740198ab0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe72c00c910 tx=0x7fe72c00cc20 comp rx=0 tx=0).stop 2024-03-13T14:45:57.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.023+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 shutdown_connections 2024-03-13T14:45:57.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.023+0000 7fe748200700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fe72806c4e0 0x7fe72806e9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:57.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.023+0000 7fe748200700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7400737f0 0x7fe740198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:57.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.023+0000 7fe748200700 1 --2- 172.21.15.176:0/2481166892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe740074dc0 0x7fe740198ab0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:45:57.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.023+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 >> 172.21.15.176:0/2481166892 conn(0x7fe7400fc110 msgr2=0x7fe740106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:45:57.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.023+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 shutdown_connections 2024-03-13T14:45:57.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:45:57.023+0000 7fe748200700 1 -- 172.21.15.176:0/2481166892 wait complete. 2024-03-13T14:45:57.013 INFO:teuthology.orchestra.run.smithi176.stderr:inline data disabled 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:boot 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 511 B/s wr, 4 op/s 2024-03-13T14:45:57.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:57.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:56 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2481166892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-13T14:45:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:57.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:boot 2024-03-13T14:45:57.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:45:57.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:45:57.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: pgmap v228: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 511 B/s wr, 4 op/s 2024-03-13T14:45:57.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:45:57.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:56 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2481166892' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-03-13T14:45:58.141 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:58 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2481166892' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-13T14:45:58.141 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:58 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:45:58.144 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T14:45:58.155 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:45:58.155 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-13T14:45:58.385 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:58 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2481166892' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-03-13T14:45:58.385 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:58 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:45:58.545 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:45:59.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:59.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:59.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:59 smithi176 ceph-mon[29064]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-03-13T14:45:59.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:45:59 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:59.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:45:59.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:45:59.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:59 smithi196 ceph-mon[37486]: pgmap v229: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 2 op/s 2024-03-13T14:45:59.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:45:59 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:standby-replay 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: Dropping low affinity active daemon mds.cephfs.smithi176.ldskqt in favor of higher affinity standby. 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: Replacing daemon mds.cephfs.smithi176.ldskqt as rank 0 with standby daemon mds.cephfs.smithi196.tcitrk 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: osdmap e47: 6 total, 6 up, 6 in 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: fsmap cephfs:1/1 {0=cephfs.smithi196.tcitrk=up:replay} 2 up:standby 2024-03-13T14:46:00.262 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:00.263 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:00 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:standby-replay 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: Dropping low affinity active daemon mds.cephfs.smithi176.ldskqt in favor of higher affinity standby. 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: Replacing daemon mds.cephfs.smithi176.ldskqt as rank 0 with standby daemon mds.cephfs.smithi196.tcitrk 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: osdmap e47: 6 total, 6 up, 6 in 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: fsmap cephfs:1/1 {0=cephfs.smithi196.tcitrk=up:replay} 2 up:standby 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:00.284 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:00 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:00.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 -- 172.21.15.176:0/2314569767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 msgr2=0x7fad30100fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:00.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 --2- 172.21.15.176:0/2314569767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad30100fe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fad24009910 tx=0x7fad24009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:00.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 -- 172.21.15.176:0/2314569767 shutdown_connections 2024-03-13T14:46:00.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 --2- 172.21.15.176:0/2314569767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad30101dc0 0x7fad30102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:00.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 --2- 172.21.15.176:0/2314569767 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad30100fe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:00.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 -- 172.21.15.176:0/2314569767 >> 172.21.15.176:0/2314569767 conn(0x7fad300fc140 msgr2=0x7fad300fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:00.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 -- 172.21.15.176:0/2314569767 shutdown_connections 2024-03-13T14:46:00.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.878+0000 7fad38400700 1 -- 172.21.15.176:0/2314569767 wait complete. 2024-03-13T14:46:00.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.879+0000 7fad38400700 1 Processor -- start 2024-03-13T14:46:00.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad38400700 1 -- start start 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad38400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad301985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad38400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad30101dc0 0x7fad30198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad38400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad30199100 con 0x7fad30100bc0 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad38400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad30199240 con 0x7fad30101dc0 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad35a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad301985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad35a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad301985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36126/0 (socket says 172.21.15.176:36126) 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.880+0000 7fad35a00700 1 -- 172.21.15.176:0/2583075268 learned_addr learned my addr 172.21.15.176:0/2583075268 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad35000700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad30101dc0 0x7fad30198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad35a00700 1 -- 172.21.15.176:0/2583075268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad30101dc0 msgr2=0x7fad30198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:00.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad35a00700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad30101dc0 0x7fad30198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:00.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad35a00700 1 -- 172.21.15.176:0/2583075268 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad240095f0 con 0x7fad30100bc0 2024-03-13T14:46:00.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad35a00700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad301985a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fad2400bda0 tx=0x7fad2400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:00.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad2e000700 1 -- 172.21.15.176:0/2583075268 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad2401d070 con 0x7fad30100bc0 2024-03-13T14:46:00.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad2e000700 1 -- 172.21.15.176:0/2583075268 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad2400f460 con 0x7fad30100bc0 2024-03-13T14:46:00.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3019dc90 con 0x7fad30100bc0 2024-03-13T14:46:00.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.881+0000 7fad2e000700 1 -- 172.21.15.176:0/2583075268 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad24021640 con 0x7fad30100bc0 2024-03-13T14:46:00.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.882+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3019e180 con 0x7fad30100bc0 2024-03-13T14:46:00.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.883+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad30105ea0 con 0x7fad30100bc0 2024-03-13T14:46:00.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.884+0000 7fad2e000700 1 -- 172.21.15.176:0/2583075268 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fad2400fa00 con 0x7fad30100bc0 2024-03-13T14:46:00.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.884+0000 7fad2e000700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad1806c4e0 0x7fad1806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:00.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.884+0000 7fad2e000700 1 -- 172.21.15.176:0/2583075268 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fad2408cb80 con 0x7fad30100bc0 2024-03-13T14:46:00.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.885+0000 7fad35000700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad1806c4e0 0x7fad1806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:00.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.885+0000 7fad35000700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad1806c4e0 0x7fad1806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad1c005950 tx=0x7fad1c0058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:00.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:00.890+0000 7fad2e000700 1 -- 172.21.15.176:0/2583075268 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fad2405ad90 con 0x7fad30100bc0 2024-03-13T14:46:01.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.159+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad3002cc70 con 0x7fad30100bc0 2024-03-13T14:46:01.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.160+0000 7fad2e000700 1 -- 172.21.15.176:0/2583075268 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1850 (secure 0 0 0) 0x7fad2405a920 con 0x7fad30100bc0 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:e16 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 16 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:01.147442+0000 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:46:01.150 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:46:01.151 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:rejoin seq 6 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{-1:14484} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:46:01.152 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:46:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad1806c4e0 msgr2=0x7fad1806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad1806c4e0 0x7fad1806e9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad1c005950 tx=0x7fad1c0058e0 comp rx=0 tx=0).stop 2024-03-13T14:46:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 msgr2=0x7fad301985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad301985a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fad2400bda0 tx=0x7fad2400be80 comp rx=0 tx=0).stop 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 shutdown_connections 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fad1806c4e0 0x7fad1806e9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad30100bc0 0x7fad301985a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 --2- 172.21.15.176:0/2583075268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad30101dc0 0x7fad30198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 >> 172.21.15.176:0/2583075268 conn(0x7fad300fc140 msgr2=0x7fad30102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 shutdown_connections 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:01.164+0000 7fad38400700 1 -- 172.21.15.176:0/2583075268 wait complete. 2024-03-13T14:46:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 16 2024-03-13T14:46:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:01 smithi196 ceph-mon[37486]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-13T14:46:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:01 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] up:boot 2024-03-13T14:46:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:01 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] up:standby 2024-03-13T14:46:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:01 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:standby 2024-03-13T14:46:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:01 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:reconnect 2024-03-13T14:46:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:01 smithi196 ceph-mon[37486]: fsmap cephfs:1/1 {0=cephfs.smithi196.tcitrk=up:reconnect} 3 up:standby 2024-03-13T14:46:01.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:01 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:46:01.474 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:01 smithi176 ceph-mon[29064]: pgmap v231: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-13T14:46:01.474 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:01 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] up:boot 2024-03-13T14:46:01.474 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:01 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] up:standby 2024-03-13T14:46:01.474 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:01 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:standby 2024-03-13T14:46:01.474 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:01 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:reconnect 2024-03-13T14:46:01.474 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:01 smithi176 ceph-mon[29064]: fsmap cephfs:1/1 {0=cephfs.smithi196.tcitrk=up:reconnect} 3 up:standby 2024-03-13T14:46:01.474 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:01 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:46:01.816 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-03-13T14:46:02.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:02 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:rejoin 2024-03-13T14:46:02.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:02 smithi176 ceph-mon[29064]: fsmap cephfs:1/1 {0=cephfs.smithi196.tcitrk=up:rejoin} 3 up:standby 2024-03-13T14:46:02.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:02 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2583075268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:46:02.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:02 smithi176 ceph-mon[29064]: daemon mds.cephfs.smithi196.tcitrk is now active in filesystem cephfs as rank 0 2024-03-13T14:46:02.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:02 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:rejoin 2024-03-13T14:46:02.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:02 smithi196 ceph-mon[37486]: fsmap cephfs:1/1 {0=cephfs.smithi196.tcitrk=up:rejoin} 3 up:standby 2024-03-13T14:46:02.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:02 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2583075268' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:46:02.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:02 smithi196 ceph-mon[37486]: daemon mds.cephfs.smithi196.tcitrk is now active in filesystem cephfs as rank 0 2024-03-13T14:46:02.614 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:03.355 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:03 smithi176 ceph-mon[29064]: pgmap v232: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-03-13T14:46:03.355 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:03 smithi176 ceph-mon[29064]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-13T14:46:03.355 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:03 smithi176 ceph-mon[29064]: Cluster is now healthy 2024-03-13T14:46:03.355 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:03 smithi176 ceph-mon[29064]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:active 2024-03-13T14:46:03.355 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:03 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T14:46:03.355 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:03 smithi176 ceph-mon[29064]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:46:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:03 smithi196 ceph-mon[37486]: pgmap v232: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-03-13T14:46:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:03 smithi196 ceph-mon[37486]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-13T14:46:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:03 smithi196 ceph-mon[37486]: Cluster is now healthy 2024-03-13T14:46:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:03 smithi196 ceph-mon[37486]: mds.? [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] up:active 2024-03-13T14:46:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:03 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T14:46:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:03 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:46:05.422 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:05 smithi176 ceph-mon[29064]: pgmap v233: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 3 op/s 2024-03-13T14:46:05.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:05 smithi196 ceph-mon[37486]: pgmap v233: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.0 KiB/s rd, 3 op/s 2024-03-13T14:46:05.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 -- 172.21.15.176:0/2964626385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 msgr2=0x7faef4103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 --2- 172.21.15.176:0/2964626385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4103a30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7faee4009910 tx=0x7faee4009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 -- 172.21.15.176:0/2964626385 shutdown_connections 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 --2- 172.21.15.176:0/2964626385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4103f70 0x7faef4106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 --2- 172.21.15.176:0/2964626385 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4103a30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 -- 172.21.15.176:0/2964626385 >> 172.21.15.176:0/2964626385 conn(0x7faef40fb050 msgr2=0x7faef40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 -- 172.21.15.176:0/2964626385 shutdown_connections 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.857+0000 7faefb000700 1 -- 172.21.15.176:0/2964626385 wait complete. 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.858+0000 7faefb000700 1 Processor -- start 2024-03-13T14:46:05.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.858+0000 7faefb000700 1 -- start start 2024-03-13T14:46:05.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faefb000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:05.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faefb000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4103f70 0x7faef4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faefb000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef4196e70 con 0x7faef4101640 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faefb000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef4196fb0 con 0x7faef4103f70 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faef3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faef3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:51914/0 (socket says 172.21.15.176:51914) 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faef3e00700 1 -- 172.21.15.176:0/505166282 learned_addr learned my addr 172.21.15.176:0/505166282 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.859+0000 7faef3400700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4103f70 0x7faef4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faef3e00700 1 -- 172.21.15.176:0/505166282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4103f70 msgr2=0x7faef4196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faef3e00700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4103f70 0x7faef4196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faef3e00700 1 -- 172.21.15.176:0/505166282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee40095f0 con 0x7faef4101640 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faef3e00700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4196310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faee4005b40 tx=0x7faee400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faef0c00700 1 -- 172.21.15.176:0/505166282 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faee401d070 con 0x7faef4101640 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faef0c00700 1 -- 172.21.15.176:0/505166282 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faee400f460 con 0x7faef4101640 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef419ba00 con 0x7faef4101640 2024-03-13T14:46:05.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faef0c00700 1 -- 172.21.15.176:0/505166282 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faee4021640 con 0x7faef4101640 2024-03-13T14:46:05.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.860+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef419bef0 con 0x7faef4101640 2024-03-13T14:46:05.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.862+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef4190520 con 0x7faef4101640 2024-03-13T14:46:05.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.863+0000 7faef0c00700 1 -- 172.21.15.176:0/505166282 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7faee400fa90 con 0x7faef4101640 2024-03-13T14:46:05.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.864+0000 7faef0c00700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7faed806c4e0 0x7faed806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:05.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.864+0000 7faef0c00700 1 -- 172.21.15.176:0/505166282 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7faee408cb70 con 0x7faef4101640 2024-03-13T14:46:05.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.864+0000 7faef3400700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7faed806c4e0 0x7faed806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:05.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.865+0000 7faef3400700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7faed806c4e0 0x7faed806e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faee0005950 tx=0x7faee00058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:05.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:05.869+0000 7faef0c00700 1 -- 172.21.15.176:0/505166282 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7faee405ae00 con 0x7faef4101640 2024-03-13T14:46:06.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.151+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faef402cc70 con 0x7faef4101640 2024-03-13T14:46:06.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.152+0000 7faef0c00700 1 -- 172.21.15.176:0/505166282 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4753 (secure 0 0 0) 0x7faee405a990 con 0x7faef4101640 2024-03-13T14:46:06.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7faed806c4e0 msgr2=0x7faed806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:06.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7faed806c4e0 0x7faed806e9a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faee0005950 tx=0x7faee00058e0 comp rx=0 tx=0).stop 2024-03-13T14:46:06.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 msgr2=0x7faef4196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:06.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4196310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7faee4005b40 tx=0x7faee400be20 comp rx=0 tx=0).stop 2024-03-13T14:46:06.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 shutdown_connections 2024-03-13T14:46:06.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7faed806c4e0 0x7faed806e9a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:06.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faef4101640 0x7faef4196310 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:06.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 --2- 172.21.15.176:0/505166282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4103f70 0x7faef4196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:06.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 >> 172.21.15.176:0/505166282 conn(0x7faef40fb050 msgr2=0x7faef40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:06.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 shutdown_connections 2024-03-13T14:46:06.146 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:06.155+0000 7faefb000700 1 -- 172.21.15.176:0/505166282 wait complete. 2024-03-13T14:46:06.146 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:46:06.160 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:46:06.954 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-03-13T14:46:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:46:07.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:46:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: pgmap v234: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.4 KiB/s rd, 5 op/s 2024-03-13T14:46:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/505166282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-13T14:46:07.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:07 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:46:07.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:07.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:07.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:46:07.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:46:07.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:07.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: pgmap v234: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.4 KiB/s rd, 5 op/s 2024-03-13T14:46:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/505166282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-13T14:46:07.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:07 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:46:07.652 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:09.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:09 smithi196 ceph-mon[37486]: pgmap v235: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 9 op/s 2024-03-13T14:46:09.452 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:09 smithi176 ceph-mon[29064]: pgmap v235: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.0 KiB/s rd, 9 op/s 2024-03-13T14:46:09.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.958+0000 7fc321e00700 1 -- 172.21.15.176:0/2701889356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c100a30 msgr2=0x7fc31c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:09.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.958+0000 7fc321e00700 1 --2- 172.21.15.176:0/2701889356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c100a30 0x7fc31c100e50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc310009910 tx=0x7fc310009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:09.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.959+0000 7fc321e00700 1 -- 172.21.15.176:0/2701889356 shutdown_connections 2024-03-13T14:46:09.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.959+0000 7fc321e00700 1 --2- 172.21.15.176:0/2701889356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc31c101c30 0x7fc31c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:09.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.959+0000 7fc321e00700 1 --2- 172.21.15.176:0/2701889356 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c100a30 0x7fc31c100e50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:09.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.959+0000 7fc321e00700 1 -- 172.21.15.176:0/2701889356 >> 172.21.15.176:0/2701889356 conn(0x7fc31c0fbfd0 msgr2=0x7fc31c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:09.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.959+0000 7fc321e00700 1 -- 172.21.15.176:0/2701889356 shutdown_connections 2024-03-13T14:46:09.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.959+0000 7fc321e00700 1 -- 172.21.15.176:0/2701889356 wait complete. 2024-03-13T14:46:09.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.960+0000 7fc321e00700 1 Processor -- start 2024-03-13T14:46:09.950 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.960+0000 7fc321e00700 1 -- start start 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.960+0000 7fc321e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc31c100a30 0x7fc31c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc321e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c101c30 0x7fc31c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc321e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c0736c0 con 0x7fc31c101c30 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc321e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31c073800 con 0x7fc31c100a30 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc31be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c101c30 0x7fc31c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc31be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c101c30 0x7fc31c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:51936/0 (socket says 172.21.15.176:51936) 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc31be00700 1 -- 172.21.15.176:0/185700606 learned_addr learned my addr 172.21.15.176:0/185700606 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc320a00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc31c100a30 0x7fc31c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc31be00700 1 -- 172.21.15.176:0/185700606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc31c100a30 msgr2=0x7fc31c074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc31be00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc31c100a30 0x7fc31c074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.961+0000 7fc31be00700 1 -- 172.21.15.176:0/185700606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100095f0 con 0x7fc31c101c30 2024-03-13T14:46:09.951 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.962+0000 7fc31be00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c101c30 0x7fc31c073180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c930 tx=0x7fc30800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:09.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.962+0000 7fc319600700 1 -- 172.21.15.176:0/185700606 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc308007ad0 con 0x7fc31c101c30 2024-03-13T14:46:09.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.962+0000 7fc319600700 1 -- 172.21.15.176:0/185700606 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc308007c30 con 0x7fc31c101c30 2024-03-13T14:46:09.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.962+0000 7fc319600700 1 -- 172.21.15.176:0/185700606 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc308018630 con 0x7fc31c101c30 2024-03-13T14:46:09.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.962+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31c073ae0 con 0x7fc31c101c30 2024-03-13T14:46:09.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.962+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31c19e1f0 con 0x7fc31c101c30 2024-03-13T14:46:09.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.963+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc31c109590 con 0x7fc31c101c30 2024-03-13T14:46:09.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.964+0000 7fc319600700 1 -- 172.21.15.176:0/185700606 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fc30800f450 con 0x7fc31c101c30 2024-03-13T14:46:09.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.965+0000 7fc319600700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc30406c4e0 0x7fc30406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:09.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.965+0000 7fc319600700 1 -- 172.21.15.176:0/185700606 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc30808ba20 con 0x7fc31c101c30 2024-03-13T14:46:09.954 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.965+0000 7fc320a00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc30406c4e0 0x7fc30406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:09.955 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.966+0000 7fc320a00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc30406c4e0 0x7fc30406e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc310006010 tx=0x7fc31000b580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:09.958 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:09.970+0000 7fc319600700 1 -- 172.21.15.176:0/185700606 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fc308059c10 con 0x7fc31c101c30 2024-03-13T14:46:10.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.277+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fc31c19ea30 con 0x7fc31c101c30 2024-03-13T14:46:10.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.278+0000 7fc319600700 1 -- 172.21.15.176:0/185700606 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v18) v1 ==== 78+0+83 (secure 0 0 0) 0x7fc3080597a0 con 0x7fc31c101c30 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc30406c4e0 msgr2=0x7fc30406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc30406c4e0 0x7fc30406e9a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc310006010 tx=0x7fc31000b580 comp rx=0 tx=0).stop 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c101c30 msgr2=0x7fc31c073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c101c30 0x7fc31c073180 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc30800c930 tx=0x7fc30800ccf0 comp rx=0 tx=0).stop 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 shutdown_connections 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fc30406c4e0 0x7fc30406e9a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc31c100a30 0x7fc31c074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 --2- 172.21.15.176:0/185700606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc31c101c30 0x7fc31c073180 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.281+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 >> 172.21.15.176:0/185700606 conn(0x7fc31c0fbfd0 msgr2=0x7fc31c104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.282+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 shutdown_connections 2024-03-13T14:46:10.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:10.282+0000 7fc321e00700 1 -- 172.21.15.176:0/185700606 wait complete. 2024-03-13T14:46:10.287 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:46:11.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:10 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:11.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:10 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:11.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:10 smithi196 ceph-mon[37486]: pgmap v236: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 8 op/s 2024-03-13T14:46:11.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:10 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/185700606' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-13T14:46:11.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:10 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:11.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:10 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:11.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:10 smithi176 ceph-mon[29064]: pgmap v236: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 8 op/s 2024-03-13T14:46:11.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:10 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/185700606' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-03-13T14:46:11.510 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-03-13T14:46:11.526 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T14:46:11.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.737+0000 7f162be00640 1 -- 172.21.15.176:0/3178608584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c101050 msgr2=0x7f162c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:11.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.737+0000 7f162be00640 1 --2- 172.21.15.176:0/3178608584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c101050 0x7f162c1014d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1618009880 tx=0x7f161802bce0 comp rx=0 tx=0).stop 2024-03-13T14:46:11.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.738+0000 7f162be00640 1 -- 172.21.15.176:0/3178608584 shutdown_connections 2024-03-13T14:46:11.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.738+0000 7f162be00640 1 --2- 172.21.15.176:0/3178608584 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c101050 0x7f162c1014d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:11.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.738+0000 7f162be00640 1 --2- 172.21.15.176:0/3178608584 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c0ffe70 0x7f162c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:11.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.738+0000 7f162be00640 1 -- 172.21.15.176:0/3178608584 >> 172.21.15.176:0/3178608584 conn(0x7f162c0fb410 msgr2=0x7f162c0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:11.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.738+0000 7f162be00640 1 -- 172.21.15.176:0/3178608584 shutdown_connections 2024-03-13T14:46:11.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.738+0000 7f162be00640 1 -- 172.21.15.176:0/3178608584 wait complete. 2024-03-13T14:46:11.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162be00640 1 Processor -- start 2024-03-13T14:46:11.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162be00640 1 -- start start 2024-03-13T14:46:11.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162be00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c0ffe70 0x7f162c195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:11.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162be00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c101050 0x7f162c195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162be00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c196250 con 0x7f162c0ffe70 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162be00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c1963c0 con 0x7f162c101050 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162aa00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c0ffe70 0x7f162c195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162aa00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c0ffe70 0x7f162c195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:51942/0 (socket says 172.21.15.176:51942) 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.739+0000 7f162aa00640 1 -- 172.21.15.176:0/1431821471 learned_addr learned my addr 172.21.15.176:0/1431821471 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f162a000640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c101050 0x7f162c195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f162aa00640 1 -- 172.21.15.176:0/1431821471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c101050 msgr2=0x7f162c195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f162aa00640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c101050 0x7f162c195c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f162aa00640 1 -- 172.21.15.176:0/1431821471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618009560 con 0x7f162c0ffe70 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f162aa00640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c0ffe70 0x7f162c195740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b3d0 tx=0x7f161c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f1623400640 1 -- 172.21.15.176:0/1431821471 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161c011800 con 0x7f162c0ffe70 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f1623400640 1 -- 172.21.15.176:0/1431821471 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f161c011e10 con 0x7f162c0ffe70 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c19ae50 con 0x7f162c0ffe70 2024-03-13T14:46:11.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f1623400640 1 -- 172.21.15.176:0/1431821471 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f161c00f510 con 0x7f162c0ffe70 2024-03-13T14:46:11.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.740+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c19b3d0 con 0x7f162c0ffe70 2024-03-13T14:46:11.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.741+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c108c30 con 0x7f162c0ffe70 2024-03-13T14:46:11.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.742+0000 7f1623400640 1 -- 172.21.15.176:0/1431821471 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f161c010380 con 0x7f162c0ffe70 2024-03-13T14:46:11.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.743+0000 7f1623400640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f161006c510 0x7f161006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:11.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.743+0000 7f1623400640 1 -- 172.21.15.176:0/1431821471 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f161c08b630 con 0x7f162c0ffe70 2024-03-13T14:46:11.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.743+0000 7f162a000640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f161006c510 0x7f161006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:11.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.744+0000 7f162a000640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f161006c510 0x7f161006e9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1618036040 tx=0x7f161802cf90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:11.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.748+0000 7f1623400640 1 -- 172.21.15.176:0/1431821471 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f161c055db0 con 0x7f162c0ffe70 2024-03-13T14:46:11.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.963+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f162c19b650 con 0x7f162c0ffe70 2024-03-13T14:46:11.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.964+0000 7f1623400640 1 -- 172.21.15.176:0/1431821471 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+15189 (secure 0 0 0) 0x7f161c0593d0 con 0x7f162c0ffe70 2024-03-13T14:46:11.954 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:46:11.954 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":47,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","created":"2024-03-13T14:36:32.120003+0000","modified":"2024-03-13T14:45:59.134398+0000","last_up_change":"2024-03-13T14:44:06.116104+0000","last_in_change":"2024-03-13T14:43:42.351790+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T14:42:22.643268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-13T14:45:37.278783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-13T14:45:37.880653+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c570fcfa-ceda-4e39-b200-5c8b0b7fab19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6803","nonce":2000924858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6805","nonce":2000924858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6809","nonce":2000924858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":2000924858},{"type":"v1","addr":"172.21.15.176:6807","nonce":2000924858}]},"public_addr":"172.21.15.176:6803/2000924858","cluster_addr":"172.21.15.176:6805/2000924858","heartbeat_back_addr":"172.21.15.176:6809/2000924858","heartbeat_front_addr":"172.21.15.176:6807/2000924858","state":["exists","up"]},{"osd":1,"uuid":"5a973902-7ecc-4a05-b917-a74e108df4db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6811","nonce":2968433057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6813","nonce":2968433057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6817","nonce":2968433057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2968433057},{"type":"v1","addr":"172.21.15.176:6815","nonce":2968433057}]},"public_addr":"172.21.15.176:6811/2968433057","cluster_addr":"172.21.15.176:6813/2968433057","heartbeat_back_addr":"172.21.15.176:6817/2968433057","heartbeat_front_addr":"172.21.15.176:6815/2968433057","state":["exists","up"]},{"osd":2,"uuid":"f72757d1-b949-45fb-abcd-c8649cec14f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6819","nonce":2465800922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6821","nonce":2465800922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6825","nonce":2465800922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2465800922},{"type":"v1","addr":"172.21.15.176:6823","nonce":2465800922}]},"public_addr":"172.21.15.176:6819/2465800922","cluster_addr":"172.21.15.176:6821/2465800922","heartbeat_back_addr":"172.21.15.176:6825/2465800922","heartbeat_front_addr":"172.21.15.176:6823/2465800922","state":["exists","up"]},{"osd":3,"uuid":"e1c42f2a-0efa-4c44-a965-9e5904aeace0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6801","nonce":3087694250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6803","nonce":3087694250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6807","nonce":3087694250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":3087694250},{"type":"v1","addr":"172.21.15.196:6805","nonce":3087694250}]},"public_addr":"172.21.15.196:6801/3087694250","cluster_addr":"172.21.15.196:6803/3087694250","heartbeat_back_addr":"172.21.15.196:6807/3087694250","heartbeat_front_addr":"172.21.15.196:6805/3087694250","state":["exists","up"]},{"osd":4,"uuid":"b4c7c89d-a349-4b39-b64a-e1080d213acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6809","nonce":111802733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6811","nonce":111802733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6815","nonce":111802733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":111802733},{"type":"v1","addr":"172.21.15.196:6813","nonce":111802733}]},"public_addr":"172.21.15.196:6809/111802733","cluster_addr":"172.21.15.196:6811/111802733","heartbeat_back_addr":"172.21.15.196:6815/111802733","heartbeat_front_addr":"172.21.15.196:6813/111802733","state":["exists","up"]},{"osd":5,"uuid":"efb96c7b-7a78-4c40-996f-f8141029c310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6817","nonce":3680704272}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6819","nonce":3680704272}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6823","nonce":3680704272}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3680704272},{"type":"v1","addr":"172.21.15.196:6821","nonce":3680704272}]},"public_addr":"172.21.15.196:6817/3680704272","cluster_addr":"172.21.15.196:6819/3680704272","heartbeat_back_addr":"172.21.15.196:6823/3680704272","heartbeat_front_addr":"172.21.15.196:6821/3680704272","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:06.529418+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:43.356501+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:19.060195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:53.551733+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:43:28.434416+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:44:05.009478+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.176:6827/3592546199":"2024-03-14T14:45:59.134341+0000","172.21.15.176:0/2332303582":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1567438135":"2024-03-14T14:37:05.490145+0000","172.21.15.176:6800/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1930887076":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6801/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:6800/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2108157434":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1319558216":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/248607826":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6801/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1477585567":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1825335586":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/698195875":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6801/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6800/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6826/3592546199":"2024-03-14T14:45:59.134341+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f161006c510 msgr2=0x7f161006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f161006c510 0x7f161006e9d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1618036040 tx=0x7f161802cf90 comp rx=0 tx=0).stop 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c0ffe70 msgr2=0x7f162c195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c0ffe70 0x7f162c195740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f161c00b3d0 tx=0x7f161c00b890 comp rx=0 tx=0).stop 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 shutdown_connections 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f161006c510 0x7f161006e9d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f162c101050 0x7f162c195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:11.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 --2- 172.21.15.176:0/1431821471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f162c0ffe70 0x7f162c195740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:11.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 >> 172.21.15.176:0/1431821471 conn(0x7f162c0fb410 msgr2=0x7f162c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:11.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 shutdown_connections 2024-03-13T14:46:11.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:11.967+0000 7f162be00640 1 -- 172.21.15.176:0/1431821471 wait complete. 2024-03-13T14:46:11.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-13T14:42:22.643268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-13T14:45:37.278783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-13T14:45:37.880653+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-13T14:46:11.974 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-13T14:46:12.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:11 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1431821471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.168+0000 7fa9e0400640 1 -- 172.21.15.176:0/206015810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 msgr2=0x7fa9d8106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.168+0000 7fa9e0400640 1 --2- 172.21.15.176:0/206015810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d8106a80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009880 tx=0x7fa9cc02bce0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.168+0000 7fa9e0400640 1 -- 172.21.15.176:0/206015810 shutdown_connections 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.168+0000 7fa9e0400640 1 --2- 172.21.15.176:0/206015810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d8107850 0x7fa9d8107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.168+0000 7fa9e0400640 1 --2- 172.21.15.176:0/206015810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d8106a80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.168+0000 7fa9e0400640 1 -- 172.21.15.176:0/206015810 >> 172.21.15.176:0/206015810 conn(0x7fa9d80757f0 msgr2=0x7fa9d8077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.168+0000 7fa9e0400640 1 -- 172.21.15.176:0/206015810 shutdown_connections 2024-03-13T14:46:12.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.169+0000 7fa9e0400640 1 -- 172.21.15.176:0/206015810 wait complete. 2024-03-13T14:46:12.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.169+0000 7fa9e0400640 1 Processor -- start 2024-03-13T14:46:12.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.169+0000 7fa9e0400640 1 -- start start 2024-03-13T14:46:12.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9e0400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d819bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:12.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9e0400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d8107850 0x7fa9d819c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:12.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9e0400640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d819ca30 con 0x7fa9d8106660 2024-03-13T14:46:12.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9e0400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d819cba0 con 0x7fa9d8107850 2024-03-13T14:46:12.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9dda00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d819bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9dda00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d819bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40744/0 (socket says 172.21.15.176:40744) 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9dda00640 1 -- 172.21.15.176:0/3585119940 learned_addr learned my addr 172.21.15.176:0/3585119940 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9dd000640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d8107850 0x7fa9d819c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9dda00640 1 -- 172.21.15.176:0/3585119940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d8107850 msgr2=0x7fa9d819c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9dda00640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d8107850 0x7fa9d819c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.170+0000 7fa9dda00640 1 -- 172.21.15.176:0/3585119940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc009560 con 0x7fa9d8106660 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.171+0000 7fa9dda00640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d819bf20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc02c220 tx=0x7fa9cc004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.171+0000 7fa9d6000640 1 -- 172.21.15.176:0/3585119940 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9cc0044a0 con 0x7fa9d8106660 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.171+0000 7fa9d6000640 1 -- 172.21.15.176:0/3585119940 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9cc0054b0 con 0x7fa9d8106660 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.171+0000 7fa9d6000640 1 -- 172.21.15.176:0/3585119940 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9cc034440 con 0x7fa9d8106660 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.171+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d81a15d0 con 0x7fa9d8106660 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.171+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d81a1ad0 con 0x7fa9d8106660 2024-03-13T14:46:12.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.172+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d810f3f0 con 0x7fa9d8106660 2024-03-13T14:46:12.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.173+0000 7fa9d6000640 1 -- 172.21.15.176:0/3585119940 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa9cc005000 con 0x7fa9d8106660 2024-03-13T14:46:12.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.174+0000 7fa9d6000640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa9c006c450 0x7fa9c006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:12.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.174+0000 7fa9d6000640 1 -- 172.21.15.176:0/3585119940 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa9cc0adf40 con 0x7fa9d8106660 2024-03-13T14:46:12.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.174+0000 7fa9dd000640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa9c006c450 0x7fa9c006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:12.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.175+0000 7fa9dd000640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa9c006c450 0x7fa9c006e910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4005fd0 tx=0x7fa9c4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:12.167 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.178+0000 7fa9d6000640 1 -- 172.21.15.176:0/3585119940 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa9cc078620 con 0x7fa9d8106660 2024-03-13T14:46:12.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.360+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9d81a1fa0 con 0x7fa9d8106660 2024-03-13T14:46:12.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.361+0000 7fa9d6000640 1 -- 172.21.15.176:0/3585119940 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9cc07bc40 con 0x7fa9d8106660 2024-03-13T14:46:12.350 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 1 2024-03-13T14:46:12.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa9c006c450 msgr2=0x7fa9c006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa9c006c450 0x7fa9c006e910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4005fd0 tx=0x7fa9c4005850 comp rx=0 tx=0).stop 2024-03-13T14:46:12.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 msgr2=0x7fa9d819bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d819bf20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc02c220 tx=0x7fa9cc004dc0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 shutdown_connections 2024-03-13T14:46:12.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa9c006c450 0x7fa9c006e910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9d8107850 0x7fa9d819c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 --2- 172.21.15.176:0/3585119940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa9d8106660 0x7fa9d819bf20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 >> 172.21.15.176:0/3585119940 conn(0x7fa9d80757f0 msgr2=0x7fa9d810aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:12.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.364+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 shutdown_connections 2024-03-13T14:46:12.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.365+0000 7fa9e0400640 1 -- 172.21.15.176:0/3585119940 wait complete. 2024-03-13T14:46:12.375 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-13T14:46:12.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:11 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1431821471' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 -- 172.21.15.176:0/1830012836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c101090 msgr2=0x7f554c101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 --2- 172.21.15.176:0/1830012836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c101090 0x7f554c101510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5538009880 tx=0x7f553802bce0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 -- 172.21.15.176:0/1830012836 shutdown_connections 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 --2- 172.21.15.176:0/1830012836 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c101090 0x7f554c101510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 --2- 172.21.15.176:0/1830012836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f554c0ffeb0 0x7f554c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 -- 172.21.15.176:0/1830012836 >> 172.21.15.176:0/1830012836 conn(0x7f554c0fb470 msgr2=0x7f554c0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 -- 172.21.15.176:0/1830012836 shutdown_connections 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.565+0000 7f5551a00640 1 -- 172.21.15.176:0/1830012836 wait complete. 2024-03-13T14:46:12.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.566+0000 7f5551a00640 1 Processor -- start 2024-03-13T14:46:12.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.566+0000 7f5551a00640 1 -- start start 2024-03-13T14:46:12.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.566+0000 7f5551a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c0ffeb0 0x7f554c195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:12.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.566+0000 7f5551a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f554c101090 0x7f554c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:12.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f5551a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c196310 con 0x7f554c0ffeb0 2024-03-13T14:46:12.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f5551a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c196480 con 0x7f554c101090 2024-03-13T14:46:12.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f554be00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c0ffeb0 0x7f554c195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f554be00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c0ffeb0 0x7f554c195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40760/0 (socket says 172.21.15.176:40760) 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f554be00640 1 -- 172.21.15.176:0/4269953387 learned_addr learned my addr 172.21.15.176:0/4269953387 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f554be00640 1 -- 172.21.15.176:0/4269953387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f554c101090 msgr2=0x7f554c195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f554be00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f554c101090 0x7f554c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f554be00640 1 -- 172.21.15.176:0/4269953387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5538009560 con 0x7f554c0ffeb0 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f554be00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c0ffeb0 0x7f554c195800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f554400b9c0 tx=0x7f554400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.567+0000 7f5548c00640 1 -- 172.21.15.176:0/4269953387 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554400caf0 con 0x7f554c0ffeb0 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.568+0000 7f5548c00640 1 -- 172.21.15.176:0/4269953387 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f554400cc50 con 0x7f554c0ffeb0 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.568+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c19af10 con 0x7f554c0ffeb0 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.568+0000 7f5548c00640 1 -- 172.21.15.176:0/4269953387 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5544012770 con 0x7f554c0ffeb0 2024-03-13T14:46:12.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.568+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c19b4c0 con 0x7f554c0ffeb0 2024-03-13T14:46:12.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.569+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554c108ae0 con 0x7f554c0ffeb0 2024-03-13T14:46:12.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.570+0000 7f5548c00640 1 -- 172.21.15.176:0/4269953387 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f55440129e0 con 0x7f554c0ffeb0 2024-03-13T14:46:12.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.570+0000 7f5548c00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f553006c450 0x7f553006e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:12.560 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.570+0000 7f5548c00640 1 -- 172.21.15.176:0/4269953387 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f554408b240 con 0x7f554c0ffeb0 2024-03-13T14:46:12.560 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.571+0000 7f554b400640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f553006c450 0x7f553006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:12.560 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.571+0000 7f554b400640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f553006c450 0x7f553006e910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f553802c7c0 tx=0x7f5538000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:12.563 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.575+0000 7f5548c00640 1 -- 172.21.15.176:0/4269953387 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f55440559d0 con 0x7f554c0ffeb0 2024-03-13T14:46:12.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.787+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f554c19b740 con 0x7f554c0ffeb0 2024-03-13T14:46:12.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.788+0000 7f5548c00640 1 -- 172.21.15.176:0/4269953387 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7f5544058ff0 con 0x7f554c0ffeb0 2024-03-13T14:46:12.777 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T14:46:12.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f553006c450 msgr2=0x7f553006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f553006c450 0x7f553006e910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f553802c7c0 tx=0x7f5538000bc0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c0ffeb0 msgr2=0x7f554c195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:12.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c0ffeb0 0x7f554c195800 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f554400b9c0 tx=0x7f554400be80 comp rx=0 tx=0).stop 2024-03-13T14:46:12.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 shutdown_connections 2024-03-13T14:46:12.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f553006c450 0x7f553006e910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f554c101090 0x7f554c195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 --2- 172.21.15.176:0/4269953387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f554c0ffeb0 0x7f554c195800 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:12.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 >> 172.21.15.176:0/4269953387 conn(0x7f554c0fb470 msgr2=0x7f554c104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:12.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 shutdown_connections 2024-03-13T14:46:12.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:12.791+0000 7f5551a00640 1 -- 172.21.15.176:0/4269953387 wait complete. 2024-03-13T14:46:12.799 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-13T14:46:13.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 -- 172.21.15.176:0/793941628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 msgr2=0x7fa304107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 --2- 172.21.15.176:0/793941628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa304107d50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009880 tx=0x7fa2ec02bce0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 -- 172.21.15.176:0/793941628 shutdown_connections 2024-03-13T14:46:13.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 --2- 172.21.15.176:0/793941628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa304107d50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 --2- 172.21.15.176:0/793941628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3041066e0 0x7fa304106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 -- 172.21.15.176:0/793941628 >> 172.21.15.176:0/793941628 conn(0x7fa3040757f0 msgr2=0x7fa304077c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:13.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 -- 172.21.15.176:0/793941628 shutdown_connections 2024-03-13T14:46:13.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.011+0000 7fa308a00640 1 -- 172.21.15.176:0/793941628 wait complete. 2024-03-13T14:46:13.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.012+0000 7fa308a00640 1 Processor -- start 2024-03-13T14:46:13.002 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.012+0000 7fa308a00640 1 -- start start 2024-03-13T14:46:13.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.012+0000 7fa308a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3041066e0 0x7fa30419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:13.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa308a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa30419c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa308a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30419cb30 con 0x7fa3041078d0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa308a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30419cca0 con 0x7fa3041066e0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa2f9400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa30419c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa2f9400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa30419c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40782/0 (socket says 172.21.15.176:40782) 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa2f9400640 1 -- 172.21.15.176:0/3878613222 learned_addr learned my addr 172.21.15.176:0/3878613222 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa2f9400640 1 -- 172.21.15.176:0/3878613222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3041066e0 msgr2=0x7fa30419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa2f9400640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3041066e0 0x7fa30419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa2f9400640 1 -- 172.21.15.176:0/3878613222 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec009560 con 0x7fa3041078d0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.013+0000 7fa2f9400640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa30419c560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000c00 tx=0x7fa2ec004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.014+0000 7fa2fb400640 1 -- 172.21.15.176:0/3878613222 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ec005000 con 0x7fa3041078d0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.014+0000 7fa2fb400640 1 -- 172.21.15.176:0/3878613222 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa2ec005610 con 0x7fa3041078d0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.014+0000 7fa2fb400640 1 -- 172.21.15.176:0/3878613222 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ec034670 con 0x7fa3041078d0 2024-03-13T14:46:13.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.014+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3041a16d0 con 0x7fa3041078d0 2024-03-13T14:46:13.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.014+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3041a1c50 con 0x7fa3041078d0 2024-03-13T14:46:13.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.015+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30410f4b0 con 0x7fa3041078d0 2024-03-13T14:46:13.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.016+0000 7fa2fb400640 1 -- 172.21.15.176:0/3878613222 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fa2ec0039a0 con 0x7fa3041078d0 2024-03-13T14:46:13.005 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.017+0000 7fa2fb400640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa2e806c450 0x7fa2e806e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:13.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.017+0000 7fa2fb400640 1 -- 172.21.15.176:0/3878613222 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa2ec0addf0 con 0x7fa3041078d0 2024-03-13T14:46:13.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.017+0000 7fa301e00640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa2e806c450 0x7fa2e806e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:13.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.017+0000 7fa301e00640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa2e806c450 0x7fa2e806e910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa304107730 tx=0x7fa2f4006c90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:13.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.021+0000 7fa2fb400640 1 -- 172.21.15.176:0/3878613222 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fa2ec0785f0 con 0x7fa3041078d0 2024-03-13T14:46:13.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.204+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa30410f640 con 0x7fa3041078d0 2024-03-13T14:46:13.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.205+0000 7fa2fb400640 1 -- 172.21.15.176:0/3878613222 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v47) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa30410f640 con 0x7fa3041078d0 2024-03-13T14:46:13.194 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T14:46:13.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa2e806c450 msgr2=0x7fa2e806e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa2e806c450 0x7fa2e806e910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa304107730 tx=0x7fa2f4006c90 comp rx=0 tx=0).stop 2024-03-13T14:46:13.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 msgr2=0x7fa30419c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa30419c560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec000c00 tx=0x7fa2ec004f00 comp rx=0 tx=0).stop 2024-03-13T14:46:13.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 shutdown_connections 2024-03-13T14:46:13.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fa2e806c450 0x7fa2e806e910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3041078d0 0x7fa30419c560 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 --2- 172.21.15.176:0/3878613222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3041066e0 0x7fa30419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 >> 172.21.15.176:0/3878613222 conn(0x7fa3040757f0 msgr2=0x7fa30410ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:13.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 shutdown_connections 2024-03-13T14:46:13.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.207+0000 7fa308a00640 1 -- 172.21.15.176:0/3878613222 wait complete. 2024-03-13T14:46:13.216 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-13T14:46:13.284 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:12 smithi176 ceph-mon[29064]: pgmap v237: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-13T14:46:13.284 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:12 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3585119940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T14:46:13.284 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:12 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/4269953387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.417+0000 7f1b27400640 1 -- 172.21.15.176:0/614952213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201022d0 msgr2=0x7f1b201026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.417+0000 7f1b27400640 1 --2- 172.21.15.176:0/614952213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201022d0 0x7f1b201026f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009880 tx=0x7f1b1402bce0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.417+0000 7f1b27400640 1 -- 172.21.15.176:0/614952213 shutdown_connections 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.417+0000 7f1b27400640 1 --2- 172.21.15.176:0/614952213 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b201034c0 0x7f1b20103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.417+0000 7f1b27400640 1 --2- 172.21.15.176:0/614952213 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201022d0 0x7f1b201026f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.417+0000 7f1b27400640 1 -- 172.21.15.176:0/614952213 >> 172.21.15.176:0/614952213 conn(0x7f1b200fd820 msgr2=0x7f1b200ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.417+0000 7f1b27400640 1 -- 172.21.15.176:0/614952213 shutdown_connections 2024-03-13T14:46:13.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.418+0000 7f1b27400640 1 -- 172.21.15.176:0/614952213 wait complete. 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.418+0000 7f1b27400640 1 Processor -- start 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b27400640 1 -- start start 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b27400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b201022d0 0x7f1b20197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b27400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201034c0 0x7f1b20197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b27400640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b20198590 con 0x7f1b201034c0 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b27400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b20198700 con 0x7f1b201022d0 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b1fe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201034c0 0x7f1b20197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:13.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b1fe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201034c0 0x7f1b20197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40808/0 (socket says 172.21.15.176:40808) 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b1fe00640 1 -- 172.21.15.176:0/2580778276 learned_addr learned my addr 172.21.15.176:0/2580778276 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b24a00640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b201022d0 0x7f1b20197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b1fe00640 1 -- 172.21.15.176:0/2580778276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b201022d0 msgr2=0x7f1b20197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b1fe00640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b201022d0 0x7f1b20197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.419+0000 7f1b1fe00640 1 -- 172.21.15.176:0/2580778276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b14009560 con 0x7f1b201034c0 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.420+0000 7f1b1fe00640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201034c0 0x7f1b20197fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c8f0 tx=0x7f1b0c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.420+0000 7f1b1d600640 1 -- 172.21.15.176:0/2580778276 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c007b80 con 0x7f1b201034c0 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.420+0000 7f1b1d600640 1 -- 172.21.15.176:0/2580778276 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1b0c007ce0 con 0x7f1b201034c0 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.420+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2019d190 con 0x7f1b201034c0 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.420+0000 7f1b1d600640 1 -- 172.21.15.176:0/2580778276 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b0c00f450 con 0x7f1b201034c0 2024-03-13T14:46:13.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.420+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2019d6f0 con 0x7f1b201034c0 2024-03-13T14:46:13.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.421+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2010b020 con 0x7f1b201034c0 2024-03-13T14:46:13.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.422+0000 7f1b1d600640 1 -- 172.21.15.176:0/2580778276 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f1b0c004f60 con 0x7f1b201034c0 2024-03-13T14:46:13.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.423+0000 7f1b1d600640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1b0806c510 0x7f1b0806e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:13.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.423+0000 7f1b1d600640 1 -- 172.21.15.176:0/2580778276 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1b0c08b610 con 0x7f1b201034c0 2024-03-13T14:46:13.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.424+0000 7f1b24a00640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1b0806c510 0x7f1b0806e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:13.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.424+0000 7f1b24a00640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1b0806c510 0x7f1b0806e9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b14006010 tx=0x7f1b1402cf90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:13.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.428+0000 7f1b1d600640 1 -- 172.21.15.176:0/2580778276 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f1b0c055d70 con 0x7f1b201034c0 2024-03-13T14:46:13.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:12 smithi196 ceph-mon[37486]: pgmap v237: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.9 KiB/s rd, 0 B/s wr, 8 op/s 2024-03-13T14:46:13.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:12 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3585119940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T14:46:13.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:12 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/4269953387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T14:46:13.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.661+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b2019d9c0 con 0x7f1b201034c0 2024-03-13T14:46:13.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.662+0000 7f1b1d600640 1 -- 172.21.15.176:0/2580778276 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4753 (secure 0 0 0) 0x7f1b0c059390 con 0x7f1b201034c0 2024-03-13T14:46:13.651 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:46:13.652 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/668575330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":668575330},{"type":"v1","addr":"172.21.15.176:6827","nonce":668575330}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T14:46:02.172324+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14484":{"gid":14484,"name":"cephfs.smithi176.hjrgov","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":3,"addr":"172.21.15.176:6829/4135138463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":4135138463},{"type":"v1","addr":"172.21.15.176:6829","nonce":4135138463}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24249":{"gid":24249,"name":"cephfs.smithi196.tcitrk","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6825/152396394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":152396394},{"type":"v1","addr":"172.21.15.196:6825","nonce":152396394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T14:46:13.652 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:46:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1b0806c510 msgr2=0x7f1b0806e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1b0806c510 0x7f1b0806e9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b14006010 tx=0x7f1b1402cf90 comp rx=0 tx=0).stop 2024-03-13T14:46:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201034c0 msgr2=0x7f1b20197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201034c0 0x7f1b20197fc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c00c8f0 tx=0x7f1b0c00cdb0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 shutdown_connections 2024-03-13T14:46:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f1b0806c510 0x7f1b0806e9d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1b201034c0 0x7f1b20197fc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 --2- 172.21.15.176:0/2580778276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b201022d0 0x7f1b20197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 >> 172.21.15.176:0/2580778276 conn(0x7f1b200fd820 msgr2=0x7f1b201066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 shutdown_connections 2024-03-13T14:46:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:13.665+0000 7f1b27400640 1 -- 172.21.15.176:0/2580778276 wait complete. 2024-03-13T14:46:13.672 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-03-13T14:46:13.672 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-13T14:46:13.699 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-13T14:46:13.699 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-03-13T14:46:13.699 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-13T14:46:13.699 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-13T14:46:13.699 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:13.699 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-13T14:46:13.700 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-13T14:46:13.700 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:13.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:13.700 DEBUG:teuthology.orchestra.run.smithi176:> ip netns list 2024-03-13T14:46:13.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:13.738 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link delete ceph-brx 2024-03-13T14:46:13.872 INFO:teuthology.orchestra.run.smithi176.stderr:Cannot find device "ceph-brx" 2024-03-13T14:46:13.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T14:46:13.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:13.876 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-13T14:46:13.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:13.933 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-03-13T14:46:14.030 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-03-13T14:46:14.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T14:46:14.033 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-13T14:46:14.034 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-13T14:46:14.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:14.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:13 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3878613222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T14:46:14.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:13 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2580778276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-13T14:46:14.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:14.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:13 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3878613222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T14:46:14.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:13 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2580778276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-13T14:46:14.449 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:15 smithi176 ceph-mon[29064]: pgmap v238: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-13T14:46:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:15 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:15 smithi196 ceph-mon[37486]: pgmap v238: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-13T14:46:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:15 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:16.037 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:46:16.037 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:46:16.037 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:46:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:46:16.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:46:16.537 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.547+0000 7f5fcb400700 1 -- 172.21.15.176:0/1318053952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4100da0 msgr2=0x7f5fc4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:16.537 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.547+0000 7f5fcb400700 1 --2- 172.21.15.176:0/1318053952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4100da0 0x7f5fc4101180 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009910 tx=0x7f5fac009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:16.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.548+0000 7f5fcb400700 1 -- 172.21.15.176:0/1318053952 shutdown_connections 2024-03-13T14:46:16.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.548+0000 7f5fcb400700 1 --2- 172.21.15.176:0/1318053952 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc4101750 0x7f5fc41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:16.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.548+0000 7f5fcb400700 1 --2- 172.21.15.176:0/1318053952 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4100da0 0x7f5fc4101180 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:16.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.548+0000 7f5fcb400700 1 -- 172.21.15.176:0/1318053952 >> 172.21.15.176:0/1318053952 conn(0x7f5fc40fc630 msgr2=0x7f5fc40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:16.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.548+0000 7f5fcb400700 1 -- 172.21.15.176:0/1318053952 shutdown_connections 2024-03-13T14:46:16.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.548+0000 7f5fcb400700 1 -- 172.21.15.176:0/1318053952 wait complete. 2024-03-13T14:46:16.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.549+0000 7f5fcb400700 1 Processor -- start 2024-03-13T14:46:16.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.549+0000 7f5fcb400700 1 -- start start 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fcb400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc4100da0 0x7f5fc4073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fcb400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4101750 0x7f5fc40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fcb400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc4073b10 con 0x7f5fc4101750 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fcb400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc4073c50 con 0x7f5fc4100da0 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fbbe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4101750 0x7f5fc40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fbbe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4101750 0x7f5fc40735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40826/0 (socket says 172.21.15.176:40826) 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fbbe00700 1 -- 172.21.15.176:0/503509021 learned_addr learned my addr 172.21.15.176:0/503509021 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.550+0000 7f5fc8a00700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc4100da0 0x7f5fc4073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fbbe00700 1 -- 172.21.15.176:0/503509021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc4100da0 msgr2=0x7f5fc4073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:16.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fbbe00700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc4100da0 0x7f5fc4073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:16.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fbbe00700 1 -- 172.21.15.176:0/503509021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0095f0 con 0x7f5fc4101750 2024-03-13T14:46:16.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fc8a00700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc4100da0 0x7f5fc4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T14:46:16.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fbbe00700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4101750 0x7f5fc40735d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400d900 tx=0x7f5fb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:16.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fc2000700 1 -- 172.21.15.176:0/503509021 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fb4009900 con 0x7f5fc4101750 2024-03-13T14:46:16.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc41a29a0 con 0x7f5fc4101750 2024-03-13T14:46:16.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fc2000700 1 -- 172.21.15.176:0/503509021 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5fb4010460 con 0x7f5fc4101750 2024-03-13T14:46:16.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.551+0000 7f5fc2000700 1 -- 172.21.15.176:0/503509021 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fb400f5f0 con 0x7f5fc4101750 2024-03-13T14:46:16.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.552+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc41a2db0 con 0x7f5fc4101750 2024-03-13T14:46:16.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.553+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc4109140 con 0x7f5fc4101750 2024-03-13T14:46:16.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.554+0000 7f5fc2000700 1 -- 172.21.15.176:0/503509021 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5fb40105d0 con 0x7f5fc4101750 2024-03-13T14:46:16.544 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.555+0000 7f5fc2000700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5fa406c4e0 0x7f5fa406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:16.544 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.555+0000 7f5fc2000700 1 -- 172.21.15.176:0/503509021 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5fb408b5f0 con 0x7f5fc4101750 2024-03-13T14:46:16.544 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.555+0000 7f5fc8a00700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5fa406c4e0 0x7f5fa406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:16.545 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.556+0000 7f5fc8a00700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5fa406c4e0 0x7f5fa406e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fac006010 tx=0x7f5fac01a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:16.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.560+0000 7f5fc2000700 1 -- 172.21.15.176:0/503509021 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5fb4055df0 con 0x7f5fc4101750 2024-03-13T14:46:16.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.817+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5fc404ea90 con 0x7f5fc4101750 2024-03-13T14:46:16.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.818+0000 7f5fc2000700 1 -- 172.21.15.176:0/503509021 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5fb4059410 con 0x7f5fc4101750 2024-03-13T14:46:16.807 INFO:teuthology.orchestra.run.smithi176.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5fa406c4e0 msgr2=0x7f5fa406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5fa406c4e0 0x7f5fa406e9a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5fac006010 tx=0x7f5fac01a040 comp rx=0 tx=0).stop 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4101750 msgr2=0x7f5fc40735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4101750 0x7f5fc40735d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400d900 tx=0x7f5fb400dcc0 comp rx=0 tx=0).stop 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 shutdown_connections 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5fa406c4e0 0x7f5fa406e9a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fc4100da0 0x7f5fc4073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 --2- 172.21.15.176:0/503509021 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5fc4101750 0x7f5fc40735d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.820+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 >> 172.21.15.176:0/503509021 conn(0x7f5fc40fc630 msgr2=0x7f5fc40fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:16.811 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.821+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 shutdown_connections 2024-03-13T14:46:16.811 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:16.821+0000 7f5fcb400700 1 -- 172.21.15.176:0/503509021 wait complete. 2024-03-13T14:46:17.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:17 smithi176 ceph-mon[29064]: pgmap v239: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-13T14:46:17.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:17 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/503509021' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-13T14:46:17.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:17 smithi196 ceph-mon[37486]: pgmap v239: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 7 op/s 2024-03-13T14:46:17.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:17 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/503509021' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-13T14:46:17.689 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-13T14:46:17.689 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-13T14:46:17.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi176.front.sepia.ceph.com 2024-03-13T14:46:17.690 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-13T14:46:17.690 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:17.690 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-13T14:46:17.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-13T14:46:17.690 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-13T14:46:17.690 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-13T14:46:17.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:17.690 DEBUG:teuthology.orchestra.run.smithi176:> ip addr 2024-03-13T14:46:17.744 INFO:teuthology.orchestra.run.smithi176.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-13T14:46:17.744 INFO:teuthology.orchestra.run.smithi176.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-13T14:46:17.744 INFO:teuthology.orchestra.run.smithi176.stdout: inet 127.0.0.1/8 scope host lo 2024-03-13T14:46:17.744 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft forever preferred_lft forever 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: inet6 ::1/128 scope host 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft forever preferred_lft forever 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:d9:36:80 brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: altname enp7s0f0 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:d9:36:81 brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: altname enp7s0f1 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:88:6f:60 brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: link/ether 0c:c4:7a:88:6f:61 brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: inet 172.21.15.176/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft 42021sec preferred_lft 42021sec 2024-03-13T14:46:17.745 INFO:teuthology.orchestra.run.smithi176.stdout: inet6 fe80::ec4:7aff:fe88:6f61/64 scope link 2024-03-13T14:46:17.746 INFO:teuthology.orchestra.run.smithi176.stdout: valid_lft forever preferred_lft forever 2024-03-13T14:46:17.747 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-13T14:46:17.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:17.747 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec bash -c ' 2024-03-13T14:46:17.747 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-03-13T14:46:17.747 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link add name ceph-brx type bridge 2024-03-13T14:46:17.747 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip addr flush dev ceph-brx 2024-03-13T14:46:17.747 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link set ceph-brx up 2024-03-13T14:46:17.747 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-13T14:46:17.747 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-03-13T14:46:18.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:18.190 DEBUG:teuthology.orchestra.run.smithi176:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-13T14:46:18.267 INFO:teuthology.orchestra.run.smithi176.stdout:1 2024-03-13T14:46:18.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:18.270 DEBUG:teuthology.orchestra.run.smithi176:> route 2024-03-13T14:46:18.325 INFO:teuthology.orchestra.run.smithi176.stdout:Kernel IP routing table 2024-03-13T14:46:18.326 INFO:teuthology.orchestra.run.smithi176.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-13T14:46:18.326 INFO:teuthology.orchestra.run.smithi176.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-13T14:46:18.326 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-13T14:46:18.326 INFO:teuthology.orchestra.run.smithi176.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-13T14:46:18.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:18.326 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec bash -c ' 2024-03-13T14:46:18.326 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-03-13T14:46:18.326 DEBUG:teuthology.orchestra.run.smithi176:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-13T14:46:18.327 DEBUG:teuthology.orchestra.run.smithi176:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-13T14:46:18.327 DEBUG:teuthology.orchestra.run.smithi176:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-13T14:46:18.327 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-03-13T14:46:18.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:18.988 DEBUG:teuthology.orchestra.run.smithi176:> ip netns list 2024-03-13T14:46:19.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:19.043 DEBUG:teuthology.orchestra.run.smithi176:> ip netns list-id 2024-03-13T14:46:19.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:19.096 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec bash -c ' 2024-03-13T14:46:19.096 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-03-13T14:46:19.097 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-13T14:46:19.097 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-13T14:46:19.097 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-03-13T14:46:19.232 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:19 smithi176 ceph-mon[29064]: pgmap v240: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-13T14:46:19.235 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-13T14:46:19.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec bash -c ' 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-13T14:46:19.236 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-03-13T14:46:19.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:19 smithi196 ceph-mon[37486]: pgmap v240: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.9 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-13T14:46:19.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:19.512 DEBUG:teuthology.orchestra.run.smithi176:> (cd / && exec bash -c ' 2024-03-13T14:46:19.512 DEBUG:teuthology.orchestra.run.smithi176:> set -e 2024-03-13T14:46:19.512 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link set brx.0 up 2024-03-13T14:46:19.512 DEBUG:teuthology.orchestra.run.smithi176:> sudo ip link set dev brx.0 master ceph-brx 2024-03-13T14:46:19.512 DEBUG:teuthology.orchestra.run.smithi176:> ') 2024-03-13T14:46:19.677 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-13T14:46:19.677 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-13T14:46:19.677 DEBUG:teuthology.orchestra.run.smithi176:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:19.722 INFO:teuthology.orchestra.run.smithi176.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-13T14:46:19.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-13T14:46:19.723 DEBUG:teuthology.orchestra.run.smithi176:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:19.818 DEBUG:teuthology.orchestra.run.smithi176:> sudo modprobe fuse 2024-03-13T14:46:19.912 DEBUG:teuthology.orchestra.run.smithi176:> cat /proc/self/mounts | awk '{print $2}' 2024-03-13T14:46:19.964 INFO:teuthology.orchestra.run.smithi176.stdout:/proc 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/sys 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/dev 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/security 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/shm 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/pts 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/run 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/cgroup 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/pstore 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/bpf 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/ 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/selinux 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/hugepages 2024-03-13T14:46:19.965 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/mqueue 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/debug 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/tracing 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/fuse/connections 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/config 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-sysctl.service 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/nfs/rpc_pipefs 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/1000 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/ceph 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b40729237e0d4ab3c7e21b102e6c46bbe559f78b5dd44957763a67dee1297f49/merged 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/68b71964c6da8fdb3123d54d1348e72ca05058f5e8bcff7c6b2520fa7802a957/merged 2024-03-13T14:46:19.966 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/0 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/7197754f57fed7d6aeba1da31d044bdc99cc10c7a441a3912d7a24009a526f0e/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/5b046d7d1031d4c8142c627de68968218c7a0d8e48146da45222c2bd7e037b35/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/294599cff23f9f17641a507a056b3f4237d7357c0a5e7841223a3feded4189fc/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/47865500b812d2dcb61d48709002f3077f5420098026621fba1568d0d1a90f85/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/d3d8dc01661f5e9661d00faa59cf75ef626ab5d0f36ad053e4c587ce08547c20/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/19500e3b0fc2da340e67a2a1c2002c3d84960cf7cc5e6620bcc1046c2f10424f/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/724c2b9e65499e63eaac600d7e321a182dc0f419c823452916196e09904aba8e/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/0083370c822baea0cafdc56bb7f42ef81d6c0eed1f18545593b6752766609ca9/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b3f5bfb76903724407fb9244797e14742337d4a6e6cb512340d4e11f4ba6469e/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/1253785850e7e080d9a8adaed3c703977c3d884f77e0f72242a4ee245cf193b1/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/88587999dcee258687820214bb82779f9edaa46a7ab3b01391a9fb84ab198637/merged 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-13T14:46:19.967 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-13T14:46:19.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:19.969 DEBUG:teuthology.orchestra.run.smithi176:> ls /sys/fs/fuse/connections 2024-03-13T14:46:20.013 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-13T14:46:20.013 DEBUG:teuthology.orchestra.run.smithi176:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-13T14:46:20.061 DEBUG:teuthology.orchestra.run.smithi176:> sudo modprobe fuse 2024-03-13T14:46:20.142 DEBUG:teuthology.orchestra.run.smithi176:> cat /proc/self/mounts | awk '{print $2}' 2024-03-13T14:46:20.186 INFO:teuthology.orchestra.run.smithi176.stdout:/proc 2024-03-13T14:46:20.186 INFO:teuthology.orchestra.run.smithi176.stdout:/sys 2024-03-13T14:46:20.186 INFO:teuthology.orchestra.run.smithi176.stdout:/dev 2024-03-13T14:46:20.186 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/security 2024-03-13T14:46:20.186 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/shm 2024-03-13T14:46:20.186 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/pts 2024-03-13T14:46:20.186 INFO:teuthology.orchestra.run.smithi176.stdout:/run 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/cgroup 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/pstore 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/bpf 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/ 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/selinux 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/hugepages 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/mqueue 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/debug 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/tracing 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/fuse/connections 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/config 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-sysctl.service 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-13T14:46:20.187 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/nfs/rpc_pipefs 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/1000 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/ceph 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b40729237e0d4ab3c7e21b102e6c46bbe559f78b5dd44957763a67dee1297f49/merged 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/68b71964c6da8fdb3123d54d1348e72ca05058f5e8bcff7c6b2520fa7802a957/merged 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/0 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/7197754f57fed7d6aeba1da31d044bdc99cc10c7a441a3912d7a24009a526f0e/merged 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/5b046d7d1031d4c8142c627de68968218c7a0d8e48146da45222c2bd7e037b35/merged 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/294599cff23f9f17641a507a056b3f4237d7357c0a5e7841223a3feded4189fc/merged 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/47865500b812d2dcb61d48709002f3077f5420098026621fba1568d0d1a90f85/merged 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/d3d8dc01661f5e9661d00faa59cf75ef626ab5d0f36ad053e4c587ce08547c20/merged 2024-03-13T14:46:20.188 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/19500e3b0fc2da340e67a2a1c2002c3d84960cf7cc5e6620bcc1046c2f10424f/merged 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/724c2b9e65499e63eaac600d7e321a182dc0f419c823452916196e09904aba8e/merged 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/0083370c822baea0cafdc56bb7f42ef81d6c0eed1f18545593b6752766609ca9/merged 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b3f5bfb76903724407fb9244797e14742337d4a6e6cb512340d4e11f4ba6469e/merged 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/1253785850e7e080d9a8adaed3c703977c3d884f77e0f72242a4ee245cf193b1/merged 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/88587999dcee258687820214bb82779f9edaa46a7ab3b01391a9fb84ab198637/merged 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-13T14:46:20.189 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-13T14:46:20.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:20.190 DEBUG:teuthology.orchestra.run.smithi176:> ls /sys/fs/fuse/connections 2024-03-13T14:46:20.279 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi176.stderr:2024-03-13T14:46:20.291+0000 7f385d613480 -1 init, newargv = 0x5579085fed60 newargc=15 2024-03-13T14:46:20.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi176.stderr:ceph-fuse[76677]: starting ceph client 2024-03-13T14:46:20.294 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi176.stderr:ceph-fuse[76677]: starting fuse 2024-03-13T14:46:21.376 DEBUG:teuthology.orchestra.run.smithi176:> sudo modprobe fuse 2024-03-13T14:46:21.385 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:20 smithi176 ceph-mon[29064]: pgmap v241: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-13T14:46:21.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:20 smithi196 ceph-mon[37486]: pgmap v241: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-13T14:46:21.478 DEBUG:teuthology.orchestra.run.smithi176:> cat /proc/self/mounts | awk '{print $2}' 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/proc 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/sys 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/dev 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/security 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/shm 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/pts 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/run 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/cgroup 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/pstore 2024-03-13T14:46:21.537 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/bpf 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/ 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/selinux 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/hugepages 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/dev/mqueue 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/debug 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/tracing 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/fs/fuse/connections 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/sys/kernel/config 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-sysctl.service 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-13T14:46:21.538 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/nfs/rpc_pipefs 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/1000 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/ceph 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b40729237e0d4ab3c7e21b102e6c46bbe559f78b5dd44957763a67dee1297f49/merged 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/68b71964c6da8fdb3123d54d1348e72ca05058f5e8bcff7c6b2520fa7802a957/merged 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/run/user/0 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/7197754f57fed7d6aeba1da31d044bdc99cc10c7a441a3912d7a24009a526f0e/merged 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/5b046d7d1031d4c8142c627de68968218c7a0d8e48146da45222c2bd7e037b35/merged 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/294599cff23f9f17641a507a056b3f4237d7357c0a5e7841223a3feded4189fc/merged 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/47865500b812d2dcb61d48709002f3077f5420098026621fba1568d0d1a90f85/merged 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/d3d8dc01661f5e9661d00faa59cf75ef626ab5d0f36ad053e4c587ce08547c20/merged 2024-03-13T14:46:21.539 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/19500e3b0fc2da340e67a2a1c2002c3d84960cf7cc5e6620bcc1046c2f10424f/merged 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/724c2b9e65499e63eaac600d7e321a182dc0f419c823452916196e09904aba8e/merged 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/0083370c822baea0cafdc56bb7f42ef81d6c0eed1f18545593b6752766609ca9/merged 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/b3f5bfb76903724407fb9244797e14742337d4a6e6cb512340d4e11f4ba6469e/merged 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/1253785850e7e080d9a8adaed3c703977c3d884f77e0f72242a4ee245cf193b1/merged 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/var/lib/containers/storage/overlay/88587999dcee258687820214bb82779f9edaa46a7ab3b01391a9fb84ab198637/merged 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-13T14:46:21.540 INFO:teuthology.orchestra.run.smithi176.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:21.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:21.541 DEBUG:teuthology.orchestra.run.smithi176:> ls /sys/fs/fuse/connections 2024-03-13T14:46:21.595 INFO:teuthology.orchestra.run.smithi176.stdout:158 2024-03-13T14:46:21.595 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [158] 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> sudo stdin-killer -- python3 -c ' 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> import glob 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> import re 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> import os 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> import subprocess 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> def _find_admin_socket(client_name): 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> files = glob.glob(asok_path) 2024-03-13T14:46:21.596 DEBUG:teuthology.orchestra.run.smithi176:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> # Given a non-glob path, it better be there 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> if "*" not in asok_path: 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> assert(len(files) == 1) 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> return files[0] 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> for f in files: 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> contents = proc_f.read() 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> if mountpoint in contents: 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> return f 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> 2024-03-13T14:46:21.597 DEBUG:teuthology.orchestra.run.smithi176:> print(_find_admin_socket("client.0")) 2024-03-13T14:46:21.598 DEBUG:teuthology.orchestra.run.smithi176:> ' 2024-03-13T14:46:21.769 INFO:teuthology.orchestra.run.smithi176.stdout:/var/run/ceph/ceph-client.0.76677.asok 2024-03-13T14:46:21.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-13T14:46:21.785 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76677.asok 2024-03-13T14:46:21.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:21.786 DEBUG:teuthology.orchestra.run.smithi176:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76677.asok status 2024-03-13T14:46:22.020 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:46:22.020 INFO:teuthology.orchestra.run.smithi176.stdout: "metadata": { 2024-03-13T14:46:22.020 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-03-13T14:46:22.020 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-13T14:46:22.020 INFO:teuthology.orchestra.run.smithi176.stdout: "entity_id": "0", 2024-03-13T14:46:22.020 INFO:teuthology.orchestra.run.smithi176.stdout: "hostname": "smithi176", 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "pid": "76677", 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "root": "/" 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "dentry_count": 0, 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "dentry_pinned_count": 0, 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "id": 14538, 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "inst": { 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "name": { 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "client", 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "num": 14538 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": { 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-03-13T14:46:22.021 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "192.168.0.1:0", 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3863864493 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": { 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "type": "v1", 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "addr": "192.168.0.1:0", 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "nonce": 3863864493 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "inst_str": "client.14538 192.168.0.1:0/3863864493", 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "addr_str": "192.168.0.1:0/3863864493", 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "inode_count": 1, 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "mds_epoch": 18, 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_epoch": 47, 2024-03-13T14:46:22.022 INFO:teuthology.orchestra.run.smithi176.stdout: "osd_epoch_barrier": 0, 2024-03-13T14:46:22.023 INFO:teuthology.orchestra.run.smithi176.stdout: "blocklisted": false, 2024-03-13T14:46:22.023 INFO:teuthology.orchestra.run.smithi176.stdout: "fs_name": "cephfs" 2024-03-13T14:46:22.023 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:46:22.035 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- ceph fs ls 2024-03-13T14:46:22.444 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:23.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:23 smithi196 ceph-mon[37486]: pgmap v242: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-13T14:46:23.613 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:23 smithi176 ceph-mon[29064]: pgmap v242: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-13T14:46:24.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 -- 172.21.15.176:0/2024022201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 msgr2=0x7f83ec105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:24.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 --2- 172.21.15.176:0/2024022201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec105e20 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009910 tx=0x7f83dc009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:24.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 -- 172.21.15.176:0/2024022201 shutdown_connections 2024-03-13T14:46:24.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 --2- 172.21.15.176:0/2024022201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0ffa40 0x7f83ec0ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:24.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 --2- 172.21.15.176:0/2024022201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec105e20 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:24.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 -- 172.21.15.176:0/2024022201 >> 172.21.15.176:0/2024022201 conn(0x7f83ec0fb570 msgr2=0x7f83ec0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:24.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 -- 172.21.15.176:0/2024022201 shutdown_connections 2024-03-13T14:46:24.548 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.557+0000 7f83f2400700 1 -- 172.21.15.176:0/2024022201 wait complete. 2024-03-13T14:46:24.548 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.558+0000 7f83f2400700 1 Processor -- start 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.558+0000 7f83f2400700 1 -- start start 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83f2400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0ffa40 0x7f83ec101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83f2400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83f2400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec102090 con 0x7f83ec105a40 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83f2400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1021d0 con 0x7f83ec0ffa40 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83eaa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83eaa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:38936/0 (socket says 172.21.15.176:38936) 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83eaa00700 1 -- 172.21.15.176:0/106707040 learned_addr learned my addr 172.21.15.176:0/106707040 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:24.549 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.559+0000 7f83eb400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0ffa40 0x7f83ec101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:24.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83eaa00700 1 -- 172.21.15.176:0/106707040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0ffa40 msgr2=0x7f83ec101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:24.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83eaa00700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0ffa40 0x7f83ec101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:24.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83eaa00700 1 -- 172.21.15.176:0/106707040 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0095f0 con 0x7f83ec105a40 2024-03-13T14:46:24.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83eb400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0ffa40 0x7f83ec101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T14:46:24.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83eaa00700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec101b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83d800d900 tx=0x7f83d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:24.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83e3e00700 1 -- 172.21.15.176:0/106707040 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d8004a00 con 0x7f83ec105a40 2024-03-13T14:46:24.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec1a2910 con 0x7f83ec105a40 2024-03-13T14:46:24.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83e3e00700 1 -- 172.21.15.176:0/106707040 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f83d80054d0 con 0x7f83ec105a40 2024-03-13T14:46:24.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.560+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec1a2d20 con 0x7f83ec105a40 2024-03-13T14:46:24.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.561+0000 7f83e3e00700 1 -- 172.21.15.176:0/106707040 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d8009d50 con 0x7f83ec105a40 2024-03-13T14:46:24.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.562+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec04ea90 con 0x7f83ec105a40 2024-03-13T14:46:24.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.563+0000 7f83e3e00700 1 -- 172.21.15.176:0/106707040 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f83d8005000 con 0x7f83ec105a40 2024-03-13T14:46:24.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.564+0000 7f83e3e00700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f83d006c4e0 0x7f83d006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:24.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.564+0000 7f83e3e00700 1 -- 172.21.15.176:0/106707040 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f83d808baa0 con 0x7f83ec105a40 2024-03-13T14:46:24.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.564+0000 7f83eb400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f83d006c4e0 0x7f83d006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:24.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.565+0000 7f83eb400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f83d006c4e0 0x7f83d006e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00b5c0 tx=0x7f83dc01a040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:24.558 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.569+0000 7f83e3e00700 1 -- 172.21.15.176:0/106707040 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f83d8056180 con 0x7f83ec105a40 2024-03-13T14:46:24.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.827+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f83ec102b40 con 0x7f83ec105a40 2024-03-13T14:46:24.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.828+0000 7f83e3e00700 1 -- 172.21.15.176:0/106707040 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f83d80597a0 con 0x7f83ec105a40 2024-03-13T14:46:24.818 INFO:teuthology.orchestra.run.smithi176.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f83d006c4e0 msgr2=0x7f83d006e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f83d006c4e0 0x7f83d006e9a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00b5c0 tx=0x7f83dc01a040 comp rx=0 tx=0).stop 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 msgr2=0x7f83ec101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec101b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f83d800d900 tx=0x7f83d800dcc0 comp rx=0 tx=0).stop 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 shutdown_connections 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f83d006c4e0 0x7f83d006e9a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83ec0ffa40 0x7f83ec101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 --2- 172.21.15.176:0/106707040 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f83ec105a40 0x7f83ec101b50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.832+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 >> 172.21.15.176:0/106707040 conn(0x7f83ec0fb570 msgr2=0x7f83ec103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.833+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 shutdown_connections 2024-03-13T14:46:24.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:24.833+0000 7f83f2400700 1 -- 172.21.15.176:0/106707040 wait complete. 2024-03-13T14:46:25.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:25 smithi196 ceph-mon[37486]: pgmap v243: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:46:25.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:25 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/106707040' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-13T14:46:25.589 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:25 smithi176 ceph-mon[29064]: pgmap v243: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:46:25.589 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:25 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/106707040' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-13T14:46:25.642 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-13T14:46:25.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:25.643 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-03-13T14:46:25.701 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:25.702 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-03-13T14:46:25.703 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-03-13T14:46:25.704 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-13T14:46:25.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:25.704 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-03-13T14:46:25.704 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-13T14:46:25.704 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add name ceph-brx type bridge 2024-03-13T14:46:25.704 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr flush dev ceph-brx 2024-03-13T14:46:25.705 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set ceph-brx up 2024-03-13T14:46:25.705 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-13T14:46:25.705 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-13T14:46:26.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:26.179 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-13T14:46:26.262 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-03-13T14:46:26.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:26.265 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-03-13T14:46:26.322 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-03-13T14:46:26.322 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-13T14:46:26.323 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-13T14:46:26.323 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-13T14:46:26.323 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-13T14:46:26.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:26.324 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-03-13T14:46:26.324 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-13T14:46:26.324 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-13T14:46:26.324 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-13T14:46:26.324 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-13T14:46:26.324 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-13T14:46:27.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:27.057 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-03-13T14:46:27.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:27.098 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-03-13T14:46:27.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:27.187 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-03-13T14:46:27.187 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-13T14:46:27.187 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-13T14:46:27.187 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-13T14:46:27.187 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-13T14:46:27.311 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:27 smithi196 ceph-mon[37486]: pgmap v244: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-13T14:46:27.369 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-13T14:46:27.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-13T14:46:27.369 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-13T14:46:27.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:27 smithi176 ceph-mon[29064]: pgmap v244: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-03-13T14:46:27.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:27.649 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-03-13T14:46:27.649 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-03-13T14:46:27.649 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-03-13T14:46:27.649 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.0 master ceph-brx 2024-03-13T14:46:27.649 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-03-13T14:46:27.795 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-13T14:46:27.795 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-13T14:46:27.796 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:27.836 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-13T14:46:27.837 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-13T14:46:27.837 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:27.929 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-03-13T14:46:28.027 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-03-13T14:46:28.077 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-03-13T14:46:28.078 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-13T14:46:28.079 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/0 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/59fd49b7cc4144c1362ac2ae464955b1c93cab7b014d552a9673109e256675b8/merged 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/3f3bc583229c3c11593e6173ededb26b42dc84eaf4022911ba21f05fbda3bec4/merged 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/d17579dc6658fb15cb0a05d0233bea329ba614fc98c60fd9be5d640d8b4aeb73/merged 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/b9e87f602b35c131bc8e6e0b818ec6d614b33045f07f861ef8e90ad995236528/merged 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/952bd057c9364b9b57bbedeb9944ad8985fe28eb75140ec34f1398705757a1e6/merged 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/498a4de3e42d49f1f55401cbec3e9d21712f540544af8e5b9fe5ec33e30b03a7/merged 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/f3dc03ed0b270adf888b5acf8367df3811e59015585d16e14a3baa83cb8b97f9/merged 2024-03-13T14:46:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/ae95ce4ac7206ce9bc70eced1c8c0393299646dc38d786a3ff10f7113e28f538/merged 2024-03-13T14:46:28.081 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/4b9ff5457c00db299f2bb899b998d0d2789684dab58082de2fe9ca654426a6c8/merged 2024-03-13T14:46:28.081 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/394d145df904eb1a773cb70551c0b5237d25a00de0fadf7248bd685ade6c740b/merged 2024-03-13T14:46:28.081 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-03-13T14:46:28.081 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-13T14:46:28.081 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-13T14:46:28.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:28.082 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-03-13T14:46:28.125 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-13T14:46:28.125 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-13T14:46:28.173 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-03-13T14:46:28.248 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-03-13T14:46:28.293 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-13T14:46:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/0 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/59fd49b7cc4144c1362ac2ae464955b1c93cab7b014d552a9673109e256675b8/merged 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/3f3bc583229c3c11593e6173ededb26b42dc84eaf4022911ba21f05fbda3bec4/merged 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/d17579dc6658fb15cb0a05d0233bea329ba614fc98c60fd9be5d640d8b4aeb73/merged 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/b9e87f602b35c131bc8e6e0b818ec6d614b33045f07f861ef8e90ad995236528/merged 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/952bd057c9364b9b57bbedeb9944ad8985fe28eb75140ec34f1398705757a1e6/merged 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/498a4de3e42d49f1f55401cbec3e9d21712f540544af8e5b9fe5ec33e30b03a7/merged 2024-03-13T14:46:28.295 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/f3dc03ed0b270adf888b5acf8367df3811e59015585d16e14a3baa83cb8b97f9/merged 2024-03-13T14:46:28.296 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/ae95ce4ac7206ce9bc70eced1c8c0393299646dc38d786a3ff10f7113e28f538/merged 2024-03-13T14:46:28.296 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/4b9ff5457c00db299f2bb899b998d0d2789684dab58082de2fe9ca654426a6c8/merged 2024-03-13T14:46:28.296 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/394d145df904eb1a773cb70551c0b5237d25a00de0fadf7248bd685ade6c740b/merged 2024-03-13T14:46:28.296 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-03-13T14:46:28.296 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-13T14:46:28.296 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-13T14:46:28.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:28.297 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-03-13T14:46:28.297 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:28.457 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-03-13T14:46:28.468+0000 7f60eb048480 -1 init, newargv = 0x55cf18494420 newargc=15 2024-03-13T14:46:28.458 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[63251]: starting ceph client 2024-03-13T14:46:28.467 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[63251]: starting fuse 2024-03-13T14:46:28.487 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:29.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:29 smithi196 ceph-mon[37486]: pgmap v245: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-13T14:46:29.486 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-03-13T14:46:29.580 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-03-13T14:46:29.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:29 smithi176 ceph-mon[29064]: pgmap v245: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-03-13T14:46:29.632 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/selinux 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysctl.service 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/nfs/rpc_pipefs 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-03-13T14:46:29.633 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/0 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/59fd49b7cc4144c1362ac2ae464955b1c93cab7b014d552a9673109e256675b8/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/3f3bc583229c3c11593e6173ededb26b42dc84eaf4022911ba21f05fbda3bec4/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/d17579dc6658fb15cb0a05d0233bea329ba614fc98c60fd9be5d640d8b4aeb73/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/b9e87f602b35c131bc8e6e0b818ec6d614b33045f07f861ef8e90ad995236528/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/952bd057c9364b9b57bbedeb9944ad8985fe28eb75140ec34f1398705757a1e6/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/498a4de3e42d49f1f55401cbec3e9d21712f540544af8e5b9fe5ec33e30b03a7/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/f3dc03ed0b270adf888b5acf8367df3811e59015585d16e14a3baa83cb8b97f9/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/ae95ce4ac7206ce9bc70eced1c8c0393299646dc38d786a3ff10f7113e28f538/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/4b9ff5457c00db299f2bb899b998d0d2789684dab58082de2fe9ca654426a6c8/merged 2024-03-13T14:46:29.634 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/containers/storage/overlay/394d145df904eb1a773cb70551c0b5237d25a00de0fadf7248bd685ade6c740b/merged 2024-03-13T14:46:29.635 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-03-13T14:46:29.635 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-13T14:46:29.635 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-13T14:46:29.635 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:29.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:29.636 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-03-13T14:46:29.681 INFO:teuthology.orchestra.run.smithi196.stdout:125 2024-03-13T14:46:29.682 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [125] 2024-03-13T14:46:29.682 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-03-13T14:46:29.682 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-03-13T14:46:29.682 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-03-13T14:46:29.682 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-03-13T14:46:29.682 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-13T14:46:29.683 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-13T14:46:29.684 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-03-13T14:46:29.684 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-03-13T14:46:29.684 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-03-13T14:46:29.684 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-13T14:46:29.684 DEBUG:teuthology.orchestra.run.smithi196:> 2024-03-13T14:46:29.684 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-03-13T14:46:29.684 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-03-13T14:46:29.905 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.63251.asok 2024-03-13T14:46:29.908 INFO:teuthology.orchestra.run.smithi196.stderr:2024-03-13T14:46:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-13T14:46:29.922 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.63251.asok 2024-03-13T14:46:29.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:29.923 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.63251.asok status 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "7fe91d5d5842e04be3b4f514d6dd990c54b29c76", 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)", 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "63251", 2024-03-13T14:46:30.154 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 24287, 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 24287 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2547707940 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-03-13T14:46:30.155 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2547707940 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.24287 192.168.0.1:0/2547707940", 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/2547707940", 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 18, 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 47, 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-03-13T14:46:30.156 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-03-13T14:46:30.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:30.170 DEBUG:teuthology.orchestra.run.smithi176:> stat --file-system '--printf=%T 2024-03-13T14:46:30.170 DEBUG:teuthology.orchestra.run.smithi176:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:30.229 INFO:teuthology.orchestra.run.smithi176.stdout:fuseblk 2024-03-13T14:46:30.230 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:30.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:30.230 DEBUG:teuthology.orchestra.run.smithi176:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:30.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:30.329 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-03-13T14:46:30.329 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:30.390 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-03-13T14:46:30.390 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:30.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T14:46:30.390 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:30.492 INFO:teuthology.run_tasks:Running task print... 2024-03-13T14:46:30.503 INFO:teuthology.task.print:**** done client 2024-03-13T14:46:30.503 INFO:teuthology.run_tasks:Running task parallel... 2024-03-13T14:46:30.565 INFO:teuthology.task.parallel:starting parallel... 2024-03-13T14:46:30.566 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-13T14:46:30.566 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-13T14:46:30.567 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:46:30.567 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-13T14:46:30.567 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-03-13T14:46:30.568 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-03-13T14:46:30.571 INFO:tasks.workunit:Pulling workunits from ref 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:46:30.572 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-13T14:46:30.572 INFO:tasks.workunit:timeout=3h 2024-03-13T14:46:30.572 INFO:tasks.workunit:cleanup=True 2024-03-13T14:46:30.572 DEBUG:teuthology.orchestra.run.smithi176:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:30.628 INFO:teuthology.orchestra.run.smithi176.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:30.628 INFO:teuthology.orchestra.run.smithi176.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-13T14:46:30.629 INFO:teuthology.orchestra.run.smithi176.stdout:Device: 9eh/158d Inode: 1 Links: 2 2024-03-13T14:46:30.629 INFO:teuthology.orchestra.run.smithi176.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:46:30.629 INFO:teuthology.orchestra.run.smithi176.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-13T14:46:30.629 INFO:teuthology.orchestra.run.smithi176.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-13T14:46:30.629 INFO:teuthology.orchestra.run.smithi176.stdout:Modify: 2024-03-13 14:45:43.416190951 +0000 2024-03-13T14:46:30.629 INFO:teuthology.orchestra.run.smithi176.stdout:Change: 2024-03-13 14:46:30.333946525 +0000 2024-03-13T14:46:30.629 INFO:teuthology.orchestra.run.smithi176.stdout: Birth: - 2024-03-13T14:46:30.629 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-13T14:46:30.629 DEBUG:teuthology.orchestra.run.smithi176:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-13T14:46:30.778 DEBUG:teuthology.orchestra.run.smithi196:> stat -- /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:30.836 INFO:teuthology.orchestra.run.smithi196.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 7dh/125d Inode: 1 Links: 3 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:fusefs_t:s0 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-03-13 14:46:30.772574692 +0000 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-03-13 14:46:30.772574692 +0000 2024-03-13T14:46:30.837 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-03-13T14:46:30.837 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-03-13T14:46:30.838 DEBUG:teuthology.orchestra.run.smithi196:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-03-13T14:46:30.950 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:46:30.950 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:46:30.966 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:30.995 INFO:tasks.workunit.client.0.smithi176.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-13T14:46:31.010 INFO:tasks.workunit.client.1.smithi196.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-03-13T14:46:31.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:30 smithi196 ceph-mon[37486]: pgmap v246: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-13T14:46:31.299 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:30 smithi176 ceph-mon[29064]: pgmap v246: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-03-13T14:46:33.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 -- 172.21.15.176:0/3949265921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810108240 msgr2=0x7f38101086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:33.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 --2- 172.21.15.176:0/3949265921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810108240 0x7f38101086c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009910 tx=0x7f37fc009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 -- 172.21.15.176:0/3949265921 shutdown_connections 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 --2- 172.21.15.176:0/3949265921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810108240 0x7f38101086c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 --2- 172.21.15.176:0/3949265921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3810106ff0 0x7f3810107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 -- 172.21.15.176:0/3949265921 >> 172.21.15.176:0/3949265921 conn(0x7f3810076080 msgr2=0x7f38100784e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 -- 172.21.15.176:0/3949265921 shutdown_connections 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.016+0000 7f3815400700 1 -- 172.21.15.176:0/3949265921 wait complete. 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.017+0000 7f3815400700 1 Processor -- start 2024-03-13T14:46:33.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f3815400700 1 -- start start 2024-03-13T14:46:33.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f3815400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810106ff0 0x7f381019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:33.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f3815400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3810108240 0x7f381019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f3815400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381019d290 con 0x7f3810106ff0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f3815400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381019d3d0 con 0x7f3810108240 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f380fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810106ff0 0x7f381019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f380fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810106ff0 0x7f381019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60142/0 (socket says 172.21.15.176:60142) 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f380fe00700 1 -- 172.21.15.176:0/3492296037 learned_addr learned my addr 172.21.15.176:0/3492296037 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.018+0000 7f380f400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3810108240 0x7f381019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f380fe00700 1 -- 172.21.15.176:0/3492296037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3810108240 msgr2=0x7f381019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f380fe00700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3810108240 0x7f381019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f380fe00700 1 -- 172.21.15.176:0/3492296037 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0095f0 con 0x7f3810106ff0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f380fe00700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810106ff0 0x7f381019c7c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f380000c930 tx=0x7f380000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f380cc00700 1 -- 172.21.15.176:0/3492296037 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3800007ad0 con 0x7f3810106ff0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f380cc00700 1 -- 172.21.15.176:0/3492296037 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3800007c30 con 0x7f3810106ff0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f380cc00700 1 -- 172.21.15.176:0/3492296037 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3800018630 con 0x7f3810106ff0 2024-03-13T14:46:33.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38101a1e90 con 0x7f3810106ff0 2024-03-13T14:46:33.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.019+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38101a2300 con 0x7f3810106ff0 2024-03-13T14:46:33.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.021+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381010fb70 con 0x7f3810106ff0 2024-03-13T14:46:33.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.022+0000 7f380cc00700 1 -- 172.21.15.176:0/3492296037 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f380000f450 con 0x7f3810106ff0 2024-03-13T14:46:33.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.022+0000 7f380cc00700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f37f406c4e0 0x7f37f406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:33.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.023+0000 7f380cc00700 1 -- 172.21.15.176:0/3492296037 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f380008ba00 con 0x7f3810106ff0 2024-03-13T14:46:33.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.023+0000 7f380f400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f37f406c4e0 0x7f37f406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:33.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.024+0000 7f380f400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f37f406c4e0 0x7f37f406e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c00 tx=0x7f37fc005fb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:33.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.028+0000 7f380cc00700 1 -- 172.21.15.176:0/3492296037 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3800059bf0 con 0x7f3810106ff0 2024-03-13T14:46:33.231 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.241+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f381010fd60 con 0x7f3810106ff0 2024-03-13T14:46:33.231 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.242+0000 7f380cc00700 1 -- 172.21.15.176:0/3492296037 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f381010fd60 con 0x7f3810106ff0 2024-03-13T14:46:33.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f37f406c4e0 msgr2=0x7f37f406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:33.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f37f406c4e0 0x7f37f406e9a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c00 tx=0x7f37fc005fb0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810106ff0 msgr2=0x7f381019c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:33.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810106ff0 0x7f381019c7c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f380000c930 tx=0x7f380000ccf0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 shutdown_connections 2024-03-13T14:46:33.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f37f406c4e0 0x7f37f406e9a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3810106ff0 0x7f381019c7c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 --2- 172.21.15.176:0/3492296037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3810108240 0x7f381019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:33.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 >> 172.21.15.176:0/3492296037 conn(0x7f3810076080 msgr2=0x7f381010b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:33.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 shutdown_connections 2024-03-13T14:46:33.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:33.245+0000 7f3815400700 1 -- 172.21.15.176:0/3492296037 wait complete. 2024-03-13T14:46:33.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:33 smithi196 ceph-mon[37486]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:46:33.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:33 smithi176 ceph-mon[29064]: pgmap v247: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:46:34.089 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-13T14:46:34.540 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:35.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:35 smithi176 ceph-mon[29064]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:46:35.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:35 smithi196 ceph-mon[37486]: pgmap v248: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:46:36.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 -- 172.21.15.176:0/1916696857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 msgr2=0x7f09a4107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:36.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 --2- 172.21.15.176:0/1916696857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a4107620 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0998009910 tx=0x7f0998009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:36.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 -- 172.21.15.176:0/1916696857 shutdown_connections 2024-03-13T14:46:36.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 --2- 172.21.15.176:0/1916696857 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a4108400 0x7f09a4108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:36.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 --2- 172.21.15.176:0/1916696857 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a4107620 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:36.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 -- 172.21.15.176:0/1916696857 >> 172.21.15.176:0/1916696857 conn(0x7f09a40761b0 msgr2=0x7f09a4078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:36.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 -- 172.21.15.176:0/1916696857 shutdown_connections 2024-03-13T14:46:36.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.534+0000 7f09ab800700 1 -- 172.21.15.176:0/1916696857 wait complete. 2024-03-13T14:46:36.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.535+0000 7f09ab800700 1 Processor -- start 2024-03-13T14:46:36.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.535+0000 7f09ab800700 1 -- start start 2024-03-13T14:46:36.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09ab800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:36.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09ab800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a4108400 0x7f09a419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:36.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09ab800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a419d590 con 0x7f09a4107200 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09ab800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a419d6d0 con 0x7f09a4108400 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09a8e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09a8e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60162/0 (socket says 172.21.15.176:60162) 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09a8e00700 1 -- 172.21.15.176:0/3591627509 learned_addr learned my addr 172.21.15.176:0/3591627509 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09a8e00700 1 -- 172.21.15.176:0/3591627509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a4108400 msgr2=0x7f09a419cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09a3e00700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a4108400 0x7f09a419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09a8e00700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a4108400 0x7f09a419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.536+0000 7f09a8e00700 1 -- 172.21.15.176:0/3591627509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09980095f0 con 0x7f09a4107200 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.537+0000 7f09a8e00700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a419ca30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f099800bda0 tx=0x7f099800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:36.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.537+0000 7f09a1600700 1 -- 172.21.15.176:0/3591627509 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099801d070 con 0x7f09a4107200 2024-03-13T14:46:36.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.537+0000 7f09a1600700 1 -- 172.21.15.176:0/3591627509 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0998004c50 con 0x7f09a4107200 2024-03-13T14:46:36.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.537+0000 7f09a1600700 1 -- 172.21.15.176:0/3591627509 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099800f690 con 0x7f09a4107200 2024-03-13T14:46:36.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.537+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a41a2120 con 0x7f09a4107200 2024-03-13T14:46:36.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.537+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a41a2690 con 0x7f09a4107200 2024-03-13T14:46:36.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.538+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a410fde0 con 0x7f09a4107200 2024-03-13T14:46:36.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.539+0000 7f09a1600700 1 -- 172.21.15.176:0/3591627509 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f099800f7f0 con 0x7f09a4107200 2024-03-13T14:46:36.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.541+0000 7f09a1600700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f098c06c4e0 0x7f098c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:36.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.541+0000 7f09a1600700 1 -- 172.21.15.176:0/3591627509 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f099808d070 con 0x7f09a4107200 2024-03-13T14:46:36.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.541+0000 7f09a3e00700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f098c06c4e0 0x7f098c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:36.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.542+0000 7f09a3e00700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f098c06c4e0 0x7f098c06e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0994006fd0 tx=0x7f0994008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:36.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.546+0000 7f09a1600700 1 -- 172.21.15.176:0/3591627509 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f099805b380 con 0x7f09a4107200 2024-03-13T14:46:36.749 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.760+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f09a41a2de0 con 0x7f09a4107200 2024-03-13T14:46:36.749 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.760+0000 7f09a1600700 1 -- 172.21.15.176:0/3591627509 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f099805af10 con 0x7f09a4107200 2024-03-13T14:46:36.752 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f098c06c4e0 msgr2=0x7f098c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:36.752 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f098c06c4e0 0x7f098c06e9a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0994006fd0 tx=0x7f0994008040 comp rx=0 tx=0).stop 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 msgr2=0x7f09a419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a419ca30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f099800bda0 tx=0x7f099800be80 comp rx=0 tx=0).stop 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 shutdown_connections 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f098c06c4e0 0x7f098c06e9a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f09a4107200 0x7f09a419ca30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 --2- 172.21.15.176:0/3591627509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09a4108400 0x7f09a419cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 >> 172.21.15.176:0/3591627509 conn(0x7f09a40761b0 msgr2=0x7f09a410b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 shutdown_connections 2024-03-13T14:46:36.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:36.763+0000 7f09ab800700 1 -- 172.21.15.176:0/3591627509 wait complete. 2024-03-13T14:46:37.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:37 smithi176 ceph-mon[29064]: pgmap v249: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-13T14:46:37.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:37 smithi196 ceph-mon[37486]: pgmap v249: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s 2024-03-13T14:46:37.546 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-13T14:46:37.976 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:39.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:39 smithi196 ceph-mon[37486]: pgmap v250: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-13T14:46:39.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:39 smithi176 ceph-mon[29064]: pgmap v250: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.168+0000 7f5f44c00700 1 -- 172.21.15.176:0/2407661419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40108400 msgr2=0x7f5f40108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.168+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2407661419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40108400 0x7f5f40108860 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c009910 tx=0x7f5f2c009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.169+0000 7f5f44c00700 1 -- 172.21.15.176:0/2407661419 shutdown_connections 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.169+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2407661419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40108400 0x7f5f40108860 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.169+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2407661419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f40107200 0x7f5f40107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.169+0000 7f5f44c00700 1 -- 172.21.15.176:0/2407661419 >> 172.21.15.176:0/2407661419 conn(0x7f5f400761b0 msgr2=0x7f5f40078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.169+0000 7f5f44c00700 1 -- 172.21.15.176:0/2407661419 shutdown_connections 2024-03-13T14:46:40.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.169+0000 7f5f44c00700 1 -- 172.21.15.176:0/2407661419 wait complete. 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.170+0000 7f5f44c00700 1 Processor -- start 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.170+0000 7f5f44c00700 1 -- start start 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.170+0000 7f5f44c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40107200 0x7f5f4019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.170+0000 7f5f44c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f40108400 0x7f5f4019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.170+0000 7f5f44c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4019d560 con 0x7f5f40107200 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.170+0000 7f5f44c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4019d6a0 con 0x7f5f40108400 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40107200 0x7f5f4019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:40.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40107200 0x7f5f4019ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60188/0 (socket says 172.21.15.176:60188) 2024-03-13T14:46:40.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3de00700 1 -- 172.21.15.176:0/2089995362 learned_addr learned my addr 172.21.15.176:0/2089995362 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:40.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3d400700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f40108400 0x7f5f4019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:40.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3de00700 1 -- 172.21.15.176:0/2089995362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f40108400 msgr2=0x7f5f4019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:40.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3de00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f40108400 0x7f5f4019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:40.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3de00700 1 -- 172.21.15.176:0/2089995362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0095f0 con 0x7f5f40107200 2024-03-13T14:46:40.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.171+0000 7f5f3d400700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f40108400 0x7f5f4019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:46:40.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.172+0000 7f5f3de00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40107200 0x7f5f4019ca00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b480 tx=0x7f5f3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:40.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.172+0000 7f5f36a00700 1 -- 172.21.15.176:0/2089995362 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f300118c0 con 0x7f5f40107200 2024-03-13T14:46:40.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.172+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f401a2150 con 0x7f5f40107200 2024-03-13T14:46:40.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.172+0000 7f5f36a00700 1 -- 172.21.15.176:0/2089995362 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5f30010380 con 0x7f5f40107200 2024-03-13T14:46:40.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.172+0000 7f5f36a00700 1 -- 172.21.15.176:0/2089995362 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3000f4f0 con 0x7f5f40107200 2024-03-13T14:46:40.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.172+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f401a2720 con 0x7f5f40107200 2024-03-13T14:46:40.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.173+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4010fe00 con 0x7f5f40107200 2024-03-13T14:46:40.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.175+0000 7f5f36a00700 1 -- 172.21.15.176:0/2089995362 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f5f300104f0 con 0x7f5f40107200 2024-03-13T14:46:40.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.175+0000 7f5f36a00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5f2406c4e0 0x7f5f2406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:40.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.176+0000 7f5f36a00700 1 -- 172.21.15.176:0/2089995362 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5f3008b990 con 0x7f5f40107200 2024-03-13T14:46:40.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.176+0000 7f5f3d400700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5f2406c4e0 0x7f5f2406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:40.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.176+0000 7f5f3d400700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5f2406c4e0 0x7f5f2406e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c006010 tx=0x7f5f2c00b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:40.168 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.179+0000 7f5f36a00700 1 -- 172.21.15.176:0/2089995362 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f5f30059b80 con 0x7f5f40107200 2024-03-13T14:46:40.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.395+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f5f401a2f10 con 0x7f5f40107200 2024-03-13T14:46:40.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.396+0000 7f5f36a00700 1 -- 172.21.15.176:0/2089995362 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f5f30059710 con 0x7f5f40107200 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5f2406c4e0 msgr2=0x7f5f2406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5f2406c4e0 0x7f5f2406e9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c006010 tx=0x7f5f2c00b540 comp rx=0 tx=0).stop 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40107200 msgr2=0x7f5f4019ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40107200 0x7f5f4019ca00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000b480 tx=0x7f5f3000b840 comp rx=0 tx=0).stop 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 shutdown_connections 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f5f2406c4e0 0x7f5f2406e9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5f40107200 0x7f5f4019ca00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 --2- 172.21.15.176:0/2089995362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f40108400 0x7f5f4019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.399+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 >> 172.21.15.176:0/2089995362 conn(0x7f5f400761b0 msgr2=0x7f5f4010b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.400+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 shutdown_connections 2024-03-13T14:46:40.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:40.400+0000 7f5f44c00700 1 -- 172.21.15.176:0/2089995362 wait complete. 2024-03-13T14:46:41.064 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-03-13T14:46:41.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:40 smithi176 ceph-mon[29064]: pgmap v251: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-13T14:46:41.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:40 smithi196 ceph-mon[37486]: pgmap v251: 65 pgs: 65 active+clean; 585 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-13T14:46:41.457 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:46:43.349 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:43 smithi176 ceph-mon[29064]: pgmap v252: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-13T14:46:43.349 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:43.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:43 smithi196 ceph-mon[37486]: pgmap v252: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-13T14:46:43.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:43.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 -- 172.21.15.176:0/2009977538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 msgr2=0x7fefd0108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:43.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 --2- 172.21.15.176:0/2009977538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd0108860 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009910 tx=0x7fefbc009c20 comp rx=0 tx=0).stop 2024-03-13T14:46:43.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 -- 172.21.15.176:0/2009977538 shutdown_connections 2024-03-13T14:46:43.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 --2- 172.21.15.176:0/2009977538 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd0108860 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:43.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 --2- 172.21.15.176:0/2009977538 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0107200 0x7fefd0107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:43.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 -- 172.21.15.176:0/2009977538 >> 172.21.15.176:0/2009977538 conn(0x7fefd00761b0 msgr2=0x7fefd0078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:46:43.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 -- 172.21.15.176:0/2009977538 shutdown_connections 2024-03-13T14:46:43.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.644+0000 7fefd4c00700 1 -- 172.21.15.176:0/2009977538 wait complete. 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.645+0000 7fefd4c00700 1 Processor -- start 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefd4c00700 1 -- start start 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefd4c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0107200 0x7fefd019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefd4c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefd4c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd019d4c0 con 0x7fefd0108400 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefd4c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd019d600 con 0x7fefd0107200 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefcd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefcd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44024/0 (socket says 172.21.15.176:44024) 2024-03-13T14:46:43.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.646+0000 7fefcd400700 1 -- 172.21.15.176:0/1575853259 learned_addr learned my addr 172.21.15.176:0/1575853259 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:46:43.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.647+0000 7fefcde00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0107200 0x7fefd019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:43.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.647+0000 7fefcd400700 1 -- 172.21.15.176:0/1575853259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0107200 msgr2=0x7fefd019c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.647+0000 7fefcd400700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0107200 0x7fefd019c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.647+0000 7fefcd400700 1 -- 172.21.15.176:0/1575853259 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0095f0 con 0x7fefd0108400 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.647+0000 7fefcd400700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd019cea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005b40 tx=0x7fefbc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.647+0000 7fefc6a00700 1 -- 172.21.15.176:0/1575853259 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefbc01d070 con 0x7fefd0108400 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.648+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd01a2050 con 0x7fefd0108400 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.648+0000 7fefc6a00700 1 -- 172.21.15.176:0/1575853259 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fefbc022470 con 0x7fefd0108400 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.648+0000 7fefc6a00700 1 -- 172.21.15.176:0/1575853259 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefbc00f690 con 0x7fefd0108400 2024-03-13T14:46:43.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.648+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd01a2540 con 0x7fefd0108400 2024-03-13T14:46:43.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.649+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd010fd60 con 0x7fefd0108400 2024-03-13T14:46:43.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.650+0000 7fefc6a00700 1 -- 172.21.15.176:0/1575853259 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fefbc0225e0 con 0x7fefd0108400 2024-03-13T14:46:43.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.651+0000 7fefc6a00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fefb406c4e0 0x7fefb406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:46:43.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.651+0000 7fefc6a00700 1 -- 172.21.15.176:0/1575853259 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fefbc08d190 con 0x7fefd0108400 2024-03-13T14:46:43.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.651+0000 7fefcde00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fefb406c4e0 0x7fefb406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:46:43.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.652+0000 7fefcde00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fefb406c4e0 0x7fefb406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefc0005fd0 tx=0x7fefc0005dd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:46:43.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.656+0000 7fefc6a00700 1 -- 172.21.15.176:0/1575853259 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fefbc05b4a0 con 0x7fefd0108400 2024-03-13T14:46:43.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:46:43.885+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fefd010ce00 con 0x7fefb406c4e0 2024-03-13T14:46:45.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:45 smithi196 ceph-mon[37486]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:46:45.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:45 smithi196 ceph-mon[37486]: pgmap v253: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-13T14:46:45.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:45 smithi176 ceph-mon[29064]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:46:45.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:45 smithi176 ceph-mon[29064]: pgmap v253: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-13T14:46:47.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:47 smithi196 ceph-mon[37486]: pgmap v254: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-13T14:46:47.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:47 smithi176 ceph-mon[29064]: pgmap v254: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-13T14:46:49.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:49 smithi176 ceph-mon[29064]: pgmap v255: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-13T14:46:49.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:49 smithi196 ceph-mon[37486]: pgmap v255: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 597 B/s wr, 2 op/s 2024-03-13T14:46:51.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:51 smithi176 ceph-mon[29064]: pgmap v256: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:46:51.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:51 smithi196 ceph-mon[37486]: pgmap v256: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:46:53.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:53 smithi176 ceph-mon[29064]: pgmap v257: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:46:53.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:53 smithi196 ceph-mon[37486]: pgmap v257: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:46:55.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:55 smithi196 ceph-mon[37486]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:46:55.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:55 smithi176 ceph-mon[29064]: pgmap v258: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:46:57.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:57 smithi196 ceph-mon[37486]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:46:57.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:57 smithi176 ceph-mon[29064]: pgmap v259: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:46:58.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:58.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:46:59.582 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:46:59 smithi196 ceph-mon[37486]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:46:59.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:46:59 smithi176 ceph-mon[29064]: pgmap v260: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:01.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:01 smithi176 ceph-mon[29064]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:01.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:01 smithi196 ceph-mon[37486]: pgmap v261: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:03 smithi196 ceph-mon[37486]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:03.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:03 smithi176 ceph-mon[29064]: pgmap v262: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:05.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:05 smithi196 ceph-mon[37486]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:05.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:05 smithi176 ceph-mon[29064]: pgmap v263: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:07.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:07 smithi196 ceph-mon[37486]: pgmap v264: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:07.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:07 smithi176 ceph-mon[29064]: pgmap v264: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:09.621 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:09 smithi176 ceph-mon[29064]: pgmap v265: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:47:09.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:09 smithi196 ceph-mon[37486]: pgmap v265: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:47:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:11 smithi176 ceph-mon[29064]: pgmap v266: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:11.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:11 smithi196 ceph-mon[37486]: pgmap v266: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:13.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:13 smithi176 ceph-mon[29064]: pgmap v267: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:13.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:13.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:13 smithi196 ceph-mon[37486]: pgmap v267: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:13.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:15 smithi196 ceph-mon[37486]: pgmap v268: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:15.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:15 smithi176 ceph-mon[29064]: pgmap v268: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:17.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:17 smithi196 ceph-mon[37486]: pgmap v269: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:17.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:17 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:47:17.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:17 smithi176 ceph-mon[29064]: pgmap v269: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:17.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:17 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:47:19.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:19 smithi176 ceph-mon[29064]: pgmap v270: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:19.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:47:19.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:47:19.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:19 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:19.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:19 smithi196 ceph-mon[37486]: pgmap v270: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:19.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:47:19.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:47:19.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:19 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:21.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:21 smithi176 ceph-mon[29064]: pgmap v271: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:21.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:21 smithi196 ceph-mon[37486]: pgmap v271: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:21.686 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.696+0000 7fefc6a00700 1 -- 172.21.15.176:0/1575853259 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fefd010ce00 con 0x7fefb406c4e0 2024-03-13T14:47:21.686 INFO:teuthology.orchestra.run.smithi176.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:47:21.688 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.699+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fefb406c4e0 msgr2=0x7fefb406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:21.688 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.699+0000 7fefd4c00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fefb406c4e0 0x7fefb406e9a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefc0005fd0 tx=0x7fefc0005dd0 comp rx=0 tx=0).stop 2024-03-13T14:47:21.688 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.699+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 msgr2=0x7fefd019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:21.688 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.699+0000 7fefd4c00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd019cea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fefbc005b40 tx=0x7fefbc00be20 comp rx=0 tx=0).stop 2024-03-13T14:47:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.700+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 shutdown_connections 2024-03-13T14:47:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.700+0000 7fefd4c00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fefb406c4e0 0x7fefb406e9a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.700+0000 7fefd4c00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fefd0107200 0x7fefd019c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.700+0000 7fefd4c00700 1 --2- 172.21.15.176:0/1575853259 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fefd0108400 0x7fefd019cea0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.700+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 >> 172.21.15.176:0/1575853259 conn(0x7fefd00761b0 msgr2=0x7fefd010b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.700+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 shutdown_connections 2024-03-13T14:47:21.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:21.700+0000 7fefd4c00700 1 -- 172.21.15.176:0/1575853259 wait complete. 2024-03-13T14:47:22.419 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-03-13T14:47:22.991 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:47:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:47:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:47:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:47:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:47:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:22 smithi176 ceph-mon[29064]: pgmap v272: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:23.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:47:23.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:23.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:47:23.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:47:23.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:47:23.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:23.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:47:23.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:22 smithi196 ceph-mon[37486]: pgmap v272: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:23.909 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:47:25.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:25.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: Upgrade: Target is version 19.0.0-1916-g9a75c81f (unknown) 2024-03-13T14:47:25.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a'] 2024-03-13T14:47:25.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:47:25.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:25.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: Upgrade: Need to upgrade myself (mgr.smithi176.tzfdwd) 2024-03-13T14:47:25.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: pgmap v273: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:25.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:24 smithi176 ceph-mon[29064]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a on smithi196 2024-03-13T14:47:25.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:25.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: Upgrade: Target is version 19.0.0-1916-g9a75c81f (unknown) 2024-03-13T14:47:25.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a'] 2024-03-13T14:47:25.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:47:25.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:25.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: Upgrade: Need to upgrade myself (mgr.smithi176.tzfdwd) 2024-03-13T14:47:25.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: pgmap v273: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:25.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:24 smithi196 ceph-mon[37486]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a on smithi196 2024-03-13T14:47:27.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:27 smithi196 ceph-mon[37486]: pgmap v274: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:27.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:27 smithi176 ceph-mon[29064]: pgmap v274: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:28.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:28.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:29.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:29 smithi196 ceph-mon[37486]: pgmap v275: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:29.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:29 smithi176 ceph-mon[29064]: pgmap v275: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:31.008 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:31 smithi196 ceph-mon[37486]: pgmap v276: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:31.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:31 smithi176 ceph-mon[29064]: pgmap v276: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:32.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 -- 172.21.15.176:0/635503603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 msgr2=0x7f3904106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:32.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 --2- 172.21.15.176:0/635503603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904106360 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009910 tx=0x7f38f0009c20 comp rx=0 tx=0).stop 2024-03-13T14:47:32.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 -- 172.21.15.176:0/635503603 shutdown_connections 2024-03-13T14:47:32.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 --2- 172.21.15.176:0/635503603 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904106360 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 --2- 172.21.15.176:0/635503603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3904101640 0x7f3904103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 -- 172.21.15.176:0/635503603 >> 172.21.15.176:0/635503603 conn(0x7f39040fb030 msgr2=0x7f39040fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:32.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 -- 172.21.15.176:0/635503603 shutdown_connections 2024-03-13T14:47:32.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.443+0000 7f3909200700 1 -- 172.21.15.176:0/635503603 wait complete. 2024-03-13T14:47:32.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.444+0000 7f3909200700 1 Processor -- start 2024-03-13T14:47:32.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3909200700 1 -- start start 2024-03-13T14:47:32.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3909200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3904101640 0x7f39041984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:32.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3909200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:32.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3909200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3904199050 con 0x7f3904103f70 2024-03-13T14:47:32.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3909200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3904199190 con 0x7f3904101640 2024-03-13T14:47:32.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3901400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:32.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3901400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57244/0 (socket says 172.21.15.176:57244) 2024-03-13T14:47:32.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.445+0000 7f3901400700 1 -- 172.21.15.176:0/787782159 learned_addr learned my addr 172.21.15.176:0/787782159 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:47:32.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.446+0000 7f3901e00700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3904101640 0x7f39041984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:32.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.446+0000 7f3901400700 1 -- 172.21.15.176:0/787782159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3904101640 msgr2=0x7f39041984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:32.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.446+0000 7f3901400700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3904101640 0x7f39041984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.446+0000 7f3901400700 1 -- 172.21.15.176:0/787782159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f00095f0 con 0x7f3904103f70 2024-03-13T14:47:32.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.446+0000 7f3901400700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904198a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b5c0 tx=0x7f38f000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:32.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.447+0000 7f38faa00700 1 -- 172.21.15.176:0/787782159 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f001d070 con 0x7f3904103f70 2024-03-13T14:47:32.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.447+0000 7f38faa00700 1 -- 172.21.15.176:0/787782159 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f38f000f460 con 0x7f3904103f70 2024-03-13T14:47:32.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.447+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390419dbe0 con 0x7f3904103f70 2024-03-13T14:47:32.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.447+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390419e0d0 con 0x7f3904103f70 2024-03-13T14:47:32.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.447+0000 7f38faa00700 1 -- 172.21.15.176:0/787782159 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f0021640 con 0x7f3904103f70 2024-03-13T14:47:32.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.448+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e4005320 con 0x7f3904103f70 2024-03-13T14:47:32.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.449+0000 7f38faa00700 1 -- 172.21.15.176:0/787782159 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f38f0003a60 con 0x7f3904103f70 2024-03-13T14:47:32.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.451+0000 7f38faa00700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f38e806c4e0 0x7f38e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:32.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.451+0000 7f38faa00700 1 -- 172.21.15.176:0/787782159 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f38f008ca50 con 0x7f3904103f70 2024-03-13T14:47:32.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.451+0000 7f3901e00700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f38e806c4e0 0x7f38e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:32.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.453+0000 7f3901e00700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f38e806c4e0 0x7f38e806e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005950 tx=0x7f38f400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:32.444 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.456+0000 7f38faa00700 1 -- 172.21.15.176:0/787782159 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f38f005ac40 con 0x7f3904103f70 2024-03-13T14:47:32.701 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.712+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e4000bf0 con 0x7f38e806c4e0 2024-03-13T14:47:32.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.714+0000 7f38faa00700 1 -- 172.21.15.176:0/787782159 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f38e4000bf0 con 0x7f38e806c4e0 2024-03-13T14:47:32.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.717+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f38e806c4e0 msgr2=0x7f38e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:32.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.717+0000 7f3909200700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f38e806c4e0 0x7f38e806e9a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005950 tx=0x7f38f400b410 comp rx=0 tx=0).stop 2024-03-13T14:47:32.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.717+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 msgr2=0x7f3904198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:32.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.717+0000 7f3909200700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904198a30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f38f000b5c0 tx=0x7f38f000bd70 comp rx=0 tx=0).stop 2024-03-13T14:47:32.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.718+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 shutdown_connections 2024-03-13T14:47:32.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.718+0000 7f3909200700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f38e806c4e0 0x7f38e806e9a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.718+0000 7f3909200700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3904101640 0x7f39041984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.718+0000 7f3909200700 1 --2- 172.21.15.176:0/787782159 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3904103f70 0x7f3904198a30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.718+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 >> 172.21.15.176:0/787782159 conn(0x7f39040fb030 msgr2=0x7f39040ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:32.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.718+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 shutdown_connections 2024-03-13T14:47:32.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.718+0000 7f3909200700 1 -- 172.21.15.176:0/787782159 wait complete. 2024-03-13T14:47:32.730 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:47:32.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.875+0000 7f94e4000700 1 -- 172.21.15.176:0/194600285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 msgr2=0x7f94dc104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:32.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.875+0000 7f94e4000700 1 --2- 172.21.15.176:0/194600285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc104230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f94c8009910 tx=0x7f94c8009c20 comp rx=0 tx=0).stop 2024-03-13T14:47:32.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.876+0000 7f94e4000700 1 -- 172.21.15.176:0/194600285 shutdown_connections 2024-03-13T14:47:32.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.876+0000 7f94e4000700 1 --2- 172.21.15.176:0/194600285 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc104230 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.876+0000 7f94e4000700 1 --2- 172.21.15.176:0/194600285 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc0ff430 0x7f94dc101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.876+0000 7f94e4000700 1 -- 172.21.15.176:0/194600285 >> 172.21.15.176:0/194600285 conn(0x7f94dc0fb030 msgr2=0x7f94dc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:32.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.876+0000 7f94e4000700 1 -- 172.21.15.176:0/194600285 shutdown_connections 2024-03-13T14:47:32.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.876+0000 7f94e4000700 1 -- 172.21.15.176:0/194600285 wait complete. 2024-03-13T14:47:32.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.877+0000 7f94e4000700 1 Processor -- start 2024-03-13T14:47:32.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.877+0000 7f94e4000700 1 -- start start 2024-03-13T14:47:32.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.878+0000 7f94e4000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc0ff430 0x7f94dc19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:32.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.878+0000 7f94e4000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:32.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.878+0000 7f94e4000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc19d490 con 0x7f94dc101da0 2024-03-13T14:47:32.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.878+0000 7f94e4000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc19d5d0 con 0x7f94dc0ff430 2024-03-13T14:47:32.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.878+0000 7f94e0c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:32.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.878+0000 7f94e0c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57250/0 (socket says 172.21.15.176:57250) 2024-03-13T14:47:32.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.878+0000 7f94e0c00700 1 -- 172.21.15.176:0/372666419 learned_addr learned my addr 172.21.15.176:0/372666419 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:47:32.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.879+0000 7f94e1600700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc0ff430 0x7f94dc19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:32.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.879+0000 7f94e0c00700 1 -- 172.21.15.176:0/372666419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc0ff430 msgr2=0x7f94dc19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:32.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.879+0000 7f94e0c00700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc0ff430 0x7f94dc19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:32.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.879+0000 7f94e0c00700 1 -- 172.21.15.176:0/372666419 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c80095f0 con 0x7f94dc101da0 2024-03-13T14:47:32.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.880+0000 7f94e0c00700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc19ce70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f94c80092e0 tx=0x7f94c800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:32.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.880+0000 7f94da000700 1 -- 172.21.15.176:0/372666419 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c801d070 con 0x7f94dc101da0 2024-03-13T14:47:32.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.880+0000 7f94da000700 1 -- 172.21.15.176:0/372666419 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94c8005dc0 con 0x7f94dc101da0 2024-03-13T14:47:32.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.880+0000 7f94da000700 1 -- 172.21.15.176:0/372666419 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94c8017630 con 0x7f94dc101da0 2024-03-13T14:47:32.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.880+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94dc1a2020 con 0x7f94dc101da0 2024-03-13T14:47:32.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.880+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc1a2510 con 0x7f94dc101da0 2024-03-13T14:47:32.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.882+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc196b10 con 0x7f94dc101da0 2024-03-13T14:47:32.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.883+0000 7f94da000700 1 -- 172.21.15.176:0/372666419 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f94c800fba0 con 0x7f94dc101da0 2024-03-13T14:47:32.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.884+0000 7f94da000700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f94c406c4e0 0x7f94c406e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:32.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.884+0000 7f94da000700 1 -- 172.21.15.176:0/372666419 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f94c808ce90 con 0x7f94dc101da0 2024-03-13T14:47:32.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.886+0000 7f94e1600700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f94c406c4e0 0x7f94c406e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:32.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.887+0000 7f94e1600700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f94c406c4e0 0x7f94c406e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009bf0 tx=0x7f94d0009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:32.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:32.892+0000 7f94da000700 1 -- 172.21.15.176:0/372666419 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f94c805b1c0 con 0x7f94dc101da0 2024-03-13T14:47:33.114 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.125+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94dc0611d0 con 0x7f94c406c4e0 2024-03-13T14:47:33.116 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.127+0000 7f94da000700 1 -- 172.21.15.176:0/372666419 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f94dc0611d0 con 0x7f94c406c4e0 2024-03-13T14:47:33.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.130+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f94c406c4e0 msgr2=0x7f94c406e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.130+0000 7f94e4000700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f94c406c4e0 0x7f94c406e9a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009bf0 tx=0x7f94d0009260 comp rx=0 tx=0).stop 2024-03-13T14:47:33.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.130+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 msgr2=0x7f94dc19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.130+0000 7f94e4000700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc19ce70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f94c80092e0 tx=0x7f94c800f740 comp rx=0 tx=0).stop 2024-03-13T14:47:33.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.131+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 shutdown_connections 2024-03-13T14:47:33.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.131+0000 7f94e4000700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f94c406c4e0 0x7f94c406e9a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.131+0000 7f94e4000700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94dc0ff430 0x7f94dc19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.131+0000 7f94e4000700 1 --2- 172.21.15.176:0/372666419 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94dc101da0 0x7f94dc19ce70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.131+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 >> 172.21.15.176:0/372666419 conn(0x7f94dc0fb030 msgr2=0x7f94dc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:33.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.132+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 shutdown_connections 2024-03-13T14:47:33.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.132+0000 7f94e4000700 1 -- 172.21.15.176:0/372666419 wait complete. 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.275+0000 7fd3a9e00700 1 -- 172.21.15.176:0/660714743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a40737f0 msgr2=0x7fd3a4073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.275+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/660714743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a40737f0 0x7fd3a4073c70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd390009910 tx=0x7fd390009c20 comp rx=0 tx=0).stop 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.276+0000 7fd3a9e00700 1 -- 172.21.15.176:0/660714743 shutdown_connections 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.276+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/660714743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a40737f0 0x7fd3a4073c70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.276+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/660714743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a4074dc0 0x7fd3a4073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.276+0000 7fd3a9e00700 1 -- 172.21.15.176:0/660714743 >> 172.21.15.176:0/660714743 conn(0x7fd3a40fbf60 msgr2=0x7fd3a40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.276+0000 7fd3a9e00700 1 -- 172.21.15.176:0/660714743 shutdown_connections 2024-03-13T14:47:33.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.276+0000 7fd3a9e00700 1 -- 172.21.15.176:0/660714743 wait complete. 2024-03-13T14:47:33.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.277+0000 7fd3a9e00700 1 Processor -- start 2024-03-13T14:47:33.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a9e00700 1 -- start start 2024-03-13T14:47:33.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a9e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a40737f0 0x7fd3a419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:33.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a4074dc0 0x7fd3a419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:33.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a9e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a419d340 con 0x7fd3a4074dc0 2024-03-13T14:47:33.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a9e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a419d480 con 0x7fd3a40737f0 2024-03-13T14:47:33.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a8a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a40737f0 0x7fd3a419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:33.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a8a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a40737f0 0x7fd3a419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:39792/0 (socket says 172.21.15.176:39792) 2024-03-13T14:47:33.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a8a00700 1 -- 172.21.15.176:0/1519281019 learned_addr learned my addr 172.21.15.176:0/1519281019 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:47:33.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.278+0000 7fd3a3e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a4074dc0 0x7fd3a419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:33.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.279+0000 7fd3a8a00700 1 -- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a4074dc0 msgr2=0x7fd3a419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.279+0000 7fd3a8a00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a4074dc0 0x7fd3a419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.279+0000 7fd3a8a00700 1 -- 172.21.15.176:0/1519281019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3900095f0 con 0x7fd3a40737f0 2024-03-13T14:47:33.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.279+0000 7fd3a8a00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a40737f0 0x7fd3a419c7e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e8f0 tx=0x7fd39800ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:33.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.280+0000 7fd3a1600700 1 -- 172.21.15.176:0/1519281019 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd398009790 con 0x7fd3a40737f0 2024-03-13T14:47:33.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.280+0000 7fd3a1600700 1 -- 172.21.15.176:0/1519281019 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd39800c810 con 0x7fd3a40737f0 2024-03-13T14:47:33.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.280+0000 7fd3a1600700 1 -- 172.21.15.176:0/1519281019 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3980055b0 con 0x7fd3a40737f0 2024-03-13T14:47:33.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.280+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a41a1f30 con 0x7fd3a40737f0 2024-03-13T14:47:33.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.280+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a41a23f0 con 0x7fd3a40737f0 2024-03-13T14:47:33.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.281+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a410fb90 con 0x7fd3a40737f0 2024-03-13T14:47:33.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.283+0000 7fd3a1600700 1 -- 172.21.15.176:0/1519281019 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd39800c980 con 0x7fd3a40737f0 2024-03-13T14:47:33.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.284+0000 7fd3a1600700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd38c06c4e0 0x7fd38c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:33.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.284+0000 7fd3a1600700 1 -- 172.21.15.176:0/1519281019 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd398014070 con 0x7fd3a40737f0 2024-03-13T14:47:33.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.285+0000 7fd3a3e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd38c06c4e0 0x7fd38c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:33.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.285+0000 7fd3a3e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd38c06c4e0 0x7fd38c06e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd390009fd0 tx=0x7fd390005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:33.287 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.298+0000 7fd3a1600700 1 -- 172.21.15.176:0/1519281019 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd398059cc0 con 0x7fd3a40737f0 2024-03-13T14:47:33.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:33 smithi176 ceph-mon[29064]: pgmap v277: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:33.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:33 smithi196 ceph-mon[37486]: pgmap v277: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:33.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.520+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3a41039a0 con 0x7fd38c06c4e0 2024-03-13T14:47:33.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.534+0000 7fd3a1600700 1 -- 172.21.15.176:0/1519281019 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7fd3a41039a0 con 0x7fd38c06c4e0 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (7m) 78s ago 9m 13.5M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (9m) 78s ago 9m 7520k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (7m) 83s ago 7m 7428k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (9m) 78s ago 9m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (7m) 83s ago 7m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (6m) 78s ago 8m 74.0M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (102s) 78s ago 101s 13.0M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:47:33.523 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (111s) 78s ago 110s 11.4M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (107s) 83s ago 106s 14.8M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (98s) 83s ago 97s 11.6M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:9283,8765,8443 running (10m) 78s ago 10m 489M - 18.2.1 5be31c24972a c3fcbaf472c1 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (7m) 83s ago 7m 441M - 18.2.1 5be31c24972a 4af30d3028b6 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (10m) 78s ago 11m 45.2M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (7m) 83s ago 7m 34.2M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (9m) 78s ago 9m 7541k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (7m) 83s ago 7m 7650k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (6m) 78s ago 6m 47.0M 3947M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (5m) 78s ago 5m 47.1M 3947M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (5m) 78s ago 5m 45.8M 3947M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (4m) 83s ago 4m 45.5M 4971M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (4m) 83s ago 4m 42.6M 4971M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:47:33.524 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (3m) 83s ago 3m 42.8M 4971M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:47:33.525 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (6m) 78s ago 8m 48.7M - 2.43.0 a07b618ecd1d 4ec38b851bbe 2024-03-13T14:47:33.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.536+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd38c06c4e0 msgr2=0x7fd38c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.536+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd38c06c4e0 0x7fd38c06e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd390009fd0 tx=0x7fd390005fd0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.536+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a40737f0 msgr2=0x7fd3a419c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.536+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a40737f0 0x7fd3a419c7e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd39800e8f0 tx=0x7fd39800ec00 comp rx=0 tx=0).stop 2024-03-13T14:47:33.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.537+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 shutdown_connections 2024-03-13T14:47:33.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.537+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd38c06c4e0 0x7fd38c06e9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.537+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3a40737f0 0x7fd3a419c7e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.537+0000 7fd3a9e00700 1 --2- 172.21.15.176:0/1519281019 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3a4074dc0 0x7fd3a419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.537+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 >> 172.21.15.176:0/1519281019 conn(0x7fd3a40fbf60 msgr2=0x7fd3a4102280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:33.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.537+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 shutdown_connections 2024-03-13T14:47:33.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.537+0000 7fd3a9e00700 1 -- 172.21.15.176:0/1519281019 wait complete. 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.681+0000 7f3af7600700 1 -- 172.21.15.176:0/2178953738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af01016e0 msgr2=0x7f3af0103ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.681+0000 7f3af7600700 1 --2- 172.21.15.176:0/2178953738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af01016e0 0x7f3af0103ad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009910 tx=0x7f3ae4009c20 comp rx=0 tx=0).stop 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.682+0000 7f3af7600700 1 -- 172.21.15.176:0/2178953738 shutdown_connections 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.682+0000 7f3af7600700 1 --2- 172.21.15.176:0/2178953738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3af0104010 0x7f3af0106400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.682+0000 7f3af7600700 1 --2- 172.21.15.176:0/2178953738 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af01016e0 0x7f3af0103ad0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.682+0000 7f3af7600700 1 -- 172.21.15.176:0/2178953738 >> 172.21.15.176:0/2178953738 conn(0x7f3af00fb030 msgr2=0x7f3af00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.682+0000 7f3af7600700 1 -- 172.21.15.176:0/2178953738 shutdown_connections 2024-03-13T14:47:33.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.682+0000 7f3af7600700 1 -- 172.21.15.176:0/2178953738 wait complete. 2024-03-13T14:47:33.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.683+0000 7f3af7600700 1 Processor -- start 2024-03-13T14:47:33.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.683+0000 7f3af7600700 1 -- start start 2024-03-13T14:47:33.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3af7600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3af01016e0 0x7f3af0196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:33.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3af7600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af0104010 0x7f3af0196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:33.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3af7600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0196e80 con 0x7f3af0104010 2024-03-13T14:47:33.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3af7600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0196fc0 con 0x7f3af01016e0 2024-03-13T14:47:33.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3aefe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af0104010 0x7f3af0196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:33.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3aefe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af0104010 0x7f3af0196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57288/0 (socket says 172.21.15.176:57288) 2024-03-13T14:47:33.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3aefe00700 1 -- 172.21.15.176:0/1965983018 learned_addr learned my addr 172.21.15.176:0/1965983018 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:47:33.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.684+0000 7f3af4c00700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3af01016e0 0x7f3af0196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:33.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.685+0000 7f3aefe00700 1 -- 172.21.15.176:0/1965983018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3af01016e0 msgr2=0x7f3af0196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.685+0000 7f3aefe00700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3af01016e0 0x7f3af0196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.685+0000 7f3aefe00700 1 -- 172.21.15.176:0/1965983018 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae40095f0 con 0x7f3af0104010 2024-03-13T14:47:33.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.685+0000 7f3aefe00700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af0104010 0x7f3af0196860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00e920 tx=0x7f3adc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:33.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.685+0000 7f3aed600700 1 -- 172.21.15.176:0/1965983018 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3adc00c9d0 con 0x7f3af0104010 2024-03-13T14:47:33.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.686+0000 7f3aed600700 1 -- 172.21.15.176:0/1965983018 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3adc010430 con 0x7f3af0104010 2024-03-13T14:47:33.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.686+0000 7f3aed600700 1 -- 172.21.15.176:0/1965983018 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3adc018650 con 0x7f3af0104010 2024-03-13T14:47:33.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.686+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af019ba70 con 0x7f3af0104010 2024-03-13T14:47:33.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.686+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af019bfc0 con 0x7f3af0104010 2024-03-13T14:47:33.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.687+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af0190530 con 0x7f3af0104010 2024-03-13T14:47:33.677 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.688+0000 7f3aed600700 1 -- 172.21.15.176:0/1965983018 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f3adc0105a0 con 0x7f3af0104010 2024-03-13T14:47:33.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.689+0000 7f3aed600700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3ad806c4e0 0x7f3ad806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:33.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.689+0000 7f3aed600700 1 -- 172.21.15.176:0/1965983018 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3adc014070 con 0x7f3af0104010 2024-03-13T14:47:33.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.690+0000 7f3af4c00700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3ad806c4e0 0x7f3ad806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:33.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.690+0000 7f3af4c00700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3ad806c4e0 0x7f3ad806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009fd0 tx=0x7f3ae4005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:33.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.694+0000 7f3aed600700 1 -- 172.21.15.176:0/1965983018 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f3adc059cb0 con 0x7f3af0104010 2024-03-13T14:47:33.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.986+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3af002cc70 con 0x7f3af0104010 2024-03-13T14:47:33.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.987+0000 7f3aed600700 1 -- 172.21.15.176:0/1965983018 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f3adc059840 con 0x7f3af0104010 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:47:33.976 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:47:33.977 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:47:33.977 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:47:33.977 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:47:33.977 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:47:33.977 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-13T14:47:33.977 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:47:33.977 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:47:33.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3ad806c4e0 msgr2=0x7f3ad806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3ad806c4e0 0x7f3ad806e9a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4009fd0 tx=0x7f3ae4005fd0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af0104010 msgr2=0x7f3af0196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:33.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af0104010 0x7f3af0196860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00e920 tx=0x7f3adc00ec30 comp rx=0 tx=0).stop 2024-03-13T14:47:33.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 shutdown_connections 2024-03-13T14:47:33.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f3ad806c4e0 0x7f3ad806e9a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.980 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3af01016e0 0x7f3af0196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.980 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 --2- 172.21.15.176:0/1965983018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3af0104010 0x7f3af0196860 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:33.980 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.990+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 >> 172.21.15.176:0/1965983018 conn(0x7f3af00fb030 msgr2=0x7f3af00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:33.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.993+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 shutdown_connections 2024-03-13T14:47:33.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:33.993+0000 7f3af7600700 1 -- 172.21.15.176:0/1965983018 wait complete. 2024-03-13T14:47:34.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.146+0000 7f34d9800700 1 -- 172.21.15.176:0/2261293414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 msgr2=0x7f34d4073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:34.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.146+0000 7f34d9800700 1 --2- 172.21.15.176:0/2261293414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d4073c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009910 tx=0x7f34c0009c20 comp rx=0 tx=0).stop 2024-03-13T14:47:34.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.147+0000 7f34d9800700 1 -- 172.21.15.176:0/2261293414 shutdown_connections 2024-03-13T14:47:34.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.147+0000 7f34d9800700 1 --2- 172.21.15.176:0/2261293414 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d4073c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:34.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.147+0000 7f34d9800700 1 --2- 172.21.15.176:0/2261293414 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34d4074dc0 0x7f34d4073220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:34.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.147+0000 7f34d9800700 1 -- 172.21.15.176:0/2261293414 >> 172.21.15.176:0/2261293414 conn(0x7f34d40fc100 msgr2=0x7f34d40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:34.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.147+0000 7f34d9800700 1 -- 172.21.15.176:0/2261293414 shutdown_connections 2024-03-13T14:47:34.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.147+0000 7f34d9800700 1 -- 172.21.15.176:0/2261293414 wait complete. 2024-03-13T14:47:34.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.148+0000 7f34d9800700 1 Processor -- start 2024-03-13T14:47:34.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.148+0000 7f34d9800700 1 -- start start 2024-03-13T14:47:34.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.148+0000 7f34d9800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:34.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.149+0000 7f34d9800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34d4074dc0 0x7f34d419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:34.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.149+0000 7f34d9800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d419d480 con 0x7f34d40737f0 2024-03-13T14:47:34.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.149+0000 7f34d9800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d419d5c0 con 0x7f34d4074dc0 2024-03-13T14:47:34.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.149+0000 7f34d2a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:34.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.149+0000 7f34d2a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:57310/0 (socket says 172.21.15.176:57310) 2024-03-13T14:47:34.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.149+0000 7f34d2a00700 1 -- 172.21.15.176:0/2853784208 learned_addr learned my addr 172.21.15.176:0/2853784208 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:47:34.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.149+0000 7f34d2000700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34d4074dc0 0x7f34d419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:34.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34d2a00700 1 -- 172.21.15.176:0/2853784208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34d4074dc0 msgr2=0x7f34d419ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:34.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34d2a00700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34d4074dc0 0x7f34d419ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:34.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34d2a00700 1 -- 172.21.15.176:0/2853784208 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00095f0 con 0x7f34d40737f0 2024-03-13T14:47:34.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34d2a00700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d419c920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f34c400e8f0 tx=0x7f34c400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:34.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34cb400700 1 -- 172.21.15.176:0/2853784208 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c4009790 con 0x7f34d40737f0 2024-03-13T14:47:34.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34cb400700 1 -- 172.21.15.176:0/2853784208 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f34c400c810 con 0x7f34d40737f0 2024-03-13T14:47:34.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d41a2070 con 0x7f34d40737f0 2024-03-13T14:47:34.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.150+0000 7f34cb400700 1 -- 172.21.15.176:0/2853784208 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34c40055b0 con 0x7f34d40737f0 2024-03-13T14:47:34.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.151+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d41a25c0 con 0x7f34d40737f0 2024-03-13T14:47:34.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.152+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d410fd10 con 0x7f34d40737f0 2024-03-13T14:47:34.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.153+0000 7f34cb400700 1 -- 172.21.15.176:0/2853784208 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f34c400c980 con 0x7f34d40737f0 2024-03-13T14:47:34.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.154+0000 7f34cb400700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f34b806c530 0x7f34b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:47:34.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.155+0000 7f34cb400700 1 -- 172.21.15.176:0/2853784208 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f34c4014070 con 0x7f34d40737f0 2024-03-13T14:47:34.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.155+0000 7f34d2000700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f34b806c530 0x7f34b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:47:34.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.156+0000 7f34d2000700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f34b806c530 0x7f34b806e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009fd0 tx=0x7f34c0005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:47:34.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.159+0000 7f34cb400700 1 -- 172.21.15.176:0/2853784208 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f34c4059c90 con 0x7f34d40737f0 2024-03-13T14:47:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:34 smithi176 ceph-mon[29064]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:34 smithi176 ceph-mon[29064]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:34 smithi176 ceph-mon[29064]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:34 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1965983018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:47:34.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.399+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34d4103b20 con 0x7f34b806c530 2024-03-13T14:47:34.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.400+0000 7f34cb400700 1 -- 172.21.15.176:0/2853784208 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f34d4103b20 con 0x7f34b806c530 2024-03-13T14:47:34.389 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:47:34.389 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:47:34.389 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:47:34.389 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:47:34.389 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [], 2024-03-13T14:47:34.390 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "0/2 daemons upgraded", 2024-03-13T14:47:34.390 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a image on host smithi196", 2024-03-13T14:47:34.390 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:47:34.390 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:47:34.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f34b806c530 msgr2=0x7f34b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:34.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f34b806c530 0x7f34b806e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009fd0 tx=0x7f34c0005fd0 comp rx=0 tx=0).stop 2024-03-13T14:47:34.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 msgr2=0x7f34d419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d419c920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f34c400e8f0 tx=0x7f34c400ec00 comp rx=0 tx=0).stop 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 shutdown_connections 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f34b806c530 0x7f34b806e9f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f34d40737f0 0x7f34d419c920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 --2- 172.21.15.176:0/2853784208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34d4074dc0 0x7f34d419ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.403+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 >> 172.21.15.176:0/2853784208 conn(0x7f34d40fc100 msgr2=0x7f34d4102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.404+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 shutdown_connections 2024-03-13T14:47:34.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:47:34.404+0000 7f34d9800700 1 -- 172.21.15.176:0/2853784208 wait complete. 2024-03-13T14:47:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:34 smithi196 ceph-mon[37486]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:34 smithi196 ceph-mon[37486]: from='client.14570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:34 smithi196 ceph-mon[37486]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:34 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1965983018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:47:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:35 smithi176 ceph-mon[29064]: pgmap v278: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:35 smithi176 ceph-mon[29064]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:35.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:35 smithi196 ceph-mon[37486]: pgmap v278: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:35.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:35 smithi196 ceph-mon[37486]: from='client.14582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:47:37.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:37 smithi176 ceph-mon[29064]: pgmap v279: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:37.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:37 smithi196 ceph-mon[37486]: pgmap v279: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:39.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:39 smithi176 ceph-mon[29064]: pgmap v280: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:39.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:39 smithi196 ceph-mon[37486]: pgmap v280: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:41.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:41 smithi196 ceph-mon[37486]: pgmap v281: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:41.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:41 smithi176 ceph-mon[29064]: pgmap v281: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:43.175 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:43 smithi176 ceph-mon[29064]: pgmap v282: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:43.176 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:43 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:43.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:43 smithi196 ceph-mon[37486]: pgmap v282: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:43.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:43 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:45.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:45 smithi176 ceph-mon[29064]: pgmap v283: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:45.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:45 smithi196 ceph-mon[37486]: pgmap v283: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:47.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:47 smithi176 ceph-mon[29064]: pgmap v284: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:47.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:47 smithi196 ceph-mon[37486]: pgmap v284: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:49.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:49 smithi176 ceph-mon[29064]: pgmap v285: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:49.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:49 smithi196 ceph-mon[37486]: pgmap v285: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:51.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:51 smithi176 ceph-mon[29064]: pgmap v286: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:51.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:51 smithi196 ceph-mon[37486]: pgmap v286: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:53.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:53 smithi196 ceph-mon[37486]: pgmap v287: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:53.620 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:53 smithi176 ceph-mon[29064]: pgmap v287: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:54.310 INFO:tasks.workunit.client.0.smithi176.stderr:Updating files: 63% (7791/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-13T14:47:54.698 INFO:tasks.workunit.client.0.smithi176.stderr:Note: switching to '9a75c81fd074f3cb252248b860d5cc88e5f3fb55'. 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr:state without impacting any branches by switching back to a branch. 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr: git switch -c 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-03-13T14:47:54.699 INFO:tasks.workunit.client.0.smithi176.stderr:Or undo this operation with: 2024-03-13T14:47:54.700 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-03-13T14:47:54.700 INFO:tasks.workunit.client.0.smithi176.stderr: git switch - 2024-03-13T14:47:54.700 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-03-13T14:47:54.700 INFO:tasks.workunit.client.0.smithi176.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-13T14:47:54.700 INFO:tasks.workunit.client.0.smithi176.stderr: 2024-03-13T14:47:54.700 INFO:tasks.workunit.client.0.smithi176.stderr:HEAD is now at 9a75c81fd07 Merge PR #52623 into wip-vshankar-testing-20240307.013758 2024-03-13T14:47:54.706 DEBUG:teuthology.orchestra.run.smithi176:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-13T14:47:54.767 INFO:tasks.workunit.client.0.smithi176.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-13T14:47:54.774 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-13T14:47:54.774 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-13T14:47:55.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:55 smithi196 ceph-mon[37486]: pgmap v288: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:55.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:55 smithi176 ceph-mon[29064]: pgmap v288: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:55.684 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-13T14:47:55.806 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-13T14:47:55.888 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-13T14:47:55.893 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-13T14:47:55.893 INFO:tasks.workunit.client.0.smithi176.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-13T14:47:55.973 INFO:tasks.workunit.client.0.smithi176.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-13T14:47:55.983 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T14:47:55.983 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-13T14:47:56.034 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-03-13T14:47:56.037 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-13T14:47:56.038 DEBUG:teuthology.orchestra.run.smithi176:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-03-13T14:47:56.134 INFO:tasks.workunit.client.0.smithi176.stderr:+ mkdir -p fsstress 2024-03-13T14:47:56.143 INFO:tasks.workunit.client.0.smithi176.stderr:+ pushd fsstress 2024-03-13T14:47:56.147 INFO:tasks.workunit.client.0.smithi176.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-13T14:47:56.147 INFO:tasks.workunit.client.0.smithi176.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-13T14:47:57.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:57 smithi176 ceph-mon[29064]: Upgrade: Updating mgr.smithi196.vdktbk 2024-03-13T14:47:57.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:57.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:47:57.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:57 smithi176 ceph-mon[29064]: pgmap v289: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:57.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:47:57.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:57 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:47:57.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:57 smithi176 ceph-mon[29064]: Deploying daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:47:57.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:57 smithi196 ceph-mon[37486]: Upgrade: Updating mgr.smithi196.vdktbk 2024-03-13T14:47:57.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:47:57.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:47:57.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:57 smithi196 ceph-mon[37486]: pgmap v289: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:47:57.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:47:57.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:57 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:47:57.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:57 smithi196 ceph-mon[37486]: Deploying daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:47:58.037 INFO:tasks.workunit.client.0.smithi176.stderr:+ tar xzf ltp-full.tgz 2024-03-13T14:47:58.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:58 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:58.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:58 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:47:59.330 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:47:59 smithi196 ceph-mon[37486]: pgmap v290: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:47:59.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:47:59 smithi176 ceph-mon[29064]: pgmap v290: 65 pgs: 65 active+clean; 588 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:48:01.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:01 smithi176 ceph-mon[29064]: pgmap v291: 65 pgs: 65 active+clean; 4.6 MiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 348 KiB/s wr, 3 op/s 2024-03-13T14:48:01.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:01 smithi196 ceph-mon[37486]: pgmap v291: 65 pgs: 65 active+clean; 4.6 MiB data, 171 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 348 KiB/s wr, 3 op/s 2024-03-13T14:48:02.138 INFO:tasks.workunit.client.1.smithi196.stderr:Updating files: 52% (6498/12271) Updating files: 53% (6504/12271) Updating files: 54% (6627/12271) Updating files: 55% (6750/12271) Updating files: 56% (6872/12271) Updating files: 57% (6995/12271) Updating files: 58% (7118/12271) Updating files: 59% (7240/12271) Updating files: 60% (7363/12271) Updating files: 61% (7486/12271) Updating files: 62% (7609/12271) Updating files: 63% (7731/12271) Updating files: 64% (7854/12271) Updating files: 65% (7977/12271) Updating files: 66% (8099/12271) Updating files: 67% (8222/12271) Updating files: 68% (8345/12271) Updating files: 69% (8467/12271) Updating files: 70% (8590/12271) Updating files: 71% (8713/12271) Updating files: 72% (8836/12271) Updating files: 73% (8958/12271) Updating files: 74% (9081/12271) Updating files: 75% (9204/12271) Updating files: 76% (9326/12271) Updating files: 77% (9449/12271) Updating files: 78% (9572/12271) Updating files: 79% (9695/12271) Updating files: 80% (9817/12271) Updating files: 81% (9940/12271) Updating files: 82% (10063/12271) Updating files: 83% (10185/12271) Updating files: 84% (10308/12271) Updating files: 84% (10314/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr:Note: switching to '9a75c81fd074f3cb252248b860d5cc88e5f3fb55'. 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr: 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr:state without impacting any branches by switching back to a branch. 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr: 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr: 2024-03-13T14:48:02.671 INFO:tasks.workunit.client.1.smithi196.stderr: git switch -c 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr: 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr:Or undo this operation with: 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr: 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr: git switch - 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr: 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr: 2024-03-13T14:48:02.672 INFO:tasks.workunit.client.1.smithi196.stderr:HEAD is now at 9a75c81fd07 Merge PR #52623 into wip-vshankar-testing-20240307.013758 2024-03-13T14:48:02.679 DEBUG:teuthology.orchestra.run.smithi196:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-03-13T14:48:03.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:03 smithi176 ceph-mon[29064]: pgmap v292: 65 pgs: 65 active+clean; 7.5 MiB data, 190 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 603 KiB/s wr, 15 op/s 2024-03-13T14:48:03.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:03 smithi196 ceph-mon[37486]: pgmap v292: 65 pgs: 65 active+clean; 7.5 MiB data, 190 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 603 KiB/s wr, 15 op/s 2024-03-13T14:48:03.839 INFO:tasks.workunit.client.1.smithi196.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-13T14:48:03.846 INFO:tasks.workunit.client.1.smithi196.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-13T14:48:03.846 INFO:tasks.workunit.client.1.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-13T14:48:04.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 -- 172.21.15.176:0/1670876987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 msgr2=0x7f6d74108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 --2- 172.21.15.176:0/1670876987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d74108750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009910 tx=0x7f6d60009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 -- 172.21.15.176:0/1670876987 shutdown_connections 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 --2- 172.21.15.176:0/1670876987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d74108750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 --2- 172.21.15.176:0/1670876987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d74072a60 0x7f6d74107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 -- 172.21.15.176:0/1670876987 >> 172.21.15.176:0/1670876987 conn(0x7f6d7406ddc0 msgr2=0x7f6d74070200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 -- 172.21.15.176:0/1670876987 shutdown_connections 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.591+0000 7f6d79a00700 1 -- 172.21.15.176:0/1670876987 wait complete. 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.593+0000 7f6d79a00700 1 Processor -- start 2024-03-13T14:48:04.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.593+0000 7f6d79a00700 1 -- start start 2024-03-13T14:48:04.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.593+0000 7f6d79a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d74072a60 0x7f6d7419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:04.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.593+0000 7f6d79a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d7419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:04.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.593+0000 7f6d79a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7419f6a0 con 0x7f6d741082d0 2024-03-13T14:48:04.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.593+0000 7f6d79a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7419f7e0 con 0x7f6d74072a60 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.593+0000 7f6d72000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d7419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.594+0000 7f6d72000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d7419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45920/0 (socket says 172.21.15.176:45920) 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.594+0000 7f6d72000700 1 -- 172.21.15.176:0/3289787840 learned_addr learned my addr 172.21.15.176:0/3289787840 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.594+0000 7f6d72a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d74072a60 0x7f6d7419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.594+0000 7f6d72000700 1 -- 172.21.15.176:0/3289787840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d74072a60 msgr2=0x7f6d7419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.594+0000 7f6d72000700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d74072a60 0x7f6d7419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.594+0000 7f6d72000700 1 -- 172.21.15.176:0/3289787840 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600095f0 con 0x7f6d741082d0 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.594+0000 7f6d72000700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d7419f080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005b40 tx=0x7f6d6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:04.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.595+0000 7f6d6b400700 1 -- 172.21.15.176:0/3289787840 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6001d070 con 0x7f6d741082d0 2024-03-13T14:48:04.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.595+0000 7f6d6b400700 1 -- 172.21.15.176:0/3289787840 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6d6000f460 con 0x7f6d741082d0 2024-03-13T14:48:04.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.595+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d741a4230 con 0x7f6d741082d0 2024-03-13T14:48:04.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.595+0000 7f6d6b400700 1 -- 172.21.15.176:0/3289787840 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d60021640 con 0x7f6d741082d0 2024-03-13T14:48:04.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.595+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d741a4720 con 0x7f6d741082d0 2024-03-13T14:48:04.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.596+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d74111e70 con 0x7f6d741082d0 2024-03-13T14:48:04.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.597+0000 7f6d6b400700 1 -- 172.21.15.176:0/3289787840 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f6d6000fa90 con 0x7f6d741082d0 2024-03-13T14:48:04.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.598+0000 7f6d6b400700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f6d5806c4e0 0x7f6d5806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:04.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.599+0000 7f6d6b400700 1 -- 172.21.15.176:0/3289787840 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6d6008cc10 con 0x7f6d741082d0 2024-03-13T14:48:04.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.599+0000 7f6d72a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f6d5806c4e0 0x7f6d5806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:04.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.600+0000 7f6d72a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f6d5806c4e0 0x7f6d5806e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005950 tx=0x7f6d640058e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:04.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.605+0000 7f6d6b400700 1 -- 172.21.15.176:0/3289787840 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f6d6005ae00 con 0x7f6d741082d0 2024-03-13T14:48:04.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.869+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d7410e7e0 con 0x7f6d5806c4e0 2024-03-13T14:48:04.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.871+0000 7f6d6b400700 1 -- 172.21.15.176:0/3289787840 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f6d7410e7e0 con 0x7f6d5806c4e0 2024-03-13T14:48:04.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.874+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f6d5806c4e0 msgr2=0x7f6d5806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:04.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.874+0000 7f6d79a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f6d5806c4e0 0x7f6d5806e9a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005950 tx=0x7f6d640058e0 comp rx=0 tx=0).stop 2024-03-13T14:48:04.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.874+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 msgr2=0x7f6d7419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:04.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.874+0000 7f6d79a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d7419f080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d60005b40 tx=0x7f6d6000be20 comp rx=0 tx=0).stop 2024-03-13T14:48:04.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.875+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 shutdown_connections 2024-03-13T14:48:04.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.875+0000 7f6d79a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f6d5806c4e0 0x7f6d5806e9a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:04.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.875+0000 7f6d79a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d74072a60 0x7f6d7419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:04.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.875+0000 7f6d79a00700 1 --2- 172.21.15.176:0/3289787840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d741082d0 0x7f6d7419f080 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:04.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.875+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 >> 172.21.15.176:0/3289787840 conn(0x7f6d7406ddc0 msgr2=0x7f6d7410d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:04.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.875+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 shutdown_connections 2024-03-13T14:48:04.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:04.875+0000 7f6d79a00700 1 -- 172.21.15.176:0/3289787840 wait complete. 2024-03-13T14:48:04.883 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:48:05.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.053+0000 7ff4be400700 1 -- 172.21.15.176:0/2579571315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b80ffab0 msgr2=0x7ff4b80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.053+0000 7ff4be400700 1 --2- 172.21.15.176:0/2579571315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b80ffab0 0x7ff4b80ffed0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009910 tx=0x7ff4a8009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:05.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.053+0000 7ff4be400700 1 -- 172.21.15.176:0/2579571315 shutdown_connections 2024-03-13T14:48:05.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.053+0000 7ff4be400700 1 --2- 172.21.15.176:0/2579571315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b8100cb0 0x7ff4b8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.053+0000 7ff4be400700 1 --2- 172.21.15.176:0/2579571315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b80ffab0 0x7ff4b80ffed0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.053+0000 7ff4be400700 1 -- 172.21.15.176:0/2579571315 >> 172.21.15.176:0/2579571315 conn(0x7ff4b80fb030 msgr2=0x7ff4b80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:05.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.053+0000 7ff4be400700 1 -- 172.21.15.176:0/2579571315 shutdown_connections 2024-03-13T14:48:05.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.054+0000 7ff4be400700 1 -- 172.21.15.176:0/2579571315 wait complete. 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.055+0000 7ff4be400700 1 Processor -- start 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.055+0000 7ff4be400700 1 -- start start 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.055+0000 7ff4be400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b80ffab0 0x7ff4b8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.055+0000 7ff4be400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b8100cb0 0x7ff4b8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.055+0000 7ff4be400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8199080 con 0x7ff4b8100cb0 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.055+0000 7ff4be400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81991c0 con 0x7ff4b80ffab0 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b8100cb0 0x7ff4b8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b8100cb0 0x7ff4b8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45940/0 (socket says 172.21.15.176:45940) 2024-03-13T14:48:05.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b6a00700 1 -- 172.21.15.176:0/561697373 learned_addr learned my addr 172.21.15.176:0/561697373 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:05.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b7400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b80ffab0 0x7ff4b8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b6a00700 1 -- 172.21.15.176:0/561697373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b80ffab0 msgr2=0x7ff4b8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b6a00700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b80ffab0 0x7ff4b8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b6a00700 1 -- 172.21.15.176:0/561697373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80095f0 con 0x7ff4b8100cb0 2024-03-13T14:48:05.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.056+0000 7ff4b7400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b80ffab0 0x7ff4b8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:48:05.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.057+0000 7ff4b6a00700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b8100cb0 0x7ff4b8198a60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c8a0 tx=0x7ff4a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:05.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.057+0000 7ff4afe00700 1 -- 172.21.15.176:0/561697373 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4a400cea0 con 0x7ff4b8100cb0 2024-03-13T14:48:05.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.057+0000 7ff4afe00700 1 -- 172.21.15.176:0/561697373 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff4a4004830 con 0x7ff4b8100cb0 2024-03-13T14:48:05.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.057+0000 7ff4afe00700 1 -- 172.21.15.176:0/561697373 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4a40055b0 con 0x7ff4b8100cb0 2024-03-13T14:48:05.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.057+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b819dc70 con 0x7ff4b8100cb0 2024-03-13T14:48:05.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.057+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b819e1c0 con 0x7ff4b8100cb0 2024-03-13T14:48:05.047 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.059+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b8108610 con 0x7ff4b8100cb0 2024-03-13T14:48:05.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.060+0000 7ff4afe00700 1 -- 172.21.15.176:0/561697373 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7ff4a4007ad0 con 0x7ff4b8100cb0 2024-03-13T14:48:05.049 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.061+0000 7ff4afe00700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff49c06c4e0 0x7ff49c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.049 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.061+0000 7ff4afe00700 1 -- 172.21.15.176:0/561697373 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff4a408b560 con 0x7ff4b8100cb0 2024-03-13T14:48:05.049 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.061+0000 7ff4b7400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff49c06c4e0 0x7ff49c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.050 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.062+0000 7ff4b7400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff49c06c4e0 0x7ff49c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006010 tx=0x7ff4a800b540 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:05.055 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.066+0000 7ff4afe00700 1 -- 172.21.15.176:0/561697373 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7ff4a4059750 con 0x7ff4b8100cb0 2024-03-13T14:48:05.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.340+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4b8068ae0 con 0x7ff49c06c4e0 2024-03-13T14:48:05.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.344+0000 7ff4afe00700 1 -- 172.21.15.176:0/561697373 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7ff4b8068ae0 con 0x7ff49c06c4e0 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff49c06c4e0 msgr2=0x7ff49c06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff49c06c4e0 0x7ff49c06e9a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006010 tx=0x7ff4a800b540 comp rx=0 tx=0).stop 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b8100cb0 msgr2=0x7ff4b8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b8100cb0 0x7ff4b8198a60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c8a0 tx=0x7ff4a400cc60 comp rx=0 tx=0).stop 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 shutdown_connections 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7ff49c06c4e0 0x7ff49c06e9a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4b80ffab0 0x7ff4b8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 --2- 172.21.15.176:0/561697373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff4b8100cb0 0x7ff4b8198a60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.347+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 >> 172.21.15.176:0/561697373 conn(0x7ff4b80fb030 msgr2=0x7ff4b80683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:05.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.348+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 shutdown_connections 2024-03-13T14:48:05.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.348+0000 7ff4be400700 1 -- 172.21.15.176:0/561697373 wait complete. 2024-03-13T14:48:05.497 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 -- 172.21.15.176:0/861492971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 msgr2=0x7fd4e80ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 --2- 172.21.15.176:0/861492971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e80ff8b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009910 tx=0x7fd4d0009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:05.498 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:05 smithi176 ceph-mon[29064]: pgmap v293: 65 pgs: 65 active+clean; 12 MiB data, 198 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1021 KiB/s wr, 29 op/s 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 -- 172.21.15.176:0/861492971 shutdown_connections 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 --2- 172.21.15.176:0/861492971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4e80ffdf0 0x7fd4e8100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 --2- 172.21.15.176:0/861492971 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e80ff8b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 -- 172.21.15.176:0/861492971 >> 172.21.15.176:0/861492971 conn(0x7fd4e80fb050 msgr2=0x7fd4e80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 -- 172.21.15.176:0/861492971 shutdown_connections 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.509+0000 7fd4edc00700 1 -- 172.21.15.176:0/861492971 wait complete. 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.510+0000 7fd4edc00700 1 Processor -- start 2024-03-13T14:48:05.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.510+0000 7fd4edc00700 1 -- start start 2024-03-13T14:48:05.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4edc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4edc00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4e80ffdf0 0x7fd4e8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4edc00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e8199160 con 0x7fd4e80ff490 2024-03-13T14:48:05.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4edc00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e81992a0 con 0x7fd4e80ffdf0 2024-03-13T14:48:05.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4e6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4e6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45962/0 (socket says 172.21.15.176:45962) 2024-03-13T14:48:05.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4e6a00700 1 -- 172.21.15.176:0/307487042 learned_addr learned my addr 172.21.15.176:0/307487042 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:05.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.511+0000 7fd4e6000700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4e80ffdf0 0x7fd4e8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.512+0000 7fd4e6a00700 1 -- 172.21.15.176:0/307487042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4e80ffdf0 msgr2=0x7fd4e8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.512+0000 7fd4e6a00700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4e80ffdf0 0x7fd4e8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.512+0000 7fd4e6a00700 1 -- 172.21.15.176:0/307487042 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00095f0 con 0x7fd4e80ff490 2024-03-13T14:48:05.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.512+0000 7fd4e6a00700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e8198600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000bd70 tx=0x7fd4d000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:05.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.512+0000 7fd4df400700 1 -- 172.21.15.176:0/307487042 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4d001d070 con 0x7fd4e80ff490 2024-03-13T14:48:05.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.513+0000 7fd4df400700 1 -- 172.21.15.176:0/307487042 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4d0022470 con 0x7fd4e80ff490 2024-03-13T14:48:05.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.513+0000 7fd4df400700 1 -- 172.21.15.176:0/307487042 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4d000f670 con 0x7fd4e80ff490 2024-03-13T14:48:05.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.513+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e819dcf0 con 0x7fd4e80ff490 2024-03-13T14:48:05.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.513+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e819e210 con 0x7fd4e80ff490 2024-03-13T14:48:05.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.514+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e810b990 con 0x7fd4e80ff490 2024-03-13T14:48:05.504 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.516+0000 7fd4df400700 1 -- 172.21.15.176:0/307487042 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7fd4d0022a50 con 0x7fd4e80ff490 2024-03-13T14:48:05.505 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.517+0000 7fd4df400700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd4cc06c4e0 0x7fd4cc06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.505 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.517+0000 7fd4df400700 1 -- 172.21.15.176:0/307487042 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd4d008cf60 con 0x7fd4e80ff490 2024-03-13T14:48:05.506 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.517+0000 7fd4e6000700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd4cc06c4e0 0x7fd4cc06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.506 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.518+0000 7fd4e6000700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd4cc06c4e0 0x7fd4cc06e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:05.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.522+0000 7fd4df400700 1 -- 172.21.15.176:0/307487042 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7fd4d005b200 con 0x7fd4e80ff490 2024-03-13T14:48:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:05 smithi196 ceph-mon[37486]: pgmap v293: 65 pgs: 65 active+clean; 12 MiB data, 198 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 1021 KiB/s wr, 29 op/s 2024-03-13T14:48:05.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.771+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4e81091e0 con 0x7fd4cc06c4e0 2024-03-13T14:48:05.796 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.808+0000 7fd4df400700 1 -- 172.21.15.176:0/307487042 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7fd4e81091e0 con 0x7fd4cc06c4e0 2024-03-13T14:48:05.797 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:48:05.797 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (7m) 110s ago 9m 13.5M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:48:05.797 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (9m) 110s ago 9m 7520k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (8m) 115s ago 8m 7428k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (9m) 110s ago 9m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (8m) 115s ago 8m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (7m) 110s ago 9m 74.0M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (2m) 110s ago 2m 13.0M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (2m) 110s ago 2m 11.4M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (2m) 115s ago 2m 14.8M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (2m) 115s ago 2m 11.6M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:9283,8765,8443 running (11m) 110s ago 11m 489M - 18.2.1 5be31c24972a c3fcbaf472c1 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (8m) 115s ago 8m 441M - 18.2.1 5be31c24972a 4af30d3028b6 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (11m) 110s ago 11m 45.2M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (8m) 115s ago 7m 34.2M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:48:05.798 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (9m) 110s ago 9m 7541k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (8m) 115s ago 8m 7650k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (7m) 110s ago 7m 47.0M 3947M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (6m) 110s ago 6m 47.1M 3947M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (5m) 110s ago 5m 45.8M 3947M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (5m) 115s ago 5m 45.5M 4971M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (4m) 115s ago 4m 42.6M 4971M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (4m) 115s ago 4m 42.8M 4971M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:48:05.799 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (7m) 110s ago 8m 48.7M - 2.43.0 a07b618ecd1d 4ec38b851bbe 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd4cc06c4e0 msgr2=0x7fd4cc06e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd4cc06c4e0 0x7fd4cc06e9a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8008040 comp rx=0 tx=0).stop 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 msgr2=0x7fd4e8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e8198600 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000bd70 tx=0x7fd4d000be50 comp rx=0 tx=0).stop 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 shutdown_connections 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7fd4cc06c4e0 0x7fd4cc06e9a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4e80ff490 0x7fd4e8198600 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 --2- 172.21.15.176:0/307487042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4e80ffdf0 0x7fd4e8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 >> 172.21.15.176:0/307487042 conn(0x7fd4e80fb050 msgr2=0x7fd4e8107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 shutdown_connections 2024-03-13T14:48:05.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.811+0000 7fd4edc00700 1 -- 172.21.15.176:0/307487042 wait complete. 2024-03-13T14:48:05.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.984+0000 7f1a07200700 1 -- 172.21.15.176:0/1693961930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00100a00 msgr2=0x7f1a00100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.984+0000 7f1a07200700 1 --2- 172.21.15.176:0/1693961930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00100a00 0x7f1a00100e20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009910 tx=0x7f19f4009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:05.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.985+0000 7f1a07200700 1 -- 172.21.15.176:0/1693961930 shutdown_connections 2024-03-13T14:48:05.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.985+0000 7f1a07200700 1 --2- 172.21.15.176:0/1693961930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00101c00 0x7f1a00102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.985+0000 7f1a07200700 1 --2- 172.21.15.176:0/1693961930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00100a00 0x7f1a00100e20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.985+0000 7f1a07200700 1 -- 172.21.15.176:0/1693961930 >> 172.21.15.176:0/1693961930 conn(0x7f1a000fbf60 msgr2=0x7f1a000fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:05.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.985+0000 7f1a07200700 1 -- 172.21.15.176:0/1693961930 shutdown_connections 2024-03-13T14:48:05.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.985+0000 7f1a07200700 1 -- 172.21.15.176:0/1693961930 wait complete. 2024-03-13T14:48:05.974 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.986+0000 7f1a07200700 1 Processor -- start 2024-03-13T14:48:05.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.986+0000 7f1a07200700 1 -- start start 2024-03-13T14:48:05.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.987+0000 7f1a07200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00100a00 0x7f1a001983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.987+0000 7f1a07200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00101c00 0x7f1a001988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.976 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.987+0000 7f1a07200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a00198f10 con 0x7f1a00101c00 2024-03-13T14:48:05.976 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.987+0000 7f1a07200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a00199050 con 0x7f1a00100a00 2024-03-13T14:48:05.976 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.987+0000 7f1a05400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00101c00 0x7f1a001988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.976 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.987+0000 7f1a05400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00101c00 0x7f1a001988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45980/0 (socket says 172.21.15.176:45980) 2024-03-13T14:48:05.976 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.987+0000 7f1a05400700 1 -- 172.21.15.176:0/2846051085 learned_addr learned my addr 172.21.15.176:0/2846051085 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f1a05e00700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00100a00 0x7f1a001983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f1a05400700 1 -- 172.21.15.176:0/2846051085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00100a00 msgr2=0x7f1a001983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f1a05400700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00100a00 0x7f1a001983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f1a05400700 1 -- 172.21.15.176:0/2846051085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40095f0 con 0x7f1a00101c00 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f1a05400700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00101c00 0x7f1a001988f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f19f000c8a0 tx=0x7f19f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f19faa00700 1 -- 172.21.15.176:0/2846051085 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19f000cea0 con 0x7f1a00101c00 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f19faa00700 1 -- 172.21.15.176:0/2846051085 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19f0004830 con 0x7f1a00101c00 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.988+0000 7f19faa00700 1 -- 172.21.15.176:0/2846051085 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19f00055b0 con 0x7f1a00101c00 2024-03-13T14:48:05.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.989+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0019db00 con 0x7f1a00101c00 2024-03-13T14:48:05.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.989+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0019e050 con 0x7f1a00101c00 2024-03-13T14:48:05.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.990+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a00105ce0 con 0x7f1a00101c00 2024-03-13T14:48:05.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.991+0000 7f19faa00700 1 -- 172.21.15.176:0/2846051085 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f19f0007ad0 con 0x7f1a00101c00 2024-03-13T14:48:05.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.992+0000 7f19faa00700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f19e806c4e0 0x7f19e806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:05.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.993+0000 7f19faa00700 1 -- 172.21.15.176:0/2846051085 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f19f008b560 con 0x7f1a00101c00 2024-03-13T14:48:05.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.993+0000 7f1a05e00700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f19e806c4e0 0x7f19e806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:05.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.993+0000 7f1a05e00700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f19e806c4e0 0x7f19e806e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f4005920 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:05.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:05.997+0000 7f19faa00700 1 -- 172.21.15.176:0/2846051085 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f19f0059750 con 0x7f1a00101c00 2024-03-13T14:48:06.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.365+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1a0002cc70 con 0x7f1a00101c00 2024-03-13T14:48:06.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.366+0000 7f19faa00700 1 -- 172.21.15.176:0/2846051085 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f19f00592e0 con 0x7f1a00101c00 2024-03-13T14:48:06.354 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:48:06.354 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 14 2024-03-13T14:48:06.355 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:48:06.356 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:48:06.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f19e806c4e0 msgr2=0x7f19e806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:06.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f19e806c4e0 0x7f19e806e9a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f4005920 comp rx=0 tx=0).stop 2024-03-13T14:48:06.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00101c00 msgr2=0x7f1a001988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:06.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00101c00 0x7f1a001988f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f19f000c8a0 tx=0x7f19f000cc60 comp rx=0 tx=0).stop 2024-03-13T14:48:06.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 shutdown_connections 2024-03-13T14:48:06.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f19e806c4e0 0x7f19e806e9a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a00100a00 0x7f1a001983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 --2- 172.21.15.176:0/2846051085 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1a00101c00 0x7f1a001988f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.369+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 >> 172.21.15.176:0/2846051085 conn(0x7f1a000fbf60 msgr2=0x7f1a00102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:06.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.370+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 shutdown_connections 2024-03-13T14:48:06.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.370+0000 7f1a07200700 1 -- 172.21.15.176:0/2846051085 wait complete. 2024-03-13T14:48:06.446 INFO:tasks.workunit.client.1.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-13T14:48:06.525 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:06 smithi176 ceph-mon[29064]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:06.526 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:06 smithi176 ceph-mon[29064]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:06.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.536+0000 7f412a400700 1 -- 172.21.15.176:0/672293995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4124101f20 msgr2=0x7f4124102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:06.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.536+0000 7f412a400700 1 --2- 172.21.15.176:0/672293995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4124101f20 0x7f4124102380 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4110009910 tx=0x7f4110009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:06.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.537+0000 7f412a400700 1 -- 172.21.15.176:0/672293995 shutdown_connections 2024-03-13T14:48:06.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.537+0000 7f412a400700 1 --2- 172.21.15.176:0/672293995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4124101f20 0x7f4124102380 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.537+0000 7f412a400700 1 --2- 172.21.15.176:0/672293995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41240feea0 0x7f41240ff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.537+0000 7f412a400700 1 -- 172.21.15.176:0/672293995 >> 172.21.15.176:0/672293995 conn(0x7f41240fa7c0 msgr2=0x7f41240fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:06.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.537+0000 7f412a400700 1 -- 172.21.15.176:0/672293995 shutdown_connections 2024-03-13T14:48:06.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.537+0000 7f412a400700 1 -- 172.21.15.176:0/672293995 wait complete. 2024-03-13T14:48:06.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.538+0000 7f412a400700 1 Processor -- start 2024-03-13T14:48:06.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.538+0000 7f412a400700 1 -- start start 2024-03-13T14:48:06.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.538+0000 7f412a400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41240feea0 0x7f4124198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:06.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.538+0000 7f412a400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4124101f20 0x7f4124198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:06.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.538+0000 7f412a400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41241990e0 con 0x7f41240feea0 2024-03-13T14:48:06.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.538+0000 7f412a400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4124199220 con 0x7f4124101f20 2024-03-13T14:48:06.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.539+0000 7f4123400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41240feea0 0x7f4124198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:06.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.539+0000 7f4123400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41240feea0 0x7f4124198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:45996/0 (socket says 172.21.15.176:45996) 2024-03-13T14:48:06.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.539+0000 7f4123400700 1 -- 172.21.15.176:0/361054546 learned_addr learned my addr 172.21.15.176:0/361054546 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:06.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.539+0000 7f4122a00700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4124101f20 0x7f4124198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:06.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.539+0000 7f4123400700 1 -- 172.21.15.176:0/361054546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4124101f20 msgr2=0x7f4124198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.539+0000 7f4123400700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4124101f20 0x7f4124198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.539+0000 7f4123400700 1 -- 172.21.15.176:0/361054546 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41100095f0 con 0x7f41240feea0 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.540+0000 7f4123400700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41240feea0 0x7f4124198580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f411400e8f0 tx=0x7f411400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.540+0000 7f411be00700 1 -- 172.21.15.176:0/361054546 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4114009790 con 0x7f41240feea0 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.540+0000 7f411be00700 1 -- 172.21.15.176:0/361054546 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f411400c810 con 0x7f41240feea0 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.540+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412419dcd0 con 0x7f41240feea0 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.540+0000 7f411be00700 1 -- 172.21.15.176:0/361054546 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4114003d20 con 0x7f41240feea0 2024-03-13T14:48:06.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.541+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412419e1c0 con 0x7f41240feea0 2024-03-13T14:48:06.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.542+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4124106000 con 0x7f41240feea0 2024-03-13T14:48:06.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.543+0000 7f411be00700 1 -- 172.21.15.176:0/361054546 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 19) v1 ==== 90314+0+0 (secure 0 0 0) 0x7f41140041f0 con 0x7f41240feea0 2024-03-13T14:48:06.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.544+0000 7f411be00700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f410806c4e0 0x7f410806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:06.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.544+0000 7f411be00700 1 -- 172.21.15.176:0/361054546 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4114014070 con 0x7f41240feea0 2024-03-13T14:48:06.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.545+0000 7f4122a00700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f410806c4e0 0x7f410806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:06.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.546+0000 7f4122a00700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f410806c4e0 0x7f410806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4110009fd0 tx=0x7f4110005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:06.538 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.550+0000 7f411be00700 1 -- 172.21.15.176:0/361054546 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180422 (secure 0 0 0) 0x7f41140595d0 con 0x7f41240feea0 2024-03-13T14:48:06.664 INFO:tasks.workunit.client.1.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-13T14:48:06.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:06 smithi196 ceph-mon[37486]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:06.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:06 smithi196 ceph-mon[37486]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:06.760 INFO:tasks.workunit.client.1.smithi196.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-03-13T14:48:06.765 INFO:tasks.workunit.client.1.smithi196.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-13T14:48:06.765 INFO:tasks.workunit.client.1.smithi196.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-13T14:48:06.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.856+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 --> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4124061bd0 con 0x7f410806c4e0 2024-03-13T14:48:06.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.859+0000 7f411be00700 1 -- 172.21.15.176:0/361054546 <== mgr.14223 v2:172.21.15.176:6800/2698132085 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f4124061bd0 con 0x7f410806c4e0 2024-03-13T14:48:06.848 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:48:06.848 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:48:06.848 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:48:06.848 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:48:06.848 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [], 2024-03-13T14:48:06.848 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "0/2 daemons upgraded", 2024-03-13T14:48:06.849 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mgr daemons", 2024-03-13T14:48:06.849 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:48:06.849 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:48:06.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f410806c4e0 msgr2=0x7f410806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f410806c4e0 0x7f410806e9a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4110009fd0 tx=0x7f4110005fd0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41240feea0 msgr2=0x7f4124198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41240feea0 0x7f4124198580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f411400e8f0 tx=0x7f411400ec00 comp rx=0 tx=0).stop 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 shutdown_connections 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:6800/2698132085,v1:172.21.15.176:6801/2698132085] conn(0x7f410806c4e0 0x7f410806e9a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f41240feea0 0x7f4124198580 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 --2- 172.21.15.176:0/361054546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4124101f20 0x7f4124198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 >> 172.21.15.176:0/361054546 conn(0x7f41240fa7c0 msgr2=0x7f4124103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 shutdown_connections 2024-03-13T14:48:06.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:06.863+0000 7f412a400700 1 -- 172.21.15.176:0/361054546 wait complete. 2024-03-13T14:48:06.853 INFO:tasks.workunit.client.1.smithi196.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-03-13T14:48:06.865 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-03-13T14:48:06.865 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-03-13T14:48:06.912 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-03-13T14:48:06.913 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-03-13T14:48:06.913 DEBUG:teuthology.orchestra.run.smithi196:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-03-13T14:48:07.021 INFO:tasks.workunit.client.1.smithi196.stderr:+ mkdir -p fsstress 2024-03-13T14:48:07.028 INFO:tasks.workunit.client.1.smithi196.stderr:+ pushd fsstress 2024-03-13T14:48:07.032 INFO:tasks.workunit.client.1.smithi196.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-13T14:48:07.032 INFO:tasks.workunit.client.1.smithi196.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-03-13T14:48:07.470 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:07 smithi196 ceph-mon[37486]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:07.471 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:07 smithi196 ceph-mon[37486]: pgmap v294: 65 pgs: 65 active+clean; 13 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s rd, 1.1 MiB/s wr, 40 op/s 2024-03-13T14:48:07.471 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:07 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2846051085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:48:07.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:07 smithi176 ceph-mon[29064]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:07.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:07 smithi176 ceph-mon[29064]: pgmap v294: 65 pgs: 65 active+clean; 13 MiB data, 203 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s rd, 1.1 MiB/s wr, 40 op/s 2024-03-13T14:48:07.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:07 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2846051085' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:48:08.355 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:08 smithi196 ceph-mon[37486]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:08.529 INFO:tasks.workunit.client.1.smithi196.stderr:+ tar xzf ltp-full.tgz 2024-03-13T14:48:08.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:08 smithi176 ceph-mon[29064]: from='client.14602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:09.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:09 smithi196 ceph-mon[37486]: pgmap v295: 65 pgs: 65 active+clean; 14 MiB data, 207 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-03-13T14:48:09.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:09 smithi176 ceph-mon[29064]: pgmap v295: 65 pgs: 65 active+clean; 14 MiB data, 207 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-03-13T14:48:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:11 smithi176 ceph-mon[29064]: pgmap v296: 65 pgs: 65 active+clean; 15 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-03-13T14:48:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:11 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:11.414 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:11 smithi196 ceph-mon[37486]: pgmap v296: 65 pgs: 65 active+clean; 15 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-03-13T14:48:11.414 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:11.414 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:11.414 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:11 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:12.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:12.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:12 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:12.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:12 smithi176 ceph-mon[29064]: pgmap v297: 65 pgs: 65 active+clean; 24 MiB data, 248 MiB used, 536 GiB / 536 GiB avail; 283 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-03-13T14:48:13.139 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:13.139 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:12 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:13.139 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:12 smithi196 ceph-mon[37486]: pgmap v297: 65 pgs: 65 active+clean; 24 MiB data, 248 MiB used, 536 GiB / 536 GiB avail; 283 KiB/s rd, 1.7 MiB/s wr, 97 op/s 2024-03-13T14:48:13.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:13 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:14.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:13 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:14.885 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:14 smithi196 ceph-mon[37486]: pgmap v298: 65 pgs: 65 active+clean; 30 MiB data, 272 MiB used, 536 GiB / 536 GiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-03-13T14:48:15.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:14 smithi176 ceph-mon[29064]: pgmap v298: 65 pgs: 65 active+clean; 30 MiB data, 272 MiB used, 536 GiB / 536 GiB avail; 281 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-03-13T14:48:16.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:16.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:16 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:16.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:16 smithi196 ceph-mon[37486]: pgmap v299: 65 pgs: 65 active+clean; 32 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-03-13T14:48:17.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:17.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:16 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:17.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:16 smithi176 ceph-mon[29064]: pgmap v299: 65 pgs: 65 active+clean; 32 MiB data, 287 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-03-13T14:48:18.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:18 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:18.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:18 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:18.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:18 smithi196 ceph-mon[37486]: pgmap v300: 65 pgs: 65 active+clean; 38 MiB data, 318 MiB used, 536 GiB / 536 GiB avail; 619 KiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-13T14:48:19.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:18 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:19.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:18 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:19.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:18 smithi176 ceph-mon[29064]: pgmap v300: 65 pgs: 65 active+clean; 38 MiB data, 318 MiB used, 536 GiB / 536 GiB avail; 619 KiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-03-13T14:48:21.029 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:21 smithi196 ceph-mon[37486]: pgmap v301: 65 pgs: 65 active+clean; 40 MiB data, 334 MiB used, 536 GiB / 536 GiB avail; 619 KiB/s rd, 2.3 MiB/s wr, 212 op/s 2024-03-13T14:48:21.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:21 smithi176 ceph-mon[29064]: pgmap v301: 65 pgs: 65 active+clean; 40 MiB data, 334 MiB used, 536 GiB / 536 GiB avail; 619 KiB/s rd, 2.3 MiB/s wr, 212 op/s 2024-03-13T14:48:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:23 smithi196 ceph-mon[37486]: pgmap v302: 65 pgs: 65 active+clean; 46 MiB data, 356 MiB used, 536 GiB / 536 GiB avail; 961 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2024-03-13T14:48:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:23 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk restarted 2024-03-13T14:48:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:23 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:48:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:48:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:48:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:48:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:48:23.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:23 smithi176 ceph-mon[29064]: pgmap v302: 65 pgs: 65 active+clean; 46 MiB data, 356 MiB used, 536 GiB / 536 GiB avail; 961 KiB/s rd, 2.8 MiB/s wr, 260 op/s 2024-03-13T14:48:23.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:23 smithi176 ceph-mon[29064]: Standby manager daemon smithi196.vdktbk restarted 2024-03-13T14:48:23.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:23 smithi176 ceph-mon[29064]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:48:23.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:48:23.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:48:23.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:48:23.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:48:24.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:24 smithi176 ceph-mon[29064]: mgrmap e20: smithi176.tzfdwd(active, since 8m), standbys: smithi196.vdktbk 2024-03-13T14:48:24.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:24 smithi196 ceph-mon[37486]: mgrmap e20: smithi176.tzfdwd(active, since 8m), standbys: smithi196.vdktbk 2024-03-13T14:48:25.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:25 smithi176 ceph-mon[29064]: pgmap v303: 65 pgs: 65 active+clean; 52 MiB data, 379 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-03-13T14:48:25.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:25 smithi196 ceph-mon[37486]: pgmap v303: 65 pgs: 65 active+clean; 52 MiB data, 379 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-03-13T14:48:27.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:27 smithi176 ceph-mon[29064]: pgmap v304: 65 pgs: 65 active+clean; 56 MiB data, 409 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-03-13T14:48:27.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:27 smithi196 ceph-mon[37486]: pgmap v304: 65 pgs: 65 active+clean; 56 MiB data, 409 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-03-13T14:48:28.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:28 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:28.668 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:28 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:29.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: pgmap v305: 65 pgs: 65 active+clean; 58 MiB data, 428 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: Upgrade: Need to upgrade myself (mgr.smithi176.tzfdwd) 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr fail", "who": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: osdmap e48: 6 total, 6 up, 6 in 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "mgr fail", "who": "smithi176.tzfdwd"}]': finished 2024-03-13T14:48:29.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: mgrmap e21: smithi196.vdktbk(active, starting, since 0.0128589s) 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:48:29.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: Manager daemon smithi196.vdktbk is now available 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/trash_purge_schedule"}]: dispatch 2024-03-13T14:48:29.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:29 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/trash_purge_schedule"}]: dispatch 2024-03-13T14:48:29.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: pgmap v305: 65 pgs: 65 active+clean; 58 MiB data, 428 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 282 op/s 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: Upgrade: Need to upgrade myself (mgr.smithi176.tzfdwd) 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr fail", "who": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: osdmap e48: 6 total, 6 up, 6 in 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.14223 172.21.15.176:0/1243286049' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "mgr fail", "who": "smithi176.tzfdwd"}]': finished 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: mgrmap e21: smithi196.vdktbk(active, starting, since 0.0128589s) 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:48:29.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: Manager daemon smithi196.vdktbk is now available 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:29.681 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:48:29.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:29.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:48:29.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:48:29.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/trash_purge_schedule"}]: dispatch 2024-03-13T14:48:29.682 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:29 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi196.vdktbk/trash_purge_schedule"}]: dispatch 2024-03-13T14:48:30.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: [13/Mar/2024:14:48:29] ENGINE Bus STARTING 2024-03-13T14:48:30.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: [13/Mar/2024:14:48:29] ENGINE Serving on https://172.21.15.196:7150 2024-03-13T14:48:30.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: [13/Mar/2024:14:48:29] ENGINE Serving on http://172.21.15.196:8765 2024-03-13T14:48:30.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: [13/Mar/2024:14:48:29] ENGINE Bus STARTED 2024-03-13T14:48:30.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: Deploying cephadm binary to smithi176 2024-03-13T14:48:30.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: Deploying cephadm binary to smithi196 2024-03-13T14:48:30.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: mgrmap e22: smithi196.vdktbk(active, since 1.01483s) 2024-03-13T14:48:30.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:30 smithi196 ceph-mon[37486]: pgmap v3: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:31.040 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:48:29] ENGINE Bus STARTING 2024-03-13T14:48:31.040 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:48:29] ENGINE Serving on https://172.21.15.196:7150 2024-03-13T14:48:31.041 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:48:29] ENGINE Serving on http://172.21.15.196:8765 2024-03-13T14:48:31.041 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: [13/Mar/2024:14:48:29] ENGINE Bus STARTED 2024-03-13T14:48:31.041 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: Deploying cephadm binary to smithi176 2024-03-13T14:48:31.041 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: Deploying cephadm binary to smithi196 2024-03-13T14:48:31.041 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: mgrmap e22: smithi196.vdktbk(active, since 1.01483s) 2024-03-13T14:48:31.041 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:30 smithi176 ceph-mon[29064]: pgmap v3: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:31.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:31 smithi176 ceph-mon[29064]: pgmap v4: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:31.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:31 smithi196 ceph-mon[37486]: pgmap v4: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:32.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:32 smithi196 ceph-mon[37486]: mgrmap e23: smithi196.vdktbk(active, since 3s) 2024-03-13T14:48:33.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:32 smithi176 ceph-mon[29064]: mgrmap e23: smithi196.vdktbk(active, since 3s) 2024-03-13T14:48:33.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:33 smithi196 ceph-mon[37486]: pgmap v5: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:34.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:33 smithi176 ceph-mon[29064]: pgmap v5: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:35.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:35 smithi176 ceph-mon[29064]: pgmap v6: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:35.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:35 smithi196 ceph-mon[37486]: pgmap v6: 65 pgs: 65 active+clean; 71 MiB data, 482 MiB used, 536 GiB / 536 GiB avail 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.030+0000 7f997a600700 1 -- 172.21.15.176:0/388865633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 msgr2=0x7f9974103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.030+0000 7f997a600700 1 --2- 172.21.15.176:0/388865633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974103a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f995c009870 tx=0x7f995c009b80 comp rx=0 tx=0).stop 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.030+0000 7f997a600700 1 -- 172.21.15.176:0/388865633 shutdown_connections 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.030+0000 7f997a600700 1 --2- 172.21.15.176:0/388865633 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9974103f70 0x7f9974106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.030+0000 7f997a600700 1 --2- 172.21.15.176:0/388865633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974103a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.030+0000 7f997a600700 1 -- 172.21.15.176:0/388865633 >> 172.21.15.176:0/388865633 conn(0x7f99740fb030 msgr2=0x7f99740fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.031+0000 7f997a600700 1 -- 172.21.15.176:0/388865633 shutdown_connections 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.031+0000 7f997a600700 1 -- 172.21.15.176:0/388865633 wait complete. 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.032+0000 7f997a600700 1 Processor -- start 2024-03-13T14:48:37.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.032+0000 7f997a600700 1 -- start start 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.032+0000 7f997a600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974194110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.032+0000 7f997a600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9974103f70 0x7f9974194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.032+0000 7f997a600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974194c70 con 0x7f9974103f70 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.032+0000 7f997a600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974194db0 con 0x7f9974101640 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.033+0000 7f9973400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974194110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.033+0000 7f9973400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974194110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:60786/0 (socket says 172.21.15.176:60786) 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.033+0000 7f9973400700 1 -- 172.21.15.176:0/2915181835 learned_addr learned my addr 172.21.15.176:0/2915181835 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.033+0000 7f9972a00700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9974103f70 0x7f9974194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.033+0000 7f9973400700 1 -- 172.21.15.176:0/2915181835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9974103f70 msgr2=0x7f9974194650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.033+0000 7f9973400700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9974103f70 0x7f9974194650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.033+0000 7f9973400700 1 -- 172.21.15.176:0/2915181835 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c009520 con 0x7f9974101640 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.034+0000 7f9973400700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974194110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f995c009230 tx=0x7f995c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:37.023 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.034+0000 7f996be00700 1 -- 172.21.15.176:0/2915181835 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c01c070 con 0x7f9974101640 2024-03-13T14:48:37.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.035+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9974199800 con 0x7f9974101640 2024-03-13T14:48:37.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.035+0000 7f996be00700 1 -- 172.21.15.176:0/2915181835 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f995c00fd90 con 0x7f9974101640 2024-03-13T14:48:37.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.035+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9974199d70 con 0x7f9974101640 2024-03-13T14:48:37.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.035+0000 7f996be00700 1 -- 172.21.15.176:0/2915181835 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c022e90 con 0x7f9974101640 2024-03-13T14:48:37.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.036+0000 7f996be00700 1 -- 172.21.15.176:0/2915181835 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 23) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f995c020070 con 0x7f9974101640 2024-03-13T14:48:37.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.037+0000 7f996be00700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f9958039230 0x7f995803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:37.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.037+0000 7f996be00700 1 -- 172.21.15.176:0/2915181835 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f995c04e6f0 con 0x7f9974101640 2024-03-13T14:48:37.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.037+0000 7f9972a00700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f9958039230 0x7f995803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:37.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.037+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99740ffb80 con 0x7f9974101640 2024-03-13T14:48:37.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.038+0000 7f9972a00700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f9958039230 0x7f995803b6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f996400f540 tx=0x7f996400b040 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:37.036 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.047+0000 7f996be00700 1 -- 172.21.15.176:0/2915181835 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f995c027080 con 0x7f9974101640 2024-03-13T14:48:37.359 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.370+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f99740fc490 con 0x7f9958039230 2024-03-13T14:48:37.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.372+0000 7f996be00700 1 -- 172.21.15.176:0/2915181835 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f99740fc490 con 0x7f9958039230 2024-03-13T14:48:37.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f9958039230 msgr2=0x7f995803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f9958039230 0x7f995803b6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f996400f540 tx=0x7f996400b040 comp rx=0 tx=0).stop 2024-03-13T14:48:37.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 msgr2=0x7f9974194110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974194110 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f995c009230 tx=0x7f995c00f690 comp rx=0 tx=0).stop 2024-03-13T14:48:37.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 shutdown_connections 2024-03-13T14:48:37.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f9958039230 0x7f995803b6f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9974101640 0x7f9974194110 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 --2- 172.21.15.176:0/2915181835 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9974103f70 0x7f9974194650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.375+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 >> 172.21.15.176:0/2915181835 conn(0x7f99740fb030 msgr2=0x7f9974103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:37.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.376+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 shutdown_connections 2024-03-13T14:48:37.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.376+0000 7f997a600700 1 -- 172.21.15.176:0/2915181835 wait complete. 2024-03-13T14:48:37.384 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:48:37.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.600+0000 7f25aaa00700 1 -- 172.21.15.176:0/3536497632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 msgr2=0x7f25a4101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.600+0000 7f25aaa00700 1 --2- 172.21.15.176:0/3536497632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a4101860 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2594009910 tx=0x7f2594009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:37.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.600+0000 7f25aaa00700 1 -- 172.21.15.176:0/3536497632 shutdown_connections 2024-03-13T14:48:37.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.600+0000 7f25aaa00700 1 --2- 172.21.15.176:0/3536497632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f25a4101da0 0x7f25a4104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.600+0000 7f25aaa00700 1 --2- 172.21.15.176:0/3536497632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a4101860 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.600+0000 7f25aaa00700 1 -- 172.21.15.176:0/3536497632 >> 172.21.15.176:0/3536497632 conn(0x7f25a40fb030 msgr2=0x7f25a40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:37.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.601+0000 7f25aaa00700 1 -- 172.21.15.176:0/3536497632 shutdown_connections 2024-03-13T14:48:37.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.601+0000 7f25aaa00700 1 -- 172.21.15.176:0/3536497632 wait complete. 2024-03-13T14:48:37.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.602+0000 7f25aaa00700 1 Processor -- start 2024-03-13T14:48:37.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.602+0000 7f25aaa00700 1 -- start start 2024-03-13T14:48:37.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.603+0000 7f25aaa00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:37.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.603+0000 7f25aaa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f25a4101da0 0x7f25a419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:37.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.603+0000 7f25aaa00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a419d450 con 0x7f25a4101da0 2024-03-13T14:48:37.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.603+0000 7f25aaa00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a419d590 con 0x7f25a40ff430 2024-03-13T14:48:37.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.604+0000 7f25a3e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:37.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.604+0000 7f25a3e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:60812/0 (socket says 172.21.15.176:60812) 2024-03-13T14:48:37.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.604+0000 7f25a3e00700 1 -- 172.21.15.176:0/543700680 learned_addr learned my addr 172.21.15.176:0/543700680 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:37.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.604+0000 7f25a3e00700 1 -- 172.21.15.176:0/543700680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f25a4101da0 msgr2=0x7f25a419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.604+0000 7f25a3e00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f25a4101da0 0x7f25a419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.604+0000 7f25a3e00700 1 -- 172.21.15.176:0/543700680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25940095f0 con 0x7f25a40ff430 2024-03-13T14:48:37.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.605+0000 7f25a3e00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a419c8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2594005b40 tx=0x7f259400be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:37.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.605+0000 7f25a0c00700 1 -- 172.21.15.176:0/543700680 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f259401d070 con 0x7f25a40ff430 2024-03-13T14:48:37.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.605+0000 7f25a0c00700 1 -- 172.21.15.176:0/543700680 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f259400f460 con 0x7f25a40ff430 2024-03-13T14:48:37.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.605+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a41a1fe0 con 0x7f25a40ff430 2024-03-13T14:48:37.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.605+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a41a24d0 con 0x7f25a40ff430 2024-03-13T14:48:37.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.605+0000 7f25a0c00700 1 -- 172.21.15.176:0/543700680 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2594021640 con 0x7f25a40ff430 2024-03-13T14:48:37.597 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.608+0000 7f25a0c00700 1 -- 172.21.15.176:0/543700680 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 23) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f259402b440 con 0x7f25a40ff430 2024-03-13T14:48:37.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.611+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584005320 con 0x7f25a40ff430 2024-03-13T14:48:37.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.611+0000 7f25a0c00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f2588039280 0x7f258803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:37.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.611+0000 7f25a0c00700 1 -- 172.21.15.176:0/543700680 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f259404f7a0 con 0x7f25a40ff430 2024-03-13T14:48:37.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.611+0000 7f25a3400700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f2588039280 0x7f258803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:37.601 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.613+0000 7f25a3400700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f2588039280 0x7f258803b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2590005d90 tx=0x7f2590005d00 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:37.608 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.618+0000 7f25a0c00700 1 -- 172.21.15.176:0/543700680 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f2594029930 con 0x7f25a40ff430 2024-03-13T14:48:37.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.925+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2584000bf0 con 0x7f2588039280 2024-03-13T14:48:37.915 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.927+0000 7f25a0c00700 1 -- 172.21.15.176:0/543700680 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f2584000bf0 con 0x7f2588039280 2024-03-13T14:48:37.918 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.930+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f2588039280 msgr2=0x7f258803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.930+0000 7f25aaa00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f2588039280 0x7f258803b740 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2590005d90 tx=0x7f2590005d00 comp rx=0 tx=0).stop 2024-03-13T14:48:37.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.930+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 msgr2=0x7f25a419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:37.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.930+0000 7f25aaa00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a419c8f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2594005b40 tx=0x7f259400be20 comp rx=0 tx=0).stop 2024-03-13T14:48:37.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.931+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 shutdown_connections 2024-03-13T14:48:37.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.931+0000 7f25aaa00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f2588039280 0x7f258803b740 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.931+0000 7f25aaa00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25a40ff430 0x7f25a419c8f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.931+0000 7f25aaa00700 1 --2- 172.21.15.176:0/543700680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f25a4101da0 0x7f25a419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:37.920 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.931+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 >> 172.21.15.176:0/543700680 conn(0x7f25a40fb030 msgr2=0x7f25a40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:37.920 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.931+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 shutdown_connections 2024-03-13T14:48:37.920 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:37.931+0000 7f25aaa00700 1 -- 172.21.15.176:0/543700680 wait complete. 2024-03-13T14:48:37.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:37 smithi196 ceph-mon[37486]: pgmap v7: 65 pgs: 65 active+clean; 93 MiB data, 529 MiB used, 536 GiB / 536 GiB avail; 593 KiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-03-13T14:48:38.086 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:37 smithi176 ceph-mon[29064]: pgmap v7: 65 pgs: 65 active+clean; 93 MiB data, 529 MiB used, 536 GiB / 536 GiB avail; 593 KiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-03-13T14:48:38.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.110+0000 7f5910800700 1 -- 172.21.15.176:0/533648135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 msgr2=0x7f5908103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:38.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.110+0000 7f5910800700 1 --2- 172.21.15.176:0/533648135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 0x7f5908103a30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009f90 tx=0x7f58fc00b330 comp rx=0 tx=0).stop 2024-03-13T14:48:38.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.110+0000 7f5910800700 1 -- 172.21.15.176:0/533648135 shutdown_connections 2024-03-13T14:48:38.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.110+0000 7f5910800700 1 --2- 172.21.15.176:0/533648135 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 0x7f5908106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.110+0000 7f5910800700 1 --2- 172.21.15.176:0/533648135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 0x7f5908103a30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.110+0000 7f5910800700 1 -- 172.21.15.176:0/533648135 >> 172.21.15.176:0/533648135 conn(0x7f59080fb050 msgr2=0x7f59080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:38.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.111+0000 7f5910800700 1 -- 172.21.15.176:0/533648135 shutdown_connections 2024-03-13T14:48:38.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.111+0000 7f5910800700 1 -- 172.21.15.176:0/533648135 wait complete. 2024-03-13T14:48:38.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.112+0000 7f5910800700 1 Processor -- start 2024-03-13T14:48:38.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.112+0000 7f5910800700 1 -- start start 2024-03-13T14:48:38.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.112+0000 7f5910800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 0x7f59081985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:38.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.112+0000 7f5910800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 0x7f5908198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:38.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.112+0000 7f5910800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5908199120 con 0x7f5908103f70 2024-03-13T14:48:38.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.112+0000 7f5910800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5908199260 con 0x7f5908101640 2024-03-13T14:48:38.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.113+0000 7f590d400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 0x7f5908198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:38.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.113+0000 7f590de00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 0x7f59081985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:38.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.113+0000 7f590d400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 0x7f5908198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59390/0 (socket says 172.21.15.176:59390) 2024-03-13T14:48:38.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.113+0000 7f590d400700 1 -- 172.21.15.176:0/3165912628 learned_addr learned my addr 172.21.15.176:0/3165912628 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:38.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.113+0000 7f590d400700 1 -- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 msgr2=0x7f59081985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:38.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.113+0000 7f590d400700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 0x7f59081985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.113+0000 7f590d400700 1 -- 172.21.15.176:0/3165912628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58fc009c40 con 0x7f5908103f70 2024-03-13T14:48:38.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.114+0000 7f590d400700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 0x7f5908198b00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58f800e940 tx=0x7f58f800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:38.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.114+0000 7f5902a00700 1 -- 172.21.15.176:0/3165912628 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f8004a00 con 0x7f5908103f70 2024-03-13T14:48:38.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.115+0000 7f5902a00700 1 -- 172.21.15.176:0/3165912628 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f58f800cc70 con 0x7f5908103f70 2024-03-13T14:48:38.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.115+0000 7f5902a00700 1 -- 172.21.15.176:0/3165912628 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f80055b0 con 0x7f5908103f70 2024-03-13T14:48:38.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.115+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590819dcc0 con 0x7f5908103f70 2024-03-13T14:48:38.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.115+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590819e290 con 0x7f5908103f70 2024-03-13T14:48:38.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.117+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58ec005320 con 0x7f5908103f70 2024-03-13T14:48:38.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.117+0000 7f5902a00700 1 -- 172.21.15.176:0/3165912628 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 23) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f58f8004b60 con 0x7f5908103f70 2024-03-13T14:48:38.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.118+0000 7f5902a00700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f58f0039280 0x7f58f003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:38.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.118+0000 7f5902a00700 1 -- 172.21.15.176:0/3165912628 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f58f8014070 con 0x7f5908103f70 2024-03-13T14:48:38.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.120+0000 7f590de00700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f58f0039280 0x7f58f003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:38.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.121+0000 7f590de00700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f58f0039280 0x7f58f003b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009790 tx=0x7f58fc009700 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:38.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.128+0000 7f5902a00700 1 -- 172.21.15.176:0/3165912628 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f58f801fb90 con 0x7f5908103f70 2024-03-13T14:48:38.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.439+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f58ec000bf0 con 0x7f58f0039280 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.457+0000 7f5902a00700 1 -- 172.21.15.176:0/3165912628 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f58ec000bf0 con 0x7f58f0039280 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (8m) 2m ago 10m 13.5M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (10m) 2m ago 10m 7520k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (8m) 22s ago 8m 7591k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (10m) 2m ago 10m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (8m) 22s ago 8m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:48:38.446 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (7m) 2m ago 9m 74.0M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (2m) 2m ago 2m 13.0M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (2m) 2m ago 2m 11.4M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (2m) 22s ago 2m 54.4M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (2m) 22s ago 2m 12.6M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:9283,8765,8443 running (11m) 2m ago 11m 489M - 18.2.1 5be31c24972a c3fcbaf472c1 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (29s) 22s ago 8m 55.9M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5187659249a7 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (11m) 2m ago 12m 45.2M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (8m) 22s ago 8m 34.8M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (10m) 2m ago 10m 7541k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (8m) 22s ago 8m 7763k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (7m) 2m ago 7m 47.0M 3947M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (7m) 2m ago 7m 47.1M 3947M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (6m) 2m ago 6m 45.8M 3947M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:48:38.447 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (5m) 22s ago 5m 59.0M 4971M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:48:38.448 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (5m) 22s ago 5m 56.5M 4971M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:48:38.448 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (4m) 22s ago 4m 55.2M 4971M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:48:38.448 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (7m) 2m ago 9m 48.7M - 2.43.0 a07b618ecd1d 4ec38b851bbe 2024-03-13T14:48:38.449 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f58f0039280 msgr2=0x7f58f003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:38.449 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f58f0039280 0x7f58f003b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009790 tx=0x7f58fc009700 comp rx=0 tx=0).stop 2024-03-13T14:48:38.449 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 msgr2=0x7f5908198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:38.449 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 0x7f5908198b00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f58f800e940 tx=0x7f58f800ec50 comp rx=0 tx=0).stop 2024-03-13T14:48:38.450 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 shutdown_connections 2024-03-13T14:48:38.450 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f58f0039280 0x7f58f003b740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.450 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5908101640 0x7f59081985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.450 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 --2- 172.21.15.176:0/3165912628 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5908103f70 0x7f5908198b00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.450 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.461+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 >> 172.21.15.176:0/3165912628 conn(0x7f59080fb050 msgr2=0x7f59080ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:38.451 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.462+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 shutdown_connections 2024-03-13T14:48:38.451 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.462+0000 7f5910800700 1 -- 172.21.15.176:0/3165912628 wait complete. 2024-03-13T14:48:38.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.644+0000 7fce87e00700 1 -- 172.21.15.176:0/1089387656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce8010a370 msgr2=0x7fce8010c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:38.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.644+0000 7fce87e00700 1 --2- 172.21.15.176:0/1089387656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce8010a370 0x7fce8010c7e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009910 tx=0x7fce6c009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:38.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.646+0000 7fce87e00700 1 -- 172.21.15.176:0/1089387656 shutdown_connections 2024-03-13T14:48:38.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.646+0000 7fce87e00700 1 --2- 172.21.15.176:0/1089387656 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce8010a370 0x7fce8010c7e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.646+0000 7fce87e00700 1 --2- 172.21.15.176:0/1089387656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce80072b80 0x7fce80109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.646+0000 7fce87e00700 1 -- 172.21.15.176:0/1089387656 >> 172.21.15.176:0/1089387656 conn(0x7fce8006d9f0 msgr2=0x7fce8006fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:38.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.646+0000 7fce87e00700 1 -- 172.21.15.176:0/1089387656 shutdown_connections 2024-03-13T14:48:38.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.646+0000 7fce87e00700 1 -- 172.21.15.176:0/1089387656 wait complete. 2024-03-13T14:48:38.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.648+0000 7fce87e00700 1 Processor -- start 2024-03-13T14:48:38.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.648+0000 7fce87e00700 1 -- start start 2024-03-13T14:48:38.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.648+0000 7fce87e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce80072b80 0x7fce801a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:38.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.648+0000 7fce87e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8010a370 0x7fce801a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:38.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.649+0000 7fce87e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce801a5cb0 con 0x7fce80072b80 2024-03-13T14:48:38.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.649+0000 7fce87e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce801a5df0 con 0x7fce8010a370 2024-03-13T14:48:38.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.649+0000 7fce85400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce80072b80 0x7fce801a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:38.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.649+0000 7fce84a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8010a370 0x7fce801a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:38.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.649+0000 7fce84a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8010a370 0x7fce801a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:60848/0 (socket says 172.21.15.176:60848) 2024-03-13T14:48:38.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.649+0000 7fce84a00700 1 -- 172.21.15.176:0/3259740096 learned_addr learned my addr 172.21.15.176:0/3259740096 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:38.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.650+0000 7fce84a00700 1 -- 172.21.15.176:0/3259740096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce80072b80 msgr2=0x7fce801a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:38.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.650+0000 7fce84a00700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce80072b80 0x7fce801a5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:38.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.650+0000 7fce84a00700 1 -- 172.21.15.176:0/3259740096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0095f0 con 0x7fce8010a370 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.650+0000 7fce85400700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce80072b80 0x7fce801a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.650+0000 7fce84a00700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8010a370 0x7fce801a5690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0092e0 tx=0x7fce6c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.651+0000 7fce7e000700 1 -- 172.21.15.176:0/3259740096 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6c01d070 con 0x7fce8010a370 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.651+0000 7fce7e000700 1 -- 172.21.15.176:0/3259740096 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fce6c005dc0 con 0x7fce8010a370 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.651+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce801aa840 con 0x7fce8010a370 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.651+0000 7fce7e000700 1 -- 172.21.15.176:0/3259740096 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6c017630 con 0x7fce8010a370 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.651+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce801aad30 con 0x7fce8010a370 2024-03-13T14:48:38.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.653+0000 7fce7e000700 1 -- 172.21.15.176:0/3259740096 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 23) v1 ==== 46247+0+0 (secure 0 0 0) 0x7fce6c017790 con 0x7fce8010a370 2024-03-13T14:48:38.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.653+0000 7fce7e000700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fce68039230 0x7fce6803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:38.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.653+0000 7fce7e000700 1 -- 172.21.15.176:0/3259740096 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fce6c04e940 con 0x7fce8010a370 2024-03-13T14:48:38.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.654+0000 7fce85400700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fce68039230 0x7fce6803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:38.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.655+0000 7fce85400700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fce68039230 0x7fce6803b6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce74011490 tx=0x7fce74009260 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:38.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.656+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce60005320 con 0x7fce8010a370 2024-03-13T14:48:38.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:38.663+0000 7fce7e000700 1 -- 172.21.15.176:0/3259740096 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fce6c020370 con 0x7fce8010a370 2024-03-13T14:48:39.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.049+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce60005cc0 con 0x7fce8010a370 2024-03-13T14:48:39.038 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:38 smithi176 ceph-mon[29064]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:39.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.050+0000 7fce7e000700 1 -- 172.21.15.176:0/3259740096 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fce6c00ee60 con 0x7fce8010a370 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 1 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:39.039 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:48:39.040 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:48:39.040 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:48:39.040 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:48:39.040 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:48:39.040 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 1 2024-03-13T14:48:39.040 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:48:39.040 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:48:39.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.053+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fce68039230 msgr2=0x7fce6803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:39.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.053+0000 7fce87e00700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fce68039230 0x7fce6803b6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce74011490 tx=0x7fce74009260 comp rx=0 tx=0).stop 2024-03-13T14:48:39.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.053+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8010a370 msgr2=0x7fce801a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:39.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.053+0000 7fce87e00700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8010a370 0x7fce801a5690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0092e0 tx=0x7fce6c00f690 comp rx=0 tx=0).stop 2024-03-13T14:48:39.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.054+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 shutdown_connections 2024-03-13T14:48:39.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.054+0000 7fce87e00700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce80072b80 0x7fce801a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.054+0000 7fce87e00700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fce68039230 0x7fce6803b6f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.054+0000 7fce87e00700 1 --2- 172.21.15.176:0/3259740096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce8010a370 0x7fce801a5690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.054+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 >> 172.21.15.176:0/3259740096 conn(0x7fce8006d9f0 msgr2=0x7fce8006fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:39.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.055+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 shutdown_connections 2024-03-13T14:48:39.044 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.056+0000 7fce87e00700 1 -- 172.21.15.176:0/3259740096 wait complete. 2024-03-13T14:48:39.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:38 smithi196 ceph-mon[37486]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:39.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.248+0000 7f14eaa00700 1 -- 172.21.15.176:0/3244236082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 msgr2=0x7f14e4073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:39.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.248+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3244236082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 0x7f14e4073c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009910 tx=0x7f14d0009c20 comp rx=0 tx=0).stop 2024-03-13T14:48:39.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.248+0000 7f14eaa00700 1 -- 172.21.15.176:0/3244236082 shutdown_connections 2024-03-13T14:48:39.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.248+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3244236082 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 0x7f14e4073c10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.248+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3244236082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 0x7f14e40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.248+0000 7f14eaa00700 1 -- 172.21.15.176:0/3244236082 >> 172.21.15.176:0/3244236082 conn(0x7f14e40fc190 msgr2=0x7f14e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:39.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.249+0000 7f14eaa00700 1 -- 172.21.15.176:0/3244236082 shutdown_connections 2024-03-13T14:48:39.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.249+0000 7f14eaa00700 1 -- 172.21.15.176:0/3244236082 wait complete. 2024-03-13T14:48:39.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.251+0000 7f14eaa00700 1 Processor -- start 2024-03-13T14:48:39.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.251+0000 7f14eaa00700 1 -- start start 2024-03-13T14:48:39.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.251+0000 7f14eaa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 0x7f14e4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:39.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.252+0000 7f14eaa00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 0x7f14e4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:39.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.252+0000 7f14eaa00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e41990d0 con 0x7f14e4073790 2024-03-13T14:48:39.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.252+0000 7f14eaa00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4199210 con 0x7f14e4074d60 2024-03-13T14:48:39.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.252+0000 7f14e3400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 0x7f14e4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:39.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.252+0000 7f14e3400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 0x7f14e4198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:60864/0 (socket says 172.21.15.176:60864) 2024-03-13T14:48:39.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.252+0000 7f14e3400700 1 -- 172.21.15.176:0/3660823315 learned_addr learned my addr 172.21.15.176:0/3660823315 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:48:39.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.252+0000 7f14e3e00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 0x7f14e4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:39.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.253+0000 7f14e3400700 1 -- 172.21.15.176:0/3660823315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 msgr2=0x7f14e4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:39.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.253+0000 7f14e3400700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 0x7f14e4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.253+0000 7f14e3400700 1 -- 172.21.15.176:0/3660823315 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d00095f0 con 0x7f14e4074d60 2024-03-13T14:48:39.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.253+0000 7f14e3400700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 0x7f14e4198ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14d0000c00 tx=0x7f14d000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:39.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.253+0000 7f14e0c00700 1 -- 172.21.15.176:0/3660823315 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d001d070 con 0x7f14e4074d60 2024-03-13T14:48:39.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.254+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e419dc60 con 0x7f14e4074d60 2024-03-13T14:48:39.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.254+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e419e150 con 0x7f14e4074d60 2024-03-13T14:48:39.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.254+0000 7f14e0c00700 1 -- 172.21.15.176:0/3660823315 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f14d0005dc0 con 0x7f14e4074d60 2024-03-13T14:48:39.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.254+0000 7f14e0c00700 1 -- 172.21.15.176:0/3660823315 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d0022470 con 0x7f14e4074d60 2024-03-13T14:48:39.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.255+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c4005320 con 0x7f14e4074d60 2024-03-13T14:48:39.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.256+0000 7f14e0c00700 1 -- 172.21.15.176:0/3660823315 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 23) v1 ==== 46247+0+0 (secure 0 0 0) 0x7f14d00225d0 con 0x7f14e4074d60 2024-03-13T14:48:39.245 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.256+0000 7f14e0c00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f14c8039230 0x7f14c803b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:48:39.245 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.257+0000 7f14e0c00700 1 -- 172.21.15.176:0/3660823315 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f14d004e7a0 con 0x7f14e4074d60 2024-03-13T14:48:39.245 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.257+0000 7f14e3e00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f14c8039230 0x7f14c803b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:48:39.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.258+0000 7f14e3e00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f14c8039230 0x7f14c803b6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009c60 tx=0x7f14d4009260 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:48:39.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.263+0000 7f14e0c00700 1 -- 172.21.15.176:0/3660823315 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f14d0020de0 con 0x7f14e4074d60 2024-03-13T14:48:39.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.583+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14c4000bf0 con 0x7f14c8039230 2024-03-13T14:48:39.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.584+0000 7f14e0c00700 1 -- 172.21.15.176:0/3660823315 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f14c4000bf0 con 0x7f14c8039230 2024-03-13T14:48:39.575 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [], 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "1/2 daemons upgraded", 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "", 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:48:39.576 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.587+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f14c8039230 msgr2=0x7f14c803b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.587+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f14c8039230 0x7f14c803b6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009c60 tx=0x7f14d4009260 comp rx=0 tx=0).stop 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.587+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 msgr2=0x7f14e4198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.587+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 0x7f14e4198ab0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14d0000c00 tx=0x7f14d000f690 comp rx=0 tx=0).stop 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.588+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 shutdown_connections 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.588+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f14e4073790 0x7f14e4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.588+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f14c8039230 0x7f14c803b6f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.588+0000 7f14eaa00700 1 --2- 172.21.15.176:0/3660823315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f14e4074d60 0x7f14e4198ab0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f14d0000c00 tx=0x7f14d000f690 comp rx=0 tx=0).stop 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.588+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 >> 172.21.15.176:0/3660823315 conn(0x7f14e40fc190 msgr2=0x7f14e4106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.588+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 shutdown_connections 2024-03-13T14:48:39.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:48:39.588+0000 7f14eaa00700 1 -- 172.21.15.176:0/3660823315 wait complete. 2024-03-13T14:48:39.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:39 smithi176 ceph-mon[29064]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:39.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:39 smithi176 ceph-mon[29064]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:39.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:39 smithi176 ceph-mon[29064]: pgmap v8: 65 pgs: 65 active+clean; 93 MiB data, 529 MiB used, 536 GiB / 536 GiB avail; 461 KiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-03-13T14:48:39.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:39 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3259740096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:48:40.063 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:39 smithi196 ceph-mon[37486]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:40.063 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:39 smithi196 ceph-mon[37486]: from='client.14638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:40.063 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:39 smithi196 ceph-mon[37486]: pgmap v8: 65 pgs: 65 active+clean; 93 MiB data, 529 MiB used, 536 GiB / 536 GiB avail; 461 KiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-03-13T14:48:40.063 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:39 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3259740096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:48:40.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:40 smithi196 ceph-mon[37486]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:40.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:40 smithi196 ceph-mon[37486]: Standby manager daemon smithi176.tzfdwd started 2024-03-13T14:48:40.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:40 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/crt"}]: dispatch 2024-03-13T14:48:40.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:40 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:48:40.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:40 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/key"}]: dispatch 2024-03-13T14:48:40.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:40 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:48:41.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:40 smithi176 ceph-mon[29064]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:48:41.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:40 smithi176 ceph-mon[29064]: Standby manager daemon smithi176.tzfdwd started 2024-03-13T14:48:41.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:40 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/crt"}]: dispatch 2024-03-13T14:48:41.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:40 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:48:41.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:40 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/key"}]: dispatch 2024-03-13T14:48:41.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:40 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/4120503192' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:48:41.946 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:41 smithi176 ceph-mon[29064]: pgmap v9: 65 pgs: 65 active+clean; 104 MiB data, 543 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-03-13T14:48:41.947 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:41 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:48:41.947 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:41 smithi176 ceph-mon[29064]: mgrmap e24: smithi196.vdktbk(active, since 12s), standbys: smithi176.tzfdwd 2024-03-13T14:48:42.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:41 smithi196 ceph-mon[37486]: pgmap v9: 65 pgs: 65 active+clean; 104 MiB data, 543 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-03-13T14:48:42.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:41 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:48:42.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:41 smithi196 ceph-mon[37486]: mgrmap e24: smithi196.vdktbk(active, since 12s), standbys: smithi176.tzfdwd 2024-03-13T14:48:43.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:43 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:43.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:43 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:43.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:43 smithi176 ceph-mon[29064]: pgmap v10: 65 pgs: 65 active+clean; 104 MiB data, 543 MiB used, 536 GiB / 536 GiB avail; 1022 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-03-13T14:48:43.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:43 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:43.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:43 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:43.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:43 smithi196 ceph-mon[37486]: pgmap v10: 65 pgs: 65 active+clean; 104 MiB data, 543 MiB used, 536 GiB / 536 GiB avail; 1022 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-03-13T14:48:44.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:44 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:44 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:44 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:44.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:44 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:48:44.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:44 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:44.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:44 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:45.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:45 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:45.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:45 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:45.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:45 smithi176 ceph-mon[29064]: pgmap v11: 65 pgs: 65 active+clean; 104 MiB data, 543 MiB used, 536 GiB / 536 GiB avail; 1022 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-03-13T14:48:45.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:45 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:45.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:45 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:45.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:45 smithi196 ceph-mon[37486]: pgmap v11: 65 pgs: 65 active+clean; 104 MiB data, 543 MiB used, 536 GiB / 536 GiB avail; 1022 KiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-03-13T14:48:47.110 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:47.110 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:47.110 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.110 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.110 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.110 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi176 to 1217M 2024-03-13T14:48:47.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:47.111 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:47 smithi196 ceph-mon[37486]: pgmap v12: 65 pgs: 65 active+clean; 113 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 432 op/s 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi176 to 1217M 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:47.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:47 smithi176 ceph-mon[29064]: pgmap v12: 65 pgs: 65 active+clean; 113 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 432 op/s 2024-03-13T14:48:50.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:49 smithi176 ceph-mon[29064]: pgmap v13: 65 pgs: 65 active+clean; 113 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2024-03-13T14:48:50.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:49 smithi196 ceph-mon[37486]: pgmap v13: 65 pgs: 65 active+clean; 113 MiB data, 611 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 275 op/s 2024-03-13T14:48:51.311 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:51 smithi196 ceph-mon[37486]: pgmap v14: 65 pgs: 65 active+clean; 122 MiB data, 662 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-03-13T14:48:51.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:51 smithi176 ceph-mon[29064]: pgmap v14: 65 pgs: 65 active+clean; 122 MiB data, 662 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 395 op/s 2024-03-13T14:48:52.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:52.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:52.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: Adjusting osd_memory_target on smithi196 to 2241M 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:48:52.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:52 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: Adjusting osd_memory_target on smithi196 to 2241M 2024-03-13T14:48:52.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:52.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:48:52.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:48:52.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:48:52.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:52 smithi176 ceph-mon[29064]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:48:53.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:53 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:48:53.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:53 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:48:53.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:53 smithi196 ceph-mon[37486]: pgmap v15: 65 pgs: 65 active+clean; 122 MiB data, 662 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-13T14:48:53.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:53 smithi176 ceph-mon[29064]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:48:53.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:53 smithi176 ceph-mon[29064]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:48:53.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:53 smithi176 ceph-mon[29064]: pgmap v15: 65 pgs: 65 active+clean; 122 MiB data, 662 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:54 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:54.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:54 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:48:55.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:55 smithi196 ceph-mon[37486]: Reconfiguring prometheus.smithi176 (dependencies changed)... 2024-03-13T14:48:55.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:55 smithi196 ceph-mon[37486]: pgmap v16: 65 pgs: 65 active+clean; 122 MiB data, 662 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-13T14:48:55.680 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:55 smithi176 ceph-mon[29064]: Reconfiguring prometheus.smithi176 (dependencies changed)... 2024-03-13T14:48:55.680 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:55 smithi176 ceph-mon[29064]: pgmap v16: 65 pgs: 65 active+clean; 122 MiB data, 662 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 267 op/s 2024-03-13T14:48:56.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:56 smithi196 ceph-mon[37486]: Reconfiguring daemon prometheus.smithi176 on smithi176 2024-03-13T14:48:56.694 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:56 smithi176 ceph-mon[29064]: Reconfiguring daemon prometheus.smithi176 on smithi176 2024-03-13T14:48:57.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:57 smithi176 ceph-mon[29064]: pgmap v17: 65 pgs: 65 active+clean; 133 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2024-03-13T14:48:57.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:57 smithi196 ceph-mon[37486]: pgmap v17: 65 pgs: 65 active+clean; 133 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 382 op/s 2024-03-13T14:48:59.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:59 smithi176 ceph-mon[29064]: pgmap v18: 65 pgs: 65 active+clean; 133 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 923 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-03-13T14:48:59.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:48:59 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:00.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:59 smithi196 ceph-mon[37486]: pgmap v18: 65 pgs: 65 active+clean; 133 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 923 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-03-13T14:49:00.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:48:59 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:01.351 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:01 smithi176 ceph-mon[29064]: pgmap v19: 65 pgs: 65 active+clean; 151 MiB data, 756 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-03-13T14:49:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:01 smithi196 ceph-mon[37486]: pgmap v19: 65 pgs: 65 active+clean; 151 MiB data, 756 MiB used, 536 GiB / 536 GiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-03-13T14:49:02.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:02 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:02.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:02 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:02.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:02 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:49:02.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:02 smithi176 ceph-mon[29064]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:49:02.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:02 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:02.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:02 smithi176 ceph-mon[29064]: Upgrade: Need to upgrade myself (mgr.smithi196.vdktbk) 2024-03-13T14:49:02.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:02 smithi176 ceph-mon[29064]: Upgrade: Need to upgrade myself (mgr.smithi196.vdktbk) 2024-03-13T14:49:02.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:02 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:02.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:02 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:02.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:02 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:49:02.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:02 smithi196 ceph-mon[37486]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:49:02.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:02 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:02.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:02 smithi196 ceph-mon[37486]: Upgrade: Need to upgrade myself (mgr.smithi196.vdktbk) 2024-03-13T14:49:02.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:02 smithi196 ceph-mon[37486]: Upgrade: Need to upgrade myself (mgr.smithi196.vdktbk) 2024-03-13T14:49:03.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:03 smithi176 ceph-mon[29064]: pgmap v20: 65 pgs: 65 active+clean; 151 MiB data, 756 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-03-13T14:49:03.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:03 smithi196 ceph-mon[37486]: pgmap v20: 65 pgs: 65 active+clean; 151 MiB data, 756 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-03-13T14:49:05.535 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: Upgrade: Updating mgr.smithi176.tzfdwd 2024-03-13T14:49:05.536 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:05.536 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:49:05.536 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:49:05.536 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:49:05.536 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:49:05.536 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: Deploying daemon mgr.smithi176.tzfdwd on smithi176 2024-03-13T14:49:05.536 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:05 smithi176 ceph-mon[29064]: pgmap v21: 65 pgs: 65 active+clean; 152 MiB data, 766 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 259 op/s 2024-03-13T14:49:05.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: Upgrade: Updating mgr.smithi176.tzfdwd 2024-03-13T14:49:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:49:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:49:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:49:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:49:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: Deploying daemon mgr.smithi176.tzfdwd on smithi176 2024-03-13T14:49:05.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:05 smithi196 ceph-mon[37486]: pgmap v21: 65 pgs: 65 active+clean; 152 MiB data, 766 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.6 MiB/s wr, 259 op/s 2024-03-13T14:49:08.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:07 smithi176 ceph-mon[29064]: pgmap v22: 65 pgs: 65 active+clean; 166 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 360 op/s 2024-03-13T14:49:08.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:07 smithi196 ceph-mon[37486]: pgmap v22: 65 pgs: 65 active+clean; 166 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 360 op/s 2024-03-13T14:49:09.737 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:09 smithi176 ceph-mon[29064]: pgmap v23: 65 pgs: 65 active+clean; 166 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 246 op/s 2024-03-13T14:49:09.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.797+0000 7f1edac00700 1 -- 172.21.15.176:0/2852992201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4101db0 msgr2=0x7f1ed4102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:09.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.797+0000 7f1edac00700 1 --2- 172.21.15.176:0/2852992201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4101db0 0x7f1ed4102210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009910 tx=0x7f1ec0009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:09.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.798+0000 7f1edac00700 1 -- 172.21.15.176:0/2852992201 shutdown_connections 2024-03-13T14:49:09.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.798+0000 7f1edac00700 1 --2- 172.21.15.176:0/2852992201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4101db0 0x7f1ed4102210 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:09.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.798+0000 7f1edac00700 1 --2- 172.21.15.176:0/2852992201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed4100bb0 0x7f1ed4100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:09.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.798+0000 7f1edac00700 1 -- 172.21.15.176:0/2852992201 >> 172.21.15.176:0/2852992201 conn(0x7f1ed40fc110 msgr2=0x7f1ed40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:09.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.799+0000 7f1edac00700 1 -- 172.21.15.176:0/2852992201 shutdown_connections 2024-03-13T14:49:09.788 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.799+0000 7f1edac00700 1 -- 172.21.15.176:0/2852992201 wait complete. 2024-03-13T14:49:09.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.800+0000 7f1edac00700 1 Processor -- start 2024-03-13T14:49:09.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.800+0000 7f1edac00700 1 -- start start 2024-03-13T14:49:09.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1edac00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4100bb0 0x7f1ed4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:09.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1edac00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed4101db0 0x7f1ed4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:09.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1edac00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed41990d0 con 0x7f1ed4100bb0 2024-03-13T14:49:09.789 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1edac00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed4199210 con 0x7f1ed4101db0 2024-03-13T14:49:09.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1ed3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4100bb0 0x7f1ed4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:09.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1ed3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4100bb0 0x7f1ed4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:41782/0 (socket says 172.21.15.176:41782) 2024-03-13T14:49:09.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1ed3e00700 1 -- 172.21.15.176:0/2029078818 learned_addr learned my addr 172.21.15.176:0/2029078818 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:09.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.801+0000 7f1ed3400700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed4101db0 0x7f1ed4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:09.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.802+0000 7f1ed3e00700 1 -- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed4101db0 msgr2=0x7f1ed4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:09.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.802+0000 7f1ed3e00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed4101db0 0x7f1ed4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:09.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.802+0000 7f1ed3e00700 1 -- 172.21.15.176:0/2029078818 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00095f0 con 0x7f1ed4100bb0 2024-03-13T14:49:09.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.803+0000 7f1ed3e00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4100bb0 0x7f1ed4198570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400e8f0 tx=0x7f1ec400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:09.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.803+0000 7f1ed0c00700 1 -- 172.21.15.176:0/2029078818 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec4009790 con 0x7f1ed4100bb0 2024-03-13T14:49:09.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.803+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed419dcc0 con 0x7f1ed4100bb0 2024-03-13T14:49:09.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.803+0000 7f1ed0c00700 1 -- 172.21.15.176:0/2029078818 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1ec400c810 con 0x7f1ed4100bb0 2024-03-13T14:49:09.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.803+0000 7f1ed0c00700 1 -- 172.21.15.176:0/2029078818 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec4003d20 con 0x7f1ed4100bb0 2024-03-13T14:49:09.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.803+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed419e1c0 con 0x7f1ed4100bb0 2024-03-13T14:49:09.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.805+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed410b920 con 0x7f1ed4100bb0 2024-03-13T14:49:09.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.806+0000 7f1ed0c00700 1 -- 172.21.15.176:0/2029078818 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 91107+0+0 (secure 0 0 0) 0x7f1ec40041f0 con 0x7f1ed4100bb0 2024-03-13T14:49:09.795 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.807+0000 7f1ed0c00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f1eb806d1b0 0x7f1eb806f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:09.795 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.807+0000 7f1ed0c00700 1 -- 172.21.15.176:0/2029078818 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1ec4014070 con 0x7f1ed4100bb0 2024-03-13T14:49:09.798 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.810+0000 7f1ed3400700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f1eb806d1b0 0x7f1eb806f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:09.798 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.811+0000 7f1ed3400700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f1eb806d1b0 0x7f1eb806f670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009fd0 tx=0x7f1ec0005fd0 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:09.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:09.816+0000 7f1ed0c00700 1 -- 172.21.15.176:0/2029078818 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f1ec405a0f0 con 0x7f1ed4100bb0 2024-03-13T14:49:10.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:09 smithi196 ceph-mon[37486]: pgmap v23: 65 pgs: 65 active+clean; 166 MiB data, 821 MiB used, 536 GiB / 536 GiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 246 op/s 2024-03-13T14:49:10.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.198+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ed4106700 con 0x7f1eb806d1b0 2024-03-13T14:49:10.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.202+0000 7f1ed0c00700 1 -- 172.21.15.176:0/2029078818 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f1ed4106700 con 0x7f1eb806d1b0 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f1eb806d1b0 msgr2=0x7f1eb806f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f1eb806d1b0 0x7f1eb806f670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009fd0 tx=0x7f1ec0005fd0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4100bb0 msgr2=0x7f1ed4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4100bb0 0x7f1ed4198570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400e8f0 tx=0x7f1ec400ec00 comp rx=0 tx=0).stop 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 shutdown_connections 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1ed4100bb0 0x7f1ed4198570 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400e8f0 tx=0x7f1ec400ec00 comp rx=0 tx=0).stop 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7f1eb806d1b0 0x7f1eb806f670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 --2- 172.21.15.176:0/2029078818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ed4101db0 0x7f1ed4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.205+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 >> 172.21.15.176:0/2029078818 conn(0x7f1ed40fc110 msgr2=0x7f1ed4104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:10.195 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.206+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 shutdown_connections 2024-03-13T14:49:10.195 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.206+0000 7f1edac00700 1 -- 172.21.15.176:0/2029078818 wait complete. 2024-03-13T14:49:10.215 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:49:10.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.386+0000 7fe6c2e00700 1 -- 172.21.15.176:0/3729325736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc074dc0 msgr2=0x7fe6bc073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.386+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/3729325736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc073220 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009910 tx=0x7fe6ac009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:10.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.386+0000 7fe6c2e00700 1 -- 172.21.15.176:0/3729325736 shutdown_connections 2024-03-13T14:49:10.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.386+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/3729325736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc0737f0 0x7fe6bc073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.386+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/3729325736 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc073220 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.386+0000 7fe6c2e00700 1 -- 172.21.15.176:0/3729325736 >> 172.21.15.176:0/3729325736 conn(0x7fe6bc0fc110 msgr2=0x7fe6bc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:10.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.387+0000 7fe6c2e00700 1 -- 172.21.15.176:0/3729325736 shutdown_connections 2024-03-13T14:49:10.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.387+0000 7fe6c2e00700 1 -- 172.21.15.176:0/3729325736 wait complete. 2024-03-13T14:49:10.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.388+0000 7fe6c2e00700 1 Processor -- start 2024-03-13T14:49:10.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6c2e00700 1 -- start start 2024-03-13T14:49:10.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6c2e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc0737f0 0x7fe6bc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6c2e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6c2e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1990f0 con 0x7fe6bc0737f0 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6c2e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc199230 con 0x7fe6bc074dc0 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6bb400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6bb400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:55096/0 (socket says 172.21.15.176:55096) 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6bb400700 1 -- 172.21.15.176:0/2643175004 learned_addr learned my addr 172.21.15.176:0/2643175004 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.389+0000 7fe6bbe00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc0737f0 0x7fe6bc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.390+0000 7fe6bbe00700 1 -- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc074dc0 msgr2=0x7fe6bc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.390+0000 7fe6bbe00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.390+0000 7fe6bbe00700 1 -- 172.21.15.176:0/2643175004 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac0095f0 con 0x7fe6bc0737f0 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.390+0000 7fe6bb400700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T14:49:10.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.390+0000 7fe6bbe00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc0737f0 0x7fe6bc198590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:10.380 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.391+0000 7fe6b8c00700 1 -- 172.21.15.176:0/2643175004 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6ac01d070 con 0x7fe6bc0737f0 2024-03-13T14:49:10.380 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.391+0000 7fe6b8c00700 1 -- 172.21.15.176:0/2643175004 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe6ac00fe60 con 0x7fe6bc0737f0 2024-03-13T14:49:10.380 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.391+0000 7fe6b8c00700 1 -- 172.21.15.176:0/2643175004 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6ac017400 con 0x7fe6bc0737f0 2024-03-13T14:49:10.380 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.392+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc19dc80 con 0x7fe6bc0737f0 2024-03-13T14:49:10.380 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.392+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc19e140 con 0x7fe6bc0737f0 2024-03-13T14:49:10.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.394+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc10b920 con 0x7fe6bc0737f0 2024-03-13T14:49:10.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.399+0000 7fe6b8c00700 1 -- 172.21.15.176:0/2643175004 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 91107+0+0 (secure 0 0 0) 0x7fe6ac0051d0 con 0x7fe6bc0737f0 2024-03-13T14:49:10.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.400+0000 7fe6b8c00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fe6a006d240 0x7fe6a006f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:10.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.400+0000 7fe6b8c00700 1 -- 172.21.15.176:0/2643175004 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe6ac08e580 con 0x7fe6bc0737f0 2024-03-13T14:49:10.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.401+0000 7fe6bb400700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fe6a006d240 0x7fe6a006f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:10.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.402+0000 7fe6bb400700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fe6a006d240 0x7fe6a006f700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8005950 tx=0x7fe6a8009260 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:10.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.406+0000 7fe6b8c00700 1 -- 172.21.15.176:0/2643175004 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fe6ac05ba40 con 0x7fe6bc0737f0 2024-03-13T14:49:10.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.739+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6bc1080b0 con 0x7fe6a006d240 2024-03-13T14:49:10.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.740+0000 7fe6b8c00700 1 -- 172.21.15.176:0/2643175004 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fe6bc1080b0 con 0x7fe6a006d240 2024-03-13T14:49:10.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.744+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fe6a006d240 msgr2=0x7fe6a006f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.744+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fe6a006d240 0x7fe6a006f700 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8005950 tx=0x7fe6a8009260 comp rx=0 tx=0).stop 2024-03-13T14:49:10.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.744+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc0737f0 msgr2=0x7fe6bc198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.744+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc0737f0 0x7fe6bc198590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac000c00 tx=0x7fe6ac00fab0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.745+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 shutdown_connections 2024-03-13T14:49:10.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.745+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6bc0737f0 0x7fe6bc198590 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.745+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fe6a006d240 0x7fe6a006f700 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.745+0000 7fe6c2e00700 1 --2- 172.21.15.176:0/2643175004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc074dc0 0x7fe6bc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.745+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 >> 172.21.15.176:0/2643175004 conn(0x7fe6bc0fc110 msgr2=0x7fe6bc106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:10.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.746+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 shutdown_connections 2024-03-13T14:49:10.735 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.746+0000 7fe6c2e00700 1 -- 172.21.15.176:0/2643175004 wait complete. 2024-03-13T14:49:10.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 -- 172.21.15.176:0/681398355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594100bb0 msgr2=0x7fd594100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 --2- 172.21.15.176:0/681398355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594100bb0 0x7fd594100fd0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd584009910 tx=0x7fd584009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 -- 172.21.15.176:0/681398355 shutdown_connections 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 --2- 172.21.15.176:0/681398355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd594101db0 0x7fd594102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 --2- 172.21.15.176:0/681398355 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594100bb0 0x7fd594100fd0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 -- 172.21.15.176:0/681398355 >> 172.21.15.176:0/681398355 conn(0x7fd5940fc110 msgr2=0x7fd5940fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 -- 172.21.15.176:0/681398355 shutdown_connections 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.936+0000 7fd59a600700 1 -- 172.21.15.176:0/681398355 wait complete. 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.937+0000 7fd59a600700 1 Processor -- start 2024-03-13T14:49:10.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.937+0000 7fd59a600700 1 -- start start 2024-03-13T14:49:10.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd59a600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd594100bb0 0x7fd594196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:10.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd59a600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594101db0 0x7fd5941968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:10.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd59a600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd594196ee0 con 0x7fd594101db0 2024-03-13T14:49:10.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd59a600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd594197020 con 0x7fd594100bb0 2024-03-13T14:49:10.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd592a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594101db0 0x7fd5941968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:10.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd592a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594101db0 0x7fd5941968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:41806/0 (socket says 172.21.15.176:41806) 2024-03-13T14:49:10.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd592a00700 1 -- 172.21.15.176:0/84414786 learned_addr learned my addr 172.21.15.176:0/84414786 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:10.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.938+0000 7fd593400700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd594100bb0 0x7fd594196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:10.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.939+0000 7fd592a00700 1 -- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd594100bb0 msgr2=0x7fd594196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:10.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.939+0000 7fd592a00700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd594100bb0 0x7fd594196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:10.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.939+0000 7fd592a00700 1 -- 172.21.15.176:0/84414786 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840095f0 con 0x7fd594101db0 2024-03-13T14:49:10.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.939+0000 7fd593400700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd594100bb0 0x7fd594196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T14:49:10.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.939+0000 7fd592a00700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594101db0 0x7fd5941968c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e920 tx=0x7fd58000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:10.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.939+0000 7fd58be00700 1 -- 172.21.15.176:0/84414786 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd58000c9d0 con 0x7fd594101db0 2024-03-13T14:49:10.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.940+0000 7fd58be00700 1 -- 172.21.15.176:0/84414786 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd580010430 con 0x7fd594101db0 2024-03-13T14:49:10.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.940+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59419bad0 con 0x7fd594101db0 2024-03-13T14:49:10.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.940+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59419c020 con 0x7fd594101db0 2024-03-13T14:49:10.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.940+0000 7fd58be00700 1 -- 172.21.15.176:0/84414786 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd580018650 con 0x7fd594101db0 2024-03-13T14:49:10.930 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.942+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd594109710 con 0x7fd594101db0 2024-03-13T14:49:10.930 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.943+0000 7fd58be00700 1 -- 172.21.15.176:0/84414786 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 91107+0+0 (secure 0 0 0) 0x7fd5800188c0 con 0x7fd594101db0 2024-03-13T14:49:10.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.944+0000 7fd58be00700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd57806d1b0 0x7fd57806f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:10.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.944+0000 7fd58be00700 1 -- 172.21.15.176:0/84414786 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd580014070 con 0x7fd594101db0 2024-03-13T14:49:10.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.947+0000 7fd593400700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd57806d1b0 0x7fd57806f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:10.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.948+0000 7fd593400700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd57806d1b0 0x7fd57806f670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd584009fd0 tx=0x7fd584005a90 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:10.937 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:10.948+0000 7fd58be00700 1 -- 172.21.15.176:0/84414786 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fd580052f30 con 0x7fd594101db0 2024-03-13T14:49:11.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.259+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd594106700 con 0x7fd57806d1b0 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.276+0000 7fd58be00700 1 -- 172.21.15.176:0/84414786 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd594106700 con 0x7fd57806d1b0 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (8m) 27s ago 10m 13.9M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (11m) 27s ago 11m 7717k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (9m) 29s ago 9m 7633k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (10m) 27s ago 10m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (9m) 29s ago 9m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (8m) 27s ago 10m 74.9M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:49:11.265 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (3m) 27s ago 3m 59.4M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (3m) 27s ago 3m 13.3M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (3m) 29s ago 3m 98.4M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (3m) 29s ago 3m 13.2M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:9283,8765,8443 running (12m) 27s ago 12m 355M - 18.2.1 5be31c24972a c3fcbaf472c1 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (62s) 29s ago 9m 486M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5187659249a7 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (12m) 27s ago 12m 47.8M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (9m) 29s ago 9m 39.2M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (10m) 27s ago 10m 7898k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (9m) 29s ago 9m 7658k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (8m) 27s ago 8m 85.0M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (7m) 27s ago 7m 89.8M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (6m) 27s ago 6m 79.5M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (6m) 29s ago 6m 94.7M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:49:11.266 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (5m) 29s ago 5m 86.8M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:49:11.267 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (5m) 29s ago 5m 84.9M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:49:11.267 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 starting - - - - 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd57806d1b0 msgr2=0x7fd57806f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd57806d1b0 0x7fd57806f670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd584009fd0 tx=0x7fd584005a90 comp rx=0 tx=0).stop 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594101db0 msgr2=0x7fd5941968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594101db0 0x7fd5941968c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e920 tx=0x7fd58000ec30 comp rx=0 tx=0).stop 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 shutdown_connections 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd57806d1b0 0x7fd57806f670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd594100bb0 0x7fd594196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 --2- 172.21.15.176:0/84414786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd594101db0 0x7fd5941968c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.280+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 >> 172.21.15.176:0/84414786 conn(0x7fd5940fc110 msgr2=0x7fd594104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:11.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.281+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 shutdown_connections 2024-03-13T14:49:11.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.281+0000 7fd59a600700 1 -- 172.21.15.176:0/84414786 wait complete. 2024-03-13T14:49:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:11 smithi176 ceph-mon[29064]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:11 smithi176 ceph-mon[29064]: pgmap v24: 65 pgs: 65 active+clean; 176 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 355 op/s 2024-03-13T14:49:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:11 smithi176 ceph-mon[29064]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:11.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:11 smithi196 ceph-mon[37486]: from='client.14650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:11.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:11 smithi196 ceph-mon[37486]: pgmap v24: 65 pgs: 65 active+clean; 176 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 355 op/s 2024-03-13T14:49:11.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:11 smithi196 ceph-mon[37486]: from='client.14654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:11.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.448+0000 7fd55e000700 1 -- 172.21.15.176:0/1567389647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 msgr2=0x7fd5581063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:11.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.448+0000 7fd55e000700 1 --2- 172.21.15.176:0/1567389647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 0x7fd5581063e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd544009910 tx=0x7fd544009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:11.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.448+0000 7fd55e000700 1 -- 172.21.15.176:0/1567389647 shutdown_connections 2024-03-13T14:49:11.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.448+0000 7fd55e000700 1 --2- 172.21.15.176:0/1567389647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 0x7fd5581063e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.448+0000 7fd55e000700 1 --2- 172.21.15.176:0/1567389647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 0x7fd558103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.448+0000 7fd55e000700 1 -- 172.21.15.176:0/1567389647 >> 172.21.15.176:0/1567389647 conn(0x7fd5580fb030 msgr2=0x7fd5580fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:11.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.449+0000 7fd55e000700 1 -- 172.21.15.176:0/1567389647 shutdown_connections 2024-03-13T14:49:11.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.449+0000 7fd55e000700 1 -- 172.21.15.176:0/1567389647 wait complete. 2024-03-13T14:49:11.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.450+0000 7fd55e000700 1 Processor -- start 2024-03-13T14:49:11.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.450+0000 7fd55e000700 1 -- start start 2024-03-13T14:49:11.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.451+0000 7fd55e000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 0x7fd558198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:11.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.451+0000 7fd55e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 0x7fd558198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:11.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.451+0000 7fd55e000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd558199090 con 0x7fd558103ff0 2024-03-13T14:49:11.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.451+0000 7fd55e000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5581991d0 con 0x7fd5581016c0 2024-03-13T14:49:11.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.451+0000 7fd557400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 0x7fd558198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:11.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.451+0000 7fd557400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 0x7fd558198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:55170/0 (socket says 172.21.15.176:55170) 2024-03-13T14:49:11.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.451+0000 7fd557400700 1 -- 172.21.15.176:0/3587459858 learned_addr learned my addr 172.21.15.176:0/3587459858 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:11.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.452+0000 7fd556a00700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 0x7fd558198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:11.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.452+0000 7fd557400700 1 -- 172.21.15.176:0/3587459858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 msgr2=0x7fd558198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:11.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.452+0000 7fd557400700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 0x7fd558198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.452+0000 7fd557400700 1 -- 172.21.15.176:0/3587459858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440095f0 con 0x7fd5581016c0 2024-03-13T14:49:11.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.452+0000 7fd557400700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 0x7fd558198530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e940 tx=0x7fd54800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:11.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.453+0000 7fd54fe00700 1 -- 172.21.15.176:0/3587459858 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54800cba0 con 0x7fd5581016c0 2024-03-13T14:49:11.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.453+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55819dc80 con 0x7fd5581016c0 2024-03-13T14:49:11.442 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.453+0000 7fd54fe00700 1 -- 172.21.15.176:0/3587459858 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd54800ee90 con 0x7fd5581016c0 2024-03-13T14:49:11.442 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.453+0000 7fd54fe00700 1 -- 172.21.15.176:0/3587459858 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd548005650 con 0x7fd5581016c0 2024-03-13T14:49:11.442 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.453+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55819e180 con 0x7fd5581016c0 2024-03-13T14:49:11.442 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.455+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5580fcc20 con 0x7fd5581016c0 2024-03-13T14:49:11.445 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.457+0000 7fd54fe00700 1 -- 172.21.15.176:0/3587459858 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 24) v1 ==== 91107+0+0 (secure 0 0 0) 0x7fd548004750 con 0x7fd5581016c0 2024-03-13T14:49:11.446 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.458+0000 7fd54fe00700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd53c06d1c0 0x7fd53c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:11.446 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.458+0000 7fd54fe00700 1 -- 172.21.15.176:0/3587459858 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd548014070 con 0x7fd5581016c0 2024-03-13T14:49:11.447 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.459+0000 7fd556a00700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd53c06d1c0 0x7fd53c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:11.448 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.460+0000 7fd556a00700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd53c06d1c0 0x7fd53c06f680 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd544009fd0 tx=0x7fd544005940 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:11.462 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.474+0000 7fd54fe00700 1 -- 172.21.15.176:0/3587459858 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fd54805a0f0 con 0x7fd5581016c0 2024-03-13T14:49:11.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.893+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd55802ced0 con 0x7fd5581016c0 2024-03-13T14:49:11.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.894+0000 7fd54fe00700 1 -- 172.21.15.176:0/3587459858 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7fd548059c80 con 0x7fd5581016c0 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 1 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:11.883 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:49:11.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:49:11.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:11.884 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:49:11.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 13, 2024-03-13T14:49:11.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 1 2024-03-13T14:49:11.884 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:49:11.884 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.898+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd53c06d1c0 msgr2=0x7fd53c06f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.898+0000 7fd55e000700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd53c06d1c0 0x7fd53c06f680 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd544009fd0 tx=0x7fd544005940 comp rx=0 tx=0).stop 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.898+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 msgr2=0x7fd558198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.898+0000 7fd55e000700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 0x7fd558198530 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd54800e940 tx=0x7fd54800ec50 comp rx=0 tx=0).stop 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.899+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 shutdown_connections 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.899+0000 7fd55e000700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fd53c06d1c0 0x7fd53c06f680 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.899+0000 7fd55e000700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd5581016c0 0x7fd558198530 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.899+0000 7fd55e000700 1 --2- 172.21.15.176:0/3587459858 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd558103ff0 0x7fd558198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:11.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.899+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 >> 172.21.15.176:0/3587459858 conn(0x7fd5580fb030 msgr2=0x7fd5580ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:11.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.899+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 shutdown_connections 2024-03-13T14:49:11.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:11.900+0000 7fd55e000700 1 -- 172.21.15.176:0/3587459858 wait complete. 2024-03-13T14:49:12.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.106+0000 7fee0b200700 1 -- 172.21.15.176:0/3421412022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04096d90 msgr2=0x7fee040971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:12.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.106+0000 7fee0b200700 1 --2- 172.21.15.176:0/3421412022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04096d90 0x7fee040971f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009910 tx=0x7fedf8009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.106+0000 7fee0b200700 1 -- 172.21.15.176:0/3421412022 shutdown_connections 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.107+0000 7fee0b200700 1 --2- 172.21.15.176:0/3421412022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04096d90 0x7fee040971f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.107+0000 7fee0b200700 1 --2- 172.21.15.176:0/3421412022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee04095b90 0x7fee04095fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.107+0000 7fee0b200700 1 -- 172.21.15.176:0/3421412022 >> 172.21.15.176:0/3421412022 conn(0x7fee04091110 msgr2=0x7fee04093570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.107+0000 7fee0b200700 1 -- 172.21.15.176:0/3421412022 shutdown_connections 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.107+0000 7fee0b200700 1 -- 172.21.15.176:0/3421412022 wait complete. 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.108+0000 7fee0b200700 1 Processor -- start 2024-03-13T14:49:12.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.108+0000 7fee0b200700 1 -- start start 2024-03-13T14:49:12.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.108+0000 7fee0b200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04095b90 0x7fee041386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:12.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.108+0000 7fee0b200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee04096d90 0x7fee04138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:12.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.108+0000 7fee0b200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee041351d0 con 0x7fee04095b90 2024-03-13T14:49:12.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.108+0000 7fee0b200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee04135340 con 0x7fee04096d90 2024-03-13T14:49:12.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.109+0000 7fee09e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04095b90 0x7fee041386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:12.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.109+0000 7fee09e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04095b90 0x7fee041386e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37020/0 (socket says 172.21.15.176:37020) 2024-03-13T14:49:12.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.109+0000 7fee09e00700 1 -- 172.21.15.176:0/4200548585 learned_addr learned my addr 172.21.15.176:0/4200548585 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:12.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.110+0000 7fee09400700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee04096d90 0x7fee04138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:12.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.110+0000 7fee09e00700 1 -- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee04096d90 msgr2=0x7fee04138c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:12.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.110+0000 7fee09e00700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee04096d90 0x7fee04138c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:12.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.110+0000 7fee09e00700 1 -- 172.21.15.176:0/4200548585 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf80095f0 con 0x7fee04095b90 2024-03-13T14:49:12.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.110+0000 7fee09e00700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04095b90 0x7fee041386e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00e8f0 tx=0x7fedfc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:12.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.111+0000 7fee02a00700 1 -- 172.21.15.176:0/4200548585 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedfc009790 con 0x7fee04095b90 2024-03-13T14:49:12.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.111+0000 7fee02a00700 1 -- 172.21.15.176:0/4200548585 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fedfc0044d0 con 0x7fee04095b90 2024-03-13T14:49:12.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.111+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee04135620 con 0x7fee04095b90 2024-03-13T14:49:12.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.111+0000 7fee02a00700 1 -- 172.21.15.176:0/4200548585 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedfc003da0 con 0x7fee04095b90 2024-03-13T14:49:12.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.112+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee04135b70 con 0x7fee04095b90 2024-03-13T14:49:12.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.113+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0409e6f0 con 0x7fee04095b90 2024-03-13T14:49:12.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.115+0000 7fee02a00700 1 -- 172.21.15.176:0/4200548585 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 24) v1 ==== 91107+0+0 (secure 0 0 0) 0x7fedfc00cba0 con 0x7fee04095b90 2024-03-13T14:49:12.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.116+0000 7fee02a00700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fedf006d250 0x7fedf006f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:12.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.116+0000 7fee02a00700 1 -- 172.21.15.176:0/4200548585 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fedfc014070 con 0x7fee04095b90 2024-03-13T14:49:12.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.117+0000 7fee09400700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fedf006d250 0x7fedf006f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:12.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.118+0000 7fee09400700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fedf006d250 0x7fedf006f710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fedf8006010 tx=0x7fedf8009f70 comp rx=0 tx=0).ready entity=mgr.24325 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:12.111 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.123+0000 7fee02a00700 1 -- 172.21.15.176:0/4200548585 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fedfc05a200 con 0x7fee04095b90 2024-03-13T14:49:12.448 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.459+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 --> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee040996d0 con 0x7fedf006d250 2024-03-13T14:49:12.451 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.463+0000 7fee02a00700 1 -- 172.21.15.176:0/4200548585 <== mgr.24325 v2:172.21.15.196:6828/1297073737 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fee040996d0 con 0x7fedf006d250 2024-03-13T14:49:12.451 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:49:12.451 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:49:12.451 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:49:12.452 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:49:12.452 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [], 2024-03-13T14:49:12.452 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "", 2024-03-13T14:49:12.452 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mgr daemons", 2024-03-13T14:49:12.452 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:49:12.452 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fedf006d250 msgr2=0x7fedf006f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fedf006d250 0x7fedf006f710 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fedf8006010 tx=0x7fedf8009f70 comp rx=0 tx=0).stop 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04095b90 msgr2=0x7fee041386e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04095b90 0x7fee041386e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00e8f0 tx=0x7fedfc00ecb0 comp rx=0 tx=0).stop 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 shutdown_connections 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fee04095b90 0x7fee041386e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:6828/1297073737,v1:172.21.15.196:6829/1297073737] conn(0x7fedf006d250 0x7fedf006f710 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 --2- 172.21.15.176:0/4200548585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee04096d90 0x7fee04138c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:12.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 >> 172.21.15.176:0/4200548585 conn(0x7fee04091110 msgr2=0x7fee04097fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:12.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 shutdown_connections 2024-03-13T14:49:12.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:12.466+0000 7fee0b200700 1 -- 172.21.15.176:0/4200548585 wait complete. 2024-03-13T14:49:12.590 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:12 smithi176 ceph-mon[29064]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:12.590 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:12 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:12.591 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:12 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:12.591 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:12 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:12.591 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:12 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3587459858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:49:12.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:12 smithi196 ceph-mon[37486]: from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:12.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:12 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:12.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:12 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:12.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:12 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:12.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:12 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3587459858' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:49:13.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:13 smithi176 ceph-mon[29064]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:13.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:13 smithi176 ceph-mon[29064]: pgmap v25: 65 pgs: 65 active+clean; 176 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-03-13T14:49:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:13 smithi196 ceph-mon[37486]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:13.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:13 smithi196 ceph-mon[37486]: pgmap v25: 65 pgs: 65 active+clean; 176 MiB data, 848 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 234 op/s 2024-03-13T14:49:14.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:14 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:14.697 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:14 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:15.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:15 smithi176 ceph-mon[29064]: pgmap v26: 65 pgs: 65 active+clean; 178 MiB data, 869 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-03-13T14:49:15.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:15 smithi196 ceph-mon[37486]: pgmap v26: 65 pgs: 65 active+clean; 178 MiB data, 869 MiB used, 536 GiB / 536 GiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 264 op/s 2024-03-13T14:49:18.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:17 smithi176 ceph-mon[29064]: pgmap v27: 65 pgs: 65 active+clean; 192 MiB data, 898 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-13T14:49:18.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:17 smithi196 ceph-mon[37486]: pgmap v27: 65 pgs: 65 active+clean; 192 MiB data, 898 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 308 op/s 2024-03-13T14:49:18.680 INFO:tasks.workunit.client.1.smithi196.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-13T14:49:18.690 INFO:tasks.workunit.client.1.smithi196.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-13T14:49:18.691 INFO:tasks.workunit.client.1.smithi196.stderr:+ make 2024-03-13T14:49:18.857 INFO:tasks.workunit.client.1.smithi196.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-13T14:49:19.886 INFO:tasks.workunit.client.1.smithi196.stderr:++ readlink -f fsstress 2024-03-13T14:49:19.891 INFO:tasks.workunit.client.1.smithi196.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-13T14:49:19.891 INFO:tasks.workunit.client.1.smithi196.stderr:+ popd 2024-03-13T14:49:19.893 INFO:tasks.workunit.client.1.smithi196.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-03-13T14:49:19.894 INFO:tasks.workunit.client.1.smithi196.stderr:+ popd 2024-03-13T14:49:19.896 INFO:tasks.workunit.client.1.smithi196.stdout:~/cephtest/mnt.1/client.1/tmp 2024-03-13T14:49:19.896 INFO:tasks.workunit.client.1.smithi196.stderr:++ mktemp -d -p . 2024-03-13T14:49:19.902 INFO:tasks.workunit.client.1.smithi196.stderr:+ T=./tmp.n2NxSjmiRJ 2024-03-13T14:49:19.902 INFO:tasks.workunit.client.1.smithi196.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.n2NxSjmiRJ -l 1 -n 1000 -p 10 -v 2024-03-13T14:49:19.910 INFO:tasks.workunit.client.1.smithi196.stdout:seed = 1711254034 2024-03-13T14:49:19.917 INFO:tasks.workunit.client.1.smithi196.stdout:0/0: dwrite - no filename 2024-03-13T14:49:19.917 INFO:tasks.workunit.client.1.smithi196.stdout:0/1: link - no file 2024-03-13T14:49:19.917 INFO:tasks.workunit.client.1.smithi196.stdout:0/2: dread - no filename 2024-03-13T14:49:19.917 INFO:tasks.workunit.client.1.smithi196.stdout:0/3: dread - no filename 2024-03-13T14:49:19.922 INFO:tasks.workunit.client.1.smithi196.stdout:0/4: mknod c0 0 2024-03-13T14:49:19.929 INFO:tasks.workunit.client.1.smithi196.stdout:3/0: dread - no filename 2024-03-13T14:49:19.930 INFO:tasks.workunit.client.1.smithi196.stdout:1/0: getdents . 0 2024-03-13T14:49:19.930 INFO:tasks.workunit.client.1.smithi196.stdout:1/1: rename - no filename 2024-03-13T14:49:19.931 INFO:tasks.workunit.client.1.smithi196.stdout:1/2: read - no filename 2024-03-13T14:49:19.931 INFO:tasks.workunit.client.1.smithi196.stdout:1/3: rename - no filename 2024-03-13T14:49:19.931 INFO:tasks.workunit.client.1.smithi196.stdout:1/4: dread - no filename 2024-03-13T14:49:19.931 INFO:tasks.workunit.client.1.smithi196.stdout:1/5: dread - no filename 2024-03-13T14:49:19.931 INFO:tasks.workunit.client.1.smithi196.stdout:1/6: dwrite - no filename 2024-03-13T14:49:19.932 INFO:tasks.workunit.client.1.smithi196.stdout:0/5: mknod c1 0 2024-03-13T14:49:19.932 INFO:tasks.workunit.client.1.smithi196.stdout:0/6: fsync - no filename 2024-03-13T14:49:19.932 INFO:tasks.workunit.client.1.smithi196.stdout:0/7: rmdir - no directory 2024-03-13T14:49:19.935 INFO:tasks.workunit.client.1.smithi196.stdout:4/0: dread - no filename 2024-03-13T14:49:19.935 INFO:tasks.workunit.client.1.smithi196.stdout:4/1: truncate - no filename 2024-03-13T14:49:19.938 INFO:tasks.workunit.client.1.smithi196.stdout:0/8: rename c0 to c2 0 2024-03-13T14:49:19.938 INFO:tasks.workunit.client.1.smithi196.stdout:0/9: dwrite - no filename 2024-03-13T14:49:19.941 INFO:tasks.workunit.client.1.smithi196.stdout:5/0: truncate - no filename 2024-03-13T14:49:19.941 INFO:tasks.workunit.client.1.smithi196.stdout:5/1: read - no filename 2024-03-13T14:49:19.941 INFO:tasks.workunit.client.1.smithi196.stdout:5/2: fdatasync - no filename 2024-03-13T14:49:19.942 INFO:tasks.workunit.client.1.smithi196.stdout:3/1: creat f0 x:0 0 0 2024-03-13T14:49:19.942 INFO:tasks.workunit.client.1.smithi196.stdout:1/7: creat f0 x:0 0 0 2024-03-13T14:49:19.943 INFO:tasks.workunit.client.1.smithi196.stdout:1/8: chown f0 2035570 1 2024-03-13T14:49:19.944 INFO:tasks.workunit.client.1.smithi196.stdout:0/10: symlink l3 0 2024-03-13T14:49:19.948 INFO:tasks.workunit.client.1.smithi196.stdout:1/9: mkdir d1 0 2024-03-13T14:49:19.949 INFO:tasks.workunit.client.1.smithi196.stdout:1/10: write f0 [767245,2378] 0 2024-03-13T14:49:19.950 INFO:tasks.workunit.client.1.smithi196.stdout:0/11: mknod c4 0 2024-03-13T14:49:19.950 INFO:tasks.workunit.client.1.smithi196.stdout:0/12: dread - no filename 2024-03-13T14:49:19.952 INFO:tasks.workunit.client.1.smithi196.stdout:1/11: write f0 [1707326,16011] 0 2024-03-13T14:49:19.954 INFO:tasks.workunit.client.1.smithi196.stdout:5/3: mkdir d0 0 2024-03-13T14:49:19.959 INFO:tasks.workunit.client.1.smithi196.stdout:0/13: mknod c5 0 2024-03-13T14:49:19.959 INFO:tasks.workunit.client.1.smithi196.stdout:0/14: rmdir - no directory 2024-03-13T14:49:19.961 INFO:tasks.workunit.client.1.smithi196.stdout:6/0: creat f0 x:0 0 0 2024-03-13T14:49:19.963 INFO:tasks.workunit.client.1.smithi196.stdout:6/1: dread - f0 zero size 2024-03-13T14:49:19.971 INFO:tasks.workunit.client.1.smithi196.stdout:3/2: dwrite f0 [0,4194304] 0 2024-03-13T14:49:19.971 INFO:tasks.workunit.client.1.smithi196.stdout:8/0: truncate - no filename 2024-03-13T14:49:19.976 INFO:tasks.workunit.client.1.smithi196.stdout:7/0: mkdir d0 0 2024-03-13T14:49:19.976 INFO:tasks.workunit.client.1.smithi196.stdout:6/2: mknod c1 0 2024-03-13T14:49:19.977 INFO:tasks.workunit.client.1.smithi196.stdout:5/4: rmdir d0 0 2024-03-13T14:49:19.977 INFO:tasks.workunit.client.1.smithi196.stdout:5/5: dread - no filename 2024-03-13T14:49:19.977 INFO:tasks.workunit.client.1.smithi196.stdout:5/6: dwrite - no filename 2024-03-13T14:49:19.977 INFO:tasks.workunit.client.1.smithi196.stdout:7/1: chown d0 26 1 2024-03-13T14:49:19.977 INFO:tasks.workunit.client.1.smithi196.stdout:7/2: truncate - no filename 2024-03-13T14:49:19.979 INFO:tasks.workunit.client.1.smithi196.stdout:6/3: fdatasync f0 0 2024-03-13T14:49:19.979 INFO:tasks.workunit.client.1.smithi196.stdout:9/0: dwrite - no filename 2024-03-13T14:49:19.979 INFO:tasks.workunit.client.1.smithi196.stdout:9/1: dread - no filename 2024-03-13T14:49:19.979 INFO:tasks.workunit.client.1.smithi196.stdout:9/2: dwrite - no filename 2024-03-13T14:49:19.979 INFO:tasks.workunit.client.1.smithi196.stdout:9/3: truncate - no filename 2024-03-13T14:49:19.979 INFO:tasks.workunit.client.1.smithi196.stdout:9/4: rename - no filename 2024-03-13T14:49:19.979 INFO:tasks.workunit.client.1.smithi196.stdout:1/12: dread f0 [0,4194304] 0 2024-03-13T14:49:19.980 INFO:tasks.workunit.client.1.smithi196.stdout:8/1: mkdir d0 0 2024-03-13T14:49:19.980 INFO:tasks.workunit.client.1.smithi196.stdout:8/2: dwrite - no filename 2024-03-13T14:49:19.982 INFO:tasks.workunit.client.1.smithi196.stdout:5/7: mknod c1 0 2024-03-13T14:49:19.984 INFO:tasks.workunit.client.1.smithi196.stdout:5/8: chown c1 0 1 2024-03-13T14:49:19.984 INFO:tasks.workunit.client.1.smithi196.stdout:5/9: dread - no filename 2024-03-13T14:49:19.984 INFO:tasks.workunit.client.1.smithi196.stdout:5/10: fsync - no filename 2024-03-13T14:49:19.984 INFO:tasks.workunit.client.1.smithi196.stdout:5/11: dread - no filename 2024-03-13T14:49:19.984 INFO:tasks.workunit.client.1.smithi196.stdout:5/12: dwrite - no filename 2024-03-13T14:49:19.984 INFO:tasks.workunit.client.1.smithi196.stdout:3/3: link f0 f1 0 2024-03-13T14:49:19.986 INFO:tasks.workunit.client.1.smithi196.stdout:5/13: chown c1 8775 1 2024-03-13T14:49:19.988 INFO:tasks.workunit.client.1.smithi196.stdout:1/13: dread f0 [0,4194304] 0 2024-03-13T14:49:19.992 INFO:tasks.workunit.client.1.smithi196.stdout:7/3: getdents d0 0 2024-03-13T14:49:19.992 INFO:tasks.workunit.client.1.smithi196.stdout:7/4: fsync - no filename 2024-03-13T14:49:19.992 INFO:tasks.workunit.client.1.smithi196.stdout:7/5: dwrite - no filename 2024-03-13T14:49:19.992 INFO:tasks.workunit.client.1.smithi196.stdout:7/6: dwrite - no filename 2024-03-13T14:49:19.992 INFO:tasks.workunit.client.1.smithi196.stdout:7/7: dread - no filename 2024-03-13T14:49:19.992 INFO:tasks.workunit.client.1.smithi196.stdout:7/8: fsync - no filename 2024-03-13T14:49:19.993 INFO:tasks.workunit.client.1.smithi196.stdout:7/9: dread - no filename 2024-03-13T14:49:19.993 INFO:tasks.workunit.client.1.smithi196.stdout:3/4: symlink l2 0 2024-03-13T14:49:19.998 INFO:tasks.workunit.client.1.smithi196.stdout:9/5: creat f0 x:0 0 0 2024-03-13T14:49:20.002 INFO:tasks.workunit.client.1.smithi196.stdout:8/3: creat d0/f1 x:0 0 0 2024-03-13T14:49:20.004 INFO:tasks.workunit.client.1.smithi196.stdout:8/4: chown d0 8300311 1 2024-03-13T14:49:20.010 INFO:tasks.workunit.client.1.smithi196.stdout:6/4: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.012 INFO:tasks.workunit.client.1.smithi196.stdout:1/14: symlink d1/l2 0 2024-03-13T14:49:20.013 INFO:tasks.workunit.client.1.smithi196.stdout:7/10: symlink d0/l1 0 2024-03-13T14:49:20.017 INFO:tasks.workunit.client.1.smithi196.stdout:8/5: symlink d0/l2 0 2024-03-13T14:49:20.025 INFO:tasks.workunit.client.1.smithi196.stdout:1/15: mknod d1/c3 0 2024-03-13T14:49:20.029 INFO:tasks.workunit.client.1.smithi196.stdout:7/11: mknod d0/c2 0 2024-03-13T14:49:20.030 INFO:tasks.workunit.client.1.smithi196.stdout:7/12: read - no filename 2024-03-13T14:49:20.030 INFO:tasks.workunit.client.1.smithi196.stdout:7/13: dread - no filename 2024-03-13T14:49:20.030 INFO:tasks.workunit.client.1.smithi196.stdout:7/14: truncate - no filename 2024-03-13T14:49:20.039 INFO:tasks.workunit.client.1.smithi196.stdout:7/15: creat d0/f3 x:0 0 0 2024-03-13T14:49:20.043 INFO:tasks.workunit.client.1.smithi196.stdout:7/16: chown d0 337775338 1 2024-03-13T14:49:20.044 INFO:tasks.workunit.client.1.smithi196.stdout:9/6: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.056 INFO:tasks.workunit.client.1.smithi196.stdout:9/7: creat f1 x:0 0 0 2024-03-13T14:49:20.056 INFO:tasks.workunit.client.1.smithi196.stdout:8/6: dwrite d0/f1 [0,4194304] 0 2024-03-13T14:49:20.077 INFO:tasks.workunit.client.1.smithi196.stdout:1/16: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.095 INFO:tasks.workunit.client.1.smithi196.stdout:7/17: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:49:20.102 INFO:tasks.workunit.client.1.smithi196.stdout:7/18: readlink d0/l1 0 2024-03-13T14:49:20.105 INFO:tasks.workunit.client.1.smithi196.stdout:8/7: dread d0/f1 [0,4194304] 0 2024-03-13T14:49:20.107 INFO:tasks.workunit.client.1.smithi196.stdout:9/8: dwrite f1 [0,4194304] 0 2024-03-13T14:49:20.115 INFO:tasks.workunit.client.1.smithi196.stdout:9/9: truncate f1 5125392 0 2024-03-13T14:49:20.116 INFO:tasks.workunit.client.1.smithi196.stdout:9/10: chown f0 462 1 2024-03-13T14:49:20.122 INFO:tasks.workunit.client.1.smithi196.stdout:8/8: mknod d0/c3 0 2024-03-13T14:49:20.122 INFO:tasks.workunit.client.1.smithi196.stdout:1/17: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.122 INFO:tasks.workunit.client.1.smithi196.stdout:1/18: chown d1/c3 622902 1 2024-03-13T14:49:20.125 INFO:tasks.workunit.client.1.smithi196.stdout:9/11: dread f1 [4194304,4194304] 0 2024-03-13T14:49:20.126 INFO:tasks.workunit.client.1.smithi196.stdout:9/12: stat f1 0 2024-03-13T14:49:20.129 INFO:tasks.workunit.client.1.smithi196.stdout:0/15: rename c2 to c6 0 2024-03-13T14:49:20.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:19 smithi176 ceph-mon[29064]: pgmap v28: 65 pgs: 65 active+clean; 192 MiB data, 898 MiB used, 536 GiB / 536 GiB avail; 941 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-03-13T14:49:20.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:19 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:20.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:19 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:20.138 INFO:tasks.workunit.client.1.smithi196.stdout:6/5: fsync f0 0 2024-03-13T14:49:20.145 INFO:tasks.workunit.client.1.smithi196.stdout:9/13: dread f0 [0,4194304] 0 2024-03-13T14:49:20.148 INFO:tasks.workunit.client.1.smithi196.stdout:5/14: getdents . 0 2024-03-13T14:49:20.148 INFO:tasks.workunit.client.1.smithi196.stdout:5/15: dwrite - no filename 2024-03-13T14:49:20.149 INFO:tasks.workunit.client.1.smithi196.stdout:3/5: truncate f1 3768264 0 2024-03-13T14:49:20.151 INFO:tasks.workunit.client.1.smithi196.stdout:9/14: mknod c2 0 2024-03-13T14:49:20.155 INFO:tasks.workunit.client.1.smithi196.stdout:3/6: chown f1 755318 1 2024-03-13T14:49:20.159 INFO:tasks.workunit.client.1.smithi196.stdout:5/16: link c1 c2 0 2024-03-13T14:49:20.160 INFO:tasks.workunit.client.1.smithi196.stdout:9/15: write f0 [3420971,130230] 0 2024-03-13T14:49:20.165 INFO:tasks.workunit.client.1.smithi196.stdout:8/9: dwrite d0/f1 [0,4194304] 0 2024-03-13T14:49:20.168 INFO:tasks.workunit.client.1.smithi196.stdout:5/17: creat f3 x:0 0 0 2024-03-13T14:49:20.169 INFO:tasks.workunit.client.1.smithi196.stdout:8/10: rename d0 to d0/d4 22 2024-03-13T14:49:20.173 INFO:tasks.workunit.client.1.smithi196.stdout:8/11: chown d0/c3 186 1 2024-03-13T14:49:20.173 INFO:tasks.workunit.client.1.smithi196.stdout:6/6: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.175 INFO:tasks.workunit.client.1.smithi196.stdout:8/12: rename d0 to d0/d5 22 2024-03-13T14:49:20.178 INFO:tasks.workunit.client.1.smithi196.stdout:5/18: chown c2 4089334 1 2024-03-13T14:49:20.178 INFO:tasks.workunit.client.1.smithi196.stdout:8/13: read d0/f1 [2324332,5083] 0 2024-03-13T14:49:20.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:19 smithi196 ceph-mon[37486]: pgmap v28: 65 pgs: 65 active+clean; 192 MiB data, 898 MiB used, 536 GiB / 536 GiB avail; 941 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-03-13T14:49:20.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:19 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:20.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:19 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:20.181 INFO:tasks.workunit.client.1.smithi196.stdout:8/14: chown d0/c3 492752902 1 2024-03-13T14:49:20.182 INFO:tasks.workunit.client.1.smithi196.stdout:7/19: truncate d0/f3 3308058 0 2024-03-13T14:49:20.184 INFO:tasks.workunit.client.1.smithi196.stdout:6/7: mknod c2 0 2024-03-13T14:49:20.189 INFO:tasks.workunit.client.1.smithi196.stdout:8/15: rmdir d0 39 2024-03-13T14:49:20.191 INFO:tasks.workunit.client.1.smithi196.stdout:3/7: fsync f0 0 2024-03-13T14:49:20.191 INFO:tasks.workunit.client.1.smithi196.stdout:9/16: dread f1 [0,4194304] 0 2024-03-13T14:49:20.204 INFO:tasks.workunit.client.1.smithi196.stdout:5/19: chown c1 561491 1 2024-03-13T14:49:20.204 INFO:tasks.workunit.client.1.smithi196.stdout:7/20: write d0/f3 [3542029,38295] 0 2024-03-13T14:49:20.206 INFO:tasks.workunit.client.1.smithi196.stdout:0/16: rename c6 to c7 0 2024-03-13T14:49:20.206 INFO:tasks.workunit.client.1.smithi196.stdout:0/17: dread - no filename 2024-03-13T14:49:20.208 INFO:tasks.workunit.client.1.smithi196.stdout:5/20: mkdir d4 0 2024-03-13T14:49:20.213 INFO:tasks.workunit.client.1.smithi196.stdout:0/18: mkdir d8 0 2024-03-13T14:49:20.213 INFO:tasks.workunit.client.1.smithi196.stdout:0/19: dread - no filename 2024-03-13T14:49:20.215 INFO:tasks.workunit.client.1.smithi196.stdout:5/21: truncate f3 118045 0 2024-03-13T14:49:20.216 INFO:tasks.workunit.client.1.smithi196.stdout:0/20: stat c4 0 2024-03-13T14:49:20.216 INFO:tasks.workunit.client.1.smithi196.stdout:6/8: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.217 INFO:tasks.workunit.client.1.smithi196.stdout:6/9: readlink - no filename 2024-03-13T14:49:20.222 INFO:tasks.workunit.client.1.smithi196.stdout:5/22: write f3 [364968,36428] 0 2024-03-13T14:49:20.227 INFO:tasks.workunit.client.1.smithi196.stdout:9/17: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.228 INFO:tasks.workunit.client.1.smithi196.stdout:6/10: write f0 [1967590,122339] 0 2024-03-13T14:49:20.230 INFO:tasks.workunit.client.1.smithi196.stdout:0/21: creat d8/f9 x:0 0 0 2024-03-13T14:49:20.231 INFO:tasks.workunit.client.1.smithi196.stdout:3/8: dwrite f0 [4194304,4194304] 0 2024-03-13T14:49:20.234 INFO:tasks.workunit.client.1.smithi196.stdout:0/22: write d8/f9 [566881,72850] 0 2024-03-13T14:49:20.241 INFO:tasks.workunit.client.1.smithi196.stdout:5/23: creat d4/f5 x:0 0 0 2024-03-13T14:49:20.246 INFO:tasks.workunit.client.1.smithi196.stdout:8/16: dwrite d0/f1 [0,4194304] 0 2024-03-13T14:49:20.247 INFO:tasks.workunit.client.1.smithi196.stdout:8/17: stat d0/l2 0 2024-03-13T14:49:20.256 INFO:tasks.workunit.client.1.smithi196.stdout:5/24: symlink d4/l6 0 2024-03-13T14:49:20.270 INFO:tasks.workunit.client.1.smithi196.stdout:5/25: creat d4/f7 x:0 0 0 2024-03-13T14:49:20.274 INFO:tasks.workunit.client.1.smithi196.stdout:7/21: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:49:20.281 INFO:tasks.workunit.client.1.smithi196.stdout:3/9: fsync f0 0 2024-03-13T14:49:20.283 INFO:tasks.workunit.client.1.smithi196.stdout:9/18: dwrite f0 [0,4194304] 0 2024-03-13T14:49:20.286 INFO:tasks.workunit.client.1.smithi196.stdout:0/23: dwrite d8/f9 [0,4194304] 0 2024-03-13T14:49:20.286 INFO:tasks.workunit.client.1.smithi196.stdout:3/10: read f0 [6566916,89168] 0 2024-03-13T14:49:20.290 INFO:tasks.workunit.client.1.smithi196.stdout:7/22: mknod d0/c4 0 2024-03-13T14:49:20.292 INFO:tasks.workunit.client.1.smithi196.stdout:8/18: dwrite d0/f1 [0,4194304] 0 2024-03-13T14:49:20.294 INFO:tasks.workunit.client.1.smithi196.stdout:3/11: write f1 [6805733,47915] 0 2024-03-13T14:49:20.297 INFO:tasks.workunit.client.1.smithi196.stdout:8/19: chown d0/l2 24559 1 2024-03-13T14:49:20.297 INFO:tasks.workunit.client.1.smithi196.stdout:9/19: truncate f0 4474448 0 2024-03-13T14:49:20.297 INFO:tasks.workunit.client.1.smithi196.stdout:8/20: readlink d0/l2 0 2024-03-13T14:49:20.298 INFO:tasks.workunit.client.1.smithi196.stdout:3/12: stat l2 0 2024-03-13T14:49:20.300 INFO:tasks.workunit.client.1.smithi196.stdout:7/23: mkdir d0/d5 0 2024-03-13T14:49:20.309 INFO:tasks.workunit.client.1.smithi196.stdout:8/21: mknod d0/c6 0 2024-03-13T14:49:20.309 INFO:tasks.workunit.client.1.smithi196.stdout:9/20: unlink c2 0 2024-03-13T14:49:20.310 INFO:tasks.workunit.client.1.smithi196.stdout:5/26: dwrite d4/f5 [0,4194304] 0 2024-03-13T14:49:20.319 INFO:tasks.workunit.client.1.smithi196.stdout:0/24: dread d8/f9 [0,4194304] 0 2024-03-13T14:49:20.321 INFO:tasks.workunit.client.1.smithi196.stdout:3/13: rename f0 to f3 0 2024-03-13T14:49:20.322 INFO:tasks.workunit.client.1.smithi196.stdout:3/14: chown f1 3 1 2024-03-13T14:49:20.323 INFO:tasks.workunit.client.1.smithi196.stdout:7/24: rename d0/d5 to d0/d6 0 2024-03-13T14:49:20.325 INFO:tasks.workunit.client.1.smithi196.stdout:7/25: read d0/f3 [1680687,47595] 0 2024-03-13T14:49:20.326 INFO:tasks.workunit.client.1.smithi196.stdout:8/22: mknod d0/c7 0 2024-03-13T14:49:20.329 INFO:tasks.workunit.client.1.smithi196.stdout:8/23: chown d0/f1 5193556 1 2024-03-13T14:49:20.331 INFO:tasks.workunit.client.1.smithi196.stdout:5/27: rename d4/f5 to d4/f8 0 2024-03-13T14:49:20.331 INFO:tasks.workunit.client.1.smithi196.stdout:9/21: dread f1 [0,4194304] 0 2024-03-13T14:49:20.342 INFO:tasks.workunit.client.1.smithi196.stdout:7/26: dread d0/f3 [0,4194304] 0 2024-03-13T14:49:20.344 INFO:tasks.workunit.client.1.smithi196.stdout:3/15: write f1 [7587749,40539] 0 2024-03-13T14:49:20.345 INFO:tasks.workunit.client.1.smithi196.stdout:8/24: creat d0/f8 x:0 0 0 2024-03-13T14:49:20.345 INFO:tasks.workunit.client.1.smithi196.stdout:3/16: readlink l2 0 2024-03-13T14:49:20.355 INFO:tasks.workunit.client.1.smithi196.stdout:3/17: rename f1 to f4 0 2024-03-13T14:49:20.356 INFO:tasks.workunit.client.1.smithi196.stdout:3/18: chown l2 12964062 1 2024-03-13T14:49:20.356 INFO:tasks.workunit.client.1.smithi196.stdout:0/25: dwrite d8/f9 [0,4194304] 0 2024-03-13T14:49:20.359 INFO:tasks.workunit.client.1.smithi196.stdout:0/26: chown c4 19520533 1 2024-03-13T14:49:20.365 INFO:tasks.workunit.client.1.smithi196.stdout:3/19: symlink l5 0 2024-03-13T14:49:20.367 INFO:tasks.workunit.client.1.smithi196.stdout:5/28: dwrite d4/f8 [0,4194304] 0 2024-03-13T14:49:20.382 INFO:tasks.workunit.client.1.smithi196.stdout:3/20: rename l2 to l6 0 2024-03-13T14:49:20.385 INFO:tasks.workunit.client.1.smithi196.stdout:3/21: chown f3 77351713 1 2024-03-13T14:49:20.388 INFO:tasks.workunit.client.1.smithi196.stdout:7/27: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:49:20.396 INFO:tasks.workunit.client.1.smithi196.stdout:9/22: truncate f1 4413259 0 2024-03-13T14:49:20.398 INFO:tasks.workunit.client.1.smithi196.stdout:0/27: dwrite d8/f9 [0,4194304] 0 2024-03-13T14:49:20.400 INFO:tasks.workunit.client.1.smithi196.stdout:0/28: chown l3 134200388 1 2024-03-13T14:49:20.410 INFO:tasks.workunit.client.1.smithi196.stdout:9/23: dread f0 [0,4194304] 0 2024-03-13T14:49:20.530 INFO:tasks.workunit.client.1.smithi196.stdout:5/29: symlink d4/l9 0 2024-03-13T14:49:20.531 INFO:tasks.workunit.client.1.smithi196.stdout:7/28: symlink d0/d6/l7 0 2024-03-13T14:49:20.533 INFO:tasks.workunit.client.1.smithi196.stdout:0/29: mkdir d8/da 0 2024-03-13T14:49:20.535 INFO:tasks.workunit.client.1.smithi196.stdout:5/30: symlink d4/la 0 2024-03-13T14:49:20.536 INFO:tasks.workunit.client.1.smithi196.stdout:5/31: fdatasync d4/f8 0 2024-03-13T14:49:20.536 INFO:tasks.workunit.client.1.smithi196.stdout:9/24: rename f0 to f3 0 2024-03-13T14:49:20.537 INFO:tasks.workunit.client.1.smithi196.stdout:5/32: read d4/f8 [1159701,39646] 0 2024-03-13T14:49:20.547 INFO:tasks.workunit.client.1.smithi196.stdout:0/30: mknod d8/cb 0 2024-03-13T14:49:20.549 INFO:tasks.workunit.client.1.smithi196.stdout:5/33: mkdir d4/db 0 2024-03-13T14:49:20.550 INFO:tasks.workunit.client.1.smithi196.stdout:3/22: dwrite f4 [4194304,4194304] 0 2024-03-13T14:49:20.553 INFO:tasks.workunit.client.1.smithi196.stdout:0/31: read d8/f9 [3804623,110621] 0 2024-03-13T14:49:20.554 INFO:tasks.workunit.client.1.smithi196.stdout:3/23: write f4 [3810742,105530] 0 2024-03-13T14:49:20.555 INFO:tasks.workunit.client.1.smithi196.stdout:7/29: link d0/d6/l7 d0/l8 0 2024-03-13T14:49:20.562 INFO:tasks.workunit.client.1.smithi196.stdout:3/24: creat f7 x:0 0 0 2024-03-13T14:49:20.566 INFO:tasks.workunit.client.1.smithi196.stdout:0/32: write d8/f9 [1232483,94330] 0 2024-03-13T14:49:20.579 INFO:tasks.workunit.client.1.smithi196.stdout:9/25: dwrite f3 [4194304,4194304] 0 2024-03-13T14:49:20.590 INFO:tasks.workunit.client.1.smithi196.stdout:3/25: dread f4 [0,4194304] 0 2024-03-13T14:49:20.590 INFO:tasks.workunit.client.1.smithi196.stdout:5/34: dwrite f3 [0,4194304] 0 2024-03-13T14:49:20.598 INFO:tasks.workunit.client.1.smithi196.stdout:5/35: unlink c1 0 2024-03-13T14:49:20.599 INFO:tasks.workunit.client.1.smithi196.stdout:5/36: readlink d4/l9 0 2024-03-13T14:49:20.600 INFO:tasks.workunit.client.1.smithi196.stdout:0/33: dread d8/f9 [0,4194304] 0 2024-03-13T14:49:20.601 INFO:tasks.workunit.client.1.smithi196.stdout:3/26: link f3 f8 0 2024-03-13T14:49:20.602 INFO:tasks.workunit.client.1.smithi196.stdout:5/37: fdatasync d4/f8 0 2024-03-13T14:49:20.602 INFO:tasks.workunit.client.1.smithi196.stdout:9/26: dread f3 [4194304,4194304] 0 2024-03-13T14:49:20.602 INFO:tasks.workunit.client.1.smithi196.stdout:0/34: chown d8/da 228 1 2024-03-13T14:49:20.604 INFO:tasks.workunit.client.1.smithi196.stdout:0/35: chown l3 3156 1 2024-03-13T14:49:20.604 INFO:tasks.workunit.client.1.smithi196.stdout:5/38: readlink d4/l6 0 2024-03-13T14:49:20.606 INFO:tasks.workunit.client.1.smithi196.stdout:3/27: symlink l9 0 2024-03-13T14:49:20.607 INFO:tasks.workunit.client.1.smithi196.stdout:5/39: write d4/f7 [102466,67627] 0 2024-03-13T14:49:20.610 INFO:tasks.workunit.client.1.smithi196.stdout:5/40: readlink d4/l9 0 2024-03-13T14:49:20.616 INFO:tasks.workunit.client.1.smithi196.stdout:5/41: symlink d4/lc 0 2024-03-13T14:49:20.623 INFO:tasks.workunit.client.1.smithi196.stdout:9/27: dread f3 [4194304,4194304] 0 2024-03-13T14:49:20.641 INFO:tasks.workunit.client.1.smithi196.stdout:5/42: dwrite d4/f7 [0,4194304] 0 2024-03-13T14:49:20.643 INFO:tasks.workunit.client.1.smithi196.stdout:5/43: chown d4/f8 25 1 2024-03-13T14:49:20.647 INFO:tasks.workunit.client.1.smithi196.stdout:9/28: dwrite f3 [0,4194304] 0 2024-03-13T14:49:20.650 INFO:tasks.workunit.client.1.smithi196.stdout:9/29: write f3 [1411324,124043] 0 2024-03-13T14:49:20.653 INFO:tasks.workunit.client.1.smithi196.stdout:5/44: creat d4/db/fd x:0 0 0 2024-03-13T14:49:20.653 INFO:tasks.workunit.client.1.smithi196.stdout:5/45: stat d4 0 2024-03-13T14:49:20.666 INFO:tasks.workunit.client.1.smithi196.stdout:9/30: dread f3 [4194304,4194304] 0 2024-03-13T14:49:20.684 INFO:tasks.workunit.client.1.smithi196.stdout:5/46: dwrite d4/db/fd [0,4194304] 0 2024-03-13T14:49:20.687 INFO:tasks.workunit.client.1.smithi196.stdout:5/47: creat d4/fe x:0 0 0 2024-03-13T14:49:20.689 INFO:tasks.workunit.client.1.smithi196.stdout:5/48: chown d4/fe 5167 1 2024-03-13T14:49:20.691 INFO:tasks.workunit.client.1.smithi196.stdout:5/49: creat d4/db/ff x:0 0 0 2024-03-13T14:49:20.692 INFO:tasks.workunit.client.1.smithi196.stdout:5/50: dread - d4/fe zero size 2024-03-13T14:49:20.707 INFO:tasks.workunit.client.1.smithi196.stdout:5/51: dread f3 [0,4194304] 0 2024-03-13T14:49:20.720 INFO:tasks.workunit.client.1.smithi196.stdout:5/52: dread d4/f7 [0,4194304] 0 2024-03-13T14:49:20.758 INFO:tasks.workunit.client.1.smithi196.stdout:7/30: truncate d0/f3 1751264 0 2024-03-13T14:49:20.762 INFO:tasks.workunit.client.1.smithi196.stdout:7/31: mkdir d0/d9 0 2024-03-13T14:49:20.766 INFO:tasks.workunit.client.1.smithi196.stdout:9/31: rename f1 to f4 0 2024-03-13T14:49:20.770 INFO:tasks.workunit.client.1.smithi196.stdout:9/32: rename f3 to f5 0 2024-03-13T14:49:20.775 INFO:tasks.workunit.client.1.smithi196.stdout:9/33: write f5 [5366777,19765] 0 2024-03-13T14:49:20.775 INFO:tasks.workunit.client.1.smithi196.stdout:3/28: getdents . 0 2024-03-13T14:49:20.777 INFO:tasks.workunit.client.1.smithi196.stdout:0/36: truncate d8/f9 1487878 0 2024-03-13T14:49:20.784 INFO:tasks.workunit.client.1.smithi196.stdout:3/29: symlink la 0 2024-03-13T14:49:20.786 INFO:tasks.workunit.client.1.smithi196.stdout:3/30: mkdir db 0 2024-03-13T14:49:20.801 INFO:tasks.workunit.client.1.smithi196.stdout:5/53: truncate f3 2703364 0 2024-03-13T14:49:20.804 INFO:tasks.workunit.client.1.smithi196.stdout:3/31: dwrite f7 [0,4194304] 0 2024-03-13T14:49:20.805 INFO:tasks.workunit.client.1.smithi196.stdout:5/54: symlink d4/l10 0 2024-03-13T14:49:20.809 INFO:tasks.workunit.client.1.smithi196.stdout:5/55: rmdir d4/db 39 2024-03-13T14:49:20.813 INFO:tasks.workunit.client.1.smithi196.stdout:3/32: link l5 db/lc 0 2024-03-13T14:49:20.814 INFO:tasks.workunit.client.1.smithi196.stdout:7/32: getdents d0 0 2024-03-13T14:49:20.821 INFO:tasks.workunit.client.1.smithi196.stdout:7/33: creat d0/fa x:0 0 0 2024-03-13T14:49:20.828 INFO:tasks.workunit.client.1.smithi196.stdout:3/33: dread f7 [0,4194304] 0 2024-03-13T14:49:20.829 INFO:tasks.workunit.client.1.smithi196.stdout:5/56: rename d4/fe to d4/f11 0 2024-03-13T14:49:20.835 INFO:tasks.workunit.client.1.smithi196.stdout:9/34: link f4 f6 0 2024-03-13T14:49:20.836 INFO:tasks.workunit.client.1.smithi196.stdout:7/34: symlink d0/d9/lb 0 2024-03-13T14:49:20.840 INFO:tasks.workunit.client.1.smithi196.stdout:5/57: unlink f3 0 2024-03-13T14:49:20.847 INFO:tasks.workunit.client.1.smithi196.stdout:5/58: write d4/db/ff [426587,58591] 0 2024-03-13T14:49:20.852 INFO:tasks.workunit.client.1.smithi196.stdout:5/59: rmdir d4/db 39 2024-03-13T14:49:20.857 INFO:tasks.workunit.client.1.smithi196.stdout:0/37: truncate d8/f9 121137 0 2024-03-13T14:49:20.860 INFO:tasks.workunit.client.1.smithi196.stdout:5/60: rename d4/l10 to d4/db/l12 0 2024-03-13T14:49:20.865 INFO:tasks.workunit.client.1.smithi196.stdout:7/35: link d0/f3 d0/d6/fc 0 2024-03-13T14:49:20.865 INFO:tasks.workunit.client.1.smithi196.stdout:5/61: write d4/f11 [629965,118706] 0 2024-03-13T14:49:20.874 INFO:tasks.workunit.client.1.smithi196.stdout:7/36: read d0/f3 [944441,78619] 0 2024-03-13T14:49:20.878 INFO:tasks.workunit.client.1.smithi196.stdout:9/35: dwrite f6 [4194304,4194304] 0 2024-03-13T14:49:20.881 INFO:tasks.workunit.client.1.smithi196.stdout:5/62: link d4/f11 d4/f13 0 2024-03-13T14:49:20.881 INFO:tasks.workunit.client.1.smithi196.stdout:9/36: mkdir d7 0 2024-03-13T14:49:20.882 INFO:tasks.workunit.client.1.smithi196.stdout:5/63: mkdir d4/db/d14 0 2024-03-13T14:49:20.882 INFO:tasks.workunit.client.1.smithi196.stdout:7/37: dread d0/f3 [0,4194304] 0 2024-03-13T14:49:20.885 INFO:tasks.workunit.client.1.smithi196.stdout:9/37: write f5 [1909522,78472] 0 2024-03-13T14:49:20.888 INFO:tasks.workunit.client.1.smithi196.stdout:5/64: chown d4/db/fd 223 1 2024-03-13T14:49:20.888 INFO:tasks.workunit.client.1.smithi196.stdout:9/38: write f6 [3820847,80460] 0 2024-03-13T14:49:20.889 INFO:tasks.workunit.client.1.smithi196.stdout:7/38: mknod d0/d6/cd 0 2024-03-13T14:49:20.891 INFO:tasks.workunit.client.1.smithi196.stdout:7/39: chown d0 4906 1 2024-03-13T14:49:20.897 INFO:tasks.workunit.client.1.smithi196.stdout:5/65: rename d4/db/d14 to d4/db/d14/d15 22 2024-03-13T14:49:20.898 INFO:tasks.workunit.client.1.smithi196.stdout:9/39: mknod d7/c8 0 2024-03-13T14:49:20.901 INFO:tasks.workunit.client.1.smithi196.stdout:0/38: dwrite d8/f9 [0,4194304] 0 2024-03-13T14:49:20.903 INFO:tasks.workunit.client.1.smithi196.stdout:5/66: rename d4/db to d4/db/d16 22 2024-03-13T14:49:20.903 INFO:tasks.workunit.client.1.smithi196.stdout:7/40: fdatasync d0/f3 0 2024-03-13T14:49:20.905 INFO:tasks.workunit.client.1.smithi196.stdout:9/40: mkdir d7/d9 0 2024-03-13T14:49:20.915 INFO:tasks.workunit.client.1.smithi196.stdout:7/41: rename d0/l8 to d0/d6/le 0 2024-03-13T14:49:20.920 INFO:tasks.workunit.client.1.smithi196.stdout:7/42: creat d0/ff x:0 0 0 2024-03-13T14:49:20.921 INFO:tasks.workunit.client.1.smithi196.stdout:5/67: getdents d4 0 2024-03-13T14:49:20.922 INFO:tasks.workunit.client.1.smithi196.stdout:7/43: truncate d0/fa 139440 0 2024-03-13T14:49:20.924 INFO:tasks.workunit.client.1.smithi196.stdout:0/39: fdatasync d8/f9 0 2024-03-13T14:49:20.925 INFO:tasks.workunit.client.1.smithi196.stdout:7/44: mknod d0/d6/c10 0 2024-03-13T14:49:20.929 INFO:tasks.workunit.client.1.smithi196.stdout:9/41: truncate f5 5466065 0 2024-03-13T14:49:20.931 INFO:tasks.workunit.client.1.smithi196.stdout:0/40: truncate d8/f9 1207145 0 2024-03-13T14:49:20.938 INFO:tasks.workunit.client.1.smithi196.stdout:0/41: creat d8/da/fc x:0 0 0 2024-03-13T14:49:20.941 INFO:tasks.workunit.client.1.smithi196.stdout:7/45: getdents d0/d6 0 2024-03-13T14:49:20.941 INFO:tasks.workunit.client.1.smithi196.stdout:5/68: dwrite d4/f8 [4194304,4194304] 0 2024-03-13T14:49:20.943 INFO:tasks.workunit.client.1.smithi196.stdout:5/69: read d4/f7 [1848672,5513] 0 2024-03-13T14:49:20.948 INFO:tasks.workunit.client.1.smithi196.stdout:0/42: symlink d8/da/ld 0 2024-03-13T14:49:20.953 INFO:tasks.workunit.client.1.smithi196.stdout:5/70: symlink d4/db/l17 0 2024-03-13T14:49:20.954 INFO:tasks.workunit.client.1.smithi196.stdout:5/71: chown d4/la 2298 1 2024-03-13T14:49:20.955 INFO:tasks.workunit.client.1.smithi196.stdout:0/43: mkdir d8/da/de 0 2024-03-13T14:49:20.957 INFO:tasks.workunit.client.1.smithi196.stdout:5/72: symlink d4/l18 0 2024-03-13T14:49:20.958 INFO:tasks.workunit.client.1.smithi196.stdout:0/44: symlink d8/da/lf 0 2024-03-13T14:49:20.965 INFO:tasks.workunit.client.1.smithi196.stdout:0/45: creat d8/da/f10 x:0 0 0 2024-03-13T14:49:20.976 INFO:tasks.workunit.client.1.smithi196.stdout:5/73: stat d4/f11 0 2024-03-13T14:49:20.980 INFO:tasks.workunit.client.1.smithi196.stdout:9/42: dwrite f5 [0,4194304] 0 2024-03-13T14:49:20.982 INFO:tasks.workunit.client.1.smithi196.stdout:7/46: dwrite d0/d6/fc [0,4194304] 0 2024-03-13T14:49:20.982 INFO:tasks.workunit.client.1.smithi196.stdout:9/43: chown f5 3295197 1 2024-03-13T14:49:20.987 INFO:tasks.workunit.client.1.smithi196.stdout:0/46: write d8/f9 [406368,73468] 0 2024-03-13T14:49:20.990 INFO:tasks.workunit.client.1.smithi196.stdout:5/74: write d4/f8 [4355919,839] 0 2024-03-13T14:49:20.996 INFO:tasks.workunit.client.1.smithi196.stdout:0/47: write d8/da/fc [106690,117772] 0 2024-03-13T14:49:20.998 INFO:tasks.workunit.client.1.smithi196.stdout:7/47: getdents d0 0 2024-03-13T14:49:21.001 INFO:tasks.workunit.client.1.smithi196.stdout:0/48: creat d8/f11 x:0 0 0 2024-03-13T14:49:21.003 INFO:tasks.workunit.client.1.smithi196.stdout:0/49: readlink l3 0 2024-03-13T14:49:21.004 INFO:tasks.workunit.client.1.smithi196.stdout:5/75: mkdir d4/db/d14/d19 0 2024-03-13T14:49:21.009 INFO:tasks.workunit.client.1.smithi196.stdout:9/44: dwrite f6 [0,4194304] 0 2024-03-13T14:49:21.010 INFO:tasks.workunit.client.1.smithi196.stdout:0/50: creat d8/f12 x:0 0 0 2024-03-13T14:49:21.012 INFO:tasks.workunit.client.1.smithi196.stdout:0/51: chown c5 502165 1 2024-03-13T14:49:21.012 INFO:tasks.workunit.client.1.smithi196.stdout:9/45: write f5 [5422065,29024] 0 2024-03-13T14:49:21.015 INFO:tasks.workunit.client.1.smithi196.stdout:9/46: write f6 [6126676,114396] 0 2024-03-13T14:49:21.017 INFO:tasks.workunit.client.1.smithi196.stdout:0/52: creat d8/da/f13 x:0 0 0 2024-03-13T14:49:21.019 INFO:tasks.workunit.client.1.smithi196.stdout:0/53: stat d8/da 0 2024-03-13T14:49:21.025 INFO:tasks.workunit.client.1.smithi196.stdout:0/54: dread - d8/f11 zero size 2024-03-13T14:49:21.027 INFO:tasks.workunit.client.1.smithi196.stdout:5/76: dread d4/db/fd [0,4194304] 0 2024-03-13T14:49:21.028 INFO:tasks.workunit.client.1.smithi196.stdout:0/55: dread d8/f9 [0,4194304] 0 2024-03-13T14:49:21.030 INFO:tasks.workunit.client.1.smithi196.stdout:7/48: truncate d0/f3 3043281 0 2024-03-13T14:49:21.032 INFO:tasks.workunit.client.1.smithi196.stdout:9/47: dread f4 [4194304,4194304] 0 2024-03-13T14:49:21.035 INFO:tasks.workunit.client.1.smithi196.stdout:5/77: link d4/f7 d4/db/f1a 0 2024-03-13T14:49:21.051 INFO:tasks.workunit.client.1.smithi196.stdout:7/49: dread d0/d6/fc [0,4194304] 0 2024-03-13T14:49:21.055 INFO:tasks.workunit.client.1.smithi196.stdout:5/78: dwrite d4/db/ff [0,4194304] 0 2024-03-13T14:49:21.060 INFO:tasks.workunit.client.1.smithi196.stdout:7/50: write d0/d6/fc [841313,66914] 0 2024-03-13T14:49:21.063 INFO:tasks.workunit.client.1.smithi196.stdout:5/79: mkdir d4/d1b 0 2024-03-13T14:49:21.066 INFO:tasks.workunit.client.1.smithi196.stdout:7/51: link d0/ff d0/f11 0 2024-03-13T14:49:21.077 INFO:tasks.workunit.client.1.smithi196.stdout:5/80: dread d4/db/ff [0,4194304] 0 2024-03-13T14:49:21.091 INFO:tasks.workunit.client.1.smithi196.stdout:5/81: dwrite d4/f8 [4194304,4194304] 0 2024-03-13T14:49:21.110 INFO:tasks.workunit.client.1.smithi196.stdout:5/82: fdatasync d4/f8 0 2024-03-13T14:49:21.114 INFO:tasks.workunit.client.1.smithi196.stdout:5/83: mkdir d4/db/d14/d19/d1c 0 2024-03-13T14:49:21.123 INFO:tasks.workunit.client.1.smithi196.stdout:5/84: dread d4/db/fd [0,4194304] 0 2024-03-13T14:49:21.124 INFO:tasks.workunit.client.1.smithi196.stdout:5/85: read d4/f8 [2381477,34091] 0 2024-03-13T14:49:21.133 INFO:tasks.workunit.client.1.smithi196.stdout:5/86: dread d4/f8 [0,4194304] 0 2024-03-13T14:49:21.152 INFO:tasks.workunit.client.1.smithi196.stdout:5/87: dwrite d4/f7 [0,4194304] 0 2024-03-13T14:49:21.156 INFO:tasks.workunit.client.1.smithi196.stdout:5/88: truncate d4/f11 698944 0 2024-03-13T14:49:21.159 INFO:tasks.workunit.client.1.smithi196.stdout:5/89: creat d4/db/f1d x:0 0 0 2024-03-13T14:49:21.161 INFO:tasks.workunit.client.1.smithi196.stdout:5/90: mknod d4/db/d14/c1e 0 2024-03-13T14:49:21.163 INFO:tasks.workunit.client.1.smithi196.stdout:5/91: read d4/db/f1a [3981966,4756] 0 2024-03-13T14:49:21.175 INFO:tasks.workunit.client.1.smithi196.stdout:5/92: dread d4/db/fd [0,4194304] 0 2024-03-13T14:49:21.178 INFO:tasks.workunit.client.1.smithi196.stdout:5/93: getdents d4 0 2024-03-13T14:49:21.179 INFO:tasks.workunit.client.1.smithi196.stdout:5/94: chown d4/db/d14 2 1 2024-03-13T14:49:21.180 INFO:tasks.workunit.client.1.smithi196.stdout:5/95: write d4/db/f1d [687962,57878] 0 2024-03-13T14:49:21.182 INFO:tasks.workunit.client.1.smithi196.stdout:5/96: symlink d4/db/l1f 0 2024-03-13T14:49:21.184 INFO:tasks.workunit.client.1.smithi196.stdout:5/97: write d4/db/ff [152665,28562] 0 2024-03-13T14:49:21.189 INFO:tasks.workunit.client.1.smithi196.stdout:5/98: getdents d4/db/d14/d19 0 2024-03-13T14:49:21.192 INFO:tasks.workunit.client.1.smithi196.stdout:5/99: chown d4/f11 1741 1 2024-03-13T14:49:21.193 INFO:tasks.workunit.client.1.smithi196.stdout:5/100: chown d4/db/ff 0 1 2024-03-13T14:49:21.194 INFO:tasks.workunit.client.1.smithi196.stdout:5/101: chown d4/l9 31148358 1 2024-03-13T14:49:21.197 INFO:tasks.workunit.client.1.smithi196.stdout:5/102: symlink d4/db/d14/d19/l20 0 2024-03-13T14:49:21.211 INFO:tasks.workunit.client.1.smithi196.stdout:5/103: dwrite d4/f8 [4194304,4194304] 0 2024-03-13T14:49:21.225 INFO:tasks.workunit.client.1.smithi196.stdout:5/104: dread d4/db/fd [0,4194304] 0 2024-03-13T14:49:21.227 INFO:tasks.workunit.client.1.smithi196.stdout:5/105: mknod d4/c21 0 2024-03-13T14:49:21.228 INFO:tasks.workunit.client.1.smithi196.stdout:5/106: write d4/f7 [198673,84790] 0 2024-03-13T14:49:21.230 INFO:tasks.workunit.client.1.smithi196.stdout:5/107: readlink d4/l6 0 2024-03-13T14:49:21.230 INFO:tasks.workunit.client.1.smithi196.stdout:5/108: chown d4 5649 1 2024-03-13T14:49:21.234 INFO:tasks.workunit.client.1.smithi196.stdout:5/109: rename c2 to d4/db/d14/c22 0 2024-03-13T14:49:21.235 INFO:tasks.workunit.client.1.smithi196.stdout:5/110: fdatasync d4/db/f1a 0 2024-03-13T14:49:21.236 INFO:tasks.workunit.client.1.smithi196.stdout:5/111: write d4/f8 [6189237,41429] 0 2024-03-13T14:49:21.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:21 smithi176 ceph-mon[29064]: pgmap v29: 65 pgs: 65 active+clean; 209 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2024-03-13T14:49:21.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:21 smithi196 ceph-mon[37486]: pgmap v29: 65 pgs: 65 active+clean; 209 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 329 op/s 2024-03-13T14:49:22.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:22 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:22.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:22 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:22.992 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:22 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:23.029 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:22 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:23.771 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:23 smithi176 ceph-mon[29064]: Standby manager daemon smithi176.tzfdwd restarted 2024-03-13T14:49:23.771 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:23 smithi176 ceph-mon[29064]: Standby manager daemon smithi176.tzfdwd started 2024-03-13T14:49:23.771 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/crt"}]: dispatch 2024-03-13T14:49:23.771 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:23 smithi176 ceph-mon[29064]: pgmap v30: 65 pgs: 65 active+clean; 209 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 999 KiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-03-13T14:49:23.771 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:49:23.771 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/key"}]: dispatch 2024-03-13T14:49:23.771 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:23 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:49:23.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:23 smithi196 ceph-mon[37486]: Standby manager daemon smithi176.tzfdwd restarted 2024-03-13T14:49:23.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:23 smithi196 ceph-mon[37486]: Standby manager daemon smithi176.tzfdwd started 2024-03-13T14:49:23.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/crt"}]: dispatch 2024-03-13T14:49:23.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:23 smithi196 ceph-mon[37486]: pgmap v30: 65 pgs: 65 active+clean; 209 MiB data, 973 MiB used, 535 GiB / 536 GiB avail; 999 KiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-03-13T14:49:23.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:49:23.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi176.tzfdwd/key"}]: dispatch 2024-03-13T14:49:23.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:23 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:49:24.256 INFO:tasks.workunit.client.1.smithi196.stdout:2/0: sync 2024-03-13T14:49:24.257 INFO:tasks.workunit.client.1.smithi196.stdout:2/1: read - no filename 2024-03-13T14:49:24.257 INFO:tasks.workunit.client.1.smithi196.stdout:2/2: dread - no filename 2024-03-13T14:49:24.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/3: mkdir d0 0 2024-03-13T14:49:24.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/4: write - no filename 2024-03-13T14:49:24.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/5: dwrite - no filename 2024-03-13T14:49:24.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/6: dread - no filename 2024-03-13T14:49:24.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/7: truncate - no filename 2024-03-13T14:49:24.269 INFO:tasks.workunit.client.1.smithi196.stdout:2/8: write - no filename 2024-03-13T14:49:24.269 INFO:tasks.workunit.client.1.smithi196.stdout:2/9: write - no filename 2024-03-13T14:49:24.274 INFO:tasks.workunit.client.1.smithi196.stdout:2/10: creat d0/f1 x:0 0 0 2024-03-13T14:49:24.277 INFO:tasks.workunit.client.1.smithi196.stdout:2/11: dread - d0/f1 zero size 2024-03-13T14:49:24.281 INFO:tasks.workunit.client.1.smithi196.stdout:2/12: truncate d0/f1 590064 0 2024-03-13T14:49:24.442 INFO:tasks.workunit.client.1.smithi196.stdout:4/2: sync 2024-03-13T14:49:24.443 INFO:tasks.workunit.client.1.smithi196.stdout:4/3: unlink - no file 2024-03-13T14:49:24.456 INFO:tasks.workunit.client.1.smithi196.stdout:4/4: creat f0 x:0 0 0 2024-03-13T14:49:24.463 INFO:tasks.workunit.client.1.smithi196.stdout:4/5: mkdir d1 0 2024-03-13T14:49:24.516 INFO:tasks.workunit.client.1.smithi196.stdout:1/19: sync 2024-03-13T14:49:24.516 INFO:tasks.workunit.client.1.smithi196.stdout:8/25: sync 2024-03-13T14:49:24.516 INFO:tasks.workunit.client.1.smithi196.stdout:3/34: sync 2024-03-13T14:49:24.516 INFO:tasks.workunit.client.1.smithi196.stdout:6/11: sync 2024-03-13T14:49:24.535 INFO:tasks.workunit.client.1.smithi196.stdout:1/20: dread f0 [0,4194304] 0 2024-03-13T14:49:24.559 INFO:tasks.workunit.client.1.smithi196.stdout:9/48: sync 2024-03-13T14:49:24.559 INFO:tasks.workunit.client.1.smithi196.stdout:0/56: sync 2024-03-13T14:49:24.559 INFO:tasks.workunit.client.1.smithi196.stdout:5/112: sync 2024-03-13T14:49:24.559 INFO:tasks.workunit.client.1.smithi196.stdout:7/52: sync 2024-03-13T14:49:24.561 INFO:tasks.workunit.client.1.smithi196.stdout:0/57: write d8/f12 [82919,119332] 0 2024-03-13T14:49:24.561 INFO:tasks.workunit.client.1.smithi196.stdout:7/53: write d0/d6/fc [1796130,106080] 0 2024-03-13T14:49:24.561 INFO:tasks.workunit.client.1.smithi196.stdout:9/49: write f5 [1501369,62001] 0 2024-03-13T14:49:24.577 INFO:tasks.workunit.client.1.smithi196.stdout:9/50: dread f4 [0,4194304] 0 2024-03-13T14:49:24.686 INFO:tasks.workunit.client.1.smithi196.stdout:4/6: creat d1/f2 x:0 0 0 2024-03-13T14:49:24.696 INFO:tasks.workunit.client.1.smithi196.stdout:8/26: mknod d0/c9 0 2024-03-13T14:49:24.707 INFO:tasks.workunit.client.1.smithi196.stdout:4/7: dwrite d1/f2 [0,4194304] 0 2024-03-13T14:49:24.708 INFO:tasks.workunit.client.1.smithi196.stdout:0/58: rmdir d8/da 39 2024-03-13T14:49:24.710 INFO:tasks.workunit.client.1.smithi196.stdout:0/59: dread - d8/f11 zero size 2024-03-13T14:49:24.712 INFO:tasks.workunit.client.1.smithi196.stdout:7/54: mknod d0/d6/c12 0 2024-03-13T14:49:24.713 INFO:tasks.workunit.client.1.smithi196.stdout:7/55: chown d0/c4 365986 1 2024-03-13T14:49:24.714 INFO:tasks.workunit.client.1.smithi196.stdout:8/27: dread d0/f1 [0,4194304] 0 2024-03-13T14:49:24.716 INFO:tasks.workunit.client.1.smithi196.stdout:8/28: write d0/f1 [1171686,67965] 0 2024-03-13T14:49:24.719 INFO:tasks.workunit.client.1.smithi196.stdout:3/35: creat db/fd x:0 0 0 2024-03-13T14:49:24.732 INFO:tasks.workunit.client.1.smithi196.stdout:5/113: truncate d4/db/f1a 1105876 0 2024-03-13T14:49:24.734 INFO:tasks.workunit.client.1.smithi196.stdout:9/51: dwrite f6 [0,4194304] 0 2024-03-13T14:49:24.735 INFO:tasks.workunit.client.1.smithi196.stdout:9/52: readlink - no filename 2024-03-13T14:49:24.738 INFO:tasks.workunit.client.1.smithi196.stdout:0/60: truncate d8/da/fc 373253 0 2024-03-13T14:49:24.738 INFO:tasks.workunit.client.1.smithi196.stdout:7/56: creat d0/f13 x:0 0 0 2024-03-13T14:49:24.741 INFO:tasks.workunit.client.1.smithi196.stdout:3/36: creat db/fe x:0 0 0 2024-03-13T14:49:24.742 INFO:tasks.workunit.client.1.smithi196.stdout:6/12: link c2 c3 0 2024-03-13T14:49:24.742 INFO:tasks.workunit.client.1.smithi196.stdout:3/37: dread - db/fe zero size 2024-03-13T14:49:24.743 INFO:tasks.workunit.client.1.smithi196.stdout:0/61: chown d8/da/fc 0 1 2024-03-13T14:49:24.751 INFO:tasks.workunit.client.1.smithi196.stdout:1/21: link d1/c3 d1/c4 0 2024-03-13T14:49:24.756 INFO:tasks.workunit.client.1.smithi196.stdout:3/38: creat db/ff x:0 0 0 2024-03-13T14:49:24.762 INFO:tasks.workunit.client.1.smithi196.stdout:5/114: creat d4/db/d14/d19/d1c/f23 x:0 0 0 2024-03-13T14:49:24.764 INFO:tasks.workunit.client.1.smithi196.stdout:9/53: creat d7/d9/fa x:0 0 0 2024-03-13T14:49:24.770 INFO:tasks.workunit.client.1.smithi196.stdout:7/57: dwrite d0/f13 [0,4194304] 0 2024-03-13T14:49:24.771 INFO:tasks.workunit.client.1.smithi196.stdout:9/54: unlink f6 0 2024-03-13T14:49:24.771 INFO:tasks.workunit.client.1.smithi196.stdout:9/55: readlink - no filename 2024-03-13T14:49:24.772 INFO:tasks.workunit.client.1.smithi196.stdout:9/56: chown d7 25 1 2024-03-13T14:49:24.775 INFO:tasks.workunit.client.1.smithi196.stdout:5/115: chown d4/db/l12 589683 1 2024-03-13T14:49:24.775 INFO:tasks.workunit.client.1.smithi196.stdout:1/22: chown d1/c4 6861228 1 2024-03-13T14:49:24.775 INFO:tasks.workunit.client.1.smithi196.stdout:0/62: dwrite d8/da/fc [0,4194304] 0 2024-03-13T14:49:24.776 INFO:tasks.workunit.client.1.smithi196.stdout:5/116: stat d4/lc 0 2024-03-13T14:49:24.778 INFO:tasks.workunit.client.1.smithi196.stdout:0/63: chown d8/da/de 4 1 2024-03-13T14:49:24.779 INFO:tasks.workunit.client.1.smithi196.stdout:3/39: truncate f4 5846028 0 2024-03-13T14:49:24.780 INFO:tasks.workunit.client.1.smithi196.stdout:0/64: read d8/da/fc [2602744,55822] 0 2024-03-13T14:49:24.790 INFO:tasks.workunit.client.1.smithi196.stdout:5/117: creat d4/db/f24 x:0 0 0 2024-03-13T14:49:24.790 INFO:tasks.workunit.client.1.smithi196.stdout:7/58: dread d0/d6/fc [0,4194304] 0 2024-03-13T14:49:24.793 INFO:tasks.workunit.client.1.smithi196.stdout:5/118: fdatasync d4/db/f1d 0 2024-03-13T14:49:24.797 INFO:tasks.workunit.client.1.smithi196.stdout:5/119: stat d4/c21 0 2024-03-13T14:49:24.800 INFO:tasks.workunit.client.1.smithi196.stdout:9/57: dwrite f5 [4194304,4194304] 0 2024-03-13T14:49:24.808 INFO:tasks.workunit.client.1.smithi196.stdout:7/59: rename d0/d6/c10 to d0/d9/c14 0 2024-03-13T14:49:24.808 INFO:tasks.workunit.client.1.smithi196.stdout:0/65: mkdir d8/da/de/d14 0 2024-03-13T14:49:24.813 INFO:tasks.workunit.client.1.smithi196.stdout:5/120: chown d4/db/l12 10896378 1 2024-03-13T14:49:24.815 INFO:tasks.workunit.client.1.smithi196.stdout:0/66: chown d8/f11 118590 1 2024-03-13T14:49:24.820 INFO:tasks.workunit.client.1.smithi196.stdout:7/60: creat d0/f15 x:0 0 0 2024-03-13T14:49:24.827 INFO:tasks.workunit.client.1.smithi196.stdout:5/121: mkdir d4/db/d14/d19/d1c/d25 0 2024-03-13T14:49:24.830 INFO:tasks.workunit.client.1.smithi196.stdout:7/61: mknod d0/c16 0 2024-03-13T14:49:24.833 INFO:tasks.workunit.client.1.smithi196.stdout:0/67: write d8/f9 [626102,51149] 0 2024-03-13T14:49:24.834 INFO:tasks.workunit.client.1.smithi196.stdout:1/23: dwrite f0 [4194304,4194304] 0 2024-03-13T14:49:24.840 INFO:tasks.workunit.client.1.smithi196.stdout:9/58: dwrite f5 [4194304,4194304] 0 2024-03-13T14:49:24.842 INFO:tasks.workunit.client.1.smithi196.stdout:9/59: read - d7/d9/fa zero size 2024-03-13T14:49:24.843 INFO:tasks.workunit.client.1.smithi196.stdout:1/24: link f0 d1/f5 0 2024-03-13T14:49:24.844 INFO:tasks.workunit.client.1.smithi196.stdout:0/68: symlink d8/da/de/d14/l15 0 2024-03-13T14:49:24.848 INFO:tasks.workunit.client.1.smithi196.stdout:9/60: symlink d7/lb 0 2024-03-13T14:49:24.854 INFO:tasks.workunit.client.1.smithi196.stdout:0/69: write d8/da/fc [817179,84899] 0 2024-03-13T14:49:24.854 INFO:tasks.workunit.client.1.smithi196.stdout:9/61: symlink d7/lc 0 2024-03-13T14:49:24.861 INFO:tasks.workunit.client.1.smithi196.stdout:7/62: dwrite d0/f15 [0,4194304] 0 2024-03-13T14:49:24.861 INFO:tasks.workunit.client.1.smithi196.stdout:9/62: chown f4 3866 1 2024-03-13T14:49:24.862 INFO:tasks.workunit.client.1.smithi196.stdout:0/70: rmdir d8/da 39 2024-03-13T14:49:24.863 INFO:tasks.workunit.client.1.smithi196.stdout:9/63: read f4 [5427418,45292] 0 2024-03-13T14:49:24.864 INFO:tasks.workunit.client.1.smithi196.stdout:1/25: link d1/l2 d1/l6 0 2024-03-13T14:49:24.866 INFO:tasks.workunit.client.1.smithi196.stdout:7/63: symlink d0/d6/l17 0 2024-03-13T14:49:24.868 INFO:tasks.workunit.client.1.smithi196.stdout:1/26: symlink d1/l7 0 2024-03-13T14:49:24.869 INFO:tasks.workunit.client.1.smithi196.stdout:5/122: dread d4/f13 [0,4194304] 0 2024-03-13T14:49:24.871 INFO:tasks.workunit.client.1.smithi196.stdout:9/64: rename d7/d9/fa to d7/fd 0 2024-03-13T14:49:24.873 INFO:tasks.workunit.client.1.smithi196.stdout:0/71: mknod d8/da/de/c16 0 2024-03-13T14:49:24.875 INFO:tasks.workunit.client.1.smithi196.stdout:0/72: truncate d8/f11 469001 0 2024-03-13T14:49:24.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:24 smithi176 ceph-mon[29064]: mgrmap e25: smithi196.vdktbk(active, since 55s), standbys: smithi176.tzfdwd 2024-03-13T14:49:24.879 INFO:tasks.workunit.client.1.smithi196.stdout:5/123: mkdir d4/db/d14/d19/d26 0 2024-03-13T14:49:24.879 INFO:tasks.workunit.client.1.smithi196.stdout:0/73: write d8/da/f13 [965665,88538] 0 2024-03-13T14:49:24.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:24 smithi196 ceph-mon[37486]: mgrmap e25: smithi196.vdktbk(active, since 55s), standbys: smithi176.tzfdwd 2024-03-13T14:49:25.014 INFO:tasks.workunit.client.1.smithi196.stdout:1/27: dwrite f0 [4194304,4194304] 0 2024-03-13T14:49:25.016 INFO:tasks.workunit.client.1.smithi196.stdout:5/124: mknod d4/db/d14/d19/d26/c27 0 2024-03-13T14:49:25.021 INFO:tasks.workunit.client.1.smithi196.stdout:1/28: creat d1/f8 x:0 0 0 2024-03-13T14:49:25.021 INFO:tasks.workunit.client.1.smithi196.stdout:5/125: truncate d4/db/f1d 1049509 0 2024-03-13T14:49:25.021 INFO:tasks.workunit.client.1.smithi196.stdout:9/65: dwrite d7/fd [0,4194304] 0 2024-03-13T14:49:25.023 INFO:tasks.workunit.client.1.smithi196.stdout:1/29: mkdir d1/d9 0 2024-03-13T14:49:25.023 INFO:tasks.workunit.client.1.smithi196.stdout:9/66: chown d7/lc 1 1 2024-03-13T14:49:25.026 INFO:tasks.workunit.client.1.smithi196.stdout:9/67: mkdir d7/de 0 2024-03-13T14:49:25.029 INFO:tasks.workunit.client.1.smithi196.stdout:1/30: mknod d1/d9/ca 0 2024-03-13T14:49:25.030 INFO:tasks.workunit.client.1.smithi196.stdout:9/68: mkdir d7/d9/df 0 2024-03-13T14:49:25.044 INFO:tasks.workunit.client.1.smithi196.stdout:9/69: dread d7/fd [0,4194304] 0 2024-03-13T14:49:25.045 INFO:tasks.workunit.client.1.smithi196.stdout:9/70: chown d7/de 23330 1 2024-03-13T14:49:25.082 INFO:tasks.workunit.client.1.smithi196.stdout:9/71: read f4 [1831658,113172] 0 2024-03-13T14:49:25.087 INFO:tasks.workunit.client.1.smithi196.stdout:9/72: mkdir d7/d10 0 2024-03-13T14:49:25.092 INFO:tasks.workunit.client.1.smithi196.stdout:9/73: rmdir d7/d10 0 2024-03-13T14:49:25.096 INFO:tasks.workunit.client.1.smithi196.stdout:9/74: write f4 [2118862,53937] 0 2024-03-13T14:49:25.097 INFO:tasks.workunit.client.1.smithi196.stdout:9/75: chown d7/fd 5 1 2024-03-13T14:49:25.102 INFO:tasks.workunit.client.1.smithi196.stdout:9/76: rename f4 to d7/d9/f11 0 2024-03-13T14:49:25.105 INFO:tasks.workunit.client.1.smithi196.stdout:9/77: creat d7/f12 x:0 0 0 2024-03-13T14:49:25.108 INFO:tasks.workunit.client.1.smithi196.stdout:9/78: symlink d7/l13 0 2024-03-13T14:49:25.108 INFO:tasks.workunit.client.1.smithi196.stdout:9/79: fdatasync d7/f12 0 2024-03-13T14:49:25.128 INFO:tasks.workunit.client.1.smithi196.stdout:9/80: dwrite d7/f12 [0,4194304] 0 2024-03-13T14:49:25.129 INFO:tasks.workunit.client.1.smithi196.stdout:9/81: fdatasync f5 0 2024-03-13T14:49:25.150 INFO:tasks.workunit.client.1.smithi196.stdout:2/13: truncate d0/f1 51517 0 2024-03-13T14:49:25.152 INFO:tasks.workunit.client.1.smithi196.stdout:9/82: dwrite d7/f12 [0,4194304] 0 2024-03-13T14:49:25.177 INFO:tasks.workunit.client.1.smithi196.stdout:7/64: sync 2024-03-13T14:49:25.197 INFO:tasks.workunit.client.1.smithi196.stdout:7/65: dwrite d0/d6/fc [0,4194304] 0 2024-03-13T14:49:25.203 INFO:tasks.workunit.client.1.smithi196.stdout:7/66: unlink d0/f3 0 2024-03-13T14:49:25.204 INFO:tasks.workunit.client.1.smithi196.stdout:7/67: write d0/f13 [898721,106384] 0 2024-03-13T14:49:25.209 INFO:tasks.workunit.client.1.smithi196.stdout:7/68: dread d0/fa [0,4194304] 0 2024-03-13T14:49:25.210 INFO:tasks.workunit.client.1.smithi196.stdout:7/69: chown d0/f15 11 1 2024-03-13T14:49:25.237 INFO:tasks.workunit.client.1.smithi196.stdout:4/8: rmdir d1 39 2024-03-13T14:49:25.256 INFO:tasks.workunit.client.1.smithi196.stdout:4/9: dwrite d1/f2 [0,4194304] 0 2024-03-13T14:49:25.257 INFO:tasks.workunit.client.1.smithi196.stdout:4/10: stat d1/f2 0 2024-03-13T14:49:25.258 INFO:tasks.workunit.client.1.smithi196.stdout:5/126: sync 2024-03-13T14:49:25.260 INFO:tasks.workunit.client.1.smithi196.stdout:8/29: truncate d0/f1 1257489 0 2024-03-13T14:49:25.263 INFO:tasks.workunit.client.1.smithi196.stdout:5/127: truncate d4/f11 88318 0 2024-03-13T14:49:25.266 INFO:tasks.workunit.client.1.smithi196.stdout:8/30: mknod d0/ca 0 2024-03-13T14:49:25.268 INFO:tasks.workunit.client.1.smithi196.stdout:6/13: truncate f0 2644013 0 2024-03-13T14:49:25.276 INFO:tasks.workunit.client.1.smithi196.stdout:4/11: dwrite f0 [0,4194304] 0 2024-03-13T14:49:25.288 INFO:tasks.workunit.client.1.smithi196.stdout:5/128: getdents d4/db 0 2024-03-13T14:49:25.288 INFO:tasks.workunit.client.1.smithi196.stdout:7/70: getdents d0/d9 0 2024-03-13T14:49:25.292 INFO:tasks.workunit.client.1.smithi196.stdout:8/31: dwrite d0/f8 [0,4194304] 0 2024-03-13T14:49:25.293 INFO:tasks.workunit.client.1.smithi196.stdout:8/32: chown d0/f8 53427 1 2024-03-13T14:49:25.304 INFO:tasks.workunit.client.1.smithi196.stdout:4/12: dread d1/f2 [0,4194304] 0 2024-03-13T14:49:25.305 INFO:tasks.workunit.client.1.smithi196.stdout:8/33: creat d0/fb x:0 0 0 2024-03-13T14:49:25.311 INFO:tasks.workunit.client.1.smithi196.stdout:4/13: mkdir d1/d3 0 2024-03-13T14:49:25.312 INFO:tasks.workunit.client.1.smithi196.stdout:4/14: chown d1/f2 2316117 1 2024-03-13T14:49:25.313 INFO:tasks.workunit.client.1.smithi196.stdout:4/15: chown f0 102591 1 2024-03-13T14:49:25.314 INFO:tasks.workunit.client.1.smithi196.stdout:4/16: fsync f0 0 2024-03-13T14:49:25.314 INFO:tasks.workunit.client.1.smithi196.stdout:4/17: readlink - no filename 2024-03-13T14:49:25.315 INFO:tasks.workunit.client.1.smithi196.stdout:1/31: sync 2024-03-13T14:49:25.315 INFO:tasks.workunit.client.1.smithi196.stdout:4/18: stat f0 0 2024-03-13T14:49:25.324 INFO:tasks.workunit.client.1.smithi196.stdout:4/19: dread d1/f2 [0,4194304] 0 2024-03-13T14:49:25.326 INFO:tasks.workunit.client.1.smithi196.stdout:4/20: write f0 [2680551,29267] 0 2024-03-13T14:49:25.329 INFO:tasks.workunit.client.1.smithi196.stdout:4/21: symlink d1/l4 0 2024-03-13T14:49:25.346 INFO:tasks.workunit.client.1.smithi196.stdout:0/74: truncate d8/da/fc 2796378 0 2024-03-13T14:49:25.351 INFO:tasks.workunit.client.1.smithi196.stdout:0/75: mkdir d8/da/de/d17 0 2024-03-13T14:49:25.356 INFO:tasks.workunit.client.1.smithi196.stdout:0/76: mknod d8/c18 0 2024-03-13T14:49:25.357 INFO:tasks.workunit.client.1.smithi196.stdout:1/32: rmdir d1 39 2024-03-13T14:49:25.367 INFO:tasks.workunit.client.1.smithi196.stdout:1/33: rename d1/d9/ca to d1/cb 0 2024-03-13T14:49:25.376 INFO:tasks.workunit.client.1.smithi196.stdout:1/34: dread d1/f5 [0,4194304] 0 2024-03-13T14:49:25.382 INFO:tasks.workunit.client.1.smithi196.stdout:9/83: fsync d7/f12 0 2024-03-13T14:49:25.383 INFO:tasks.workunit.client.1.smithi196.stdout:9/84: chown d7/lc 15547 1 2024-03-13T14:49:25.384 INFO:tasks.workunit.client.1.smithi196.stdout:9/85: chown f5 2 1 2024-03-13T14:49:25.391 INFO:tasks.workunit.client.1.smithi196.stdout:9/86: creat d7/d9/df/f14 x:0 0 0 2024-03-13T14:49:25.392 INFO:tasks.workunit.client.1.smithi196.stdout:9/87: read - d7/d9/df/f14 zero size 2024-03-13T14:49:25.394 INFO:tasks.workunit.client.1.smithi196.stdout:9/88: write d7/d9/df/f14 [291581,29430] 0 2024-03-13T14:49:25.399 INFO:tasks.workunit.client.1.smithi196.stdout:9/89: creat d7/f15 x:0 0 0 2024-03-13T14:49:25.405 INFO:tasks.workunit.client.1.smithi196.stdout:7/71: rmdir d0 39 2024-03-13T14:49:25.406 INFO:tasks.workunit.client.1.smithi196.stdout:9/90: mkdir d7/de/d16 0 2024-03-13T14:49:25.406 INFO:tasks.workunit.client.1.smithi196.stdout:9/91: chown d7 116 1 2024-03-13T14:49:25.419 INFO:tasks.workunit.client.1.smithi196.stdout:9/92: dread d7/f12 [0,4194304] 0 2024-03-13T14:49:25.421 INFO:tasks.workunit.client.1.smithi196.stdout:7/72: dread d0/f15 [0,4194304] 0 2024-03-13T14:49:25.423 INFO:tasks.workunit.client.1.smithi196.stdout:9/93: mkdir d7/d9/d17 0 2024-03-13T14:49:25.428 INFO:tasks.workunit.client.1.smithi196.stdout:8/34: dread d0/f1 [0,4194304] 0 2024-03-13T14:49:25.435 INFO:tasks.workunit.client.1.smithi196.stdout:7/73: dread d0/f13 [0,4194304] 0 2024-03-13T14:49:25.435 INFO:tasks.workunit.client.1.smithi196.stdout:6/14: unlink f0 0 2024-03-13T14:49:25.437 INFO:tasks.workunit.client.1.smithi196.stdout:6/15: chown c1 216 1 2024-03-13T14:49:25.445 INFO:tasks.workunit.client.1.smithi196.stdout:6/16: creat f4 x:0 0 0 2024-03-13T14:49:25.447 INFO:tasks.workunit.client.1.smithi196.stdout:7/74: rename d0/d9/c14 to d0/c18 0 2024-03-13T14:49:25.447 INFO:tasks.workunit.client.1.smithi196.stdout:8/35: dread d0/f8 [0,4194304] 0 2024-03-13T14:49:25.451 INFO:tasks.workunit.client.1.smithi196.stdout:7/75: read - d0/ff zero size 2024-03-13T14:49:25.451 INFO:tasks.workunit.client.1.smithi196.stdout:6/17: symlink l5 0 2024-03-13T14:49:25.454 INFO:tasks.workunit.client.1.smithi196.stdout:9/94: dwrite d7/d9/df/f14 [0,4194304] 0 2024-03-13T14:49:25.456 INFO:tasks.workunit.client.1.smithi196.stdout:9/95: readlink d7/lc 0 2024-03-13T14:49:25.457 INFO:tasks.workunit.client.1.smithi196.stdout:4/22: dwrite f0 [4194304,4194304] 0 2024-03-13T14:49:25.461 INFO:tasks.workunit.client.1.smithi196.stdout:9/96: write d7/f15 [158339,67216] 0 2024-03-13T14:49:25.470 INFO:tasks.workunit.client.1.smithi196.stdout:6/18: mkdir d6 0 2024-03-13T14:49:25.641 INFO:tasks.workunit.client.1.smithi196.stdout:9/97: truncate d7/f12 2516387 0 2024-03-13T14:49:25.642 INFO:tasks.workunit.client.1.smithi196.stdout:0/77: fsync d8/da/fc 0 2024-03-13T14:49:25.642 INFO:tasks.workunit.client.1.smithi196.stdout:3/40: truncate f4 2203452 0 2024-03-13T14:49:25.643 INFO:tasks.workunit.client.1.smithi196.stdout:1/35: truncate f0 5528422 0 2024-03-13T14:49:25.646 INFO:tasks.workunit.client.1.smithi196.stdout:2/14: write d0/f1 [189483,106314] 0 2024-03-13T14:49:25.649 INFO:tasks.workunit.client.1.smithi196.stdout:8/36: dwrite d0/f8 [0,4194304] 0 2024-03-13T14:49:25.654 INFO:tasks.workunit.client.1.smithi196.stdout:0/78: rename d8/da/de to d8/da/de/d19 22 2024-03-13T14:49:25.660 INFO:tasks.workunit.client.1.smithi196.stdout:3/41: write f7 [5096084,826] 0 2024-03-13T14:49:25.665 INFO:tasks.workunit.client.1.smithi196.stdout:9/98: mknod d7/de/d16/c18 0 2024-03-13T14:49:25.670 INFO:tasks.workunit.client.1.smithi196.stdout:6/19: dwrite f4 [0,4194304] 0 2024-03-13T14:49:25.671 INFO:tasks.workunit.client.1.smithi196.stdout:7/76: creat d0/d6/f19 x:0 0 0 2024-03-13T14:49:25.674 INFO:tasks.workunit.client.1.smithi196.stdout:1/36: getdents d1 0 2024-03-13T14:49:25.676 INFO:tasks.workunit.client.1.smithi196.stdout:5/129: dwrite d4/f11 [0,4194304] 0 2024-03-13T14:49:25.680 INFO:tasks.workunit.client.1.smithi196.stdout:0/79: rename d8/da/de/d14 to d8/da/d1a 0 2024-03-13T14:49:25.683 INFO:tasks.workunit.client.1.smithi196.stdout:2/15: getdents d0 0 2024-03-13T14:49:25.683 INFO:tasks.workunit.client.1.smithi196.stdout:5/130: chown d4/db/d14/d19/d26 4 1 2024-03-13T14:49:25.685 INFO:tasks.workunit.client.1.smithi196.stdout:4/23: sync 2024-03-13T14:49:25.686 INFO:tasks.workunit.client.1.smithi196.stdout:9/99: truncate d7/f12 1216172 0 2024-03-13T14:49:25.689 INFO:tasks.workunit.client.1.smithi196.stdout:4/24: read d1/f2 [1439619,10640] 0 2024-03-13T14:49:25.690 INFO:tasks.workunit.client.1.smithi196.stdout:9/100: write f5 [1583780,20133] 0 2024-03-13T14:49:25.690 INFO:tasks.workunit.client.1.smithi196.stdout:1/37: unlink d1/cb 0 2024-03-13T14:49:25.694 INFO:tasks.workunit.client.1.smithi196.stdout:0/80: rename d8/c18 to d8/da/de/c1b 0 2024-03-13T14:49:25.698 INFO:tasks.workunit.client.1.smithi196.stdout:6/20: mkdir d6/d7 0 2024-03-13T14:49:25.703 INFO:tasks.workunit.client.1.smithi196.stdout:0/81: rmdir d8/da 39 2024-03-13T14:49:25.704 INFO:tasks.workunit.client.1.smithi196.stdout:2/16: read d0/f1 [172904,58732] 0 2024-03-13T14:49:25.707 INFO:tasks.workunit.client.1.smithi196.stdout:6/21: creat d6/f8 x:0 0 0 2024-03-13T14:49:25.707 INFO:tasks.workunit.client.1.smithi196.stdout:5/131: dwrite d4/f8 [0,4194304] 0 2024-03-13T14:49:25.713 INFO:tasks.workunit.client.1.smithi196.stdout:2/17: write d0/f1 [558039,28426] 0 2024-03-13T14:49:25.715 INFO:tasks.workunit.client.1.smithi196.stdout:2/18: chown d0 51625 1 2024-03-13T14:49:25.720 INFO:tasks.workunit.client.1.smithi196.stdout:6/22: mkdir d6/d9 0 2024-03-13T14:49:25.721 INFO:tasks.workunit.client.1.smithi196.stdout:2/19: truncate d0/f1 1351770 0 2024-03-13T14:49:25.722 INFO:tasks.workunit.client.1.smithi196.stdout:4/25: link d1/l4 d1/l5 0 2024-03-13T14:49:25.726 INFO:tasks.workunit.client.1.smithi196.stdout:8/37: write d0/f1 [1780364,8958] 0 2024-03-13T14:49:25.730 INFO:tasks.workunit.client.1.smithi196.stdout:0/82: dwrite d8/f11 [0,4194304] 0 2024-03-13T14:49:25.730 INFO:tasks.workunit.client.1.smithi196.stdout:6/23: creat d6/fa x:0 0 0 2024-03-13T14:49:25.737 INFO:tasks.workunit.client.1.smithi196.stdout:7/77: write d0/ff [259122,24807] 0 2024-03-13T14:49:25.740 INFO:tasks.workunit.client.1.smithi196.stdout:6/24: chown f4 22565986 1 2024-03-13T14:49:25.740 INFO:tasks.workunit.client.1.smithi196.stdout:9/101: fsync d7/f12 0 2024-03-13T14:49:25.750 INFO:tasks.workunit.client.1.smithi196.stdout:6/25: stat c3 0 2024-03-13T14:49:25.750 INFO:tasks.workunit.client.1.smithi196.stdout:5/132: dwrite d4/db/f24 [0,4194304] 0 2024-03-13T14:49:25.754 INFO:tasks.workunit.client.1.smithi196.stdout:5/133: fdatasync d4/f13 0 2024-03-13T14:49:25.755 INFO:tasks.workunit.client.1.smithi196.stdout:2/20: dread d0/f1 [0,4194304] 0 2024-03-13T14:49:25.764 INFO:tasks.workunit.client.1.smithi196.stdout:4/26: getdents d1/d3 0 2024-03-13T14:49:25.765 INFO:tasks.workunit.client.1.smithi196.stdout:0/83: creat d8/f1c x:0 0 0 2024-03-13T14:49:25.770 INFO:tasks.workunit.client.1.smithi196.stdout:6/26: mkdir d6/d7/db 0 2024-03-13T14:49:25.772 INFO:tasks.workunit.client.1.smithi196.stdout:8/38: dread d0/f8 [0,4194304] 0 2024-03-13T14:49:25.779 INFO:tasks.workunit.client.1.smithi196.stdout:0/84: rename c1 to d8/da/de/c1d 0 2024-03-13T14:49:25.782 INFO:tasks.workunit.client.1.smithi196.stdout:6/27: write d6/f8 [593454,33906] 0 2024-03-13T14:49:25.782 INFO:tasks.workunit.client.1.smithi196.stdout:8/39: chown d0/c9 74315480 1 2024-03-13T14:49:25.782 INFO:tasks.workunit.client.1.smithi196.stdout:1/38: dread f0 [0,4194304] 0 2024-03-13T14:49:25.786 INFO:tasks.workunit.client.1.smithi196.stdout:2/21: dread d0/f1 [0,4194304] 0 2024-03-13T14:49:25.791 INFO:tasks.workunit.client.1.smithi196.stdout:8/40: dread - d0/fb zero size 2024-03-13T14:49:25.794 INFO:tasks.workunit.client.1.smithi196.stdout:4/27: truncate d1/f2 2276139 0 2024-03-13T14:49:25.798 INFO:tasks.workunit.client.1.smithi196.stdout:0/85: dread d8/f9 [0,4194304] 0 2024-03-13T14:49:25.798 INFO:tasks.workunit.client.1.smithi196.stdout:1/39: write d1/f8 [145810,35733] 0 2024-03-13T14:49:25.798 INFO:tasks.workunit.client.1.smithi196.stdout:6/28: mkdir d6/d7/db/dc 0 2024-03-13T14:49:25.799 INFO:tasks.workunit.client.1.smithi196.stdout:7/78: dwrite d0/ff [0,4194304] 0 2024-03-13T14:49:25.803 INFO:tasks.workunit.client.1.smithi196.stdout:6/29: fdatasync d6/fa 0 2024-03-13T14:49:25.804 INFO:tasks.workunit.client.1.smithi196.stdout:3/42: dwrite f4 [0,4194304] 0 2024-03-13T14:49:25.804 INFO:tasks.workunit.client.1.smithi196.stdout:8/41: symlink d0/lc 0 2024-03-13T14:49:25.808 INFO:tasks.workunit.client.1.smithi196.stdout:9/102: dwrite d7/d9/f11 [0,4194304] 0 2024-03-13T14:49:25.808 INFO:tasks.workunit.client.1.smithi196.stdout:0/86: write d8/da/f13 [1582043,39343] 0 2024-03-13T14:49:25.809 INFO:tasks.workunit.client.1.smithi196.stdout:3/43: fsync db/fd 0 2024-03-13T14:49:25.815 INFO:tasks.workunit.client.1.smithi196.stdout:7/79: symlink d0/l1a 0 2024-03-13T14:49:25.818 INFO:tasks.workunit.client.1.smithi196.stdout:9/103: readlink d7/lc 0 2024-03-13T14:49:25.818 INFO:tasks.workunit.client.1.smithi196.stdout:1/40: creat d1/d9/fc x:0 0 0 2024-03-13T14:49:25.820 INFO:tasks.workunit.client.1.smithi196.stdout:9/104: chown d7/d9 1632458 1 2024-03-13T14:49:25.827 INFO:tasks.workunit.client.1.smithi196.stdout:0/87: write d8/da/f10 [541815,34655] 0 2024-03-13T14:49:25.827 INFO:tasks.workunit.client.1.smithi196.stdout:7/80: mknod d0/d9/c1b 0 2024-03-13T14:49:25.829 INFO:tasks.workunit.client.1.smithi196.stdout:1/41: creat d1/d9/fd x:0 0 0 2024-03-13T14:49:25.830 INFO:tasks.workunit.client.1.smithi196.stdout:9/105: creat d7/d9/df/f19 x:0 0 0 2024-03-13T14:49:25.832 INFO:tasks.workunit.client.1.smithi196.stdout:3/44: link l6 db/l10 0 2024-03-13T14:49:25.835 INFO:tasks.workunit.client.1.smithi196.stdout:7/81: creat d0/f1c x:0 0 0 2024-03-13T14:49:25.838 INFO:tasks.workunit.client.1.smithi196.stdout:2/22: dwrite d0/f1 [0,4194304] 0 2024-03-13T14:49:25.839 INFO:tasks.workunit.client.1.smithi196.stdout:9/106: chown d7/de/d16/c18 1125051 1 2024-03-13T14:49:25.841 INFO:tasks.workunit.client.1.smithi196.stdout:9/107: chown d7/f15 418567703 1 2024-03-13T14:49:25.844 INFO:tasks.workunit.client.1.smithi196.stdout:1/42: mknod d1/d9/ce 0 2024-03-13T14:49:25.848 INFO:tasks.workunit.client.1.smithi196.stdout:6/30: dread f4 [0,4194304] 0 2024-03-13T14:49:25.849 INFO:tasks.workunit.client.1.smithi196.stdout:7/82: write d0/f13 [3660975,33272] 0 2024-03-13T14:49:25.849 INFO:tasks.workunit.client.1.smithi196.stdout:2/23: creat d0/f2 x:0 0 0 2024-03-13T14:49:25.852 INFO:tasks.workunit.client.1.smithi196.stdout:9/108: creat d7/d9/df/f1a x:0 0 0 2024-03-13T14:49:25.855 INFO:tasks.workunit.client.1.smithi196.stdout:1/43: creat d1/ff x:0 0 0 2024-03-13T14:49:25.858 INFO:tasks.workunit.client.1.smithi196.stdout:7/83: rmdir d0/d9 39 2024-03-13T14:49:25.860 INFO:tasks.workunit.client.1.smithi196.stdout:0/88: link d8/da/f13 d8/f1e 0 2024-03-13T14:49:25.861 INFO:tasks.workunit.client.1.smithi196.stdout:2/24: mkdir d0/d3 0 2024-03-13T14:49:25.864 INFO:tasks.workunit.client.1.smithi196.stdout:1/44: fsync d1/d9/fd 0 2024-03-13T14:49:25.872 INFO:tasks.workunit.client.1.smithi196.stdout:2/25: dread - d0/f2 zero size 2024-03-13T14:49:25.880 INFO:tasks.workunit.client.1.smithi196.stdout:6/31: dwrite d6/fa [0,4194304] 0 2024-03-13T14:49:25.883 INFO:tasks.workunit.client.1.smithi196.stdout:1/45: chown d1/l2 0 1 2024-03-13T14:49:25.883 INFO:tasks.workunit.client.1.smithi196.stdout:3/45: dwrite db/fd [0,4194304] 0 2024-03-13T14:49:25.883 INFO:tasks.workunit.client.1.smithi196.stdout:7/84: creat d0/d6/f1d x:0 0 0 2024-03-13T14:49:25.887 INFO:tasks.workunit.client.1.smithi196.stdout:6/32: stat d6/d7/db/dc 0 2024-03-13T14:49:25.896 INFO:tasks.workunit.client.1.smithi196.stdout:1/46: getdents d1 0 2024-03-13T14:49:25.897 INFO:tasks.workunit.client.1.smithi196.stdout:6/33: mkdir d6/d7/dd 0 2024-03-13T14:49:25.901 INFO:tasks.workunit.client.1.smithi196.stdout:4/28: dwrite d1/f2 [0,4194304] 0 2024-03-13T14:49:25.910 INFO:tasks.workunit.client.1.smithi196.stdout:7/85: truncate d0/f15 3045280 0 2024-03-13T14:49:25.910 INFO:tasks.workunit.client.1.smithi196.stdout:1/47: truncate d1/d9/fd 560005 0 2024-03-13T14:49:25.910 INFO:tasks.workunit.client.1.smithi196.stdout:6/34: fdatasync f4 0 2024-03-13T14:49:25.913 INFO:tasks.workunit.client.1.smithi196.stdout:2/26: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:25.913 INFO:tasks.workunit.client.1.smithi196.stdout:2/27: readlink - no filename 2024-03-13T14:49:25.915 INFO:tasks.workunit.client.1.smithi196.stdout:4/29: getdents d1/d3 0 2024-03-13T14:49:25.916 INFO:tasks.workunit.client.1.smithi196.stdout:0/89: dwrite d8/da/fc [0,4194304] 0 2024-03-13T14:49:25.917 INFO:tasks.workunit.client.1.smithi196.stdout:1/48: mknod d1/d9/c10 0 2024-03-13T14:49:25.918 INFO:tasks.workunit.client.1.smithi196.stdout:0/90: fdatasync d8/f11 0 2024-03-13T14:49:25.919 INFO:tasks.workunit.client.1.smithi196.stdout:7/86: truncate d0/d6/f1d 332626 0 2024-03-13T14:49:25.921 INFO:tasks.workunit.client.1.smithi196.stdout:4/30: write d1/f2 [4306523,24052] 0 2024-03-13T14:49:25.921 INFO:tasks.workunit.client.1.smithi196.stdout:7/87: chown d0/f11 456 1 2024-03-13T14:49:25.923 INFO:tasks.workunit.client.1.smithi196.stdout:1/49: truncate d1/d9/fd 1346043 0 2024-03-13T14:49:25.927 INFO:tasks.workunit.client.1.smithi196.stdout:6/35: dread d6/f8 [0,4194304] 0 2024-03-13T14:49:25.927 INFO:tasks.workunit.client.1.smithi196.stdout:2/28: rmdir d0/d3 0 2024-03-13T14:49:25.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:25 smithi196 ceph-mon[37486]: pgmap v31: 65 pgs: 65 active+clean; 250 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 254 op/s 2024-03-13T14:49:25.937 INFO:tasks.workunit.client.1.smithi196.stdout:0/91: dread d8/da/f10 [0,4194304] 0 2024-03-13T14:49:25.940 INFO:tasks.workunit.client.1.smithi196.stdout:4/31: getdents d1/d3 0 2024-03-13T14:49:25.940 INFO:tasks.workunit.client.1.smithi196.stdout:0/92: chown d8/f1e 942 1 2024-03-13T14:49:25.950 INFO:tasks.workunit.client.1.smithi196.stdout:4/32: creat d1/f6 x:0 0 0 2024-03-13T14:49:25.951 INFO:tasks.workunit.client.1.smithi196.stdout:7/88: dread d0/f11 [0,4194304] 0 2024-03-13T14:49:25.956 INFO:tasks.workunit.client.1.smithi196.stdout:2/29: dread d0/f1 [0,4194304] 0 2024-03-13T14:49:25.956 INFO:tasks.workunit.client.1.smithi196.stdout:6/36: creat d6/d7/db/dc/fe x:0 0 0 2024-03-13T14:49:25.963 INFO:tasks.workunit.client.1.smithi196.stdout:2/30: unlink d0/f1 0 2024-03-13T14:49:25.965 INFO:tasks.workunit.client.1.smithi196.stdout:6/37: write d6/d7/db/dc/fe [827884,115159] 0 2024-03-13T14:49:25.966 INFO:tasks.workunit.client.1.smithi196.stdout:7/89: mkdir d0/d9/d1e 0 2024-03-13T14:49:25.970 INFO:tasks.workunit.client.1.smithi196.stdout:2/31: symlink d0/l4 0 2024-03-13T14:49:25.974 INFO:tasks.workunit.client.1.smithi196.stdout:2/32: fdatasync d0/f2 0 2024-03-13T14:49:25.976 INFO:tasks.workunit.client.1.smithi196.stdout:0/93: dwrite d8/da/fc [0,4194304] 0 2024-03-13T14:49:25.977 INFO:tasks.workunit.client.1.smithi196.stdout:6/38: mkdir d6/d7/df 0 2024-03-13T14:49:25.978 INFO:tasks.workunit.client.1.smithi196.stdout:1/50: dwrite f0 [0,4194304] 0 2024-03-13T14:49:25.978 INFO:tasks.workunit.client.1.smithi196.stdout:4/33: dwrite d1/f2 [0,4194304] 0 2024-03-13T14:49:25.979 INFO:tasks.workunit.client.1.smithi196.stdout:6/39: chown d6/d9 10860077 1 2024-03-13T14:49:25.984 INFO:tasks.workunit.client.1.smithi196.stdout:2/33: creat d0/f5 x:0 0 0 2024-03-13T14:49:25.985 INFO:tasks.workunit.client.1.smithi196.stdout:1/51: write f0 [4743251,16353] 0 2024-03-13T14:49:25.988 INFO:tasks.workunit.client.1.smithi196.stdout:4/34: write d1/f2 [2421016,117064] 0 2024-03-13T14:49:25.989 INFO:tasks.workunit.client.1.smithi196.stdout:1/52: readlink d1/l7 0 2024-03-13T14:49:25.990 INFO:tasks.workunit.client.1.smithi196.stdout:6/40: write d6/f8 [98532,122367] 0 2024-03-13T14:49:25.991 INFO:tasks.workunit.client.1.smithi196.stdout:2/34: chown d0/f2 3 1 2024-03-13T14:49:25.993 INFO:tasks.workunit.client.1.smithi196.stdout:1/53: read - d1/ff zero size 2024-03-13T14:49:25.999 INFO:tasks.workunit.client.1.smithi196.stdout:6/41: read d6/f8 [425479,86439] 0 2024-03-13T14:49:26.000 INFO:tasks.workunit.client.1.smithi196.stdout:7/90: dread d0/f11 [0,4194304] 0 2024-03-13T14:49:26.002 INFO:tasks.workunit.client.1.smithi196.stdout:6/42: read d6/fa [1016667,76431] 0 2024-03-13T14:49:26.005 INFO:tasks.workunit.client.1.smithi196.stdout:7/91: chown d0/l1 2 1 2024-03-13T14:49:26.005 INFO:tasks.workunit.client.1.smithi196.stdout:1/54: mkdir d1/d11 0 2024-03-13T14:49:26.009 INFO:tasks.workunit.client.1.smithi196.stdout:2/35: truncate d0/f2 1274435 0 2024-03-13T14:49:26.012 INFO:tasks.workunit.client.1.smithi196.stdout:1/55: unlink d1/d9/c10 0 2024-03-13T14:49:26.012 INFO:tasks.workunit.client.1.smithi196.stdout:6/43: write d6/d7/db/dc/fe [532407,13638] 0 2024-03-13T14:49:26.013 INFO:tasks.workunit.client.1.smithi196.stdout:0/94: dwrite d8/da/fc [0,4194304] 0 2024-03-13T14:49:26.017 INFO:tasks.workunit.client.1.smithi196.stdout:7/92: read d0/fa [18180,22066] 0 2024-03-13T14:49:26.020 INFO:tasks.workunit.client.1.smithi196.stdout:4/35: dwrite f0 [4194304,4194304] 0 2024-03-13T14:49:26.022 INFO:tasks.workunit.client.1.smithi196.stdout:4/36: chown d1/d3 30757 1 2024-03-13T14:49:26.025 INFO:tasks.workunit.client.1.smithi196.stdout:6/44: creat d6/f10 x:0 0 0 2024-03-13T14:49:26.028 INFO:tasks.workunit.client.1.smithi196.stdout:0/95: fdatasync d8/da/f10 0 2024-03-13T14:49:26.029 INFO:tasks.workunit.client.1.smithi196.stdout:0/96: stat c4 0 2024-03-13T14:49:26.040 INFO:tasks.workunit.client.1.smithi196.stdout:6/45: write d6/f8 [201240,94774] 0 2024-03-13T14:49:26.040 INFO:tasks.workunit.client.1.smithi196.stdout:7/93: symlink d0/l1f 0 2024-03-13T14:49:26.041 INFO:tasks.workunit.client.1.smithi196.stdout:0/97: creat d8/da/d1a/f1f x:0 0 0 2024-03-13T14:49:26.042 INFO:tasks.workunit.client.1.smithi196.stdout:6/46: chown l5 99 1 2024-03-13T14:49:26.044 INFO:tasks.workunit.client.1.smithi196.stdout:4/37: mkdir d1/d3/d7 0 2024-03-13T14:49:26.045 INFO:tasks.workunit.client.1.smithi196.stdout:0/98: write d8/da/d1a/f1f [348230,99714] 0 2024-03-13T14:49:26.045 INFO:tasks.workunit.client.1.smithi196.stdout:6/47: read f4 [3871367,62191] 0 2024-03-13T14:49:26.047 INFO:tasks.workunit.client.1.smithi196.stdout:7/94: rename d0/ff to d0/d9/f20 0 2024-03-13T14:49:26.047 INFO:tasks.workunit.client.1.smithi196.stdout:4/38: write f0 [4254825,9870] 0 2024-03-13T14:49:26.047 INFO:tasks.workunit.client.1.smithi196.stdout:1/56: link d1/c3 d1/d9/c12 0 2024-03-13T14:49:26.049 INFO:tasks.workunit.client.1.smithi196.stdout:4/39: dread - d1/f6 zero size 2024-03-13T14:49:26.050 INFO:tasks.workunit.client.1.smithi196.stdout:1/57: chown d1/ff 9517224 1 2024-03-13T14:49:26.053 INFO:tasks.workunit.client.1.smithi196.stdout:0/99: symlink d8/da/l20 0 2024-03-13T14:49:26.054 INFO:tasks.workunit.client.1.smithi196.stdout:6/48: getdents d6/d7/df 0 2024-03-13T14:49:26.055 INFO:tasks.workunit.client.1.smithi196.stdout:6/49: chown d6/d7/db 31886442 1 2024-03-13T14:49:26.056 INFO:tasks.workunit.client.1.smithi196.stdout:4/40: unlink f0 0 2024-03-13T14:49:26.057 INFO:tasks.workunit.client.1.smithi196.stdout:4/41: stat d1/d3/d7 0 2024-03-13T14:49:26.059 INFO:tasks.workunit.client.1.smithi196.stdout:1/58: mkdir d1/d11/d13 0 2024-03-13T14:49:26.060 INFO:tasks.workunit.client.1.smithi196.stdout:0/100: mknod d8/c21 0 2024-03-13T14:49:26.063 INFO:tasks.workunit.client.1.smithi196.stdout:4/42: symlink d1/l8 0 2024-03-13T14:49:26.064 INFO:tasks.workunit.client.1.smithi196.stdout:6/50: mkdir d6/d9/d11 0 2024-03-13T14:49:26.067 INFO:tasks.workunit.client.1.smithi196.stdout:6/51: write d6/f8 [1514185,57273] 0 2024-03-13T14:49:26.067 INFO:tasks.workunit.client.1.smithi196.stdout:1/59: rename d1/d9/ce to d1/d11/d13/c14 0 2024-03-13T14:49:26.069 INFO:tasks.workunit.client.1.smithi196.stdout:0/101: creat d8/da/de/f22 x:0 0 0 2024-03-13T14:49:26.069 INFO:tasks.workunit.client.1.smithi196.stdout:6/52: write f4 [4065605,69780] 0 2024-03-13T14:49:26.069 INFO:tasks.workunit.client.1.smithi196.stdout:6/53: readlink l5 0 2024-03-13T14:49:26.071 INFO:tasks.workunit.client.1.smithi196.stdout:6/54: truncate d6/f10 404584 0 2024-03-13T14:49:26.072 INFO:tasks.workunit.client.1.smithi196.stdout:7/95: fsync d0/d9/f20 0 2024-03-13T14:49:26.074 INFO:tasks.workunit.client.1.smithi196.stdout:1/60: creat d1/f15 x:0 0 0 2024-03-13T14:49:26.080 INFO:tasks.workunit.client.1.smithi196.stdout:1/61: creat d1/d11/d13/f16 x:0 0 0 2024-03-13T14:49:26.080 INFO:tasks.workunit.client.1.smithi196.stdout:6/55: symlink d6/d7/dd/l12 0 2024-03-13T14:49:26.082 INFO:tasks.workunit.client.1.smithi196.stdout:6/56: chown d6/f8 2 1 2024-03-13T14:49:26.085 INFO:tasks.workunit.client.1.smithi196.stdout:7/96: creat d0/d6/f21 x:0 0 0 2024-03-13T14:49:26.088 INFO:tasks.workunit.client.1.smithi196.stdout:6/57: getdents d6/d7/df 0 2024-03-13T14:49:26.090 INFO:tasks.workunit.client.1.smithi196.stdout:7/97: creat d0/d9/d1e/f22 x:0 0 0 2024-03-13T14:49:26.094 INFO:tasks.workunit.client.1.smithi196.stdout:6/58: creat d6/d9/d11/f13 x:0 0 0 2024-03-13T14:49:26.095 INFO:tasks.workunit.client.1.smithi196.stdout:1/62: rmdir d1/d11/d13 39 2024-03-13T14:49:26.098 INFO:tasks.workunit.client.1.smithi196.stdout:6/59: write d6/f8 [460804,128656] 0 2024-03-13T14:49:26.100 INFO:tasks.workunit.client.1.smithi196.stdout:1/63: write d1/d9/fc [349045,72477] 0 2024-03-13T14:49:26.106 INFO:tasks.workunit.client.1.smithi196.stdout:1/64: fdatasync d1/f8 0 2024-03-13T14:49:26.106 INFO:tasks.workunit.client.1.smithi196.stdout:0/102: dwrite d8/f1e [0,4194304] 0 2024-03-13T14:49:26.112 INFO:tasks.workunit.client.1.smithi196.stdout:6/60: rename d6/d7/db to d6/d7/d14 0 2024-03-13T14:49:26.114 INFO:tasks.workunit.client.1.smithi196.stdout:6/61: stat d6/d9/d11/f13 0 2024-03-13T14:49:26.115 INFO:tasks.workunit.client.1.smithi196.stdout:0/103: creat d8/da/f23 x:0 0 0 2024-03-13T14:49:26.115 INFO:tasks.workunit.client.1.smithi196.stdout:7/98: dwrite d0/f11 [0,4194304] 0 2024-03-13T14:49:26.120 INFO:tasks.workunit.client.1.smithi196.stdout:6/62: fsync d6/d7/d14/dc/fe 0 2024-03-13T14:49:26.120 INFO:tasks.workunit.client.1.smithi196.stdout:1/65: chown d1/d9/c12 79 1 2024-03-13T14:49:26.120 INFO:tasks.workunit.client.1.smithi196.stdout:1/66: chown d1/l6 558385525 1 2024-03-13T14:49:26.123 INFO:tasks.workunit.client.1.smithi196.stdout:0/104: dread d8/da/d1a/f1f [0,4194304] 0 2024-03-13T14:49:26.125 INFO:tasks.workunit.client.1.smithi196.stdout:6/63: symlink d6/d9/d11/l15 0 2024-03-13T14:49:26.126 INFO:tasks.workunit.client.1.smithi196.stdout:0/105: dread - d8/f1c zero size 2024-03-13T14:49:26.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:25 smithi176 ceph-mon[29064]: pgmap v31: 65 pgs: 65 active+clean; 250 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 254 op/s 2024-03-13T14:49:26.129 INFO:tasks.workunit.client.1.smithi196.stdout:6/64: write d6/d9/d11/f13 [292475,80976] 0 2024-03-13T14:49:26.134 INFO:tasks.workunit.client.1.smithi196.stdout:1/67: creat d1/d11/d13/f17 x:0 0 0 2024-03-13T14:49:26.135 INFO:tasks.workunit.client.1.smithi196.stdout:1/68: write f0 [4644924,17060] 0 2024-03-13T14:49:26.136 INFO:tasks.workunit.client.1.smithi196.stdout:0/106: mknod d8/da/d1a/c24 0 2024-03-13T14:49:26.137 INFO:tasks.workunit.client.1.smithi196.stdout:6/65: mknod d6/c16 0 2024-03-13T14:49:26.142 INFO:tasks.workunit.client.1.smithi196.stdout:6/66: write d6/f10 [822500,63197] 0 2024-03-13T14:49:26.144 INFO:tasks.workunit.client.1.smithi196.stdout:0/107: mkdir d8/da/d25 0 2024-03-13T14:49:26.144 INFO:tasks.workunit.client.1.smithi196.stdout:1/69: creat d1/d9/f18 x:0 0 0 2024-03-13T14:49:26.148 INFO:tasks.workunit.client.1.smithi196.stdout:1/70: fsync d1/d9/fc 0 2024-03-13T14:49:26.150 INFO:tasks.workunit.client.1.smithi196.stdout:7/99: dwrite d0/f1c [0,4194304] 0 2024-03-13T14:49:26.150 INFO:tasks.workunit.client.1.smithi196.stdout:1/71: chown d1/d11/d13 65480 1 2024-03-13T14:49:26.151 INFO:tasks.workunit.client.1.smithi196.stdout:6/67: rmdir d6/d9 39 2024-03-13T14:49:26.152 INFO:tasks.workunit.client.1.smithi196.stdout:6/68: readlink l5 0 2024-03-13T14:49:26.157 INFO:tasks.workunit.client.1.smithi196.stdout:1/72: unlink d1/f15 0 2024-03-13T14:49:26.166 INFO:tasks.workunit.client.1.smithi196.stdout:7/100: dread d0/f1c [0,4194304] 0 2024-03-13T14:49:26.172 INFO:tasks.workunit.client.1.smithi196.stdout:6/69: mkdir d6/d9/d17 0 2024-03-13T14:49:26.174 INFO:tasks.workunit.client.1.smithi196.stdout:0/108: getdents d8/da/d1a 0 2024-03-13T14:49:26.182 INFO:tasks.workunit.client.1.smithi196.stdout:0/109: mkdir d8/da/d25/d26 0 2024-03-13T14:49:26.187 INFO:tasks.workunit.client.1.smithi196.stdout:6/70: truncate d6/f10 20019 0 2024-03-13T14:49:26.188 INFO:tasks.workunit.client.1.smithi196.stdout:0/110: creat d8/da/d25/f27 x:0 0 0 2024-03-13T14:49:26.188 INFO:tasks.workunit.client.1.smithi196.stdout:1/73: dwrite d1/d9/fc [0,4194304] 0 2024-03-13T14:49:26.189 INFO:tasks.workunit.client.1.smithi196.stdout:7/101: dread d0/f1c [0,4194304] 0 2024-03-13T14:49:26.190 INFO:tasks.workunit.client.1.smithi196.stdout:0/111: dread - d8/da/de/f22 zero size 2024-03-13T14:49:26.190 INFO:tasks.workunit.client.1.smithi196.stdout:1/74: readlink d1/l7 0 2024-03-13T14:49:26.191 INFO:tasks.workunit.client.1.smithi196.stdout:0/112: stat c4 0 2024-03-13T14:49:26.194 INFO:tasks.workunit.client.1.smithi196.stdout:0/113: fdatasync d8/f11 0 2024-03-13T14:49:26.195 INFO:tasks.workunit.client.1.smithi196.stdout:6/71: truncate d6/d7/d14/dc/fe 1552030 0 2024-03-13T14:49:26.196 INFO:tasks.workunit.client.1.smithi196.stdout:1/75: creat d1/d11/d13/f19 x:0 0 0 2024-03-13T14:49:26.199 INFO:tasks.workunit.client.1.smithi196.stdout:6/72: read d6/d7/d14/dc/fe [1235965,112692] 0 2024-03-13T14:49:26.200 INFO:tasks.workunit.client.1.smithi196.stdout:1/76: unlink d1/l7 0 2024-03-13T14:49:26.201 INFO:tasks.workunit.client.1.smithi196.stdout:0/114: creat d8/f28 x:0 0 0 2024-03-13T14:49:26.204 INFO:tasks.workunit.client.1.smithi196.stdout:7/102: dread d0/f11 [0,4194304] 0 2024-03-13T14:49:26.207 INFO:tasks.workunit.client.1.smithi196.stdout:1/77: unlink d1/d9/fc 0 2024-03-13T14:49:26.210 INFO:tasks.workunit.client.1.smithi196.stdout:7/103: rmdir d0 39 2024-03-13T14:49:26.210 INFO:tasks.workunit.client.1.smithi196.stdout:1/78: dread d1/f8 [0,4194304] 0 2024-03-13T14:49:26.212 INFO:tasks.workunit.client.1.smithi196.stdout:6/73: symlink d6/d7/df/l18 0 2024-03-13T14:49:26.213 INFO:tasks.workunit.client.1.smithi196.stdout:1/79: write d1/f8 [4984,15206] 0 2024-03-13T14:49:26.215 INFO:tasks.workunit.client.1.smithi196.stdout:1/80: readlink d1/l2 0 2024-03-13T14:49:26.215 INFO:tasks.workunit.client.1.smithi196.stdout:0/115: dread d8/da/fc [0,4194304] 0 2024-03-13T14:49:26.217 INFO:tasks.workunit.client.1.smithi196.stdout:1/81: readlink d1/l2 0 2024-03-13T14:49:26.218 INFO:tasks.workunit.client.1.smithi196.stdout:7/104: creat d0/d6/f23 x:0 0 0 2024-03-13T14:49:26.226 INFO:tasks.workunit.client.1.smithi196.stdout:6/74: getdents d6/d7/d14 0 2024-03-13T14:49:26.229 INFO:tasks.workunit.client.1.smithi196.stdout:6/75: mkdir d6/d9/d17/d19 0 2024-03-13T14:49:26.236 INFO:tasks.workunit.client.1.smithi196.stdout:6/76: write d6/d7/d14/dc/fe [1644751,18888] 0 2024-03-13T14:49:26.236 INFO:tasks.workunit.client.1.smithi196.stdout:1/82: rmdir d1/d9 39 2024-03-13T14:49:26.275 INFO:tasks.workunit.client.1.smithi196.stdout:5/134: sync 2024-03-13T14:49:26.276 INFO:tasks.workunit.client.1.smithi196.stdout:3/46: sync 2024-03-13T14:49:26.276 INFO:tasks.workunit.client.1.smithi196.stdout:9/109: sync 2024-03-13T14:49:26.276 INFO:tasks.workunit.client.1.smithi196.stdout:8/42: sync 2024-03-13T14:49:26.312 INFO:tasks.workunit.client.1.smithi196.stdout:9/110: fdatasync f5 0 2024-03-13T14:49:26.312 INFO:tasks.workunit.client.1.smithi196.stdout:6/77: write d6/d7/d14/dc/fe [7697,67518] 0 2024-03-13T14:49:26.316 INFO:tasks.workunit.client.1.smithi196.stdout:3/47: read f8 [4109983,128957] 0 2024-03-13T14:49:26.316 INFO:tasks.workunit.client.1.smithi196.stdout:9/111: stat d7/d9/d17 0 2024-03-13T14:49:26.316 INFO:tasks.workunit.client.1.smithi196.stdout:8/43: rmdir d0 39 2024-03-13T14:49:26.318 INFO:tasks.workunit.client.1.smithi196.stdout:7/105: dwrite d0/d9/f20 [0,4194304] 0 2024-03-13T14:49:26.325 INFO:tasks.workunit.client.1.smithi196.stdout:1/83: getdents d1/d11/d13 0 2024-03-13T14:49:26.328 INFO:tasks.workunit.client.1.smithi196.stdout:5/135: unlink d4/db/f1d 0 2024-03-13T14:49:26.330 INFO:tasks.workunit.client.1.smithi196.stdout:6/78: mknod d6/d7/d14/c1a 0 2024-03-13T14:49:26.332 INFO:tasks.workunit.client.1.smithi196.stdout:3/48: mknod db/c11 0 2024-03-13T14:49:26.345 INFO:tasks.workunit.client.1.smithi196.stdout:5/136: symlink d4/db/d14/d19/l28 0 2024-03-13T14:49:26.347 INFO:tasks.workunit.client.1.smithi196.stdout:6/79: unlink d6/d9/d11/l15 0 2024-03-13T14:49:26.348 INFO:tasks.workunit.client.1.smithi196.stdout:6/80: fsync d6/f8 0 2024-03-13T14:49:26.349 INFO:tasks.workunit.client.1.smithi196.stdout:3/49: symlink db/l12 0 2024-03-13T14:49:26.352 INFO:tasks.workunit.client.1.smithi196.stdout:8/44: rename d0/c9 to d0/cd 0 2024-03-13T14:49:26.355 INFO:tasks.workunit.client.1.smithi196.stdout:7/106: rename d0/f15 to d0/d9/f24 0 2024-03-13T14:49:26.355 INFO:tasks.workunit.client.1.smithi196.stdout:8/45: write d0/f1 [1905292,88071] 0 2024-03-13T14:49:26.356 INFO:tasks.workunit.client.1.smithi196.stdout:6/81: dread d6/f8 [0,4194304] 0 2024-03-13T14:49:26.358 INFO:tasks.workunit.client.1.smithi196.stdout:8/46: dread - d0/fb zero size 2024-03-13T14:49:26.359 INFO:tasks.workunit.client.1.smithi196.stdout:5/137: rmdir d4/db/d14 39 2024-03-13T14:49:26.362 INFO:tasks.workunit.client.1.smithi196.stdout:9/112: dwrite d7/f12 [0,4194304] 0 2024-03-13T14:49:26.363 INFO:tasks.workunit.client.1.smithi196.stdout:3/50: unlink db/c11 0 2024-03-13T14:49:26.370 INFO:tasks.workunit.client.1.smithi196.stdout:1/84: creat d1/d11/f1a x:0 0 0 2024-03-13T14:49:26.370 INFO:tasks.workunit.client.1.smithi196.stdout:1/85: dread - d1/d11/d13/f19 zero size 2024-03-13T14:49:26.370 INFO:tasks.workunit.client.1.smithi196.stdout:8/47: mknod d0/ce 0 2024-03-13T14:49:26.377 INFO:tasks.workunit.client.1.smithi196.stdout:3/51: write f4 [148133,51122] 0 2024-03-13T14:49:26.377 INFO:tasks.workunit.client.1.smithi196.stdout:7/107: dwrite d0/f11 [0,4194304] 0 2024-03-13T14:49:26.377 INFO:tasks.workunit.client.1.smithi196.stdout:7/108: chown d0 3580 1 2024-03-13T14:49:26.379 INFO:tasks.workunit.client.1.smithi196.stdout:5/138: link d4/f11 d4/d1b/f29 0 2024-03-13T14:49:26.385 INFO:tasks.workunit.client.1.smithi196.stdout:6/82: mknod d6/c1b 0 2024-03-13T14:49:26.398 INFO:tasks.workunit.client.1.smithi196.stdout:7/109: symlink d0/d6/l25 0 2024-03-13T14:49:26.400 INFO:tasks.workunit.client.1.smithi196.stdout:8/48: dwrite d0/f8 [0,4194304] 0 2024-03-13T14:49:26.400 INFO:tasks.workunit.client.1.smithi196.stdout:6/83: mkdir d6/d7/d14/dc/d1c 0 2024-03-13T14:49:26.401 INFO:tasks.workunit.client.1.smithi196.stdout:8/49: chown d0/c6 1885767914 1 2024-03-13T14:49:26.402 INFO:tasks.workunit.client.1.smithi196.stdout:6/84: chown d6/d7/df 206 1 2024-03-13T14:49:26.404 INFO:tasks.workunit.client.1.smithi196.stdout:6/85: rename d6/d7/d14/dc to d6/d7/d14/dc/d1c/d1d 22 2024-03-13T14:49:26.408 INFO:tasks.workunit.client.1.smithi196.stdout:5/139: creat d4/db/d14/d19/d1c/d25/f2a x:0 0 0 2024-03-13T14:49:26.411 INFO:tasks.workunit.client.1.smithi196.stdout:5/140: dread - d4/db/d14/d19/d1c/f23 zero size 2024-03-13T14:49:26.414 INFO:tasks.workunit.client.1.smithi196.stdout:9/113: write d7/d9/df/f14 [5004087,23728] 0 2024-03-13T14:49:26.421 INFO:tasks.workunit.client.1.smithi196.stdout:6/86: dread d6/d7/d14/dc/fe [0,4194304] 0 2024-03-13T14:49:26.423 INFO:tasks.workunit.client.1.smithi196.stdout:7/110: creat d0/d6/f26 x:0 0 0 2024-03-13T14:49:26.425 INFO:tasks.workunit.client.1.smithi196.stdout:8/50: dwrite d0/f8 [0,4194304] 0 2024-03-13T14:49:26.425 INFO:tasks.workunit.client.1.smithi196.stdout:8/51: stat d0 0 2024-03-13T14:49:26.427 INFO:tasks.workunit.client.1.smithi196.stdout:3/52: truncate f3 3689038 0 2024-03-13T14:49:26.428 INFO:tasks.workunit.client.1.smithi196.stdout:5/141: mkdir d4/d1b/d2b 0 2024-03-13T14:49:26.430 INFO:tasks.workunit.client.1.smithi196.stdout:3/53: write db/fe [713921,110060] 0 2024-03-13T14:49:26.436 INFO:tasks.workunit.client.1.smithi196.stdout:7/111: creat d0/d9/f27 x:0 0 0 2024-03-13T14:49:26.436 INFO:tasks.workunit.client.1.smithi196.stdout:9/114: truncate f5 2466742 0 2024-03-13T14:49:26.437 INFO:tasks.workunit.client.1.smithi196.stdout:6/87: rename d6/d7/df/l18 to d6/d7/d14/dc/l1e 0 2024-03-13T14:49:26.450 INFO:tasks.workunit.client.1.smithi196.stdout:6/88: dread d6/f10 [0,4194304] 0 2024-03-13T14:49:26.450 INFO:tasks.workunit.client.1.smithi196.stdout:8/52: fsync d0/f8 0 2024-03-13T14:49:26.450 INFO:tasks.workunit.client.1.smithi196.stdout:6/89: chown d6/d9/d17/d19 101681603 1 2024-03-13T14:49:26.456 INFO:tasks.workunit.client.1.smithi196.stdout:9/115: symlink d7/d9/d17/l1b 0 2024-03-13T14:49:26.465 INFO:tasks.workunit.client.1.smithi196.stdout:3/54: dwrite f3 [0,4194304] 0 2024-03-13T14:49:26.473 INFO:tasks.workunit.client.1.smithi196.stdout:7/112: rename d0/c18 to d0/d6/c28 0 2024-03-13T14:49:26.477 INFO:tasks.workunit.client.1.smithi196.stdout:3/55: creat db/f13 x:0 0 0 2024-03-13T14:49:26.479 INFO:tasks.workunit.client.1.smithi196.stdout:8/53: link d0/l2 d0/lf 0 2024-03-13T14:49:26.480 INFO:tasks.workunit.client.1.smithi196.stdout:7/113: fdatasync d0/d6/f1d 0 2024-03-13T14:49:26.484 INFO:tasks.workunit.client.1.smithi196.stdout:7/114: stat d0/d9/f27 0 2024-03-13T14:49:26.484 INFO:tasks.workunit.client.1.smithi196.stdout:2/36: sync 2024-03-13T14:49:26.484 INFO:tasks.workunit.client.1.smithi196.stdout:4/43: sync 2024-03-13T14:49:26.484 INFO:tasks.workunit.client.1.smithi196.stdout:9/116: dwrite d7/d9/f11 [0,4194304] 0 2024-03-13T14:49:26.484 INFO:tasks.workunit.client.1.smithi196.stdout:6/90: dwrite d6/fa [0,4194304] 0 2024-03-13T14:49:26.485 INFO:tasks.workunit.client.1.smithi196.stdout:4/44: fdatasync d1/f6 0 2024-03-13T14:49:26.487 INFO:tasks.workunit.client.1.smithi196.stdout:2/37: chown d0 4 1 2024-03-13T14:49:26.488 INFO:tasks.workunit.client.1.smithi196.stdout:9/117: chown d7/l13 51647644 1 2024-03-13T14:49:26.490 INFO:tasks.workunit.client.1.smithi196.stdout:5/142: dwrite d4/f13 [0,4194304] 0 2024-03-13T14:49:26.490 INFO:tasks.workunit.client.1.smithi196.stdout:8/54: mkdir d0/d10 0 2024-03-13T14:49:26.493 INFO:tasks.workunit.client.1.smithi196.stdout:4/45: readlink d1/l5 0 2024-03-13T14:49:26.502 INFO:tasks.workunit.client.1.smithi196.stdout:7/115: creat d0/d9/d1e/f29 x:0 0 0 2024-03-13T14:49:26.503 INFO:tasks.workunit.client.1.smithi196.stdout:9/118: creat d7/d9/df/f1c x:0 0 0 2024-03-13T14:49:26.507 INFO:tasks.workunit.client.1.smithi196.stdout:5/143: unlink d4/db/d14/d19/l28 0 2024-03-13T14:49:26.515 INFO:tasks.workunit.client.1.smithi196.stdout:3/56: getdents db 0 2024-03-13T14:49:26.518 INFO:tasks.workunit.client.1.smithi196.stdout:9/119: mkdir d7/d9/df/d1d 0 2024-03-13T14:49:26.520 INFO:tasks.workunit.client.1.smithi196.stdout:4/46: link d1/f2 d1/d3/d7/f9 0 2024-03-13T14:49:26.523 INFO:tasks.workunit.client.1.smithi196.stdout:5/144: mknod d4/db/d14/d19/d26/c2c 0 2024-03-13T14:49:26.534 INFO:tasks.workunit.client.1.smithi196.stdout:2/38: write d0/f2 [1026058,57940] 0 2024-03-13T14:49:26.535 INFO:tasks.workunit.client.1.smithi196.stdout:2/39: stat d0 0 2024-03-13T14:49:26.537 INFO:tasks.workunit.client.1.smithi196.stdout:2/40: truncate d0/f5 282418 0 2024-03-13T14:49:26.538 INFO:tasks.workunit.client.1.smithi196.stdout:4/47: link d1/f6 d1/d3/fa 0 2024-03-13T14:49:26.539 INFO:tasks.workunit.client.1.smithi196.stdout:2/41: readlink d0/l4 0 2024-03-13T14:49:26.540 INFO:tasks.workunit.client.1.smithi196.stdout:2/42: stat d0/f5 0 2024-03-13T14:49:26.549 INFO:tasks.workunit.client.1.smithi196.stdout:7/116: dwrite d0/fa [0,4194304] 0 2024-03-13T14:49:26.551 INFO:tasks.workunit.client.1.smithi196.stdout:5/145: dwrite d4/db/fd [0,4194304] 0 2024-03-13T14:49:26.552 INFO:tasks.workunit.client.1.smithi196.stdout:4/48: truncate d1/f2 3331560 0 2024-03-13T14:49:26.556 INFO:tasks.workunit.client.1.smithi196.stdout:3/57: truncate f3 3454223 0 2024-03-13T14:49:26.557 INFO:tasks.workunit.client.1.smithi196.stdout:4/49: readlink d1/l8 0 2024-03-13T14:49:26.561 INFO:tasks.workunit.client.1.smithi196.stdout:7/117: chown d0/d6/l17 67 1 2024-03-13T14:49:26.567 INFO:tasks.workunit.client.1.smithi196.stdout:5/146: dread d4/db/ff [0,4194304] 0 2024-03-13T14:49:26.570 INFO:tasks.workunit.client.1.smithi196.stdout:4/50: mknod d1/d3/d7/cb 0 2024-03-13T14:49:26.572 INFO:tasks.workunit.client.1.smithi196.stdout:7/118: symlink d0/l2a 0 2024-03-13T14:49:26.579 INFO:tasks.workunit.client.1.smithi196.stdout:2/43: truncate d0/f2 1073873 0 2024-03-13T14:49:26.587 INFO:tasks.workunit.client.1.smithi196.stdout:2/44: mkdir d0/d6 0 2024-03-13T14:49:26.590 INFO:tasks.workunit.client.1.smithi196.stdout:5/147: dwrite d4/db/ff [0,4194304] 0 2024-03-13T14:49:26.593 INFO:tasks.workunit.client.1.smithi196.stdout:5/148: write d4/db/f24 [2192107,79413] 0 2024-03-13T14:49:26.593 INFO:tasks.workunit.client.1.smithi196.stdout:7/119: dread d0/d6/fc [0,4194304] 0 2024-03-13T14:49:26.595 INFO:tasks.workunit.client.1.smithi196.stdout:7/120: chown d0/d6/l25 4010 1 2024-03-13T14:49:26.598 INFO:tasks.workunit.client.1.smithi196.stdout:5/149: write d4/d1b/f29 [638546,102040] 0 2024-03-13T14:49:26.600 INFO:tasks.workunit.client.1.smithi196.stdout:5/150: fdatasync d4/f8 0 2024-03-13T14:49:26.601 INFO:tasks.workunit.client.1.smithi196.stdout:7/121: symlink d0/d9/d1e/l2b 0 2024-03-13T14:49:26.604 INFO:tasks.workunit.client.1.smithi196.stdout:2/45: write d0/f2 [1932922,57309] 0 2024-03-13T14:49:26.618 INFO:tasks.workunit.client.1.smithi196.stdout:5/151: dread d4/db/ff [0,4194304] 0 2024-03-13T14:49:26.620 INFO:tasks.workunit.client.1.smithi196.stdout:2/46: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:26.622 INFO:tasks.workunit.client.1.smithi196.stdout:2/47: stat d0/f5 0 2024-03-13T14:49:26.625 INFO:tasks.workunit.client.1.smithi196.stdout:5/152: mkdir d4/db/d14/d19/d1c/d2d 0 2024-03-13T14:49:26.625 INFO:tasks.workunit.client.1.smithi196.stdout:2/48: write d0/f5 [453920,13046] 0 2024-03-13T14:49:26.627 INFO:tasks.workunit.client.1.smithi196.stdout:5/153: read d4/d1b/f29 [661948,11583] 0 2024-03-13T14:49:26.629 INFO:tasks.workunit.client.1.smithi196.stdout:7/122: dwrite d0/d9/f27 [0,4194304] 0 2024-03-13T14:49:26.631 INFO:tasks.workunit.client.1.smithi196.stdout:7/123: write d0/d9/f20 [3656342,3606] 0 2024-03-13T14:49:26.633 INFO:tasks.workunit.client.1.smithi196.stdout:2/49: dread d0/f5 [0,4194304] 0 2024-03-13T14:49:26.637 INFO:tasks.workunit.client.1.smithi196.stdout:0/116: sync 2024-03-13T14:49:26.637 INFO:tasks.workunit.client.1.smithi196.stdout:5/154: mknod d4/d1b/c2e 0 2024-03-13T14:49:26.637 INFO:tasks.workunit.client.1.smithi196.stdout:1/86: sync 2024-03-13T14:49:26.643 INFO:tasks.workunit.client.1.smithi196.stdout:2/50: rmdir d0 39 2024-03-13T14:49:26.644 INFO:tasks.workunit.client.1.smithi196.stdout:4/51: dread d1/f2 [0,4194304] 0 2024-03-13T14:49:26.646 INFO:tasks.workunit.client.1.smithi196.stdout:7/124: creat d0/d6/f2c x:0 0 0 2024-03-13T14:49:26.649 INFO:tasks.workunit.client.1.smithi196.stdout:4/52: unlink d1/l4 0 2024-03-13T14:49:26.650 INFO:tasks.workunit.client.1.smithi196.stdout:4/53: dread - d1/f6 zero size 2024-03-13T14:49:26.652 INFO:tasks.workunit.client.1.smithi196.stdout:5/155: dread d4/db/f24 [0,4194304] 0 2024-03-13T14:49:26.652 INFO:tasks.workunit.client.1.smithi196.stdout:4/54: chown d1/d3/d7/f9 2808798 1 2024-03-13T14:49:26.652 INFO:tasks.workunit.client.1.smithi196.stdout:7/125: mkdir d0/d9/d1e/d2d 0 2024-03-13T14:49:26.659 INFO:tasks.workunit.client.1.smithi196.stdout:4/55: unlink d1/d3/d7/f9 0 2024-03-13T14:49:26.661 INFO:tasks.workunit.client.1.smithi196.stdout:5/156: symlink d4/db/d14/d19/d1c/d25/l2f 0 2024-03-13T14:49:26.662 INFO:tasks.workunit.client.1.smithi196.stdout:4/56: dread - d1/d3/fa zero size 2024-03-13T14:49:26.663 INFO:tasks.workunit.client.1.smithi196.stdout:7/126: symlink d0/d6/l2e 0 2024-03-13T14:49:26.666 INFO:tasks.workunit.client.1.smithi196.stdout:5/157: write d4/db/ff [5166056,30282] 0 2024-03-13T14:49:26.668 INFO:tasks.workunit.client.1.smithi196.stdout:2/51: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:26.672 INFO:tasks.workunit.client.1.smithi196.stdout:7/127: symlink d0/l2f 0 2024-03-13T14:49:26.677 INFO:tasks.workunit.client.1.smithi196.stdout:5/158: rename d4/db/f1a to d4/d1b/d2b/f30 0 2024-03-13T14:49:26.689 INFO:tasks.workunit.client.1.smithi196.stdout:7/128: dread d0/d9/f24 [0,4194304] 0 2024-03-13T14:49:26.691 INFO:tasks.workunit.client.1.smithi196.stdout:5/159: dread d4/d1b/d2b/f30 [0,4194304] 0 2024-03-13T14:49:26.692 INFO:tasks.workunit.client.1.smithi196.stdout:7/129: chown d0/d6/f1d 4975280 1 2024-03-13T14:49:26.692 INFO:tasks.workunit.client.1.smithi196.stdout:2/52: dwrite d0/f5 [0,4194304] 0 2024-03-13T14:49:26.699 INFO:tasks.workunit.client.1.smithi196.stdout:7/130: creat d0/d9/f30 x:0 0 0 2024-03-13T14:49:26.704 INFO:tasks.workunit.client.1.smithi196.stdout:7/131: unlink d0/d9/d1e/f22 0 2024-03-13T14:49:26.708 INFO:tasks.workunit.client.1.smithi196.stdout:7/132: mkdir d0/d9/d31 0 2024-03-13T14:49:26.710 INFO:tasks.workunit.client.1.smithi196.stdout:2/53: dread d0/f5 [0,4194304] 0 2024-03-13T14:49:26.711 INFO:tasks.workunit.client.1.smithi196.stdout:5/160: dwrite d4/f11 [0,4194304] 0 2024-03-13T14:49:26.711 INFO:tasks.workunit.client.1.smithi196.stdout:7/133: write d0/d6/f23 [781929,124439] 0 2024-03-13T14:49:26.712 INFO:tasks.workunit.client.1.smithi196.stdout:2/54: readlink d0/l4 0 2024-03-13T14:49:26.712 INFO:tasks.workunit.client.1.smithi196.stdout:5/161: readlink d4/l18 0 2024-03-13T14:49:26.715 INFO:tasks.workunit.client.1.smithi196.stdout:5/162: mknod d4/d1b/c31 0 2024-03-13T14:49:26.716 INFO:tasks.workunit.client.1.smithi196.stdout:2/55: mkdir d0/d7 0 2024-03-13T14:49:26.717 INFO:tasks.workunit.client.1.smithi196.stdout:7/134: mknod d0/d9/d1e/d2d/c32 0 2024-03-13T14:49:26.720 INFO:tasks.workunit.client.1.smithi196.stdout:7/135: chown d0/f1c 1581 1 2024-03-13T14:49:26.733 INFO:tasks.workunit.client.1.smithi196.stdout:2/56: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:26.756 INFO:tasks.workunit.client.1.smithi196.stdout:5/163: mknod d4/db/d14/d19/d1c/d25/c32 0 2024-03-13T14:49:26.762 INFO:tasks.workunit.client.1.smithi196.stdout:2/57: fsync d0/f5 0 2024-03-13T14:49:26.764 INFO:tasks.workunit.client.1.smithi196.stdout:5/164: mkdir d4/db/d14/d19/d26/d33 0 2024-03-13T14:49:26.766 INFO:tasks.workunit.client.1.smithi196.stdout:5/165: dread - d4/db/d14/d19/d1c/f23 zero size 2024-03-13T14:49:26.767 INFO:tasks.workunit.client.1.smithi196.stdout:7/136: mkdir d0/d6/d33 0 2024-03-13T14:49:26.771 INFO:tasks.workunit.client.1.smithi196.stdout:5/166: fsync d4/d1b/d2b/f30 0 2024-03-13T14:49:26.773 INFO:tasks.workunit.client.1.smithi196.stdout:2/58: link d0/f5 d0/d6/f8 0 2024-03-13T14:49:26.779 INFO:tasks.workunit.client.1.smithi196.stdout:2/59: unlink d0/f5 0 2024-03-13T14:49:26.782 INFO:tasks.workunit.client.1.smithi196.stdout:5/167: rename d4/db/d14/d19/d1c/d25/l2f to d4/db/d14/d19/l34 0 2024-03-13T14:49:26.801 INFO:tasks.workunit.client.1.smithi196.stdout:6/91: sync 2024-03-13T14:49:26.843 INFO:tasks.workunit.client.1.smithi196.stdout:2/60: mkdir d0/d7/d9 0 2024-03-13T14:49:26.844 INFO:tasks.workunit.client.1.smithi196.stdout:5/168: symlink d4/db/d14/d19/l35 0 2024-03-13T14:49:26.847 INFO:tasks.workunit.client.1.smithi196.stdout:7/137: dwrite d0/d6/f26 [0,4194304] 0 2024-03-13T14:49:26.847 INFO:tasks.workunit.client.1.smithi196.stdout:6/92: chown d6/d7/d14/c1a 3 1 2024-03-13T14:49:26.848 INFO:tasks.workunit.client.1.smithi196.stdout:5/169: creat d4/db/d14/d19/d1c/d25/f36 x:0 0 0 2024-03-13T14:49:26.852 INFO:tasks.workunit.client.1.smithi196.stdout:7/138: creat d0/d9/f34 x:0 0 0 2024-03-13T14:49:26.913 INFO:tasks.workunit.client.1.smithi196.stdout:5/170: getdents d4/d1b 0 2024-03-13T14:49:26.914 INFO:tasks.workunit.client.1.smithi196.stdout:7/139: mkdir d0/d6/d35 0 2024-03-13T14:49:26.916 INFO:tasks.workunit.client.1.smithi196.stdout:2/61: fsync d0/d6/f8 0 2024-03-13T14:49:26.927 INFO:tasks.workunit.client.1.smithi196.stdout:6/93: dwrite d6/d9/d11/f13 [0,4194304] 0 2024-03-13T14:49:26.929 INFO:tasks.workunit.client.1.smithi196.stdout:6/94: fdatasync d6/fa 0 2024-03-13T14:49:26.945 INFO:tasks.workunit.client.1.smithi196.stdout:7/140: dwrite d0/f13 [0,4194304] 0 2024-03-13T14:49:26.952 INFO:tasks.workunit.client.1.smithi196.stdout:5/171: dwrite d4/db/d14/d19/d1c/d25/f2a [0,4194304] 0 2024-03-13T14:49:26.954 INFO:tasks.workunit.client.1.smithi196.stdout:7/141: write d0/d6/f2c [194912,91613] 0 2024-03-13T14:49:26.960 INFO:tasks.workunit.client.1.smithi196.stdout:6/95: dwrite d6/f8 [0,4194304] 0 2024-03-13T14:49:26.962 INFO:tasks.workunit.client.1.smithi196.stdout:7/142: dread - d0/d9/d1e/f29 zero size 2024-03-13T14:49:26.962 INFO:tasks.workunit.client.1.smithi196.stdout:5/172: dread - d4/db/d14/d19/d1c/d25/f36 zero size 2024-03-13T14:49:26.965 INFO:tasks.workunit.client.1.smithi196.stdout:7/143: fdatasync d0/f1c 0 2024-03-13T14:49:26.966 INFO:tasks.workunit.client.1.smithi196.stdout:2/62: dread d0/d6/f8 [0,4194304] 0 2024-03-13T14:49:26.968 INFO:tasks.workunit.client.1.smithi196.stdout:7/144: stat d0/d9/f20 0 2024-03-13T14:49:26.968 INFO:tasks.workunit.client.1.smithi196.stdout:5/173: mkdir d4/db/d14/d37 0 2024-03-13T14:49:26.970 INFO:tasks.workunit.client.1.smithi196.stdout:5/174: chown d4/db/d14/d19/d26 2277398 1 2024-03-13T14:49:26.977 INFO:tasks.workunit.client.1.smithi196.stdout:7/145: truncate d0/f1c 1866457 0 2024-03-13T14:49:26.989 INFO:tasks.workunit.client.1.smithi196.stdout:7/146: link d0/d9/lb d0/d6/l36 0 2024-03-13T14:49:26.991 INFO:tasks.workunit.client.1.smithi196.stdout:6/96: dwrite d6/d7/d14/dc/fe [0,4194304] 0 2024-03-13T14:49:26.992 INFO:tasks.workunit.client.1.smithi196.stdout:2/63: dread d0/d6/f8 [0,4194304] 0 2024-03-13T14:49:26.999 INFO:tasks.workunit.client.1.smithi196.stdout:3/58: sync 2024-03-13T14:49:27.000 INFO:tasks.workunit.client.1.smithi196.stdout:8/55: sync 2024-03-13T14:49:27.005 INFO:tasks.workunit.client.1.smithi196.stdout:5/175: dwrite d4/db/d14/d19/d1c/f23 [0,4194304] 0 2024-03-13T14:49:27.019 INFO:tasks.workunit.client.1.smithi196.stdout:3/59: dread f3 [0,4194304] 0 2024-03-13T14:49:27.021 INFO:tasks.workunit.client.1.smithi196.stdout:8/56: creat d0/d10/f11 x:0 0 0 2024-03-13T14:49:27.023 INFO:tasks.workunit.client.1.smithi196.stdout:6/97: fsync d6/f8 0 2024-03-13T14:49:27.026 INFO:tasks.workunit.client.1.smithi196.stdout:6/98: read d6/d7/d14/dc/fe [746305,55691] 0 2024-03-13T14:49:27.033 INFO:tasks.workunit.client.1.smithi196.stdout:6/99: read d6/d7/d14/dc/fe [4158008,101325] 0 2024-03-13T14:49:27.034 INFO:tasks.workunit.client.1.smithi196.stdout:2/64: write d0/d6/f8 [2517617,5104] 0 2024-03-13T14:49:27.039 INFO:tasks.workunit.client.1.smithi196.stdout:6/100: fsync d6/f10 0 2024-03-13T14:49:27.045 INFO:tasks.workunit.client.1.smithi196.stdout:2/65: creat d0/d7/d9/fa x:0 0 0 2024-03-13T14:49:27.047 INFO:tasks.workunit.client.1.smithi196.stdout:6/101: creat d6/d9/f1f x:0 0 0 2024-03-13T14:49:27.047 INFO:tasks.workunit.client.1.smithi196.stdout:3/60: dwrite f7 [0,4194304] 0 2024-03-13T14:49:27.055 INFO:tasks.workunit.client.1.smithi196.stdout:2/66: unlink d0/d7/d9/fa 0 2024-03-13T14:49:27.055 INFO:tasks.workunit.client.1.smithi196.stdout:3/61: mknod db/c14 0 2024-03-13T14:49:27.057 INFO:tasks.workunit.client.1.smithi196.stdout:7/147: dwrite d0/f1c [0,4194304] 0 2024-03-13T14:49:27.057 INFO:tasks.workunit.client.1.smithi196.stdout:5/176: dwrite d4/f7 [0,4194304] 0 2024-03-13T14:49:27.058 INFO:tasks.workunit.client.1.smithi196.stdout:2/67: readlink d0/l4 0 2024-03-13T14:49:27.062 INFO:tasks.workunit.client.1.smithi196.stdout:3/62: read f7 [177621,83478] 0 2024-03-13T14:49:27.067 INFO:tasks.workunit.client.1.smithi196.stdout:6/102: rename d6/d7/d14/dc/fe to d6/d9/d17/d19/f20 0 2024-03-13T14:49:27.067 INFO:tasks.workunit.client.1.smithi196.stdout:5/177: mkdir d4/db/d14/d19/d1c/d25/d38 0 2024-03-13T14:49:27.069 INFO:tasks.workunit.client.1.smithi196.stdout:3/63: mkdir db/d15 0 2024-03-13T14:49:27.070 INFO:tasks.workunit.client.1.smithi196.stdout:6/103: chown d6/d7/dd/l12 12 1 2024-03-13T14:49:27.072 INFO:tasks.workunit.client.1.smithi196.stdout:3/64: symlink db/l16 0 2024-03-13T14:49:27.074 INFO:tasks.workunit.client.1.smithi196.stdout:7/148: creat d0/f37 x:0 0 0 2024-03-13T14:49:27.075 INFO:tasks.workunit.client.1.smithi196.stdout:5/178: mknod d4/db/c39 0 2024-03-13T14:49:27.075 INFO:tasks.workunit.client.1.smithi196.stdout:3/65: write db/ff [47282,12519] 0 2024-03-13T14:49:27.078 INFO:tasks.workunit.client.1.smithi196.stdout:5/179: fsync d4/f8 0 2024-03-13T14:49:27.078 INFO:tasks.workunit.client.1.smithi196.stdout:7/149: write d0/d6/f19 [826030,52283] 0 2024-03-13T14:49:27.078 INFO:tasks.workunit.client.1.smithi196.stdout:6/104: symlink d6/d7/d14/dc/l21 0 2024-03-13T14:49:27.079 INFO:tasks.workunit.client.1.smithi196.stdout:7/150: readlink d0/l2f 0 2024-03-13T14:49:27.083 INFO:tasks.workunit.client.1.smithi196.stdout:6/105: creat d6/d9/d17/d19/f22 x:0 0 0 2024-03-13T14:49:27.084 INFO:tasks.workunit.client.1.smithi196.stdout:6/106: fdatasync f4 0 2024-03-13T14:49:27.095 INFO:tasks.workunit.client.1.smithi196.stdout:7/151: getdents d0 0 2024-03-13T14:49:27.096 INFO:tasks.workunit.client.1.smithi196.stdout:3/66: write f4 [1088441,47269] 0 2024-03-13T14:49:27.098 INFO:tasks.workunit.client.1.smithi196.stdout:6/107: dread d6/d9/d17/d19/f20 [0,4194304] 0 2024-03-13T14:49:27.100 INFO:tasks.workunit.client.1.smithi196.stdout:3/67: creat db/d15/f17 x:0 0 0 2024-03-13T14:49:27.101 INFO:tasks.workunit.client.1.smithi196.stdout:6/108: rmdir d6/d7/d14/dc 39 2024-03-13T14:49:27.102 INFO:tasks.workunit.client.1.smithi196.stdout:6/109: chown d6/f10 1114599 1 2024-03-13T14:49:27.103 INFO:tasks.workunit.client.1.smithi196.stdout:3/68: mknod db/c18 0 2024-03-13T14:49:27.106 INFO:tasks.workunit.client.1.smithi196.stdout:6/110: readlink d6/d7/d14/dc/l21 0 2024-03-13T14:49:27.111 INFO:tasks.workunit.client.1.smithi196.stdout:6/111: mkdir d6/d7/d14/d23 0 2024-03-13T14:49:27.114 INFO:tasks.workunit.client.1.smithi196.stdout:6/112: dread d6/f10 [0,4194304] 0 2024-03-13T14:49:27.116 INFO:tasks.workunit.client.1.smithi196.stdout:7/152: dwrite d0/d6/f23 [0,4194304] 0 2024-03-13T14:49:27.118 INFO:tasks.workunit.client.1.smithi196.stdout:6/113: truncate d6/d9/d17/d19/f22 504680 0 2024-03-13T14:49:27.119 INFO:tasks.workunit.client.1.smithi196.stdout:3/69: dread f7 [0,4194304] 0 2024-03-13T14:49:27.119 INFO:tasks.workunit.client.1.smithi196.stdout:7/153: mknod d0/d6/d35/c38 0 2024-03-13T14:49:27.121 INFO:tasks.workunit.client.1.smithi196.stdout:3/70: write db/fd [2305443,97691] 0 2024-03-13T14:49:27.125 INFO:tasks.workunit.client.1.smithi196.stdout:6/114: creat d6/d7/d14/f24 x:0 0 0 2024-03-13T14:49:27.125 INFO:tasks.workunit.client.1.smithi196.stdout:7/154: getdents d0/d9/d31 0 2024-03-13T14:49:27.127 INFO:tasks.workunit.client.1.smithi196.stdout:6/115: unlink d6/d7/d14/f24 0 2024-03-13T14:49:27.129 INFO:tasks.workunit.client.1.smithi196.stdout:7/155: creat d0/d6/d33/f39 x:0 0 0 2024-03-13T14:49:27.130 INFO:tasks.workunit.client.1.smithi196.stdout:6/116: creat d6/d7/d14/dc/d1c/f25 x:0 0 0 2024-03-13T14:49:27.136 INFO:tasks.workunit.client.1.smithi196.stdout:7/156: dread d0/d9/f24 [0,4194304] 0 2024-03-13T14:49:27.138 INFO:tasks.workunit.client.1.smithi196.stdout:7/157: readlink d0/d9/d1e/l2b 0 2024-03-13T14:49:27.140 INFO:tasks.workunit.client.1.smithi196.stdout:4/57: sync 2024-03-13T14:49:27.140 INFO:tasks.workunit.client.1.smithi196.stdout:0/117: sync 2024-03-13T14:49:27.140 INFO:tasks.workunit.client.1.smithi196.stdout:9/120: sync 2024-03-13T14:49:27.140 INFO:tasks.workunit.client.1.smithi196.stdout:1/87: sync 2024-03-13T14:49:27.141 INFO:tasks.workunit.client.1.smithi196.stdout:7/158: mknod d0/c3a 0 2024-03-13T14:49:27.150 INFO:tasks.workunit.client.1.smithi196.stdout:9/121: write d7/fd [4068348,19938] 0 2024-03-13T14:49:27.154 INFO:tasks.workunit.client.1.smithi196.stdout:0/118: unlink d8/da/f23 0 2024-03-13T14:49:27.156 INFO:tasks.workunit.client.1.smithi196.stdout:4/58: mknod d1/d3/d7/cc 0 2024-03-13T14:49:27.156 INFO:tasks.workunit.client.1.smithi196.stdout:6/117: dwrite d6/fa [0,4194304] 0 2024-03-13T14:49:27.162 INFO:tasks.workunit.client.1.smithi196.stdout:0/119: symlink d8/da/d1a/l29 0 2024-03-13T14:49:27.164 INFO:tasks.workunit.client.1.smithi196.stdout:4/59: creat d1/d3/d7/fd x:0 0 0 2024-03-13T14:49:27.165 INFO:tasks.workunit.client.1.smithi196.stdout:1/88: creat d1/d11/f1b x:0 0 0 2024-03-13T14:49:27.170 INFO:tasks.workunit.client.1.smithi196.stdout:6/118: symlink d6/d7/d14/d23/l26 0 2024-03-13T14:49:27.171 INFO:tasks.workunit.client.1.smithi196.stdout:7/159: dwrite d0/d6/f26 [0,4194304] 0 2024-03-13T14:49:27.171 INFO:tasks.workunit.client.1.smithi196.stdout:1/89: unlink d1/ff 0 2024-03-13T14:49:27.177 INFO:tasks.workunit.client.0.smithi176.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-03-13T14:49:27.180 INFO:tasks.workunit.client.1.smithi196.stdout:4/60: mknod d1/ce 0 2024-03-13T14:49:27.181 INFO:tasks.workunit.client.1.smithi196.stdout:6/119: mkdir d6/d7/df/d27 0 2024-03-13T14:49:27.182 INFO:tasks.workunit.client.1.smithi196.stdout:4/61: dread - d1/d3/fa zero size 2024-03-13T14:49:27.186 INFO:tasks.workunit.client.1.smithi196.stdout:1/90: mknod d1/d11/d13/c1c 0 2024-03-13T14:49:27.186 INFO:tasks.workunit.client.1.smithi196.stdout:1/91: fsync d1/f5 0 2024-03-13T14:49:27.187 INFO:tasks.workunit.client.1.smithi196.stdout:6/120: mkdir d6/d7/d14/d23/d28 0 2024-03-13T14:49:27.187 INFO:tasks.workunit.client.1.smithi196.stdout:0/120: link d8/da/de/c1d d8/da/de/d17/c2a 0 2024-03-13T14:49:27.189 INFO:tasks.workunit.client.1.smithi196.stdout:6/121: fdatasync d6/f8 0 2024-03-13T14:49:27.192 INFO:tasks.workunit.client.1.smithi196.stdout:0/121: readlink d8/da/d1a/l29 0 2024-03-13T14:49:27.193 INFO:tasks.workunit.client.1.smithi196.stdout:1/92: creat d1/d11/f1d x:0 0 0 2024-03-13T14:49:27.195 INFO:tasks.workunit.client.1.smithi196.stdout:7/160: dwrite d0/d6/f21 [0,4194304] 0 2024-03-13T14:49:27.196 INFO:tasks.workunit.client.1.smithi196.stdout:0/122: read d8/da/d1a/f1f [238225,9998] 0 2024-03-13T14:49:27.196 INFO:tasks.workunit.client.1.smithi196.stdout:6/122: creat d6/d9/d17/d19/f29 x:0 0 0 2024-03-13T14:49:27.197 INFO:tasks.workunit.client.0.smithi176.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-13T14:49:27.198 INFO:tasks.workunit.client.0.smithi176.stderr:+ make 2024-03-13T14:49:27.203 INFO:tasks.workunit.client.1.smithi196.stdout:1/93: rename d1/f8 to d1/d11/d13/f1e 0 2024-03-13T14:49:27.212 INFO:tasks.workunit.client.1.smithi196.stdout:7/161: creat d0/d9/d1e/d2d/f3b x:0 0 0 2024-03-13T14:49:27.212 INFO:tasks.workunit.client.1.smithi196.stdout:1/94: creat d1/d11/d13/f1f x:0 0 0 2024-03-13T14:49:27.213 INFO:tasks.workunit.client.1.smithi196.stdout:0/123: dread d8/f1e [0,4194304] 0 2024-03-13T14:49:27.218 INFO:tasks.workunit.client.1.smithi196.stdout:6/123: getdents d6/d7/d14/dc/d1c 0 2024-03-13T14:49:27.219 INFO:tasks.workunit.client.1.smithi196.stdout:7/162: write d0/f37 [227825,127669] 0 2024-03-13T14:49:27.220 INFO:tasks.workunit.client.1.smithi196.stdout:1/95: write d1/d11/d13/f1f [184042,77911] 0 2024-03-13T14:49:27.221 INFO:tasks.workunit.client.1.smithi196.stdout:0/124: creat d8/da/de/f2b x:0 0 0 2024-03-13T14:49:27.228 INFO:tasks.workunit.client.1.smithi196.stdout:1/96: write d1/d11/f1d [21829,12421] 0 2024-03-13T14:49:27.228 INFO:tasks.workunit.client.1.smithi196.stdout:7/163: creat d0/d6/d33/f3c x:0 0 0 2024-03-13T14:49:27.230 INFO:tasks.workunit.client.1.smithi196.stdout:0/125: unlink d8/f1c 0 2024-03-13T14:49:27.234 INFO:tasks.workunit.client.1.smithi196.stdout:4/62: dwrite d1/f6 [0,4194304] 0 2024-03-13T14:49:27.236 INFO:tasks.workunit.client.1.smithi196.stdout:1/97: write d1/d9/f18 [327520,117639] 0 2024-03-13T14:49:27.240 INFO:tasks.workunit.client.1.smithi196.stdout:7/164: rename d0/d9/f34 to d0/d9/d1e/d2d/f3d 0 2024-03-13T14:49:27.240 INFO:tasks.workunit.client.1.smithi196.stdout:0/126: mknod d8/da/de/d17/c2c 0 2024-03-13T14:49:27.241 INFO:tasks.workunit.client.1.smithi196.stdout:4/63: mknod d1/cf 0 2024-03-13T14:49:27.247 INFO:tasks.workunit.client.1.smithi196.stdout:1/98: link d1/d11/d13/f19 d1/d9/f20 0 2024-03-13T14:49:27.249 INFO:tasks.workunit.client.1.smithi196.stdout:7/165: truncate d0/d6/d33/f39 467111 0 2024-03-13T14:49:27.250 INFO:tasks.workunit.client.1.smithi196.stdout:0/127: creat d8/da/d25/d26/f2d x:0 0 0 2024-03-13T14:49:27.250 INFO:tasks.workunit.client.1.smithi196.stdout:6/124: dwrite d6/d9/d17/d19/f22 [0,4194304] 0 2024-03-13T14:49:27.255 INFO:tasks.workunit.client.1.smithi196.stdout:0/128: write d8/da/d25/f27 [1044954,89929] 0 2024-03-13T14:49:27.258 INFO:tasks.workunit.client.1.smithi196.stdout:0/129: chown d8/da/f13 1 1 2024-03-13T14:49:27.264 INFO:tasks.workunit.client.1.smithi196.stdout:4/64: link d1/ce d1/d3/d7/c10 0 2024-03-13T14:49:27.267 INFO:tasks.workunit.client.1.smithi196.stdout:4/65: mknod d1/d3/c11 0 2024-03-13T14:49:27.276 INFO:tasks.workunit.client.1.smithi196.stdout:7/166: dread d0/d6/f21 [0,4194304] 0 2024-03-13T14:49:27.276 INFO:tasks.workunit.client.1.smithi196.stdout:4/66: chown d1/l8 31 1 2024-03-13T14:49:27.279 INFO:tasks.workunit.client.1.smithi196.stdout:8/57: sync 2024-03-13T14:49:27.286 INFO:tasks.workunit.client.1.smithi196.stdout:6/125: dread d6/d9/d17/d19/f20 [0,4194304] 0 2024-03-13T14:49:27.286 INFO:tasks.workunit.client.1.smithi196.stdout:4/67: chown d1/d3/d7/cc 13759531 1 2024-03-13T14:49:27.289 INFO:tasks.workunit.client.1.smithi196.stdout:8/58: mkdir d0/d10/d12 0 2024-03-13T14:49:27.291 INFO:tasks.workunit.client.1.smithi196.stdout:7/167: getdents d0/d9/d31 0 2024-03-13T14:49:27.291 INFO:tasks.workunit.client.1.smithi196.stdout:4/68: rmdir d1 39 2024-03-13T14:49:27.296 INFO:tasks.workunit.client.1.smithi196.stdout:3/71: fdatasync f4 0 2024-03-13T14:49:27.297 INFO:tasks.workunit.client.1.smithi196.stdout:1/99: dwrite d1/d11/d13/f19 [0,4194304] 0 2024-03-13T14:49:27.298 INFO:tasks.workunit.client.0.smithi176.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-03-13T14:49:27.298 INFO:tasks.workunit.client.1.smithi196.stdout:8/59: symlink d0/l13 0 2024-03-13T14:49:27.298 INFO:tasks.workunit.client.1.smithi196.stdout:7/168: chown d0 20431864 1 2024-03-13T14:49:27.301 INFO:tasks.workunit.client.1.smithi196.stdout:0/130: dwrite d8/da/d25/f27 [0,4194304] 0 2024-03-13T14:49:27.301 INFO:tasks.workunit.client.1.smithi196.stdout:2/68: sync 2024-03-13T14:49:27.301 INFO:tasks.workunit.client.1.smithi196.stdout:5/180: sync 2024-03-13T14:49:27.303 INFO:tasks.workunit.client.1.smithi196.stdout:9/122: dread d7/f15 [0,4194304] 0 2024-03-13T14:49:27.305 INFO:tasks.workunit.client.1.smithi196.stdout:6/126: unlink d6/d9/d17/d19/f29 0 2024-03-13T14:49:27.306 INFO:tasks.workunit.client.1.smithi196.stdout:4/69: creat d1/d3/f12 x:0 0 0 2024-03-13T14:49:27.312 INFO:tasks.workunit.client.1.smithi196.stdout:8/60: truncate d0/d10/f11 966247 0 2024-03-13T14:49:27.313 INFO:tasks.workunit.client.1.smithi196.stdout:9/123: write d7/d9/df/f14 [5323824,1500] 0 2024-03-13T14:49:27.315 INFO:tasks.workunit.client.1.smithi196.stdout:3/72: mkdir db/d19 0 2024-03-13T14:49:27.318 INFO:tasks.workunit.client.1.smithi196.stdout:7/169: dread d0/d6/d33/f39 [0,4194304] 0 2024-03-13T14:49:27.319 INFO:tasks.workunit.client.1.smithi196.stdout:0/131: rmdir d8/da/de 39 2024-03-13T14:49:27.319 INFO:tasks.workunit.client.1.smithi196.stdout:4/70: creat d1/d3/f13 x:0 0 0 2024-03-13T14:49:27.322 INFO:tasks.workunit.client.1.smithi196.stdout:3/73: read - db/f13 zero size 2024-03-13T14:49:27.322 INFO:tasks.workunit.client.1.smithi196.stdout:9/124: mknod d7/c1e 0 2024-03-13T14:49:27.323 INFO:tasks.workunit.client.1.smithi196.stdout:8/61: mkdir d0/d10/d14 0 2024-03-13T14:49:27.331 INFO:tasks.workunit.client.1.smithi196.stdout:9/125: write d7/d9/df/f14 [5274694,116397] 0 2024-03-13T14:49:27.332 INFO:tasks.workunit.client.1.smithi196.stdout:0/132: dread d8/f12 [0,4194304] 0 2024-03-13T14:49:27.344 INFO:tasks.workunit.client.1.smithi196.stdout:8/62: rename d0/fb to d0/d10/f15 0 2024-03-13T14:49:27.346 INFO:tasks.workunit.client.1.smithi196.stdout:7/170: symlink d0/d6/l3e 0 2024-03-13T14:49:27.346 INFO:tasks.workunit.client.1.smithi196.stdout:6/127: getdents d6/d7/d14/dc 0 2024-03-13T14:49:27.351 INFO:tasks.workunit.client.1.smithi196.stdout:6/128: fdatasync d6/f8 0 2024-03-13T14:49:27.351 INFO:tasks.workunit.client.1.smithi196.stdout:8/63: fdatasync d0/d10/f11 0 2024-03-13T14:49:27.353 INFO:tasks.workunit.client.1.smithi196.stdout:3/74: dread f7 [0,4194304] 0 2024-03-13T14:49:27.361 INFO:tasks.workunit.client.1.smithi196.stdout:8/64: write d0/d10/f11 [48146,28187] 0 2024-03-13T14:49:27.362 INFO:tasks.workunit.client.1.smithi196.stdout:1/100: dwrite d1/d11/d13/f1e [0,4194304] 0 2024-03-13T14:49:27.364 INFO:tasks.workunit.client.1.smithi196.stdout:4/71: dread d1/f2 [0,4194304] 0 2024-03-13T14:49:27.367 INFO:tasks.workunit.client.1.smithi196.stdout:0/133: dread d8/da/d25/f27 [0,4194304] 0 2024-03-13T14:49:27.370 INFO:tasks.workunit.client.1.smithi196.stdout:2/69: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:27.370 INFO:tasks.workunit.client.1.smithi196.stdout:1/101: readlink d1/l6 0 2024-03-13T14:49:27.374 INFO:tasks.workunit.client.1.smithi196.stdout:8/65: write d0/d10/f11 [1100675,119709] 0 2024-03-13T14:49:27.375 INFO:tasks.workunit.client.1.smithi196.stdout:3/75: creat db/d15/f1a x:0 0 0 2024-03-13T14:49:27.375 INFO:tasks.workunit.client.1.smithi196.stdout:6/129: dread - d6/d7/d14/dc/d1c/f25 zero size 2024-03-13T14:49:27.376 INFO:tasks.workunit.client.1.smithi196.stdout:9/126: dwrite d7/d9/df/f1c [0,4194304] 0 2024-03-13T14:49:27.381 INFO:tasks.workunit.client.1.smithi196.stdout:8/66: write d0/f1 [2025182,27724] 0 2024-03-13T14:49:27.381 INFO:tasks.workunit.client.1.smithi196.stdout:6/130: stat d6/d9/d17/d19/f20 0 2024-03-13T14:49:27.383 INFO:tasks.workunit.client.1.smithi196.stdout:8/67: write d0/f1 [1514759,27] 0 2024-03-13T14:49:27.384 INFO:tasks.workunit.client.1.smithi196.stdout:6/131: write d6/d7/d14/dc/d1c/f25 [694150,108078] 0 2024-03-13T14:49:27.386 INFO:tasks.workunit.client.1.smithi196.stdout:7/171: dread d0/f1c [0,4194304] 0 2024-03-13T14:49:27.388 INFO:tasks.workunit.client.1.smithi196.stdout:0/134: fsync d8/da/d25/f27 0 2024-03-13T14:49:27.388 INFO:tasks.workunit.client.1.smithi196.stdout:6/132: read d6/d9/d11/f13 [1411962,125469] 0 2024-03-13T14:49:27.395 INFO:tasks.workunit.client.1.smithi196.stdout:3/76: symlink db/d15/l1b 0 2024-03-13T14:49:27.397 INFO:tasks.workunit.client.1.smithi196.stdout:3/77: stat db/d15/f17 0 2024-03-13T14:49:27.400 INFO:tasks.workunit.client.1.smithi196.stdout:8/68: mkdir d0/d16 0 2024-03-13T14:49:27.404 INFO:tasks.workunit.client.1.smithi196.stdout:1/102: dwrite d1/d11/d13/f17 [0,4194304] 0 2024-03-13T14:49:27.404 INFO:tasks.workunit.client.1.smithi196.stdout:4/72: dwrite d1/f6 [0,4194304] 0 2024-03-13T14:49:27.405 INFO:tasks.workunit.client.1.smithi196.stdout:8/69: truncate d0/d10/f11 2022939 0 2024-03-13T14:49:27.407 INFO:tasks.workunit.client.1.smithi196.stdout:4/73: fdatasync d1/d3/d7/fd 0 2024-03-13T14:49:27.411 INFO:tasks.workunit.client.1.smithi196.stdout:1/103: write d1/d9/f18 [15714,60221] 0 2024-03-13T14:49:27.417 INFO:tasks.workunit.client.1.smithi196.stdout:1/104: write d1/d11/d13/f1e [740098,64014] 0 2024-03-13T14:49:27.422 INFO:tasks.workunit.client.1.smithi196.stdout:1/105: chown d1/d9/fd 1031057642 1 2024-03-13T14:49:27.423 INFO:tasks.workunit.client.1.smithi196.stdout:4/74: chown d1/d3/d7/c10 2 1 2024-03-13T14:49:27.433 INFO:tasks.workunit.client.1.smithi196.stdout:8/70: mkdir d0/d10/d12/d17 0 2024-03-13T14:49:27.435 INFO:tasks.workunit.client.1.smithi196.stdout:1/106: write d1/d11/d13/f17 [4806699,106520] 0 2024-03-13T14:49:27.437 INFO:tasks.workunit.client.1.smithi196.stdout:9/127: link f5 d7/de/f1f 0 2024-03-13T14:49:27.438 INFO:tasks.workunit.client.1.smithi196.stdout:0/135: dwrite d8/da/de/f22 [0,4194304] 0 2024-03-13T14:49:27.441 INFO:tasks.workunit.client.1.smithi196.stdout:6/133: rename d6/d7/d14 to d6/d2a 0 2024-03-13T14:49:27.449 INFO:tasks.workunit.client.1.smithi196.stdout:7/172: getdents d0/d9/d1e 0 2024-03-13T14:49:27.450 INFO:tasks.workunit.client.1.smithi196.stdout:9/128: creat d7/f20 x:0 0 0 2024-03-13T14:49:27.454 INFO:tasks.workunit.client.1.smithi196.stdout:6/134: creat d6/d9/d17/d19/f2b x:0 0 0 2024-03-13T14:49:27.456 INFO:tasks.workunit.client.1.smithi196.stdout:3/78: getdents db 0 2024-03-13T14:49:27.459 INFO:tasks.workunit.client.1.smithi196.stdout:3/79: chown db/fd 2810 1 2024-03-13T14:49:27.460 INFO:tasks.workunit.client.1.smithi196.stdout:0/136: rename d8/da/ld to d8/da/d25/l2e 0 2024-03-13T14:49:27.462 INFO:tasks.workunit.client.1.smithi196.stdout:1/107: dwrite d1/d11/f1b [0,4194304] 0 2024-03-13T14:49:27.464 INFO:tasks.workunit.client.1.smithi196.stdout:0/137: fdatasync d8/da/d25/f27 0 2024-03-13T14:49:27.465 INFO:tasks.workunit.client.1.smithi196.stdout:7/173: write d0/d9/d1e/d2d/f3d [220159,87586] 0 2024-03-13T14:49:27.465 INFO:tasks.workunit.client.1.smithi196.stdout:4/75: link d1/l8 d1/d3/l14 0 2024-03-13T14:49:27.473 INFO:tasks.workunit.client.1.smithi196.stdout:1/108: rmdir d1/d11/d13 39 2024-03-13T14:49:27.473 INFO:tasks.workunit.client.1.smithi196.stdout:6/135: creat d6/d7/dd/f2c x:0 0 0 2024-03-13T14:49:27.474 INFO:tasks.workunit.client.1.smithi196.stdout:0/138: read d8/da/d25/f27 [535706,65606] 0 2024-03-13T14:49:27.475 INFO:tasks.workunit.client.1.smithi196.stdout:7/174: write d0/d6/f26 [3327668,127761] 0 2024-03-13T14:49:27.480 INFO:tasks.workunit.client.1.smithi196.stdout:4/76: symlink d1/d3/l15 0 2024-03-13T14:49:27.480 INFO:tasks.workunit.client.1.smithi196.stdout:1/109: chown d1/d11/d13/c1c 3130926 1 2024-03-13T14:49:27.482 INFO:tasks.workunit.client.1.smithi196.stdout:6/136: creat d6/d9/d17/d19/f2d x:0 0 0 2024-03-13T14:49:27.484 INFO:tasks.workunit.client.1.smithi196.stdout:7/175: write d0/d9/d1e/f29 [225417,34587] 0 2024-03-13T14:49:27.487 INFO:tasks.workunit.client.1.smithi196.stdout:1/110: chown d1/l6 51437 1 2024-03-13T14:49:27.490 INFO:tasks.workunit.client.1.smithi196.stdout:6/137: write d6/d9/d17/d19/f2b [727494,103303] 0 2024-03-13T14:49:27.490 INFO:tasks.workunit.client.1.smithi196.stdout:4/77: write d1/d3/d7/fd [376172,15058] 0 2024-03-13T14:49:27.492 INFO:tasks.workunit.client.1.smithi196.stdout:1/111: fsync d1/d11/d13/f19 0 2024-03-13T14:49:27.495 INFO:tasks.workunit.client.1.smithi196.stdout:7/176: write d0/f37 [329192,117035] 0 2024-03-13T14:49:27.498 INFO:tasks.workunit.client.1.smithi196.stdout:3/80: dwrite db/ff [0,4194304] 0 2024-03-13T14:49:27.508 INFO:tasks.workunit.client.1.smithi196.stdout:7/177: dread d0/d6/f2c [0,4194304] 0 2024-03-13T14:49:27.513 INFO:tasks.workunit.client.1.smithi196.stdout:6/138: creat d6/d7/df/f2e x:0 0 0 2024-03-13T14:49:27.513 INFO:tasks.workunit.client.1.smithi196.stdout:0/139: dread d8/f1e [0,4194304] 0 2024-03-13T14:49:27.514 INFO:tasks.workunit.client.1.smithi196.stdout:0/140: stat d8/f12 0 2024-03-13T14:49:27.514 INFO:tasks.workunit.client.1.smithi196.stdout:6/139: fsync d6/f8 0 2024-03-13T14:49:27.516 INFO:tasks.workunit.client.1.smithi196.stdout:3/81: creat db/d15/f1c x:0 0 0 2024-03-13T14:49:27.518 INFO:tasks.workunit.client.1.smithi196.stdout:4/78: dread d1/d3/fa [0,4194304] 0 2024-03-13T14:49:27.520 INFO:tasks.workunit.client.1.smithi196.stdout:1/112: dwrite f0 [0,4194304] 0 2024-03-13T14:49:27.523 INFO:tasks.workunit.client.1.smithi196.stdout:4/79: truncate d1/d3/fa 5041258 0 2024-03-13T14:49:27.528 INFO:tasks.workunit.client.1.smithi196.stdout:0/141: creat d8/da/d25/f2f x:0 0 0 2024-03-13T14:49:27.529 INFO:tasks.workunit.client.1.smithi196.stdout:1/113: unlink d1/d11/d13/f1f 0 2024-03-13T14:49:27.530 INFO:tasks.workunit.client.1.smithi196.stdout:0/142: write d8/da/f10 [1182865,73459] 0 2024-03-13T14:49:27.531 INFO:tasks.workunit.client.1.smithi196.stdout:4/80: creat d1/d3/f16 x:0 0 0 2024-03-13T14:49:27.534 INFO:tasks.workunit.client.1.smithi196.stdout:3/82: creat db/d19/f1d x:0 0 0 2024-03-13T14:49:27.534 INFO:tasks.workunit.client.1.smithi196.stdout:1/114: dread - d1/d11/d13/f16 zero size 2024-03-13T14:49:27.536 INFO:tasks.workunit.client.1.smithi196.stdout:4/81: read - d1/d3/f13 zero size 2024-03-13T14:49:27.539 INFO:tasks.workunit.client.1.smithi196.stdout:0/143: dread d8/da/d1a/f1f [0,4194304] 0 2024-03-13T14:49:27.541 INFO:tasks.workunit.client.1.smithi196.stdout:3/83: chown db/c18 36338263 1 2024-03-13T14:49:27.541 INFO:tasks.workunit.client.1.smithi196.stdout:1/115: fdatasync d1/d11/d13/f17 0 2024-03-13T14:49:27.548 INFO:tasks.workunit.client.1.smithi196.stdout:0/144: symlink d8/da/de/l30 0 2024-03-13T14:49:27.548 INFO:tasks.workunit.client.1.smithi196.stdout:3/84: unlink db/c18 0 2024-03-13T14:49:27.549 INFO:tasks.workunit.client.1.smithi196.stdout:3/85: stat db/fe 0 2024-03-13T14:49:27.549 INFO:tasks.workunit.client.1.smithi196.stdout:0/145: chown l3 3915296 1 2024-03-13T14:49:27.551 INFO:tasks.workunit.client.1.smithi196.stdout:1/116: unlink d1/d11/d13/f1e 0 2024-03-13T14:49:27.553 INFO:tasks.workunit.client.1.smithi196.stdout:6/140: dwrite d6/d2a/dc/d1c/f25 [0,4194304] 0 2024-03-13T14:49:27.553 INFO:tasks.workunit.client.1.smithi196.stdout:7/178: dwrite d0/d6/fc [0,4194304] 0 2024-03-13T14:49:27.558 INFO:tasks.workunit.client.1.smithi196.stdout:6/141: write d6/d7/dd/f2c [506305,11562] 0 2024-03-13T14:49:27.558 INFO:tasks.workunit.client.1.smithi196.stdout:3/86: mknod db/d15/c1e 0 2024-03-13T14:49:27.563 INFO:tasks.workunit.client.1.smithi196.stdout:0/146: creat d8/da/d1a/f31 x:0 0 0 2024-03-13T14:49:27.567 INFO:tasks.workunit.client.1.smithi196.stdout:7/179: mkdir d0/d9/d1e/d3f 0 2024-03-13T14:49:27.567 INFO:tasks.workunit.client.1.smithi196.stdout:5/181: sync 2024-03-13T14:49:27.576 INFO:tasks.workunit.client.1.smithi196.stdout:7/180: stat d0/d6/cd 0 2024-03-13T14:49:27.577 INFO:tasks.workunit.client.1.smithi196.stdout:1/117: dwrite d1/d11/f1a [0,4194304] 0 2024-03-13T14:49:27.580 INFO:tasks.workunit.client.1.smithi196.stdout:7/181: write d0/d6/f1d [395176,28928] 0 2024-03-13T14:49:27.580 INFO:tasks.workunit.client.1.smithi196.stdout:5/182: creat d4/db/d14/d19/d26/f3a x:0 0 0 2024-03-13T14:49:27.581 INFO:tasks.workunit.client.1.smithi196.stdout:5/183: dread - d4/db/d14/d19/d26/f3a zero size 2024-03-13T14:49:27.583 INFO:tasks.workunit.client.1.smithi196.stdout:6/142: creat d6/d2a/d23/d28/f2f x:0 0 0 2024-03-13T14:49:27.590 INFO:tasks.workunit.client.1.smithi196.stdout:1/118: symlink d1/d9/l21 0 2024-03-13T14:49:27.594 INFO:tasks.workunit.client.1.smithi196.stdout:5/184: write d4/db/f24 [3527457,16319] 0 2024-03-13T14:49:27.596 INFO:tasks.workunit.client.1.smithi196.stdout:6/143: rmdir d6/d7/dd 39 2024-03-13T14:49:27.601 INFO:tasks.workunit.client.1.smithi196.stdout:0/147: rename d8/f28 to d8/da/f32 0 2024-03-13T14:49:27.602 INFO:tasks.workunit.client.1.smithi196.stdout:4/82: write d1/f2 [1025151,110489] 0 2024-03-13T14:49:27.603 INFO:tasks.workunit.client.1.smithi196.stdout:0/148: readlink d8/da/d1a/l29 0 2024-03-13T14:49:27.606 INFO:tasks.workunit.client.1.smithi196.stdout:5/185: creat d4/db/d14/d19/f3b x:0 0 0 2024-03-13T14:49:27.608 INFO:tasks.workunit.client.1.smithi196.stdout:6/144: creat d6/d9/f30 x:0 0 0 2024-03-13T14:49:27.608 INFO:tasks.workunit.client.1.smithi196.stdout:5/186: read d4/f8 [7712547,16526] 0 2024-03-13T14:49:27.610 INFO:tasks.workunit.client.1.smithi196.stdout:6/145: read d6/f10 [17015,73175] 0 2024-03-13T14:49:27.614 INFO:tasks.workunit.client.1.smithi196.stdout:4/83: creat d1/d3/f17 x:0 0 0 2024-03-13T14:49:27.616 INFO:tasks.workunit.client.1.smithi196.stdout:4/84: truncate d1/d3/f16 476480 0 2024-03-13T14:49:27.618 INFO:tasks.workunit.client.1.smithi196.stdout:4/85: write d1/f2 [2237460,59385] 0 2024-03-13T14:49:27.618 INFO:tasks.workunit.client.1.smithi196.stdout:0/149: mkdir d8/da/de/d33 0 2024-03-13T14:49:27.630 INFO:tasks.workunit.client.1.smithi196.stdout:1/119: link d1/d11/d13/f17 d1/d11/f22 0 2024-03-13T14:49:27.632 INFO:tasks.workunit.client.1.smithi196.stdout:1/120: fsync d1/f5 0 2024-03-13T14:49:27.633 INFO:tasks.workunit.client.1.smithi196.stdout:1/121: fsync f0 0 2024-03-13T14:49:27.637 INFO:tasks.workunit.client.1.smithi196.stdout:0/150: dwrite d8/da/d25/d26/f2d [0,4194304] 0 2024-03-13T14:49:27.637 INFO:tasks.workunit.client.1.smithi196.stdout:4/86: rename d1/d3/d7/cb to d1/d3/d7/c18 0 2024-03-13T14:49:27.650 INFO:tasks.workunit.client.1.smithi196.stdout:5/187: rename d4/d1b/d2b to d4/db/d3c 0 2024-03-13T14:49:27.655 INFO:tasks.workunit.client.1.smithi196.stdout:7/182: dwrite d0/d9/f24 [0,4194304] 0 2024-03-13T14:49:27.656 INFO:tasks.workunit.client.1.smithi196.stdout:5/188: fsync d4/db/d14/d19/d1c/d25/f36 0 2024-03-13T14:49:27.660 INFO:tasks.workunit.client.1.smithi196.stdout:7/183: readlink d0/d6/l2e 0 2024-03-13T14:49:27.663 INFO:tasks.workunit.client.1.smithi196.stdout:5/189: write d4/d1b/f29 [4825664,56576] 0 2024-03-13T14:49:27.666 INFO:tasks.workunit.client.1.smithi196.stdout:5/190: truncate d4/db/d14/d19/f3b 67287 0 2024-03-13T14:49:27.668 INFO:tasks.workunit.client.1.smithi196.stdout:1/122: dread d1/d11/f1b [0,4194304] 0 2024-03-13T14:49:27.673 INFO:tasks.workunit.client.1.smithi196.stdout:0/151: creat d8/da/de/d33/f34 x:0 0 0 2024-03-13T14:49:27.678 INFO:tasks.workunit.client.1.smithi196.stdout:0/152: read d8/da/f13 [3695898,67124] 0 2024-03-13T14:49:27.682 INFO:tasks.workunit.client.1.smithi196.stdout:5/191: unlink d4/f7 0 2024-03-13T14:49:27.684 INFO:tasks.workunit.client.1.smithi196.stdout:5/192: write d4/db/f24 [3488642,26356] 0 2024-03-13T14:49:27.687 INFO:tasks.workunit.client.1.smithi196.stdout:5/193: dread - d4/db/d14/d19/d1c/d25/f36 zero size 2024-03-13T14:49:27.687 INFO:tasks.workunit.client.1.smithi196.stdout:1/123: symlink d1/d11/d13/l23 0 2024-03-13T14:49:27.689 INFO:tasks.workunit.client.1.smithi196.stdout:6/146: truncate d6/d9/d11/f13 3107734 0 2024-03-13T14:49:27.691 INFO:tasks.workunit.client.1.smithi196.stdout:4/87: getdents d1/d3/d7 0 2024-03-13T14:49:27.695 INFO:tasks.workunit.client.1.smithi196.stdout:6/147: truncate d6/d9/f1f 311189 0 2024-03-13T14:49:27.695 INFO:tasks.workunit.client.1.smithi196.stdout:6/148: chown d6 120 1 2024-03-13T14:49:27.697 INFO:tasks.workunit.client.1.smithi196.stdout:6/149: chown d6/f10 55269 1 2024-03-13T14:49:27.700 INFO:tasks.workunit.client.1.smithi196.stdout:1/124: rename d1/d9/l21 to d1/d11/d13/l24 0 2024-03-13T14:49:27.703 INFO:tasks.workunit.client.1.smithi196.stdout:4/88: creat d1/d3/d7/f19 x:0 0 0 2024-03-13T14:49:27.705 INFO:tasks.workunit.client.1.smithi196.stdout:6/150: creat d6/d9/d11/f31 x:0 0 0 2024-03-13T14:49:27.707 INFO:tasks.workunit.client.1.smithi196.stdout:5/194: rename d4/db/d14/d19/d1c/d25/c32 to d4/d1b/c3d 0 2024-03-13T14:49:27.707 INFO:tasks.workunit.client.1.smithi196.stdout:6/151: stat d6/c16 0 2024-03-13T14:49:27.707 INFO:tasks.workunit.client.1.smithi196.stdout:4/89: chown d1/d3/d7/fd 18854997 1 2024-03-13T14:49:27.708 INFO:tasks.workunit.client.1.smithi196.stdout:0/153: rmdir d8/da 39 2024-03-13T14:49:27.709 INFO:tasks.workunit.client.1.smithi196.stdout:1/125: dread d1/d9/f18 [0,4194304] 0 2024-03-13T14:49:27.710 INFO:tasks.workunit.client.1.smithi196.stdout:6/152: dread - d6/d2a/d23/d28/f2f zero size 2024-03-13T14:49:27.712 INFO:tasks.workunit.client.1.smithi196.stdout:5/195: write d4/d1b/f29 [2810837,105964] 0 2024-03-13T14:49:27.716 INFO:tasks.workunit.client.1.smithi196.stdout:6/153: read d6/d9/d17/d19/f22 [225868,80635] 0 2024-03-13T14:49:27.716 INFO:tasks.workunit.client.1.smithi196.stdout:7/184: truncate d0/d6/f21 3950485 0 2024-03-13T14:49:27.717 INFO:tasks.workunit.client.1.smithi196.stdout:4/90: creat d1/d3/d7/f1a x:0 0 0 2024-03-13T14:49:27.718 INFO:tasks.workunit.client.1.smithi196.stdout:1/126: dread d1/d11/f1d [0,4194304] 0 2024-03-13T14:49:27.718 INFO:tasks.workunit.client.1.smithi196.stdout:5/196: chown d4/db/l1f 160980 1 2024-03-13T14:49:27.725 INFO:tasks.workunit.client.1.smithi196.stdout:6/154: write d6/d9/d17/d19/f2b [867528,68320] 0 2024-03-13T14:49:27.725 INFO:tasks.workunit.client.1.smithi196.stdout:0/154: mkdir d8/da/d25/d26/d35 0 2024-03-13T14:49:27.728 INFO:tasks.workunit.client.1.smithi196.stdout:0/155: readlink d8/da/d1a/l15 0 2024-03-13T14:49:27.731 INFO:tasks.workunit.client.1.smithi196.stdout:0/156: dread - d8/da/f32 zero size 2024-03-13T14:49:27.733 INFO:tasks.workunit.client.1.smithi196.stdout:1/127: creat d1/d11/d13/f25 x:0 0 0 2024-03-13T14:49:27.738 INFO:tasks.workunit.client.1.smithi196.stdout:1/128: write d1/f5 [1483300,109856] 0 2024-03-13T14:49:27.739 INFO:tasks.workunit.client.1.smithi196.stdout:1/129: chown d1/d11/d13/f16 3803815 1 2024-03-13T14:49:27.742 INFO:tasks.workunit.client.1.smithi196.stdout:4/91: unlink d1/l5 0 2024-03-13T14:49:27.750 INFO:tasks.workunit.client.1.smithi196.stdout:6/155: dread f4 [0,4194304] 0 2024-03-13T14:49:27.753 INFO:tasks.workunit.client.1.smithi196.stdout:7/185: dread d0/d6/f23 [0,4194304] 0 2024-03-13T14:49:27.754 INFO:tasks.workunit.client.1.smithi196.stdout:6/156: write d6/d9/f1f [266428,16476] 0 2024-03-13T14:49:27.756 INFO:tasks.workunit.client.1.smithi196.stdout:7/186: readlink d0/d6/l3e 0 2024-03-13T14:49:27.763 INFO:tasks.workunit.client.1.smithi196.stdout:5/197: dwrite d4/db/ff [0,4194304] 0 2024-03-13T14:49:27.766 INFO:tasks.workunit.client.1.smithi196.stdout:0/157: dwrite d8/da/d25/d26/f2d [0,4194304] 0 2024-03-13T14:49:27.771 INFO:tasks.workunit.client.1.smithi196.stdout:4/92: dwrite d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:27.771 INFO:tasks.workunit.client.1.smithi196.stdout:6/157: write d6/f10 [70609,53767] 0 2024-03-13T14:49:27.779 INFO:tasks.workunit.client.1.smithi196.stdout:0/158: creat d8/da/de/d17/f36 x:0 0 0 2024-03-13T14:49:27.786 INFO:tasks.workunit.client.1.smithi196.stdout:5/198: creat d4/db/d14/d19/d1c/d25/d38/f3e x:0 0 0 2024-03-13T14:49:27.789 INFO:tasks.workunit.client.1.smithi196.stdout:4/93: truncate d1/d3/f17 330450 0 2024-03-13T14:49:27.790 INFO:tasks.workunit.client.1.smithi196.stdout:5/199: read d4/db/d14/d19/d1c/f23 [779676,40082] 0 2024-03-13T14:49:27.792 INFO:tasks.workunit.client.1.smithi196.stdout:4/94: write d1/d3/f12 [629682,44089] 0 2024-03-13T14:49:27.793 INFO:tasks.workunit.client.1.smithi196.stdout:0/159: mkdir d8/d37 0 2024-03-13T14:49:27.798 INFO:tasks.workunit.client.1.smithi196.stdout:7/187: dwrite d0/d9/f20 [0,4194304] 0 2024-03-13T14:49:27.798 INFO:tasks.workunit.client.1.smithi196.stdout:4/95: stat d1/d3/f16 0 2024-03-13T14:49:27.798 INFO:tasks.workunit.client.1.smithi196.stdout:4/96: chown d1 5 1 2024-03-13T14:49:27.800 INFO:tasks.workunit.client.1.smithi196.stdout:5/200: write d4/db/d14/d19/d26/f3a [14162,57703] 0 2024-03-13T14:49:27.801 INFO:tasks.workunit.client.1.smithi196.stdout:0/160: write d8/da/de/f22 [1167016,129461] 0 2024-03-13T14:49:27.806 INFO:tasks.workunit.client.1.smithi196.stdout:7/188: stat d0/l1f 0 2024-03-13T14:49:27.807 INFO:tasks.workunit.client.1.smithi196.stdout:0/161: chown d8/da/de/c16 65670677 1 2024-03-13T14:49:27.808 INFO:tasks.workunit.client.1.smithi196.stdout:6/158: dwrite d6/fa [0,4194304] 0 2024-03-13T14:49:27.809 INFO:tasks.workunit.client.1.smithi196.stdout:4/97: rename d1/f2 to d1/d3/d7/f1b 0 2024-03-13T14:49:27.810 INFO:tasks.workunit.client.1.smithi196.stdout:5/201: creat d4/db/d14/d19/d1c/f3f x:0 0 0 2024-03-13T14:49:27.811 INFO:tasks.workunit.client.1.smithi196.stdout:7/189: creat d0/d6/d33/f40 x:0 0 0 2024-03-13T14:49:27.813 INFO:tasks.workunit.client.1.smithi196.stdout:0/162: write d8/f9 [936509,85278] 0 2024-03-13T14:49:27.813 INFO:tasks.workunit.client.1.smithi196.stdout:6/159: creat d6/d9/d11/f32 x:0 0 0 2024-03-13T14:49:27.815 INFO:tasks.workunit.client.1.smithi196.stdout:2/70: sync 2024-03-13T14:49:27.817 INFO:tasks.workunit.client.1.smithi196.stdout:7/190: readlink d0/l1 0 2024-03-13T14:49:27.817 INFO:tasks.workunit.client.1.smithi196.stdout:0/163: chown d8/da/d1a/l15 1246 1 2024-03-13T14:49:27.823 INFO:tasks.workunit.client.1.smithi196.stdout:6/160: read d6/d9/f1f [246826,101897] 0 2024-03-13T14:49:27.826 INFO:tasks.workunit.client.1.smithi196.stdout:5/202: symlink d4/db/d14/d19/d1c/d25/l40 0 2024-03-13T14:49:27.830 INFO:tasks.workunit.client.1.smithi196.stdout:7/191: chown d0/d6/c12 90827070 1 2024-03-13T14:49:27.830 INFO:tasks.workunit.client.1.smithi196.stdout:5/203: stat d4/db/d14/d19/l35 0 2024-03-13T14:49:27.831 INFO:tasks.workunit.client.1.smithi196.stdout:7/192: readlink d0/d6/l17 0 2024-03-13T14:49:27.835 INFO:tasks.workunit.client.1.smithi196.stdout:7/193: chown d0/d9/d1e/d2d/c32 17076605 1 2024-03-13T14:49:27.837 INFO:tasks.workunit.client.1.smithi196.stdout:5/204: write d4/db/d14/d19/d1c/d25/f2a [1752193,31265] 0 2024-03-13T14:49:27.842 INFO:tasks.workunit.client.1.smithi196.stdout:2/71: creat d0/d6/fb x:0 0 0 2024-03-13T14:49:27.844 INFO:tasks.workunit.client.1.smithi196.stdout:0/164: rename d8/da/de/f22 to d8/da/f38 0 2024-03-13T14:49:27.847 INFO:tasks.workunit.client.1.smithi196.stdout:4/98: getdents d1/d3/d7 0 2024-03-13T14:49:27.857 INFO:tasks.workunit.client.1.smithi196.stdout:5/205: truncate d4/db/d14/d19/d1c/d25/d38/f3e 110764 0 2024-03-13T14:49:27.858 INFO:tasks.workunit.client.1.smithi196.stdout:6/161: dwrite d6/d9/d11/f32 [0,4194304] 0 2024-03-13T14:49:27.858 INFO:tasks.workunit.client.1.smithi196.stdout:7/194: link d0/d6/d33/f39 d0/d6/d35/f41 0 2024-03-13T14:49:27.858 INFO:tasks.workunit.client.1.smithi196.stdout:0/165: mknod d8/da/de/d17/c39 0 2024-03-13T14:49:27.862 INFO:tasks.workunit.client.1.smithi196.stdout:7/195: stat d0/d9/d1e/d2d/f3b 0 2024-03-13T14:49:27.866 INFO:tasks.workunit.client.1.smithi196.stdout:0/166: rename c5 to d8/da/de/d33/c3a 0 2024-03-13T14:49:27.869 INFO:tasks.workunit.client.1.smithi196.stdout:6/162: creat d6/d2a/dc/d1c/f33 x:0 0 0 2024-03-13T14:49:27.870 INFO:tasks.workunit.client.1.smithi196.stdout:5/206: write d4/db/d14/d19/d1c/d25/f2a [10172,59312] 0 2024-03-13T14:49:27.875 INFO:tasks.workunit.client.1.smithi196.stdout:2/72: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:27.875 INFO:tasks.workunit.client.1.smithi196.stdout:5/207: chown d4/f11 31500 1 2024-03-13T14:49:27.880 INFO:tasks.workunit.client.1.smithi196.stdout:4/99: dwrite d1/d3/d7/fd [0,4194304] 0 2024-03-13T14:49:27.881 INFO:tasks.workunit.client.1.smithi196.stdout:0/167: mkdir d8/da/d25/d26/d35/d3b 0 2024-03-13T14:49:27.881 INFO:tasks.workunit.client.1.smithi196.stdout:6/163: mkdir d6/d2a/dc/d1c/d34 0 2024-03-13T14:49:27.882 INFO:tasks.workunit.client.1.smithi196.stdout:5/208: readlink d4/db/l12 0 2024-03-13T14:49:27.887 INFO:tasks.workunit.client.1.smithi196.stdout:2/73: mknod d0/d6/cc 0 2024-03-13T14:49:27.889 INFO:tasks.workunit.client.1.smithi196.stdout:4/100: chown d1/d3/f13 49607 1 2024-03-13T14:49:27.889 INFO:tasks.workunit.client.1.smithi196.stdout:2/74: readlink d0/l4 0 2024-03-13T14:49:27.889 INFO:tasks.workunit.client.1.smithi196.stdout:6/164: write d6/d7/dd/f2c [1518397,16124] 0 2024-03-13T14:49:27.890 INFO:tasks.workunit.client.1.smithi196.stdout:0/168: mknod d8/da/d1a/c3c 0 2024-03-13T14:49:27.893 INFO:tasks.workunit.client.1.smithi196.stdout:0/169: readlink d8/da/d1a/l15 0 2024-03-13T14:49:27.896 INFO:tasks.workunit.client.1.smithi196.stdout:4/101: unlink d1/d3/c11 0 2024-03-13T14:49:27.899 INFO:tasks.workunit.client.1.smithi196.stdout:0/170: stat d8/da/de/d33/c3a 0 2024-03-13T14:49:27.902 INFO:tasks.workunit.client.1.smithi196.stdout:6/165: creat d6/d7/df/f35 x:0 0 0 2024-03-13T14:49:27.910 INFO:tasks.workunit.client.1.smithi196.stdout:5/209: dread d4/f8 [0,4194304] 0 2024-03-13T14:49:27.912 INFO:tasks.workunit.client.1.smithi196.stdout:2/75: dwrite d0/d6/f8 [0,4194304] 0 2024-03-13T14:49:27.919 INFO:tasks.workunit.client.1.smithi196.stdout:6/166: truncate f4 1466696 0 2024-03-13T14:49:27.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: pgmap v32: 65 pgs: 65 active+clean; 283 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 17 MiB/s wr, 264 op/s 2024-03-13T14:49:27.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:27.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:27.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:49:27.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr fail", "who": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr fail", "who": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: osdmap e49: 6 total, 6 up, 6 in 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd='[{"prefix": "mgr fail", "who": "smithi196.vdktbk"}]': finished 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: mgrmap e26: smithi176.tzfdwd(active, starting, since 0.013968s) 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:49:27.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:49:27.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:49:27.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:49:27.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:27 smithi196 ceph-mon[37486]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:49:27.931 INFO:tasks.workunit.client.1.smithi196.stdout:6/167: symlink d6/d9/l36 0 2024-03-13T14:49:27.934 INFO:tasks.workunit.client.1.smithi196.stdout:0/171: dwrite d8/da/de/d17/f36 [0,4194304] 0 2024-03-13T14:49:27.935 INFO:tasks.workunit.client.1.smithi196.stdout:7/196: dwrite d0/d6/f21 [4194304,4194304] 0 2024-03-13T14:49:27.935 INFO:tasks.workunit.client.1.smithi196.stdout:6/168: stat d6/d9/d17 0 2024-03-13T14:49:27.937 INFO:tasks.workunit.client.1.smithi196.stdout:4/102: dwrite d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:27.950 INFO:tasks.workunit.client.1.smithi196.stdout:2/76: rename d0/l4 to d0/d6/ld 0 2024-03-13T14:49:27.951 INFO:tasks.workunit.client.1.smithi196.stdout:6/169: mknod d6/d7/dd/c37 0 2024-03-13T14:49:27.954 INFO:tasks.workunit.client.1.smithi196.stdout:0/172: unlink d8/da/d1a/l29 0 2024-03-13T14:49:27.958 INFO:tasks.workunit.client.1.smithi196.stdout:2/77: creat d0/fe x:0 0 0 2024-03-13T14:49:27.959 INFO:tasks.workunit.client.1.smithi196.stdout:6/170: write d6/d9/f1f [584973,91694] 0 2024-03-13T14:49:27.966 INFO:tasks.workunit.client.1.smithi196.stdout:4/103: dwrite d1/d3/d7/f1a [0,4194304] 0 2024-03-13T14:49:27.966 INFO:tasks.workunit.client.1.smithi196.stdout:6/171: mknod d6/d9/d11/c38 0 2024-03-13T14:49:27.970 INFO:tasks.workunit.client.1.smithi196.stdout:4/104: mknod d1/d3/c1c 0 2024-03-13T14:49:27.975 INFO:tasks.workunit.client.1.smithi196.stdout:6/172: mkdir d6/d7/df/d27/d39 0 2024-03-13T14:49:27.977 INFO:tasks.workunit.client.1.smithi196.stdout:4/105: creat d1/d3/d7/f1d x:0 0 0 2024-03-13T14:49:27.979 INFO:tasks.workunit.client.1.smithi196.stdout:0/173: dwrite d8/da/d1a/f31 [0,4194304] 0 2024-03-13T14:49:27.981 INFO:tasks.workunit.client.1.smithi196.stdout:0/174: write d8/f9 [409823,4165] 0 2024-03-13T14:49:27.988 INFO:tasks.workunit.client.1.smithi196.stdout:4/106: mkdir d1/d1e 0 2024-03-13T14:49:27.989 INFO:tasks.workunit.client.1.smithi196.stdout:0/175: write d8/da/f13 [2962124,11668] 0 2024-03-13T14:49:27.990 INFO:tasks.workunit.client.1.smithi196.stdout:6/173: getdents d6/d7/dd 0 2024-03-13T14:49:27.991 INFO:tasks.workunit.client.1.smithi196.stdout:0/176: readlink d8/da/d1a/l15 0 2024-03-13T14:49:27.992 INFO:tasks.workunit.client.1.smithi196.stdout:4/107: write d1/f6 [4937455,33359] 0 2024-03-13T14:49:27.997 INFO:tasks.workunit.client.1.smithi196.stdout:0/177: fsync d8/da/f38 0 2024-03-13T14:49:28.005 INFO:tasks.workunit.client.1.smithi196.stdout:4/108: rename d1/l8 to d1/d3/l1f 0 2024-03-13T14:49:28.008 INFO:tasks.workunit.client.1.smithi196.stdout:6/174: dread d6/d9/d17/d19/f22 [0,4194304] 0 2024-03-13T14:49:28.010 INFO:tasks.workunit.client.1.smithi196.stdout:6/175: write d6/fa [809637,87371] 0 2024-03-13T14:49:28.012 INFO:tasks.workunit.client.1.smithi196.stdout:4/109: dread d1/d3/f12 [0,4194304] 0 2024-03-13T14:49:28.020 INFO:tasks.workunit.client.1.smithi196.stdout:0/178: truncate d8/da/d1a/f31 2311226 0 2024-03-13T14:49:28.021 INFO:tasks.workunit.client.1.smithi196.stdout:0/179: write d8/da/f10 [190048,113127] 0 2024-03-13T14:49:28.025 INFO:tasks.workunit.client.1.smithi196.stdout:6/176: truncate d6/fa 4038202 0 2024-03-13T14:49:28.029 INFO:tasks.workunit.client.1.smithi196.stdout:4/110: dread d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:28.031 INFO:tasks.workunit.client.1.smithi196.stdout:6/177: creat d6/d7/df/f3a x:0 0 0 2024-03-13T14:49:28.036 INFO:tasks.workunit.client.1.smithi196.stdout:6/178: creat d6/d2a/dc/d1c/f3b x:0 0 0 2024-03-13T14:49:28.038 INFO:tasks.workunit.client.1.smithi196.stdout:4/111: rename d1/d3/l15 to d1/l20 0 2024-03-13T14:49:28.040 INFO:tasks.workunit.client.1.smithi196.stdout:0/180: dwrite d8/da/de/d17/f36 [4194304,4194304] 0 2024-03-13T14:49:28.041 INFO:tasks.workunit.client.1.smithi196.stdout:4/112: read d1/d3/d7/fd [1369119,23793] 0 2024-03-13T14:49:28.046 INFO:tasks.workunit.client.1.smithi196.stdout:6/179: dread f4 [0,4194304] 0 2024-03-13T14:49:28.047 INFO:tasks.workunit.client.1.smithi196.stdout:6/180: chown d6/c16 1468702 1 2024-03-13T14:49:28.049 INFO:tasks.workunit.client.1.smithi196.stdout:0/181: creat d8/da/d25/d26/d35/f3d x:0 0 0 2024-03-13T14:49:28.050 INFO:tasks.workunit.client.1.smithi196.stdout:6/181: write d6/d9/d11/f32 [3844978,94670] 0 2024-03-13T14:49:28.051 INFO:tasks.workunit.client.1.smithi196.stdout:0/182: write d8/f11 [3688959,79740] 0 2024-03-13T14:49:28.059 INFO:tasks.workunit.client.1.smithi196.stdout:0/183: creat d8/da/f3e x:0 0 0 2024-03-13T14:49:28.060 INFO:tasks.workunit.client.1.smithi196.stdout:8/71: sync 2024-03-13T14:49:28.063 INFO:tasks.workunit.client.1.smithi196.stdout:8/72: dread - d0/d10/f15 zero size 2024-03-13T14:49:28.066 INFO:tasks.workunit.client.1.smithi196.stdout:8/73: mknod d0/c18 0 2024-03-13T14:49:28.070 INFO:tasks.workunit.client.1.smithi196.stdout:6/182: dwrite d6/d2a/dc/d1c/f33 [0,4194304] 0 2024-03-13T14:49:28.072 INFO:tasks.workunit.client.1.smithi196.stdout:8/74: read d0/f8 [1561092,44062] 0 2024-03-13T14:49:28.079 INFO:tasks.workunit.client.1.smithi196.stdout:0/184: dread d8/da/de/d17/f36 [0,4194304] 0 2024-03-13T14:49:28.080 INFO:tasks.workunit.client.1.smithi196.stdout:4/113: truncate d1/d3/f12 87668 0 2024-03-13T14:49:28.086 INFO:tasks.workunit.client.1.smithi196.stdout:6/183: dread d6/d9/d17/d19/f22 [0,4194304] 0 2024-03-13T14:49:28.086 INFO:tasks.workunit.client.1.smithi196.stdout:4/114: unlink d1/d3/f16 0 2024-03-13T14:49:28.087 INFO:tasks.workunit.client.1.smithi196.stdout:6/184: dread - d6/d7/df/f2e zero size 2024-03-13T14:49:28.089 INFO:tasks.workunit.client.1.smithi196.stdout:0/185: getdents d8/da/d25/d26/d35/d3b 0 2024-03-13T14:49:28.092 INFO:tasks.workunit.client.1.smithi196.stdout:4/115: creat d1/d3/d7/f21 x:0 0 0 2024-03-13T14:49:28.092 INFO:tasks.workunit.client.1.smithi196.stdout:6/185: mknod d6/d2a/dc/d1c/c3c 0 2024-03-13T14:49:28.093 INFO:tasks.workunit.client.1.smithi196.stdout:0/186: mknod d8/da/d1a/c3f 0 2024-03-13T14:49:28.093 INFO:tasks.workunit.client.1.smithi196.stdout:6/186: fdatasync d6/f8 0 2024-03-13T14:49:28.095 INFO:tasks.workunit.client.1.smithi196.stdout:6/187: fdatasync d6/d7/dd/f2c 0 2024-03-13T14:49:28.096 INFO:tasks.workunit.client.1.smithi196.stdout:4/116: write d1/d3/d7/f19 [760937,126652] 0 2024-03-13T14:49:28.100 INFO:tasks.workunit.client.1.smithi196.stdout:6/188: write d6/d9/d17/d19/f2b [425753,8781] 0 2024-03-13T14:49:28.103 INFO:tasks.workunit.client.1.smithi196.stdout:0/187: rename d8/da/d1a/c3f to d8/da/d25/c40 0 2024-03-13T14:49:28.106 INFO:tasks.workunit.client.1.smithi196.stdout:0/188: read d8/da/d1a/f31 [1223915,42200] 0 2024-03-13T14:49:28.107 INFO:tasks.workunit.client.1.smithi196.stdout:4/117: creat d1/f22 x:0 0 0 2024-03-13T14:49:28.111 INFO:tasks.workunit.client.1.smithi196.stdout:4/118: mknod d1/c23 0 2024-03-13T14:49:28.113 INFO:tasks.workunit.client.1.smithi196.stdout:0/189: getdents d8 0 2024-03-13T14:49:28.117 INFO:tasks.workunit.client.1.smithi196.stdout:4/119: write d1/d3/d7/f19 [1669627,92557] 0 2024-03-13T14:49:28.119 INFO:tasks.workunit.client.1.smithi196.stdout:6/189: dwrite d6/d7/df/f2e [0,4194304] 0 2024-03-13T14:49:28.120 INFO:tasks.workunit.client.1.smithi196.stdout:4/120: chown d1/d3/d7/cc 27787541 1 2024-03-13T14:49:28.121 INFO:tasks.workunit.client.0.smithi176.stderr:++ readlink -f fsstress 2024-03-13T14:49:28.122 INFO:tasks.workunit.client.1.smithi196.stdout:6/190: stat d6/d9/l36 0 2024-03-13T14:49:28.126 INFO:tasks.workunit.client.1.smithi196.stdout:4/121: symlink d1/l24 0 2024-03-13T14:49:28.127 INFO:tasks.workunit.client.0.smithi176.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-03-13T14:49:28.127 INFO:tasks.workunit.client.0.smithi176.stderr:+ popd 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: pgmap v32: 65 pgs: 65 active+clean; 283 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 17 MiB/s wr, 264 op/s 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 172.21.15.196:0/2397844530' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr fail", "who": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "mgr fail", "who": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: osdmap e49: 6 total, 6 up, 6 in 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.24325 ' entity='mgr.smithi196.vdktbk' cmd='[{"prefix": "mgr fail", "who": "smithi196.vdktbk"}]': finished 2024-03-13T14:49:28.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: mgrmap e26: smithi176.tzfdwd(active, starting, since 0.013968s) 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:49:28.130 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:49:28.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:49:28.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:49:28.131 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:27 smithi176 ceph-mon[29064]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:49:28.131 INFO:tasks.workunit.client.1.smithi196.stdout:0/190: dwrite d8/f11 [0,4194304] 0 2024-03-13T14:49:28.132 INFO:tasks.workunit.client.0.smithi176.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-03-13T14:49:28.132 INFO:tasks.workunit.client.0.smithi176.stderr:+ popd 2024-03-13T14:49:28.133 INFO:tasks.workunit.client.0.smithi176.stdout:~/cephtest/mnt.0/client.0/tmp 2024-03-13T14:49:28.134 INFO:tasks.workunit.client.0.smithi176.stderr:++ mktemp -d -p . 2024-03-13T14:49:28.139 INFO:tasks.workunit.client.1.smithi196.stdout:4/122: symlink d1/l25 0 2024-03-13T14:49:28.141 INFO:tasks.workunit.client.0.smithi176.stderr:+ T=./tmp.VggskuEev9 2024-03-13T14:49:28.141 INFO:tasks.workunit.client.0.smithi176.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.VggskuEev9 -l 1 -n 1000 -p 10 -v 2024-03-13T14:49:28.147 INFO:tasks.workunit.client.1.smithi196.stdout:4/123: unlink d1/d3/d7/fd 0 2024-03-13T14:49:28.149 INFO:tasks.workunit.client.1.smithi196.stdout:6/191: dwrite d6/d7/dd/f2c [0,4194304] 0 2024-03-13T14:49:28.150 INFO:tasks.workunit.client.1.smithi196.stdout:4/124: fsync d1/d3/d7/f1a 0 2024-03-13T14:49:28.151 INFO:tasks.workunit.client.0.smithi176.stdout:seed = 1710440374 2024-03-13T14:49:28.152 INFO:tasks.workunit.client.1.smithi196.stdout:0/191: dread d8/da/d25/d26/f2d [0,4194304] 0 2024-03-13T14:49:28.155 INFO:tasks.workunit.client.1.smithi196.stdout:0/192: chown d8/da/f38 192897573 1 2024-03-13T14:49:28.156 INFO:tasks.workunit.client.1.smithi196.stdout:6/192: dread - d6/d7/df/f3a zero size 2024-03-13T14:49:28.160 INFO:tasks.workunit.client.1.smithi196.stdout:0/193: mknod d8/da/de/c41 0 2024-03-13T14:49:28.162 INFO:tasks.workunit.client.0.smithi176.stdout:0/0: symlink l0 0 2024-03-13T14:49:28.163 INFO:tasks.workunit.client.0.smithi176.stdout:0/1: dread - no filename 2024-03-13T14:49:28.163 INFO:tasks.workunit.client.1.smithi196.stdout:0/194: write d8/da/f3e [900919,37321] 0 2024-03-13T14:49:28.165 INFO:tasks.workunit.client.1.smithi196.stdout:6/193: truncate d6/d9/d17/d19/f22 261139 0 2024-03-13T14:49:28.170 INFO:tasks.workunit.client.0.smithi176.stdout:1/0: getdents . 0 2024-03-13T14:49:28.170 INFO:tasks.workunit.client.0.smithi176.stdout:1/1: dread - no filename 2024-03-13T14:49:28.170 INFO:tasks.workunit.client.0.smithi176.stdout:0/2: symlink l1 0 2024-03-13T14:49:28.170 INFO:tasks.workunit.client.0.smithi176.stdout:0/3: dread - no filename 2024-03-13T14:49:28.170 INFO:tasks.workunit.client.0.smithi176.stdout:0/4: read - no filename 2024-03-13T14:49:28.170 INFO:tasks.workunit.client.0.smithi176.stdout:0/5: dread - no filename 2024-03-13T14:49:28.171 INFO:tasks.workunit.client.0.smithi176.stdout:1/2: chown . 12359847 1 2024-03-13T14:49:28.172 INFO:tasks.workunit.client.0.smithi176.stdout:3/0: fdatasync - no filename 2024-03-13T14:49:28.172 INFO:tasks.workunit.client.0.smithi176.stdout:3/1: stat - no entries 2024-03-13T14:49:28.172 INFO:tasks.workunit.client.0.smithi176.stdout:0/6: mkdir d2 0 2024-03-13T14:49:28.172 INFO:tasks.workunit.client.0.smithi176.stdout:0/7: write - no filename 2024-03-13T14:49:28.173 INFO:tasks.workunit.client.1.smithi196.stdout:4/125: dwrite d1/d3/d7/f1a [0,4194304] 0 2024-03-13T14:49:28.176 INFO:tasks.workunit.client.1.smithi196.stdout:0/195: rename d8/f11 to d8/da/d25/d26/f42 0 2024-03-13T14:49:28.185 INFO:tasks.workunit.client.1.smithi196.stdout:6/194: dread d6/d9/d17/d19/f20 [0,4194304] 0 2024-03-13T14:49:28.186 INFO:tasks.workunit.client.1.smithi196.stdout:6/195: chown c2 5951 1 2024-03-13T14:49:28.187 INFO:tasks.workunit.client.0.smithi176.stdout:2/0: mknod c0 0 2024-03-13T14:49:28.187 INFO:tasks.workunit.client.0.smithi176.stdout:4/0: write - no filename 2024-03-13T14:49:28.188 INFO:tasks.workunit.client.0.smithi176.stdout:1/3: symlink l0 0 2024-03-13T14:49:28.188 INFO:tasks.workunit.client.0.smithi176.stdout:4/1: chown . 18928596 1 2024-03-13T14:49:28.188 INFO:tasks.workunit.client.1.smithi196.stdout:4/126: dread d1/d3/d7/f1b [0,4194304] 0 2024-03-13T14:49:28.190 INFO:tasks.workunit.client.1.smithi196.stdout:6/196: unlink d6/d9/d11/f32 0 2024-03-13T14:49:28.190 INFO:tasks.workunit.client.1.smithi196.stdout:4/127: fsync d1/f22 0 2024-03-13T14:49:28.192 INFO:tasks.workunit.client.0.smithi176.stdout:3/2: symlink l0 0 2024-03-13T14:49:28.193 INFO:tasks.workunit.client.0.smithi176.stdout:0/8: creat d2/f3 x:0 0 0 2024-03-13T14:49:28.195 INFO:tasks.workunit.client.1.smithi196.stdout:4/128: read d1/d3/d7/f1b [2040879,72103] 0 2024-03-13T14:49:28.197 INFO:tasks.workunit.client.0.smithi176.stdout:5/0: write - no filename 2024-03-13T14:49:28.197 INFO:tasks.workunit.client.0.smithi176.stdout:5/1: link - no file 2024-03-13T14:49:28.198 INFO:tasks.workunit.client.0.smithi176.stdout:1/4: creat f1 x:0 0 0 2024-03-13T14:49:28.199 INFO:tasks.workunit.client.0.smithi176.stdout:2/1: mkdir d1 0 2024-03-13T14:49:28.199 INFO:tasks.workunit.client.0.smithi176.stdout:1/5: dread - f1 zero size 2024-03-13T14:49:28.200 INFO:tasks.workunit.client.1.smithi196.stdout:4/129: unlink d1/d3/d7/f1b 0 2024-03-13T14:49:28.202 INFO:tasks.workunit.client.0.smithi176.stdout:2/2: rename d1 to d1/d2 22 2024-03-13T14:49:28.206 INFO:tasks.workunit.client.0.smithi176.stdout:1/6: write f1 [321613,81492] 0 2024-03-13T14:49:28.206 INFO:tasks.workunit.client.1.smithi196.stdout:6/197: symlink d6/d2a/dc/d1c/d34/l3d 0 2024-03-13T14:49:28.206 INFO:tasks.workunit.client.0.smithi176.stdout:3/3: creat f1 x:0 0 0 2024-03-13T14:49:28.209 INFO:tasks.workunit.client.0.smithi176.stdout:0/9: unlink l1 0 2024-03-13T14:49:28.219 INFO:tasks.workunit.client.0.smithi176.stdout:4/2: mknod c0 0 2024-03-13T14:49:28.219 INFO:tasks.workunit.client.0.smithi176.stdout:4/3: dread - no filename 2024-03-13T14:49:28.219 INFO:tasks.workunit.client.0.smithi176.stdout:4/4: fsync - no filename 2024-03-13T14:49:28.220 INFO:tasks.workunit.client.0.smithi176.stdout:4/5: chown c0 453 1 2024-03-13T14:49:28.224 INFO:tasks.workunit.client.1.smithi196.stdout:0/196: write d8/da/d25/d26/f2d [1203930,18661] 0 2024-03-13T14:49:28.225 INFO:tasks.workunit.client.0.smithi176.stdout:1/7: mkdir d2 0 2024-03-13T14:49:28.225 INFO:tasks.workunit.client.0.smithi176.stdout:1/8: stat f1 0 2024-03-13T14:49:28.226 INFO:tasks.workunit.client.0.smithi176.stdout:1/9: chown l0 200307 1 2024-03-13T14:49:28.229 INFO:tasks.workunit.client.0.smithi176.stdout:3/4: creat f2 x:0 0 0 2024-03-13T14:49:28.230 INFO:tasks.workunit.client.1.smithi196.stdout:0/197: chown c7 441205 1 2024-03-13T14:49:28.230 INFO:tasks.workunit.client.1.smithi196.stdout:6/198: rmdir d6 39 2024-03-13T14:49:28.232 INFO:tasks.workunit.client.0.smithi176.stdout:3/5: write f2 [222245,127218] 0 2024-03-13T14:49:28.232 INFO:tasks.workunit.client.0.smithi176.stdout:3/6: rmdir - no directory 2024-03-13T14:49:28.232 INFO:tasks.workunit.client.0.smithi176.stdout:5/2: creat f0 x:0 0 0 2024-03-13T14:49:28.234 INFO:tasks.workunit.client.0.smithi176.stdout:3/7: dread - f1 zero size 2024-03-13T14:49:28.236 INFO:tasks.workunit.client.0.smithi176.stdout:3/8: dread - f1 zero size 2024-03-13T14:49:28.236 INFO:tasks.workunit.client.0.smithi176.stdout:7/0: fdatasync - no filename 2024-03-13T14:49:28.237 INFO:tasks.workunit.client.0.smithi176.stdout:7/1: chown . 1724591 1 2024-03-13T14:49:28.237 INFO:tasks.workunit.client.0.smithi176.stdout:7/2: link - no file 2024-03-13T14:49:28.237 INFO:tasks.workunit.client.0.smithi176.stdout:7/3: write - no filename 2024-03-13T14:49:28.238 INFO:tasks.workunit.client.1.smithi196.stdout:6/199: chown d6/d2a/dc/d1c/f33 48321811 1 2024-03-13T14:49:28.239 INFO:tasks.workunit.client.0.smithi176.stdout:2/3: symlink d1/l3 0 2024-03-13T14:49:28.239 INFO:tasks.workunit.client.0.smithi176.stdout:4/6: creat f1 x:0 0 0 2024-03-13T14:49:28.240 INFO:tasks.workunit.client.0.smithi176.stdout:5/3: creat f1 x:0 0 0 2024-03-13T14:49:28.241 INFO:tasks.workunit.client.0.smithi176.stdout:6/0: mkdir d0 0 2024-03-13T14:49:28.242 INFO:tasks.workunit.client.0.smithi176.stdout:5/4: chown f1 3471951 1 2024-03-13T14:49:28.244 INFO:tasks.workunit.client.0.smithi176.stdout:5/5: dread - f1 zero size 2024-03-13T14:49:28.244 INFO:tasks.workunit.client.0.smithi176.stdout:8/0: link - no file 2024-03-13T14:49:28.244 INFO:tasks.workunit.client.0.smithi176.stdout:8/1: dwrite - no filename 2024-03-13T14:49:28.244 INFO:tasks.workunit.client.0.smithi176.stdout:8/2: dwrite - no filename 2024-03-13T14:49:28.245 INFO:tasks.workunit.client.0.smithi176.stdout:3/9: mknod c3 0 2024-03-13T14:49:28.245 INFO:tasks.workunit.client.1.smithi196.stdout:4/130: dwrite d1/d3/f12 [0,4194304] 0 2024-03-13T14:49:28.246 INFO:tasks.workunit.client.0.smithi176.stdout:5/6: truncate f0 539828 0 2024-03-13T14:49:28.247 INFO:tasks.workunit.client.0.smithi176.stdout:5/7: chown f1 2 1 2024-03-13T14:49:28.248 INFO:tasks.workunit.client.0.smithi176.stdout:3/10: chown f1 36274 1 2024-03-13T14:49:28.248 INFO:tasks.workunit.client.0.smithi176.stdout:5/8: fdatasync f0 0 2024-03-13T14:49:28.249 INFO:tasks.workunit.client.1.smithi196.stdout:6/200: creat d6/d2a/dc/f3e x:0 0 0 2024-03-13T14:49:28.251 INFO:tasks.workunit.client.0.smithi176.stdout:4/7: creat f2 x:0 0 0 2024-03-13T14:49:28.251 INFO:tasks.workunit.client.0.smithi176.stdout:2/4: creat d1/f4 x:0 0 0 2024-03-13T14:49:28.252 INFO:tasks.workunit.client.1.smithi196.stdout:0/198: dwrite d8/da/f10 [0,4194304] 0 2024-03-13T14:49:28.253 INFO:tasks.workunit.client.0.smithi176.stdout:4/8: truncate f2 504295 0 2024-03-13T14:49:28.255 INFO:tasks.workunit.client.1.smithi196.stdout:0/199: dread - d8/da/de/f2b zero size 2024-03-13T14:49:28.258 INFO:tasks.workunit.client.0.smithi176.stdout:9/0: truncate - no filename 2024-03-13T14:49:28.261 INFO:tasks.workunit.client.1.smithi196.stdout:6/201: creat d6/d2a/d23/d28/f3f x:0 0 0 2024-03-13T14:49:28.261 INFO:tasks.workunit.client.1.smithi196.stdout:0/200: write d8/da/f13 [3042097,45025] 0 2024-03-13T14:49:28.261 INFO:tasks.workunit.client.1.smithi196.stdout:4/131: mknod d1/d1e/c26 0 2024-03-13T14:49:28.263 INFO:tasks.workunit.client.1.smithi196.stdout:4/132: chown d1/d3/f17 2625967 1 2024-03-13T14:49:28.266 INFO:tasks.workunit.client.0.smithi176.stdout:3/11: mkdir d4 0 2024-03-13T14:49:28.267 INFO:tasks.workunit.client.1.smithi196.stdout:4/133: write d1/d3/d7/f19 [2381152,21132] 0 2024-03-13T14:49:28.271 INFO:tasks.workunit.client.0.smithi176.stdout:7/4: mknod c0 0 2024-03-13T14:49:28.271 INFO:tasks.workunit.client.0.smithi176.stdout:3/12: dread f2 [0,4194304] 0 2024-03-13T14:49:28.272 INFO:tasks.workunit.client.0.smithi176.stdout:7/5: dwrite - no filename 2024-03-13T14:49:28.274 INFO:tasks.workunit.client.0.smithi176.stdout:6/1: mknod d0/c1 0 2024-03-13T14:49:28.274 INFO:tasks.workunit.client.0.smithi176.stdout:6/2: dwrite - no filename 2024-03-13T14:49:28.274 INFO:tasks.workunit.client.0.smithi176.stdout:3/13: chown d4 7040944 1 2024-03-13T14:49:28.276 INFO:tasks.workunit.client.0.smithi176.stdout:2/5: creat d1/f5 x:0 0 0 2024-03-13T14:49:28.277 INFO:tasks.workunit.client.0.smithi176.stdout:8/3: mkdir d0 0 2024-03-13T14:49:28.278 INFO:tasks.workunit.client.0.smithi176.stdout:2/6: fdatasync d1/f4 0 2024-03-13T14:49:28.280 INFO:tasks.workunit.client.0.smithi176.stdout:9/1: mkdir d0 0 2024-03-13T14:49:28.280 INFO:tasks.workunit.client.0.smithi176.stdout:9/2: write - no filename 2024-03-13T14:49:28.280 INFO:tasks.workunit.client.0.smithi176.stdout:9/3: write - no filename 2024-03-13T14:49:28.281 INFO:tasks.workunit.client.0.smithi176.stdout:6/3: symlink d0/l2 0 2024-03-13T14:49:28.284 INFO:tasks.workunit.client.0.smithi176.stdout:3/14: creat d4/f5 x:0 0 0 2024-03-13T14:49:28.285 INFO:tasks.workunit.client.1.smithi196.stdout:6/202: dread d6/d7/dd/f2c [0,4194304] 0 2024-03-13T14:49:28.287 INFO:tasks.workunit.client.0.smithi176.stdout:8/4: mkdir d0/d1 0 2024-03-13T14:49:28.287 INFO:tasks.workunit.client.0.smithi176.stdout:8/5: read - no filename 2024-03-13T14:49:28.287 INFO:tasks.workunit.client.0.smithi176.stdout:8/6: truncate - no filename 2024-03-13T14:49:28.287 INFO:tasks.workunit.client.0.smithi176.stdout:8/7: dread - no filename 2024-03-13T14:49:28.289 INFO:tasks.workunit.client.0.smithi176.stdout:6/4: mknod d0/c3 0 2024-03-13T14:49:28.290 INFO:tasks.workunit.client.0.smithi176.stdout:6/5: chown d0/l2 0 1 2024-03-13T14:49:28.291 INFO:tasks.workunit.client.0.smithi176.stdout:6/6: dread - no filename 2024-03-13T14:49:28.291 INFO:tasks.workunit.client.0.smithi176.stdout:9/4: getdents d0 0 2024-03-13T14:49:28.291 INFO:tasks.workunit.client.0.smithi176.stdout:9/5: read - no filename 2024-03-13T14:49:28.293 INFO:tasks.workunit.client.0.smithi176.stdout:4/9: getdents . 0 2024-03-13T14:49:28.293 INFO:tasks.workunit.client.0.smithi176.stdout:2/7: link d1/l3 d1/l6 0 2024-03-13T14:49:28.294 INFO:tasks.workunit.client.1.smithi196.stdout:4/134: dwrite d1/d3/d7/f1d [0,4194304] 0 2024-03-13T14:49:28.295 INFO:tasks.workunit.client.1.smithi196.stdout:3/87: sync 2024-03-13T14:49:28.296 INFO:tasks.workunit.client.1.smithi196.stdout:1/130: sync 2024-03-13T14:49:28.296 INFO:tasks.workunit.client.1.smithi196.stdout:9/129: sync 2024-03-13T14:49:28.297 INFO:tasks.workunit.client.0.smithi176.stdout:6/7: mknod d0/c4 0 2024-03-13T14:49:28.297 INFO:tasks.workunit.client.0.smithi176.stdout:6/8: fsync - no filename 2024-03-13T14:49:28.297 INFO:tasks.workunit.client.0.smithi176.stdout:6/9: fsync - no filename 2024-03-13T14:49:28.298 INFO:tasks.workunit.client.1.smithi196.stdout:3/88: write db/f13 [256093,88608] 0 2024-03-13T14:49:28.300 INFO:tasks.workunit.client.1.smithi196.stdout:3/89: stat db/ff 0 2024-03-13T14:49:28.303 INFO:tasks.workunit.client.1.smithi196.stdout:4/135: mknod d1/d3/c27 0 2024-03-13T14:49:28.307 INFO:tasks.workunit.client.0.smithi176.stdout:2/8: mknod d1/c7 0 2024-03-13T14:49:28.308 INFO:tasks.workunit.client.1.smithi196.stdout:9/130: rmdir d7/d9 39 2024-03-13T14:49:28.311 INFO:tasks.workunit.client.1.smithi196.stdout:0/201: truncate d8/da/d25/d26/f42 2559869 0 2024-03-13T14:49:28.311 INFO:tasks.workunit.client.0.smithi176.stdout:4/10: symlink l3 0 2024-03-13T14:49:28.312 INFO:tasks.workunit.client.1.smithi196.stdout:6/203: getdents d6/d9/d17 0 2024-03-13T14:49:28.313 INFO:tasks.workunit.client.0.smithi176.stdout:9/6: getdents d0 0 2024-03-13T14:49:28.313 INFO:tasks.workunit.client.1.smithi196.stdout:3/90: mknod db/d19/c1f 0 2024-03-13T14:49:28.315 INFO:tasks.workunit.client.0.smithi176.stdout:3/15: dwrite d4/f5 [0,4194304] 0 2024-03-13T14:49:28.316 INFO:tasks.workunit.client.0.smithi176.stdout:3/16: chown f2 154175359 1 2024-03-13T14:49:28.318 INFO:tasks.workunit.client.1.smithi196.stdout:3/91: write db/f13 [622519,46155] 0 2024-03-13T14:49:28.319 INFO:tasks.workunit.client.0.smithi176.stdout:3/17: write d4/f5 [1292746,67648] 0 2024-03-13T14:49:28.320 INFO:tasks.workunit.client.0.smithi176.stdout:8/8: rmdir d0/d1 0 2024-03-13T14:49:28.320 INFO:tasks.workunit.client.0.smithi176.stdout:8/9: stat d0 0 2024-03-13T14:49:28.324 INFO:tasks.workunit.client.0.smithi176.stdout:2/9: rename d1/f5 to d1/f8 0 2024-03-13T14:49:28.325 INFO:tasks.workunit.client.1.smithi196.stdout:6/204: chown d6/d9/d11/f13 2771 1 2024-03-13T14:49:28.328 INFO:tasks.workunit.client.1.smithi196.stdout:1/131: creat d1/f26 x:0 0 0 2024-03-13T14:49:28.330 INFO:tasks.workunit.client.1.smithi196.stdout:1/132: chown d1/d11 6387942 1 2024-03-13T14:49:28.337 INFO:tasks.workunit.client.1.smithi196.stdout:3/92: fsync f7 0 2024-03-13T14:49:28.341 INFO:tasks.workunit.client.0.smithi176.stdout:4/11: mknod c4 0 2024-03-13T14:49:28.342 INFO:tasks.workunit.client.0.smithi176.stdout:2/10: dread - d1/f4 zero size 2024-03-13T14:49:28.343 INFO:tasks.workunit.client.0.smithi176.stdout:3/18: truncate f2 890323 0 2024-03-13T14:49:28.344 INFO:tasks.workunit.client.1.smithi196.stdout:6/205: mkdir d6/d2a/dc/d1c/d34/d40 0 2024-03-13T14:49:28.344 INFO:tasks.workunit.client.0.smithi176.stdout:8/10: creat d0/f2 x:0 0 0 2024-03-13T14:49:28.344 INFO:tasks.workunit.client.0.smithi176.stdout:4/12: write f1 [306083,74533] 0 2024-03-13T14:49:28.344 INFO:tasks.workunit.client.0.smithi176.stdout:2/11: dread - d1/f8 zero size 2024-03-13T14:49:28.349 INFO:tasks.workunit.client.1.smithi196.stdout:3/93: creat db/f20 x:0 0 0 2024-03-13T14:49:28.349 INFO:tasks.workunit.client.0.smithi176.stdout:2/12: chown d1/f4 412 1 2024-03-13T14:49:28.350 INFO:tasks.workunit.client.1.smithi196.stdout:0/202: getdents d8/da/d25/d26 0 2024-03-13T14:49:28.353 INFO:tasks.workunit.client.1.smithi196.stdout:3/94: truncate db/d15/f17 231090 0 2024-03-13T14:49:28.360 INFO:tasks.workunit.client.1.smithi196.stdout:6/206: mknod d6/d2a/dc/d1c/d34/d40/c41 0 2024-03-13T14:49:28.362 INFO:tasks.workunit.client.1.smithi196.stdout:3/95: creat db/d15/f21 x:0 0 0 2024-03-13T14:49:28.363 INFO:tasks.workunit.client.1.smithi196.stdout:1/133: getdents d1/d9 0 2024-03-13T14:49:28.367 INFO:tasks.workunit.client.1.smithi196.stdout:9/131: dwrite d7/de/f1f [0,4194304] 0 2024-03-13T14:49:28.368 INFO:tasks.workunit.client.1.smithi196.stdout:3/96: write db/d15/f17 [1031985,25488] 0 2024-03-13T14:49:28.372 INFO:tasks.workunit.client.1.smithi196.stdout:3/97: readlink la 0 2024-03-13T14:49:28.375 INFO:tasks.workunit.client.1.smithi196.stdout:1/134: rename d1/d11/d13 to d1/d9/d27 0 2024-03-13T14:49:28.388 INFO:tasks.workunit.client.1.smithi196.stdout:3/98: chown db/lc 27886 1 2024-03-13T14:49:28.391 INFO:tasks.workunit.client.1.smithi196.stdout:3/99: read db/fd [1759695,63289] 0 2024-03-13T14:49:28.393 INFO:tasks.workunit.client.1.smithi196.stdout:1/135: mkdir d1/d28 0 2024-03-13T14:49:28.398 INFO:tasks.workunit.client.1.smithi196.stdout:9/132: getdents d7/d9/df/d1d 0 2024-03-13T14:49:28.400 INFO:tasks.workunit.client.1.smithi196.stdout:9/133: write d7/d9/df/f19 [69123,10569] 0 2024-03-13T14:49:28.403 INFO:tasks.workunit.client.1.smithi196.stdout:3/100: mkdir db/d19/d22 0 2024-03-13T14:49:28.405 INFO:tasks.workunit.client.1.smithi196.stdout:3/101: write f3 [1192248,40771] 0 2024-03-13T14:49:28.406 INFO:tasks.workunit.client.1.smithi196.stdout:1/136: write d1/d9/f18 [947333,34019] 0 2024-03-13T14:49:28.411 INFO:tasks.workunit.client.1.smithi196.stdout:6/207: write f4 [1442267,106095] 0 2024-03-13T14:49:28.411 INFO:tasks.workunit.client.1.smithi196.stdout:3/102: write db/ff [2994945,86971] 0 2024-03-13T14:49:28.412 INFO:tasks.workunit.client.1.smithi196.stdout:1/137: write d1/d11/f1a [5135599,114408] 0 2024-03-13T14:49:28.412 INFO:tasks.workunit.client.1.smithi196.stdout:3/103: chown db/f20 2019 1 2024-03-13T14:49:28.413 INFO:tasks.workunit.client.1.smithi196.stdout:9/134: rmdir d7/de/d16 39 2024-03-13T14:49:28.417 INFO:tasks.workunit.client.1.smithi196.stdout:3/104: fsync db/d15/f21 0 2024-03-13T14:49:28.417 INFO:tasks.workunit.client.1.smithi196.stdout:9/135: read d7/d9/df/f1c [2734101,34561] 0 2024-03-13T14:49:28.421 INFO:tasks.workunit.client.1.smithi196.stdout:6/208: creat d6/d9/d17/d19/f42 x:0 0 0 2024-03-13T14:49:28.428 INFO:tasks.workunit.client.1.smithi196.stdout:1/138: unlink d1/c3 0 2024-03-13T14:49:28.437 INFO:tasks.workunit.client.1.smithi196.stdout:0/203: write d8/da/d25/d26/f42 [516198,23658] 0 2024-03-13T14:49:28.437 INFO:tasks.workunit.client.1.smithi196.stdout:3/105: dwrite f8 [0,4194304] 0 2024-03-13T14:49:28.438 INFO:tasks.workunit.client.1.smithi196.stdout:9/136: creat d7/d9/df/f21 x:0 0 0 2024-03-13T14:49:28.444 INFO:tasks.workunit.client.1.smithi196.stdout:6/209: symlink d6/d9/d17/d19/l43 0 2024-03-13T14:49:28.452 INFO:tasks.workunit.client.1.smithi196.stdout:0/204: mkdir d8/da/de/d43 0 2024-03-13T14:49:28.467 INFO:tasks.workunit.client.1.smithi196.stdout:9/137: mkdir d7/d9/df/d1d/d22 0 2024-03-13T14:49:28.469 INFO:tasks.workunit.client.1.smithi196.stdout:6/210: mkdir d6/d7/df/d27/d39/d44 0 2024-03-13T14:49:28.470 INFO:tasks.workunit.client.1.smithi196.stdout:9/138: truncate d7/d9/df/f1a 435252 0 2024-03-13T14:49:28.481 INFO:tasks.workunit.client.1.smithi196.stdout:5/210: sync 2024-03-13T14:49:28.481 INFO:tasks.workunit.client.1.smithi196.stdout:0/205: write d8/da/f38 [2792488,25578] 0 2024-03-13T14:49:28.483 INFO:tasks.workunit.client.1.smithi196.stdout:9/139: creat d7/d9/df/d1d/f23 x:0 0 0 2024-03-13T14:49:28.486 INFO:tasks.workunit.client.1.smithi196.stdout:1/139: getdents d1/d9/d27 0 2024-03-13T14:49:28.487 INFO:tasks.workunit.client.1.smithi196.stdout:9/140: dread - d7/d9/df/f21 zero size 2024-03-13T14:49:28.489 INFO:tasks.workunit.client.1.smithi196.stdout:6/211: mknod d6/c45 0 2024-03-13T14:49:28.489 INFO:tasks.workunit.client.1.smithi196.stdout:3/106: dwrite db/fd [0,4194304] 0 2024-03-13T14:49:28.489 INFO:tasks.workunit.client.1.smithi196.stdout:5/211: write d4/db/d14/d19/d1c/d25/d38/f3e [355718,50910] 0 2024-03-13T14:49:28.490 INFO:tasks.workunit.client.1.smithi196.stdout:0/206: chown d8/da/de/f2b 16 1 2024-03-13T14:49:28.492 INFO:tasks.workunit.client.1.smithi196.stdout:3/107: readlink db/l12 0 2024-03-13T14:49:28.494 INFO:tasks.workunit.client.1.smithi196.stdout:3/108: dread - db/d15/f1a zero size 2024-03-13T14:49:28.494 INFO:tasks.workunit.client.1.smithi196.stdout:9/141: read d7/de/f1f [2999152,31989] 0 2024-03-13T14:49:28.494 INFO:tasks.workunit.client.1.smithi196.stdout:1/140: stat d1/d9 0 2024-03-13T14:49:28.495 INFO:tasks.workunit.client.1.smithi196.stdout:6/212: chown d6/d2a/dc/d1c/f3b 33968245 1 2024-03-13T14:49:28.499 INFO:tasks.workunit.client.1.smithi196.stdout:5/212: write d4/db/d14/d19/d1c/f3f [306794,46969] 0 2024-03-13T14:49:28.508 INFO:tasks.workunit.client.1.smithi196.stdout:6/213: rename c2 to d6/d2a/dc/d1c/d34/c46 0 2024-03-13T14:49:28.515 INFO:tasks.workunit.client.1.smithi196.stdout:6/214: truncate d6/d7/df/f2e 5219922 0 2024-03-13T14:49:28.520 INFO:tasks.workunit.client.1.smithi196.stdout:0/207: dread d8/f1e [0,4194304] 0 2024-03-13T14:49:28.521 INFO:tasks.workunit.client.1.smithi196.stdout:9/142: dread d7/de/f1f [0,4194304] 0 2024-03-13T14:49:28.522 INFO:tasks.workunit.client.1.smithi196.stdout:6/215: mkdir d6/d2a/dc/d47 0 2024-03-13T14:49:28.525 INFO:tasks.workunit.client.1.smithi196.stdout:8/75: sync 2024-03-13T14:49:28.525 INFO:tasks.workunit.client.1.smithi196.stdout:4/136: sync 2024-03-13T14:49:28.525 INFO:tasks.workunit.client.1.smithi196.stdout:7/197: sync 2024-03-13T14:49:28.525 INFO:tasks.workunit.client.1.smithi196.stdout:2/78: sync 2024-03-13T14:49:28.533 INFO:tasks.workunit.client.1.smithi196.stdout:4/137: dread d1/d3/f17 [0,4194304] 0 2024-03-13T14:49:28.536 INFO:tasks.workunit.client.1.smithi196.stdout:8/76: creat d0/d10/f19 x:0 0 0 2024-03-13T14:49:28.541 INFO:tasks.workunit.client.1.smithi196.stdout:3/109: dwrite db/d15/f1c [0,4194304] 0 2024-03-13T14:49:28.541 INFO:tasks.workunit.client.1.smithi196.stdout:1/141: dwrite d1/d11/f1a [0,4194304] 0 2024-03-13T14:49:28.542 INFO:tasks.workunit.client.1.smithi196.stdout:6/216: rename d6/d2a/dc/d1c/f33 to d6/d9/d11/f48 0 2024-03-13T14:49:28.542 INFO:tasks.workunit.client.1.smithi196.stdout:4/138: chown d1/d3/c27 1 1 2024-03-13T14:49:28.545 INFO:tasks.workunit.client.1.smithi196.stdout:6/217: chown d6/d7/df/d27/d39/d44 631159 1 2024-03-13T14:49:28.545 INFO:tasks.workunit.client.1.smithi196.stdout:4/139: write d1/d3/f12 [1287338,46503] 0 2024-03-13T14:49:28.547 INFO:tasks.workunit.client.1.smithi196.stdout:5/213: dread d4/db/fd [0,4194304] 0 2024-03-13T14:49:28.547 INFO:tasks.workunit.client.1.smithi196.stdout:9/143: mkdir d7/d24 0 2024-03-13T14:49:28.553 INFO:tasks.workunit.client.1.smithi196.stdout:7/198: creat d0/d9/d31/f42 x:0 0 0 2024-03-13T14:49:28.564 INFO:tasks.workunit.client.1.smithi196.stdout:1/142: fsync d1/d9/d27/f16 0 2024-03-13T14:49:28.567 INFO:tasks.workunit.client.1.smithi196.stdout:0/208: truncate d8/f12 151614 0 2024-03-13T14:49:28.576 INFO:tasks.workunit.client.1.smithi196.stdout:4/140: mknod d1/d3/c28 0 2024-03-13T14:49:28.580 INFO:tasks.workunit.client.1.smithi196.stdout:3/110: rename db/d15/f1c to db/d19/d22/f23 0 2024-03-13T14:49:28.582 INFO:tasks.workunit.client.1.smithi196.stdout:5/214: dwrite d4/db/d14/d19/f3b [0,4194304] 0 2024-03-13T14:49:28.583 INFO:tasks.workunit.client.1.smithi196.stdout:6/218: write d6/d7/dd/f2c [1035380,79765] 0 2024-03-13T14:49:28.584 INFO:tasks.workunit.client.1.smithi196.stdout:1/143: dread f0 [0,4194304] 0 2024-03-13T14:49:28.584 INFO:tasks.workunit.client.1.smithi196.stdout:5/215: fsync d4/f11 0 2024-03-13T14:49:28.588 INFO:tasks.workunit.client.1.smithi196.stdout:5/216: chown d4/db/d14/d19/d26/f3a 667398105 1 2024-03-13T14:49:28.592 INFO:tasks.workunit.client.1.smithi196.stdout:8/77: mkdir d0/d10/d12/d17/d1a 0 2024-03-13T14:49:28.599 INFO:tasks.workunit.client.1.smithi196.stdout:7/199: chown d0/c4 69 1 2024-03-13T14:49:28.599 INFO:tasks.workunit.client.1.smithi196.stdout:8/78: write d0/d10/f11 [851396,49864] 0 2024-03-13T14:49:28.604 INFO:tasks.workunit.client.1.smithi196.stdout:8/79: fdatasync d0/f1 0 2024-03-13T14:49:28.606 INFO:tasks.workunit.client.1.smithi196.stdout:0/209: mkdir d8/da/de/d33/d44 0 2024-03-13T14:49:28.609 INFO:tasks.workunit.client.1.smithi196.stdout:1/144: dread d1/d9/d27/f19 [0,4194304] 0 2024-03-13T14:49:28.618 INFO:tasks.workunit.client.1.smithi196.stdout:4/141: unlink d1/d3/c27 0 2024-03-13T14:49:28.621 INFO:tasks.workunit.client.1.smithi196.stdout:3/111: mknod db/d19/c24 0 2024-03-13T14:49:28.623 INFO:tasks.workunit.client.1.smithi196.stdout:6/219: rmdir d6/d2a 39 2024-03-13T14:49:28.628 INFO:tasks.workunit.client.1.smithi196.stdout:6/220: fdatasync d6/d9/f1f 0 2024-03-13T14:49:28.628 INFO:tasks.workunit.client.1.smithi196.stdout:9/144: symlink d7/d24/l25 0 2024-03-13T14:49:28.636 INFO:tasks.workunit.client.1.smithi196.stdout:6/221: chown d6/d9/d11/f31 6 1 2024-03-13T14:49:28.636 INFO:tasks.workunit.client.1.smithi196.stdout:7/200: dwrite d0/d9/d1e/f29 [0,4194304] 0 2024-03-13T14:49:28.645 INFO:tasks.workunit.client.1.smithi196.stdout:1/145: dread d1/d11/f1a [0,4194304] 0 2024-03-13T14:49:28.645 INFO:tasks.workunit.client.1.smithi196.stdout:9/145: dread d7/d9/df/f1a [0,4194304] 0 2024-03-13T14:49:28.645 INFO:tasks.workunit.client.1.smithi196.stdout:3/112: dwrite f8 [4194304,4194304] 0 2024-03-13T14:49:28.651 INFO:tasks.workunit.client.1.smithi196.stdout:8/80: mkdir d0/d10/d12/d1b 0 2024-03-13T14:49:28.655 INFO:tasks.workunit.client.1.smithi196.stdout:0/210: stat d8/da/de/d17/c2a 0 2024-03-13T14:49:28.666 INFO:tasks.workunit.client.1.smithi196.stdout:0/211: dwrite d8/da/d25/d26/f42 [0,4194304] 0 2024-03-13T14:49:28.788 INFO:tasks.workunit.client.1.smithi196.stdout:5/217: mkdir d4/db/d14/d37/d41 0 2024-03-13T14:49:28.789 INFO:tasks.workunit.client.1.smithi196.stdout:5/218: fdatasync d4/f11 0 2024-03-13T14:49:28.794 INFO:tasks.workunit.client.1.smithi196.stdout:4/142: rmdir d1/d3 39 2024-03-13T14:49:28.798 INFO:tasks.workunit.client.1.smithi196.stdout:1/146: fsync d1/d9/f18 0 2024-03-13T14:49:28.803 INFO:tasks.workunit.client.1.smithi196.stdout:8/81: fsync d0/d10/f15 0 2024-03-13T14:49:28.813 INFO:tasks.workunit.client.1.smithi196.stdout:1/147: dwrite d1/d9/d27/f25 [0,4194304] 0 2024-03-13T14:49:28.849 INFO:tasks.workunit.client.0.smithi176.stdout:9/7: symlink d0/l1 0 2024-03-13T14:49:28.849 INFO:tasks.workunit.client.0.smithi176.stdout:9/8: fsync - no filename 2024-03-13T14:49:28.851 INFO:tasks.workunit.client.0.smithi176.stdout:8/11: creat d0/f3 x:0 0 0 2024-03-13T14:49:28.851 INFO:tasks.workunit.client.0.smithi176.stdout:8/12: chown d0 128839415 1 2024-03-13T14:49:28.853 INFO:tasks.workunit.client.0.smithi176.stdout:4/13: rename f2 to f5 0 2024-03-13T14:49:28.853 INFO:tasks.workunit.client.0.smithi176.stdout:8/13: write d0/f3 [407298,20633] 0 2024-03-13T14:49:28.856 INFO:tasks.workunit.client.0.smithi176.stdout:3/19: rename c3 to d4/c6 0 2024-03-13T14:49:28.857 INFO:tasks.workunit.client.0.smithi176.stdout:2/13: readlink d1/l6 0 2024-03-13T14:49:28.859 INFO:tasks.workunit.client.0.smithi176.stdout:6/10: getdents d0 0 2024-03-13T14:49:28.860 INFO:tasks.workunit.client.0.smithi176.stdout:9/9: symlink d0/l2 0 2024-03-13T14:49:28.860 INFO:tasks.workunit.client.0.smithi176.stdout:9/10: read - no filename 2024-03-13T14:49:28.860 INFO:tasks.workunit.client.0.smithi176.stdout:9/11: write - no filename 2024-03-13T14:49:28.862 INFO:tasks.workunit.client.0.smithi176.stdout:4/14: mknod c6 0 2024-03-13T14:49:28.864 INFO:tasks.workunit.client.0.smithi176.stdout:4/15: chown l3 4 1 2024-03-13T14:49:28.866 INFO:tasks.workunit.client.0.smithi176.stdout:4/16: read f1 [233190,26057] 0 2024-03-13T14:49:28.870 INFO:tasks.workunit.client.0.smithi176.stdout:4/17: mkdir d7 0 2024-03-13T14:49:28.871 INFO:tasks.workunit.client.0.smithi176.stdout:9/12: link d0/l1 d0/l3 0 2024-03-13T14:49:28.873 INFO:tasks.workunit.client.0.smithi176.stdout:9/13: rmdir d0 39 2024-03-13T14:49:28.873 INFO:tasks.workunit.client.0.smithi176.stdout:9/14: fdatasync - no filename 2024-03-13T14:49:28.874 INFO:tasks.workunit.client.1.smithi196.stdout:6/222: mknod d6/d7/c49 0 2024-03-13T14:49:28.874 INFO:tasks.workunit.client.1.smithi196.stdout:7/201: rename d0/d6/d35/f41 to d0/d9/d1e/d3f/f43 0 2024-03-13T14:49:28.876 INFO:tasks.workunit.client.1.smithi196.stdout:8/82: mkdir d0/d1c 0 2024-03-13T14:49:28.876 INFO:tasks.workunit.client.1.smithi196.stdout:0/212: creat d8/da/de/d43/f45 x:0 0 0 2024-03-13T14:49:28.876 INFO:tasks.workunit.client.1.smithi196.stdout:5/219: creat d4/db/d14/d37/d41/f42 x:0 0 0 2024-03-13T14:49:28.877 INFO:tasks.workunit.client.0.smithi176.stdout:9/15: chown d0 47933 1 2024-03-13T14:49:28.879 INFO:tasks.workunit.client.0.smithi176.stdout:4/18: dread f5 [0,4194304] 0 2024-03-13T14:49:28.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:28.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:28.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:28.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:49:28.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:49:28.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:28 smithi176 ceph-mon[29064]: mgrmap e27: smithi176.tzfdwd(active, since 1.02775s) 2024-03-13T14:49:28.880 INFO:tasks.workunit.client.1.smithi196.stdout:8/83: write d0/d10/f19 [322198,87999] 0 2024-03-13T14:49:28.881 INFO:tasks.workunit.client.1.smithi196.stdout:6/223: chown d6/d2a/d23/d28/f2f 4922 1 2024-03-13T14:49:28.884 INFO:tasks.workunit.client.1.smithi196.stdout:7/202: dread d0/d9/d1e/d3f/f43 [0,4194304] 0 2024-03-13T14:49:28.884 INFO:tasks.workunit.client.0.smithi176.stdout:9/16: readlink d0/l1 0 2024-03-13T14:49:28.885 INFO:tasks.workunit.client.0.smithi176.stdout:9/17: dread - no filename 2024-03-13T14:49:28.885 INFO:tasks.workunit.client.0.smithi176.stdout:9/18: dwrite - no filename 2024-03-13T14:49:28.888 INFO:tasks.workunit.client.0.smithi176.stdout:9/19: chown d0/l2 18 1 2024-03-13T14:49:28.888 INFO:tasks.workunit.client.1.smithi196.stdout:0/213: mknod d8/da/de/d17/c46 0 2024-03-13T14:49:28.891 INFO:tasks.workunit.client.1.smithi196.stdout:6/224: symlink d6/d7/df/l4a 0 2024-03-13T14:49:28.892 INFO:tasks.workunit.client.1.smithi196.stdout:0/214: write d8/da/d25/f27 [2741640,91849] 0 2024-03-13T14:49:28.892 INFO:tasks.workunit.client.0.smithi176.stdout:2/14: dwrite d1/f8 [0,4194304] 0 2024-03-13T14:49:28.893 INFO:tasks.workunit.client.0.smithi176.stdout:9/20: mknod d0/c4 0 2024-03-13T14:49:28.894 INFO:tasks.workunit.client.1.smithi196.stdout:6/225: dread - d6/d9/f30 zero size 2024-03-13T14:49:28.897 INFO:tasks.workunit.client.1.smithi196.stdout:1/148: rename d1/d11 to d1/d29 0 2024-03-13T14:49:28.899 INFO:tasks.workunit.client.1.smithi196.stdout:1/149: fdatasync d1/d9/d27/f25 0 2024-03-13T14:49:28.900 INFO:tasks.workunit.client.1.smithi196.stdout:7/203: readlink d0/d9/lb 0 2024-03-13T14:49:28.903 INFO:tasks.workunit.client.0.smithi176.stdout:9/21: mkdir d0/d5 0 2024-03-13T14:49:28.904 INFO:tasks.workunit.client.1.smithi196.stdout:8/84: symlink d0/d10/d14/l1d 0 2024-03-13T14:49:28.906 INFO:tasks.workunit.client.1.smithi196.stdout:0/215: fdatasync d8/da/f32 0 2024-03-13T14:49:28.911 INFO:tasks.workunit.client.1.smithi196.stdout:8/85: write d0/d10/f11 [956860,17871] 0 2024-03-13T14:49:28.912 INFO:tasks.workunit.client.1.smithi196.stdout:0/216: write d8/da/d25/f2f [423556,121660] 0 2024-03-13T14:49:28.913 INFO:tasks.workunit.client.0.smithi176.stdout:3/20: chown d4/c6 1972905 1 2024-03-13T14:49:28.919 INFO:tasks.workunit.client.0.smithi176.stdout:4/19: dwrite f1 [0,4194304] 0 2024-03-13T14:49:28.921 INFO:tasks.workunit.client.0.smithi176.stdout:9/22: rename d0/l1 to d0/l6 0 2024-03-13T14:49:28.921 INFO:tasks.workunit.client.0.smithi176.stdout:9/23: fsync - no filename 2024-03-13T14:49:28.924 INFO:tasks.workunit.client.0.smithi176.stdout:3/21: symlink d4/l7 0 2024-03-13T14:49:28.926 INFO:tasks.workunit.client.0.smithi176.stdout:3/22: rename d4 to d4/d8 22 2024-03-13T14:49:28.926 INFO:tasks.workunit.client.0.smithi176.stdout:2/15: dwrite d1/f8 [0,4194304] 0 2024-03-13T14:49:28.926 INFO:tasks.workunit.client.1.smithi196.stdout:2/79: sync 2024-03-13T14:49:28.927 INFO:tasks.workunit.client.1.smithi196.stdout:5/220: getdents d4/db/d3c 0 2024-03-13T14:49:28.930 INFO:tasks.workunit.client.0.smithi176.stdout:4/20: fsync f5 0 2024-03-13T14:49:28.931 INFO:tasks.workunit.client.1.smithi196.stdout:6/226: rename d6/d2a/dc/d1c/d34/d40 to d6/d4b 0 2024-03-13T14:49:28.931 INFO:tasks.workunit.client.1.smithi196.stdout:1/150: creat d1/f2a x:0 0 0 2024-03-13T14:49:28.935 INFO:tasks.workunit.client.1.smithi196.stdout:6/227: fsync d6/d2a/dc/d1c/f25 0 2024-03-13T14:49:28.936 INFO:tasks.workunit.client.1.smithi196.stdout:5/221: creat d4/db/d14/d19/d26/f43 x:0 0 0 2024-03-13T14:49:28.939 INFO:tasks.workunit.client.1.smithi196.stdout:7/204: getdents d0 0 2024-03-13T14:49:28.941 INFO:tasks.workunit.client.1.smithi196.stdout:8/86: rename d0/d10/d12/d17/d1a to d0/d10/d12/d1e 0 2024-03-13T14:49:28.943 INFO:tasks.workunit.client.1.smithi196.stdout:2/80: mknod d0/d6/cf 0 2024-03-13T14:49:28.944 INFO:tasks.workunit.client.1.smithi196.stdout:0/217: dwrite d8/da/d25/f27 [0,4194304] 0 2024-03-13T14:49:28.946 INFO:tasks.workunit.client.1.smithi196.stdout:2/81: read d0/f2 [1933771,11586] 0 2024-03-13T14:49:28.949 INFO:tasks.workunit.client.0.smithi176.stdout:2/16: dwrite d1/f8 [0,4194304] 0 2024-03-13T14:49:28.950 INFO:tasks.workunit.client.1.smithi196.stdout:0/218: fsync d8/da/f10 0 2024-03-13T14:49:28.950 INFO:tasks.workunit.client.1.smithi196.stdout:7/205: dread d0/d6/f2c [0,4194304] 0 2024-03-13T14:49:28.950 INFO:tasks.workunit.client.0.smithi176.stdout:2/17: readlink d1/l6 0 2024-03-13T14:49:28.952 INFO:tasks.workunit.client.1.smithi196.stdout:1/151: creat d1/d28/f2b x:0 0 0 2024-03-13T14:49:28.954 INFO:tasks.workunit.client.1.smithi196.stdout:8/87: creat d0/d10/d14/f1f x:0 0 0 2024-03-13T14:49:28.958 INFO:tasks.workunit.client.1.smithi196.stdout:0/219: write d8/da/de/d43/f45 [237773,39714] 0 2024-03-13T14:49:28.960 INFO:tasks.workunit.client.1.smithi196.stdout:5/222: rename d4/db/l17 to d4/db/d14/d19/l44 0 2024-03-13T14:49:28.963 INFO:tasks.workunit.client.1.smithi196.stdout:2/82: symlink d0/d7/d9/l10 0 2024-03-13T14:49:28.964 INFO:tasks.workunit.client.0.smithi176.stdout:2/18: rename d1/l6 to d1/l9 0 2024-03-13T14:49:28.966 INFO:tasks.workunit.client.1.smithi196.stdout:6/228: dread d6/d2a/dc/d1c/f25 [0,4194304] 0 2024-03-13T14:49:28.967 INFO:tasks.workunit.client.1.smithi196.stdout:8/88: mknod d0/d10/d12/c20 0 2024-03-13T14:49:28.969 INFO:tasks.workunit.client.1.smithi196.stdout:6/229: chown d6/d2a 59430 1 2024-03-13T14:49:28.970 INFO:tasks.workunit.client.1.smithi196.stdout:1/152: fsync d1/d9/d27/f17 0 2024-03-13T14:49:28.972 INFO:tasks.workunit.client.1.smithi196.stdout:8/89: chown d0/d10/d12/d1b 11439951 1 2024-03-13T14:49:28.974 INFO:tasks.workunit.client.0.smithi176.stdout:9/24: chown d0/l6 4146683 1 2024-03-13T14:49:28.974 INFO:tasks.workunit.client.0.smithi176.stdout:9/25: read - no filename 2024-03-13T14:49:28.974 INFO:tasks.workunit.client.1.smithi196.stdout:6/230: write d6/d9/d17/d19/f2d [1008701,103237] 0 2024-03-13T14:49:28.977 INFO:tasks.workunit.client.1.smithi196.stdout:9/146: sync 2024-03-13T14:49:28.977 INFO:tasks.workunit.client.1.smithi196.stdout:3/113: sync 2024-03-13T14:49:28.977 INFO:tasks.workunit.client.1.smithi196.stdout:4/143: sync 2024-03-13T14:49:28.979 INFO:tasks.workunit.client.1.smithi196.stdout:3/114: fsync db/d19/f1d 0 2024-03-13T14:49:28.985 INFO:tasks.workunit.client.1.smithi196.stdout:7/206: dwrite d0/d9/f30 [0,4194304] 0 2024-03-13T14:49:28.991 INFO:tasks.workunit.client.1.smithi196.stdout:7/207: write d0/d9/d31/f42 [468312,70051] 0 2024-03-13T14:49:28.999 INFO:tasks.workunit.client.0.smithi176.stdout:2/19: stat d1/l3 0 2024-03-13T14:49:28.999 INFO:tasks.workunit.client.0.smithi176.stdout:2/20: dread - d1/f4 zero size 2024-03-13T14:49:29.000 INFO:tasks.workunit.client.0.smithi176.stdout:3/23: write f2 [1021905,78450] 0 2024-03-13T14:49:29.003 INFO:tasks.workunit.client.0.smithi176.stdout:9/26: symlink d0/d5/l7 0 2024-03-13T14:49:29.013 INFO:tasks.workunit.client.0.smithi176.stdout:3/24: mknod d4/c9 0 2024-03-13T14:49:29.014 INFO:tasks.workunit.client.0.smithi176.stdout:3/25: chown d4/c6 30 1 2024-03-13T14:49:29.016 INFO:tasks.workunit.client.0.smithi176.stdout:4/21: fdatasync f1 0 2024-03-13T14:49:29.021 INFO:tasks.workunit.client.0.smithi176.stdout:9/27: creat d0/d5/f8 x:0 0 0 2024-03-13T14:49:29.030 INFO:tasks.workunit.client.0.smithi176.stdout:3/26: dwrite f2 [0,4194304] 0 2024-03-13T14:49:29.031 INFO:tasks.workunit.client.1.smithi196.stdout:2/83: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:29.032 INFO:tasks.workunit.client.1.smithi196.stdout:4/144: write d1/d3/f12 [2232873,101159] 0 2024-03-13T14:49:29.032 INFO:tasks.workunit.client.0.smithi176.stdout:3/27: read d4/f5 [4186946,64772] 0 2024-03-13T14:49:29.033 INFO:tasks.workunit.client.1.smithi196.stdout:7/208: dwrite d0/d9/d31/f42 [0,4194304] 0 2024-03-13T14:49:29.033 INFO:tasks.workunit.client.1.smithi196.stdout:6/231: rmdir d6/d9/d17 39 2024-03-13T14:49:29.033 INFO:tasks.workunit.client.0.smithi176.stdout:3/28: chown d4/c9 12 1 2024-03-13T14:49:29.033 INFO:tasks.workunit.client.0.smithi176.stdout:3/29: dread - f1 zero size 2024-03-13T14:49:29.034 INFO:tasks.workunit.client.0.smithi176.stdout:3/30: chown f1 1343 1 2024-03-13T14:49:29.039 INFO:tasks.workunit.client.0.smithi176.stdout:9/28: mknod d0/d5/c9 0 2024-03-13T14:49:29.041 INFO:tasks.workunit.client.1.smithi196.stdout:9/147: unlink d7/d9/df/f21 0 2024-03-13T14:49:29.046 INFO:tasks.workunit.client.1.smithi196.stdout:7/209: truncate d0/d6/d33/f40 662628 0 2024-03-13T14:49:29.046 INFO:tasks.workunit.client.0.smithi176.stdout:9/29: dread - d0/d5/f8 zero size 2024-03-13T14:49:29.048 INFO:tasks.workunit.client.0.smithi176.stdout:9/30: chown d0/d5/l7 30919028 1 2024-03-13T14:49:29.052 INFO:tasks.workunit.client.1.smithi196.stdout:0/220: link d8/da/de/d33/f34 d8/da/d25/d26/d35/d3b/f47 0 2024-03-13T14:49:29.053 INFO:tasks.workunit.client.0.smithi176.stdout:4/22: mkdir d7/d8 0 2024-03-13T14:49:29.056 INFO:tasks.workunit.client.0.smithi176.stdout:3/31: dread d4/f5 [0,4194304] 0 2024-03-13T14:49:29.057 INFO:tasks.workunit.client.1.smithi196.stdout:0/221: readlink d8/da/de/l30 0 2024-03-13T14:49:29.058 INFO:tasks.workunit.client.0.smithi176.stdout:9/31: rename d0/l2 to d0/la 0 2024-03-13T14:49:29.061 INFO:tasks.workunit.client.0.smithi176.stdout:9/32: dread - d0/d5/f8 zero size 2024-03-13T14:49:29.061 INFO:tasks.workunit.client.0.smithi176.stdout:4/23: symlink d7/l9 0 2024-03-13T14:49:29.065 INFO:tasks.workunit.client.0.smithi176.stdout:9/33: write d0/d5/f8 [695706,109675] 0 2024-03-13T14:49:29.066 INFO:tasks.workunit.client.1.smithi196.stdout:0/222: dread - d8/da/d25/d26/d35/d3b/f47 zero size 2024-03-13T14:49:29.067 INFO:tasks.workunit.client.1.smithi196.stdout:4/145: dwrite d1/d3/d7/f19 [0,4194304] 0 2024-03-13T14:49:29.068 INFO:tasks.workunit.client.1.smithi196.stdout:6/232: creat d6/d2a/d23/f4c x:0 0 0 2024-03-13T14:49:29.068 INFO:tasks.workunit.client.0.smithi176.stdout:4/24: mknod d7/ca 0 2024-03-13T14:49:29.069 INFO:tasks.workunit.client.0.smithi176.stdout:4/25: write f5 [425315,86620] 0 2024-03-13T14:49:29.070 INFO:tasks.workunit.client.1.smithi196.stdout:1/153: creat d1/f2c x:0 0 0 2024-03-13T14:49:29.072 INFO:tasks.workunit.client.1.smithi196.stdout:5/223: dread d4/db/f24 [0,4194304] 0 2024-03-13T14:49:29.072 INFO:tasks.workunit.client.1.smithi196.stdout:0/223: write d8/da/f32 [610179,23054] 0 2024-03-13T14:49:29.072 INFO:tasks.workunit.client.1.smithi196.stdout:7/210: write d0/d6/f23 [5021116,101299] 0 2024-03-13T14:49:29.075 INFO:tasks.workunit.client.1.smithi196.stdout:4/146: write d1/d3/d7/f1a [3265471,50810] 0 2024-03-13T14:49:29.076 INFO:tasks.workunit.client.1.smithi196.stdout:6/233: truncate d6/d2a/d23/f4c 158571 0 2024-03-13T14:49:29.080 INFO:tasks.workunit.client.1.smithi196.stdout:4/147: chown d1/d3/d7/f1a 7124218 1 2024-03-13T14:49:29.087 INFO:tasks.workunit.client.1.smithi196.stdout:8/90: getdents d0/d10/d12 0 2024-03-13T14:49:29.090 INFO:tasks.workunit.client.1.smithi196.stdout:0/224: mkdir d8/da/d25/d26/d48 0 2024-03-13T14:49:29.094 INFO:tasks.workunit.client.0.smithi176.stdout:4/26: dwrite f5 [0,4194304] 0 2024-03-13T14:49:29.095 INFO:tasks.workunit.client.1.smithi196.stdout:5/224: dread d4/db/f24 [0,4194304] 0 2024-03-13T14:49:29.096 INFO:tasks.workunit.client.1.smithi196.stdout:7/211: unlink d0/d9/d1e/d3f/f43 0 2024-03-13T14:49:29.097 INFO:tasks.workunit.client.1.smithi196.stdout:7/212: chown d0/c16 158194 1 2024-03-13T14:49:29.097 INFO:tasks.workunit.client.1.smithi196.stdout:4/148: rmdir d1/d3 39 2024-03-13T14:49:29.099 INFO:tasks.workunit.client.1.smithi196.stdout:8/91: creat d0/d10/f21 x:0 0 0 2024-03-13T14:49:29.100 INFO:tasks.workunit.client.1.smithi196.stdout:7/213: dread - d0/d9/d1e/d2d/f3b zero size 2024-03-13T14:49:29.101 INFO:tasks.workunit.client.1.smithi196.stdout:7/214: readlink d0/l2a 0 2024-03-13T14:49:29.103 INFO:tasks.workunit.client.1.smithi196.stdout:7/215: fdatasync d0/d9/d1e/d2d/f3b 0 2024-03-13T14:49:29.105 INFO:tasks.workunit.client.1.smithi196.stdout:0/225: write d8/da/d1a/f31 [1708914,39118] 0 2024-03-13T14:49:29.108 INFO:tasks.workunit.client.0.smithi176.stdout:3/32: fdatasync f2 0 2024-03-13T14:49:29.109 INFO:tasks.workunit.client.0.smithi176.stdout:2/21: fdatasync d1/f8 0 2024-03-13T14:49:29.113 INFO:tasks.workunit.client.1.smithi196.stdout:5/225: rename d4/db/l12 to d4/db/d14/l45 0 2024-03-13T14:49:29.113 INFO:tasks.workunit.client.0.smithi176.stdout:2/22: truncate d1/f4 511764 0 2024-03-13T14:49:29.115 INFO:tasks.workunit.client.0.smithi176.stdout:9/34: rename d0/d5 to d0/db 0 2024-03-13T14:49:29.117 INFO:tasks.workunit.client.0.smithi176.stdout:4/27: link c6 d7/d8/cb 0 2024-03-13T14:49:29.118 INFO:tasks.workunit.client.1.smithi196.stdout:5/226: write d4/db/d14/d19/d26/f43 [388208,74374] 0 2024-03-13T14:49:29.120 INFO:tasks.workunit.client.0.smithi176.stdout:2/23: creat d1/fa x:0 0 0 2024-03-13T14:49:29.125 INFO:tasks.workunit.client.0.smithi176.stdout:9/35: mknod d0/db/cc 0 2024-03-13T14:49:29.125 INFO:tasks.workunit.client.0.smithi176.stdout:3/33: dread d4/f5 [0,4194304] 0 2024-03-13T14:49:29.126 INFO:tasks.workunit.client.0.smithi176.stdout:9/36: write d0/db/f8 [1460377,58726] 0 2024-03-13T14:49:29.127 INFO:tasks.workunit.client.1.smithi196.stdout:0/226: mknod d8/da/d25/c49 0 2024-03-13T14:49:29.129 INFO:tasks.workunit.client.1.smithi196.stdout:0/227: readlink l3 0 2024-03-13T14:49:29.130 INFO:tasks.workunit.client.0.smithi176.stdout:4/28: creat d7/fc x:0 0 0 2024-03-13T14:49:29.130 INFO:tasks.workunit.client.0.smithi176.stdout:4/29: dread - d7/fc zero size 2024-03-13T14:49:29.134 INFO:tasks.workunit.client.1.smithi196.stdout:8/92: rename d0/l13 to d0/d10/l22 0 2024-03-13T14:49:29.134 INFO:tasks.workunit.client.0.smithi176.stdout:2/24: creat d1/fb x:0 0 0 2024-03-13T14:49:29.135 INFO:tasks.workunit.client.1.smithi196.stdout:4/149: dwrite d1/d3/d7/f1d [4194304,4194304] 0 2024-03-13T14:49:29.140 INFO:tasks.workunit.client.1.smithi196.stdout:9/148: write d7/de/f1f [1758851,28976] 0 2024-03-13T14:49:29.142 INFO:tasks.workunit.client.1.smithi196.stdout:2/84: truncate d0/d6/f8 1041690 0 2024-03-13T14:49:29.146 INFO:tasks.workunit.client.0.smithi176.stdout:3/34: creat d4/fa x:0 0 0 2024-03-13T14:49:29.146 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:29.146 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:49:29.146 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:29.147 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:49:29.147 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:49:29.147 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:28 smithi196 ceph-mon[37486]: mgrmap e27: smithi176.tzfdwd(active, since 1.02775s) 2024-03-13T14:49:29.148 INFO:tasks.workunit.client.0.smithi176.stdout:3/35: write f2 [4271068,2307] 0 2024-03-13T14:49:29.148 INFO:tasks.workunit.client.1.smithi196.stdout:5/227: unlink d4/db/d14/d19/d1c/d25/l40 0 2024-03-13T14:49:29.149 INFO:tasks.workunit.client.0.smithi176.stdout:3/36: stat d4/c6 0 2024-03-13T14:49:29.154 INFO:tasks.workunit.client.1.smithi196.stdout:0/228: dwrite d8/da/f32 [0,4194304] 0 2024-03-13T14:49:29.157 INFO:tasks.workunit.client.0.smithi176.stdout:9/37: dread d0/db/f8 [0,4194304] 0 2024-03-13T14:49:29.157 INFO:tasks.workunit.client.1.smithi196.stdout:8/93: dwrite d0/f1 [0,4194304] 0 2024-03-13T14:49:29.158 INFO:tasks.workunit.client.0.smithi176.stdout:2/25: mkdir d1/dc 0 2024-03-13T14:49:29.159 INFO:tasks.workunit.client.1.smithi196.stdout:0/229: fsync d8/da/d25/d26/f2d 0 2024-03-13T14:49:29.159 INFO:tasks.workunit.client.1.smithi196.stdout:6/234: write d6/fa [3406963,15164] 0 2024-03-13T14:49:29.161 INFO:tasks.workunit.client.1.smithi196.stdout:8/94: dread - d0/d10/f21 zero size 2024-03-13T14:49:29.164 INFO:tasks.workunit.client.0.smithi176.stdout:3/37: mknod d4/cb 0 2024-03-13T14:49:29.167 INFO:tasks.workunit.client.1.smithi196.stdout:2/85: symlink d0/d7/l11 0 2024-03-13T14:49:29.169 INFO:tasks.workunit.client.1.smithi196.stdout:2/86: fsync d0/f2 0 2024-03-13T14:49:29.169 INFO:tasks.workunit.client.1.smithi196.stdout:5/228: rmdir d4/db 39 2024-03-13T14:49:29.173 INFO:tasks.workunit.client.0.smithi176.stdout:9/38: mknod d0/db/cd 0 2024-03-13T14:49:29.175 INFO:tasks.workunit.client.0.smithi176.stdout:3/38: mknod d4/cc 0 2024-03-13T14:49:29.177 INFO:tasks.workunit.client.0.smithi176.stdout:3/39: dread - d4/fa zero size 2024-03-13T14:49:29.180 INFO:tasks.workunit.client.1.smithi196.stdout:9/149: dread d7/d9/df/f1c [0,4194304] 0 2024-03-13T14:49:29.181 INFO:tasks.workunit.client.0.smithi176.stdout:2/26: mknod d1/dc/cd 0 2024-03-13T14:49:29.182 INFO:tasks.workunit.client.0.smithi176.stdout:2/27: fsync d1/f4 0 2024-03-13T14:49:29.183 INFO:tasks.workunit.client.1.smithi196.stdout:7/216: truncate d0/d9/d31/f42 3033484 0 2024-03-13T14:49:29.186 INFO:tasks.workunit.client.1.smithi196.stdout:6/235: creat d6/d7/df/d27/f4d x:0 0 0 2024-03-13T14:49:29.187 INFO:tasks.workunit.client.0.smithi176.stdout:9/39: symlink d0/le 0 2024-03-13T14:49:29.188 INFO:tasks.workunit.client.1.smithi196.stdout:8/95: symlink d0/d10/d14/l23 0 2024-03-13T14:49:29.190 INFO:tasks.workunit.client.0.smithi176.stdout:3/40: symlink d4/ld 0 2024-03-13T14:49:29.191 INFO:tasks.workunit.client.1.smithi196.stdout:0/230: creat d8/da/de/f4a x:0 0 0 2024-03-13T14:49:29.193 INFO:tasks.workunit.client.0.smithi176.stdout:2/28: mknod d1/ce 0 2024-03-13T14:49:29.194 INFO:tasks.workunit.client.0.smithi176.stdout:9/40: mknod d0/cf 0 2024-03-13T14:49:29.194 INFO:tasks.workunit.client.1.smithi196.stdout:5/229: chown d4/db/d14/d19/d1c/d25 11 1 2024-03-13T14:49:29.195 INFO:tasks.workunit.client.0.smithi176.stdout:2/29: read d1/f8 [1257950,18605] 0 2024-03-13T14:49:29.196 INFO:tasks.workunit.client.1.smithi196.stdout:4/150: creat d1/f29 x:0 0 0 2024-03-13T14:49:29.197 INFO:tasks.workunit.client.1.smithi196.stdout:5/230: write d4/f8 [1101720,49708] 0 2024-03-13T14:49:29.199 INFO:tasks.workunit.client.1.smithi196.stdout:4/151: readlink d1/l24 0 2024-03-13T14:49:29.200 INFO:tasks.workunit.client.1.smithi196.stdout:0/231: symlink d8/da/de/l4b 0 2024-03-13T14:49:29.202 INFO:tasks.workunit.client.0.smithi176.stdout:4/30: dwrite f5 [4194304,4194304] 0 2024-03-13T14:49:29.202 INFO:tasks.workunit.client.1.smithi196.stdout:9/150: symlink d7/d9/df/d1d/d22/l26 0 2024-03-13T14:49:29.202 INFO:tasks.workunit.client.0.smithi176.stdout:2/30: creat d1/ff x:0 0 0 2024-03-13T14:49:29.206 INFO:tasks.workunit.client.1.smithi196.stdout:4/152: read d1/d3/f13 [2730877,124609] 0 2024-03-13T14:49:29.206 INFO:tasks.workunit.client.0.smithi176.stdout:9/41: dread d0/db/f8 [0,4194304] 0 2024-03-13T14:49:29.207 INFO:tasks.workunit.client.1.smithi196.stdout:5/231: fsync d4/db/d14/d19/d1c/f3f 0 2024-03-13T14:49:29.210 INFO:tasks.workunit.client.1.smithi196.stdout:2/87: rename d0/d6/ld to d0/d6/l12 0 2024-03-13T14:49:29.212 INFO:tasks.workunit.client.0.smithi176.stdout:3/41: dwrite d4/fa [0,4194304] 0 2024-03-13T14:49:29.213 INFO:tasks.workunit.client.1.smithi196.stdout:4/153: truncate d1/d3/d7/f21 960928 0 2024-03-13T14:49:29.213 INFO:tasks.workunit.client.1.smithi196.stdout:0/232: symlink d8/da/d1a/l4c 0 2024-03-13T14:49:29.213 INFO:tasks.workunit.client.1.smithi196.stdout:5/232: chown d4/db/d14/d37/d41/f42 5 1 2024-03-13T14:49:29.213 INFO:tasks.workunit.client.1.smithi196.stdout:8/96: dwrite d0/d10/f21 [0,4194304] 0 2024-03-13T14:49:29.220 INFO:tasks.workunit.client.1.smithi196.stdout:9/151: creat d7/d24/f27 x:0 0 0 2024-03-13T14:49:29.222 INFO:tasks.workunit.client.1.smithi196.stdout:3/115: sync 2024-03-13T14:49:29.224 INFO:tasks.workunit.client.0.smithi176.stdout:3/42: rename d4/c9 to d4/ce 0 2024-03-13T14:49:29.230 INFO:tasks.workunit.client.0.smithi176.stdout:3/43: write d4/fa [397115,101182] 0 2024-03-13T14:49:29.236 INFO:tasks.workunit.client.1.smithi196.stdout:4/154: dread d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:29.241 INFO:tasks.workunit.client.1.smithi196.stdout:5/233: dwrite d4/db/ff [0,4194304] 0 2024-03-13T14:49:29.241 INFO:tasks.workunit.client.1.smithi196.stdout:4/155: write d1/d3/d7/f1d [437169,109313] 0 2024-03-13T14:49:29.242 INFO:tasks.workunit.client.1.smithi196.stdout:9/152: mkdir d7/d24/d28 0 2024-03-13T14:49:29.245 INFO:tasks.workunit.client.1.smithi196.stdout:5/234: fsync d4/f8 0 2024-03-13T14:49:29.246 INFO:tasks.workunit.client.1.smithi196.stdout:3/116: symlink db/l25 0 2024-03-13T14:49:29.253 INFO:tasks.workunit.client.1.smithi196.stdout:7/217: truncate d0/d6/f21 405112 0 2024-03-13T14:49:29.254 INFO:tasks.workunit.client.0.smithi176.stdout:9/42: dwrite d0/db/f8 [0,4194304] 0 2024-03-13T14:49:29.254 INFO:tasks.workunit.client.1.smithi196.stdout:8/97: getdents d0/d10/d12/d1e 0 2024-03-13T14:49:29.254 INFO:tasks.workunit.client.1.smithi196.stdout:0/233: link d8/da/f13 d8/da/de/d17/f4d 0 2024-03-13T14:49:29.255 INFO:tasks.workunit.client.1.smithi196.stdout:8/98: fdatasync d0/d10/f19 0 2024-03-13T14:49:29.261 INFO:tasks.workunit.client.1.smithi196.stdout:5/235: unlink d4/db/d14/d19/d1c/d25/f36 0 2024-03-13T14:49:29.264 INFO:tasks.workunit.client.1.smithi196.stdout:8/99: write d0/d10/f19 [692711,83336] 0 2024-03-13T14:49:29.264 INFO:tasks.workunit.client.1.smithi196.stdout:1/154: sync 2024-03-13T14:49:29.264 INFO:tasks.workunit.client.1.smithi196.stdout:7/218: mknod d0/d9/d1e/c44 0 2024-03-13T14:49:29.267 INFO:tasks.workunit.client.0.smithi176.stdout:9/43: creat d0/f10 x:0 0 0 2024-03-13T14:49:29.267 INFO:tasks.workunit.client.0.smithi176.stdout:3/44: dwrite d4/f5 [4194304,4194304] 0 2024-03-13T14:49:29.268 INFO:tasks.workunit.client.1.smithi196.stdout:5/236: read d4/f13 [1778729,48601] 0 2024-03-13T14:49:29.273 INFO:tasks.workunit.client.1.smithi196.stdout:6/236: truncate d6/d9/d17/d19/f2d 820827 0 2024-03-13T14:49:29.279 INFO:tasks.workunit.client.0.smithi176.stdout:3/45: mknod d4/cf 0 2024-03-13T14:49:29.285 INFO:tasks.workunit.client.1.smithi196.stdout:2/88: getdents d0/d7 0 2024-03-13T14:49:29.286 INFO:tasks.workunit.client.1.smithi196.stdout:7/219: dread d0/d6/f1d [0,4194304] 0 2024-03-13T14:49:29.286 INFO:tasks.workunit.client.1.smithi196.stdout:9/153: creat d7/de/f29 x:0 0 0 2024-03-13T14:49:29.289 INFO:tasks.workunit.client.0.smithi176.stdout:9/44: getdents d0/db 0 2024-03-13T14:49:29.291 INFO:tasks.workunit.client.1.smithi196.stdout:9/154: read f5 [1380385,20513] 0 2024-03-13T14:49:29.291 INFO:tasks.workunit.client.0.smithi176.stdout:3/46: dread d4/fa [0,4194304] 0 2024-03-13T14:49:29.292 INFO:tasks.workunit.client.1.smithi196.stdout:0/234: creat d8/da/f4e x:0 0 0 2024-03-13T14:49:29.295 INFO:tasks.workunit.client.1.smithi196.stdout:2/89: write d0/fe [143708,122775] 0 2024-03-13T14:49:29.296 INFO:tasks.workunit.client.1.smithi196.stdout:0/235: readlink d8/da/d1a/l15 0 2024-03-13T14:49:29.297 INFO:tasks.workunit.client.1.smithi196.stdout:9/155: chown d7/fd 24179036 1 2024-03-13T14:49:29.301 INFO:tasks.workunit.client.1.smithi196.stdout:4/156: dwrite d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:29.303 INFO:tasks.workunit.client.1.smithi196.stdout:8/100: link d0/d10/d12/c20 d0/d16/c24 0 2024-03-13T14:49:29.309 INFO:tasks.workunit.client.1.smithi196.stdout:5/237: dread d4/db/d14/d19/f3b [0,4194304] 0 2024-03-13T14:49:29.309 INFO:tasks.workunit.client.1.smithi196.stdout:1/155: link d1/d9/d27/f17 d1/d28/f2d 0 2024-03-13T14:49:29.311 INFO:tasks.workunit.client.1.smithi196.stdout:8/101: chown d0/ca 7719212 1 2024-03-13T14:49:29.314 INFO:tasks.workunit.client.1.smithi196.stdout:0/236: rename d8/da/de/d17/f36 to d8/da/d25/d26/d35/f4f 0 2024-03-13T14:49:29.317 INFO:tasks.workunit.client.1.smithi196.stdout:7/220: creat d0/d6/f45 x:0 0 0 2024-03-13T14:49:29.319 INFO:tasks.workunit.client.1.smithi196.stdout:6/237: dwrite d6/d2a/d23/f4c [0,4194304] 0 2024-03-13T14:49:29.321 INFO:tasks.workunit.client.1.smithi196.stdout:1/156: creat d1/d9/d27/f2e x:0 0 0 2024-03-13T14:49:29.322 INFO:tasks.workunit.client.1.smithi196.stdout:1/157: truncate d1/d9/f18 1157032 0 2024-03-13T14:49:29.324 INFO:tasks.workunit.client.1.smithi196.stdout:0/237: stat d8/da/d1a/l15 0 2024-03-13T14:49:29.327 INFO:tasks.workunit.client.0.smithi176.stdout:9/45: fdatasync d0/db/f8 0 2024-03-13T14:49:29.333 INFO:tasks.workunit.client.1.smithi196.stdout:3/117: truncate f4 5483690 0 2024-03-13T14:49:29.337 INFO:tasks.workunit.client.0.smithi176.stdout:9/46: symlink d0/db/l11 0 2024-03-13T14:49:29.339 INFO:tasks.workunit.client.0.smithi176.stdout:9/47: write d0/f10 [731672,42832] 0 2024-03-13T14:49:29.339 INFO:tasks.workunit.client.1.smithi196.stdout:6/238: dwrite d6/d2a/dc/d1c/f25 [0,4194304] 0 2024-03-13T14:49:29.343 INFO:tasks.workunit.client.1.smithi196.stdout:2/90: creat d0/d6/f13 x:0 0 0 2024-03-13T14:49:29.348 INFO:tasks.workunit.client.1.smithi196.stdout:7/221: readlink d0/d6/le 0 2024-03-13T14:49:29.349 INFO:tasks.workunit.client.1.smithi196.stdout:7/222: readlink d0/d6/l25 0 2024-03-13T14:49:29.349 INFO:tasks.workunit.client.1.smithi196.stdout:3/118: dread db/fd [0,4194304] 0 2024-03-13T14:49:29.354 INFO:tasks.workunit.client.0.smithi176.stdout:9/48: creat d0/f12 x:0 0 0 2024-03-13T14:49:29.356 INFO:tasks.workunit.client.1.smithi196.stdout:5/238: dwrite d4/db/d14/d19/f3b [0,4194304] 0 2024-03-13T14:49:29.360 INFO:tasks.workunit.client.1.smithi196.stdout:5/239: dread - d4/db/d14/d37/d41/f42 zero size 2024-03-13T14:49:29.376 INFO:tasks.workunit.client.1.smithi196.stdout:6/239: mkdir d6/d9/d17/d19/d4e 0 2024-03-13T14:49:29.388 INFO:tasks.workunit.client.1.smithi196.stdout:7/223: write d0/d6/f1d [876768,45489] 0 2024-03-13T14:49:29.398 INFO:tasks.workunit.client.1.smithi196.stdout:5/240: creat d4/db/d14/d19/d1c/d25/d38/f46 x:0 0 0 2024-03-13T14:49:29.398 INFO:tasks.workunit.client.1.smithi196.stdout:4/157: truncate d1/d3/fa 127660 0 2024-03-13T14:49:29.400 INFO:tasks.workunit.client.1.smithi196.stdout:0/238: mkdir d8/da/d25/d26/d35/d3b/d50 0 2024-03-13T14:49:29.400 INFO:tasks.workunit.client.1.smithi196.stdout:5/241: chown d4/db/f24 180 1 2024-03-13T14:49:29.401 INFO:tasks.workunit.client.1.smithi196.stdout:7/224: dread d0/d6/f19 [0,4194304] 0 2024-03-13T14:49:29.407 INFO:tasks.workunit.client.1.smithi196.stdout:8/102: rmdir d0/d10/d12/d1b 0 2024-03-13T14:49:29.407 INFO:tasks.workunit.client.1.smithi196.stdout:6/240: symlink d6/d2a/dc/d1c/d34/l4f 0 2024-03-13T14:49:29.418 INFO:tasks.workunit.client.1.smithi196.stdout:9/156: getdents d7/de/d16 0 2024-03-13T14:49:29.419 INFO:tasks.workunit.client.1.smithi196.stdout:4/158: rmdir d1/d3/d7 39 2024-03-13T14:49:29.425 INFO:tasks.workunit.client.1.smithi196.stdout:8/103: rename d0/d10/d14/l23 to d0/d10/d14/l25 0 2024-03-13T14:49:29.427 INFO:tasks.workunit.client.1.smithi196.stdout:2/91: creat d0/d6/f14 x:0 0 0 2024-03-13T14:49:29.432 INFO:tasks.workunit.client.1.smithi196.stdout:5/242: getdents d4/db/d14/d19/d1c/d2d 0 2024-03-13T14:49:29.432 INFO:tasks.workunit.client.1.smithi196.stdout:0/239: dwrite d8/da/f4e [0,4194304] 0 2024-03-13T14:49:29.436 INFO:tasks.workunit.client.1.smithi196.stdout:2/92: write d0/d6/f14 [854814,22370] 0 2024-03-13T14:49:29.437 INFO:tasks.workunit.client.1.smithi196.stdout:1/158: dwrite d1/d28/f2d [4194304,4194304] 0 2024-03-13T14:49:29.441 INFO:tasks.workunit.client.1.smithi196.stdout:6/241: dread d6/d9/d11/f48 [0,4194304] 0 2024-03-13T14:49:29.443 INFO:tasks.workunit.client.1.smithi196.stdout:8/104: truncate d0/d10/f11 2561339 0 2024-03-13T14:49:29.446 INFO:tasks.workunit.client.1.smithi196.stdout:6/242: dread - d6/d9/d17/d19/f42 zero size 2024-03-13T14:49:29.447 INFO:tasks.workunit.client.1.smithi196.stdout:8/105: fsync d0/d10/f21 0 2024-03-13T14:49:29.453 INFO:tasks.workunit.client.1.smithi196.stdout:2/93: dread d0/fe [0,4194304] 0 2024-03-13T14:49:29.456 INFO:tasks.workunit.client.1.smithi196.stdout:2/94: fsync d0/f2 0 2024-03-13T14:49:29.457 INFO:tasks.workunit.client.1.smithi196.stdout:0/240: creat d8/da/de/d17/f51 x:0 0 0 2024-03-13T14:49:29.458 INFO:tasks.workunit.client.1.smithi196.stdout:7/225: dread d0/d6/f26 [0,4194304] 0 2024-03-13T14:49:29.462 INFO:tasks.workunit.client.1.smithi196.stdout:0/241: readlink d8/da/l20 0 2024-03-13T14:49:29.464 INFO:tasks.workunit.client.1.smithi196.stdout:1/159: mknod d1/d9/d27/c2f 0 2024-03-13T14:49:29.466 INFO:tasks.workunit.client.1.smithi196.stdout:1/160: chown d1/f26 902605000 1 2024-03-13T14:49:29.469 INFO:tasks.workunit.client.1.smithi196.stdout:7/226: dread d0/d6/d33/f40 [0,4194304] 0 2024-03-13T14:49:29.472 INFO:tasks.workunit.client.1.smithi196.stdout:5/243: mkdir d4/db/d14/d19/d1c/d25/d38/d47 0 2024-03-13T14:49:29.475 INFO:tasks.workunit.client.1.smithi196.stdout:7/227: dread d0/d6/d33/f40 [0,4194304] 0 2024-03-13T14:49:29.477 INFO:tasks.workunit.client.1.smithi196.stdout:5/244: write d4/db/d14/d19/f3b [2447378,62955] 0 2024-03-13T14:49:29.481 INFO:tasks.workunit.client.1.smithi196.stdout:6/243: mkdir d6/d4b/d50 0 2024-03-13T14:49:29.483 INFO:tasks.workunit.client.1.smithi196.stdout:6/244: stat d6/d9/d17 0 2024-03-13T14:49:29.483 INFO:tasks.workunit.client.1.smithi196.stdout:4/159: rename d1/c23 to d1/c2a 0 2024-03-13T14:49:29.488 INFO:tasks.workunit.client.1.smithi196.stdout:6/245: write d6/d7/df/f2e [6098949,71712] 0 2024-03-13T14:49:29.489 INFO:tasks.workunit.client.1.smithi196.stdout:2/95: dwrite d0/d6/fb [0,4194304] 0 2024-03-13T14:49:29.489 INFO:tasks.workunit.client.1.smithi196.stdout:4/160: write d1/d3/f13 [4423942,104450] 0 2024-03-13T14:49:29.498 INFO:tasks.workunit.client.1.smithi196.stdout:9/157: write d7/d9/df/f1a [1337355,59034] 0 2024-03-13T14:49:29.500 INFO:tasks.workunit.client.1.smithi196.stdout:5/245: mknod d4/db/d14/d19/c48 0 2024-03-13T14:49:29.511 INFO:tasks.workunit.client.1.smithi196.stdout:7/228: dread d0/f11 [0,4194304] 0 2024-03-13T14:49:29.511 INFO:tasks.workunit.client.1.smithi196.stdout:4/161: dwrite d1/f22 [0,4194304] 0 2024-03-13T14:49:29.517 INFO:tasks.workunit.client.1.smithi196.stdout:2/96: dread d0/d6/f8 [0,4194304] 0 2024-03-13T14:49:29.518 INFO:tasks.workunit.client.1.smithi196.stdout:0/242: mknod d8/da/c52 0 2024-03-13T14:49:29.529 INFO:tasks.workunit.client.1.smithi196.stdout:7/229: dread d0/d9/f30 [0,4194304] 0 2024-03-13T14:49:29.537 INFO:tasks.workunit.client.1.smithi196.stdout:0/243: dread d8/da/de/d43/f45 [0,4194304] 0 2024-03-13T14:49:29.541 INFO:tasks.workunit.client.1.smithi196.stdout:4/162: symlink d1/d1e/l2b 0 2024-03-13T14:49:29.546 INFO:tasks.workunit.client.1.smithi196.stdout:2/97: truncate d0/fe 911851 0 2024-03-13T14:49:29.549 INFO:tasks.workunit.client.1.smithi196.stdout:6/246: link d6/d9/f1f d6/d4b/d50/f51 0 2024-03-13T14:49:29.560 INFO:tasks.workunit.client.1.smithi196.stdout:5/246: dread d4/f11 [0,4194304] 0 2024-03-13T14:49:29.563 INFO:tasks.workunit.client.1.smithi196.stdout:5/247: chown d4/d1b/c2e 8330956 1 2024-03-13T14:49:29.565 INFO:tasks.workunit.client.1.smithi196.stdout:5/248: read d4/db/ff [1235245,127628] 0 2024-03-13T14:49:29.566 INFO:tasks.workunit.client.1.smithi196.stdout:7/230: mkdir d0/d9/d31/d46 0 2024-03-13T14:49:29.568 INFO:tasks.workunit.client.1.smithi196.stdout:3/119: dwrite f4 [0,4194304] 0 2024-03-13T14:49:29.571 INFO:tasks.workunit.client.1.smithi196.stdout:3/120: chown db/d19/d22 1 1 2024-03-13T14:49:29.573 INFO:tasks.workunit.client.1.smithi196.stdout:9/158: write d7/de/f1f [2167379,125232] 0 2024-03-13T14:49:29.575 INFO:tasks.workunit.client.1.smithi196.stdout:4/163: fdatasync d1/d3/d7/f21 0 2024-03-13T14:49:29.577 INFO:tasks.workunit.client.1.smithi196.stdout:2/98: readlink d0/d6/l12 0 2024-03-13T14:49:29.578 INFO:tasks.workunit.client.1.smithi196.stdout:4/164: stat d1/d3/d7/c18 0 2024-03-13T14:49:29.582 INFO:tasks.workunit.client.1.smithi196.stdout:4/165: truncate d1/f29 578594 0 2024-03-13T14:49:29.590 INFO:tasks.workunit.client.1.smithi196.stdout:7/231: read d0/d6/d33/f39 [158975,111609] 0 2024-03-13T14:49:29.592 INFO:tasks.workunit.client.1.smithi196.stdout:5/249: creat d4/db/d14/d19/d1c/d25/f49 x:0 0 0 2024-03-13T14:49:29.592 INFO:tasks.workunit.client.1.smithi196.stdout:1/161: truncate d1/d9/fd 380385 0 2024-03-13T14:49:29.595 INFO:tasks.workunit.client.1.smithi196.stdout:0/244: rename d8/f12 to d8/da/d25/d26/d35/d3b/d50/f53 0 2024-03-13T14:49:29.596 INFO:tasks.workunit.client.1.smithi196.stdout:3/121: fdatasync db/d19/d22/f23 0 2024-03-13T14:49:29.598 INFO:tasks.workunit.client.1.smithi196.stdout:3/122: chown l5 87321 1 2024-03-13T14:49:29.602 INFO:tasks.workunit.client.1.smithi196.stdout:6/247: mknod d6/d7/df/d27/d39/d44/c52 0 2024-03-13T14:49:29.610 INFO:tasks.workunit.client.1.smithi196.stdout:1/162: creat d1/d9/d27/f30 x:0 0 0 2024-03-13T14:49:29.612 INFO:tasks.workunit.client.1.smithi196.stdout:5/250: dwrite d4/db/d14/d19/d1c/d25/d38/f3e [0,4194304] 0 2024-03-13T14:49:29.615 INFO:tasks.workunit.client.1.smithi196.stdout:9/159: rename d7/d9/df/f1c to d7/de/d16/f2a 0 2024-03-13T14:49:29.617 INFO:tasks.workunit.client.1.smithi196.stdout:9/160: chown d7/f15 12418 1 2024-03-13T14:49:29.622 INFO:tasks.workunit.client.1.smithi196.stdout:5/251: dread d4/db/d14/d19/d26/f43 [0,4194304] 0 2024-03-13T14:49:29.623 INFO:tasks.workunit.client.1.smithi196.stdout:6/248: creat d6/d2a/dc/f53 x:0 0 0 2024-03-13T14:49:29.628 INFO:tasks.workunit.client.1.smithi196.stdout:9/161: creat d7/d24/f2b x:0 0 0 2024-03-13T14:49:29.630 INFO:tasks.workunit.client.1.smithi196.stdout:9/162: dread - d7/d24/f27 zero size 2024-03-13T14:49:29.634 INFO:tasks.workunit.client.1.smithi196.stdout:3/123: rename f8 to db/d19/f26 0 2024-03-13T14:49:29.642 INFO:tasks.workunit.client.1.smithi196.stdout:5/252: mkdir d4/db/d4a 0 2024-03-13T14:49:29.645 INFO:tasks.workunit.client.0.smithi176.stdout:9/49: mkdir d0/d13 0 2024-03-13T14:49:29.647 INFO:tasks.workunit.client.0.smithi176.stdout:9/50: write d0/db/f8 [5182549,25886] 0 2024-03-13T14:49:29.648 INFO:tasks.workunit.client.1.smithi196.stdout:1/163: link d1/d9/d27/f16 d1/f31 0 2024-03-13T14:49:29.650 INFO:tasks.workunit.client.1.smithi196.stdout:9/163: mknod d7/d24/d28/c2c 0 2024-03-13T14:49:29.654 INFO:tasks.workunit.client.1.smithi196.stdout:6/249: creat d6/f54 x:0 0 0 2024-03-13T14:49:29.655 INFO:tasks.workunit.client.1.smithi196.stdout:1/164: write d1/d9/d27/f30 [538971,98170] 0 2024-03-13T14:49:29.661 INFO:tasks.workunit.client.0.smithi176.stdout:9/51: mknod d0/db/c14 0 2024-03-13T14:49:29.665 INFO:tasks.workunit.client.1.smithi196.stdout:6/250: write d6/d9/d17/d19/f20 [5032199,62345] 0 2024-03-13T14:49:29.668 INFO:tasks.workunit.client.1.smithi196.stdout:1/165: creat d1/d28/f32 x:0 0 0 2024-03-13T14:49:29.672 INFO:tasks.workunit.client.0.smithi176.stdout:9/52: rename d0/l6 to d0/l15 0 2024-03-13T14:49:29.680 INFO:tasks.workunit.client.0.smithi176.stdout:9/53: unlink d0/db/f8 0 2024-03-13T14:49:29.682 INFO:tasks.workunit.client.1.smithi196.stdout:1/166: stat d1/d28 0 2024-03-13T14:49:29.682 INFO:tasks.workunit.client.1.smithi196.stdout:7/232: write d0/d6/d33/f40 [92203,35026] 0 2024-03-13T14:49:29.683 INFO:tasks.workunit.client.1.smithi196.stdout:8/106: sync 2024-03-13T14:49:29.684 INFO:tasks.workunit.client.1.smithi196.stdout:0/245: mkdir d8/da/d25/d26/d54 0 2024-03-13T14:49:29.685 INFO:tasks.workunit.client.1.smithi196.stdout:3/124: dwrite db/d19/f26 [0,4194304] 0 2024-03-13T14:49:29.690 INFO:tasks.workunit.client.1.smithi196.stdout:5/253: dwrite d4/db/d14/d19/d26/f43 [0,4194304] 0 2024-03-13T14:49:29.693 INFO:tasks.workunit.client.1.smithi196.stdout:6/251: rename d6/d2a/dc/f53 to d6/d2a/dc/d47/f55 0 2024-03-13T14:49:29.696 INFO:tasks.workunit.client.1.smithi196.stdout:8/107: creat d0/d10/f26 x:0 0 0 2024-03-13T14:49:29.696 INFO:tasks.workunit.client.1.smithi196.stdout:4/166: truncate d1/d3/d7/f19 981600 0 2024-03-13T14:49:29.703 INFO:tasks.workunit.client.1.smithi196.stdout:8/108: fsync d0/d10/f11 0 2024-03-13T14:49:29.707 INFO:tasks.workunit.client.1.smithi196.stdout:3/125: rename db/d19/c1f to db/d19/c27 0 2024-03-13T14:49:29.715 INFO:tasks.workunit.client.1.smithi196.stdout:9/164: dwrite d7/d9/df/d1d/f23 [0,4194304] 0 2024-03-13T14:49:29.725 INFO:tasks.workunit.client.1.smithi196.stdout:6/252: unlink d6/d7/df/d27/f4d 0 2024-03-13T14:49:29.730 INFO:tasks.workunit.client.1.smithi196.stdout:2/99: dwrite d0/fe [0,4194304] 0 2024-03-13T14:49:29.744 INFO:tasks.workunit.client.1.smithi196.stdout:1/167: dwrite d1/f2a [0,4194304] 0 2024-03-13T14:49:29.752 INFO:tasks.workunit.client.1.smithi196.stdout:9/165: dread d7/d9/df/f1a [0,4194304] 0 2024-03-13T14:49:29.752 INFO:tasks.workunit.client.1.smithi196.stdout:7/233: dwrite d0/d9/d1e/d2d/f3b [0,4194304] 0 2024-03-13T14:49:29.754 INFO:tasks.workunit.client.1.smithi196.stdout:4/167: getdents d1/d3 0 2024-03-13T14:49:29.755 INFO:tasks.workunit.client.1.smithi196.stdout:0/246: write d8/da/d25/d26/d35/d3b/f47 [465322,69175] 0 2024-03-13T14:49:29.758 INFO:tasks.workunit.client.1.smithi196.stdout:9/166: truncate d7/f15 469951 0 2024-03-13T14:49:29.767 INFO:tasks.workunit.client.1.smithi196.stdout:2/100: truncate d0/d6/f8 1011177 0 2024-03-13T14:49:29.769 INFO:tasks.workunit.client.1.smithi196.stdout:8/109: dwrite d0/d10/f11 [0,4194304] 0 2024-03-13T14:49:29.770 INFO:tasks.workunit.client.1.smithi196.stdout:5/254: getdents d4/db/d14/d19/d1c 0 2024-03-13T14:49:29.770 INFO:tasks.workunit.client.1.smithi196.stdout:7/234: fdatasync d0/d9/f20 0 2024-03-13T14:49:29.771 INFO:tasks.workunit.client.1.smithi196.stdout:9/167: read - d7/de/f29 zero size 2024-03-13T14:49:29.772 INFO:tasks.workunit.client.1.smithi196.stdout:2/101: write d0/f2 [2399344,1781] 0 2024-03-13T14:49:29.775 INFO:tasks.workunit.client.1.smithi196.stdout:9/168: dread - d7/de/f29 zero size 2024-03-13T14:49:29.777 INFO:tasks.workunit.client.1.smithi196.stdout:6/253: dread d6/d9/d17/d19/f20 [0,4194304] 0 2024-03-13T14:49:29.778 INFO:tasks.workunit.client.1.smithi196.stdout:4/168: creat d1/d3/d7/f2c x:0 0 0 2024-03-13T14:49:29.778 INFO:tasks.workunit.client.1.smithi196.stdout:9/169: readlink d7/d24/l25 0 2024-03-13T14:49:29.779 INFO:tasks.workunit.client.1.smithi196.stdout:1/168: mkdir d1/d9/d33 0 2024-03-13T14:49:29.779 INFO:tasks.workunit.client.1.smithi196.stdout:8/110: read d0/d10/f11 [2218819,53715] 0 2024-03-13T14:49:29.780 INFO:tasks.workunit.client.1.smithi196.stdout:3/126: truncate db/ff 132212 0 2024-03-13T14:49:29.782 INFO:tasks.workunit.client.1.smithi196.stdout:0/247: read d8/da/de/d43/f45 [84494,110010] 0 2024-03-13T14:49:29.785 INFO:tasks.workunit.client.1.smithi196.stdout:9/170: readlink d7/d9/df/d1d/d22/l26 0 2024-03-13T14:49:29.792 INFO:tasks.workunit.client.1.smithi196.stdout:2/102: mkdir d0/d7/d15 0 2024-03-13T14:49:29.793 INFO:tasks.workunit.client.1.smithi196.stdout:1/169: mkdir d1/d28/d34 0 2024-03-13T14:49:29.793 INFO:tasks.workunit.client.1.smithi196.stdout:7/235: unlink d0/f13 0 2024-03-13T14:49:29.794 INFO:tasks.workunit.client.1.smithi196.stdout:5/255: symlink d4/db/d4a/l4b 0 2024-03-13T14:49:29.796 INFO:tasks.workunit.client.1.smithi196.stdout:3/127: truncate db/fd 4266952 0 2024-03-13T14:49:29.796 INFO:tasks.workunit.client.1.smithi196.stdout:2/103: read d0/f2 [4118073,48030] 0 2024-03-13T14:49:29.806 INFO:tasks.workunit.client.1.smithi196.stdout:1/170: truncate f0 459559 0 2024-03-13T14:49:29.811 INFO:tasks.workunit.client.1.smithi196.stdout:1/171: fsync d1/d28/f2b 0 2024-03-13T14:49:29.811 INFO:tasks.workunit.client.1.smithi196.stdout:6/254: dwrite d6/d7/df/f2e [0,4194304] 0 2024-03-13T14:49:29.812 INFO:tasks.workunit.client.1.smithi196.stdout:0/248: mkdir d8/da/d25/d26/d35/d3b/d50/d55 0 2024-03-13T14:49:29.813 INFO:tasks.workunit.client.1.smithi196.stdout:3/128: creat db/d19/f28 x:0 0 0 2024-03-13T14:49:29.822 INFO:tasks.workunit.client.1.smithi196.stdout:5/256: rmdir d4/db/d14/d19/d1c/d25/d38 39 2024-03-13T14:49:29.828 INFO:tasks.workunit.client.1.smithi196.stdout:7/236: fsync d0/f11 0 2024-03-13T14:49:29.831 INFO:tasks.workunit.client.1.smithi196.stdout:6/255: dread - d6/d7/df/f3a zero size 2024-03-13T14:49:29.834 INFO:tasks.workunit.client.1.smithi196.stdout:3/129: creat db/f29 x:0 0 0 2024-03-13T14:49:29.838 INFO:tasks.workunit.client.1.smithi196.stdout:8/111: dwrite d0/f1 [0,4194304] 0 2024-03-13T14:49:29.838 INFO:tasks.workunit.client.1.smithi196.stdout:4/169: write d1/d3/fa [1127543,12652] 0 2024-03-13T14:49:29.841 INFO:tasks.workunit.client.1.smithi196.stdout:5/257: rename d4/db/d14/d19/f3b to d4/db/d14/d19/d1c/d2d/f4c 0 2024-03-13T14:49:29.844 INFO:tasks.workunit.client.1.smithi196.stdout:8/112: write d0/f8 [4974926,69296] 0 2024-03-13T14:49:29.849 INFO:tasks.workunit.client.1.smithi196.stdout:0/249: dread d8/da/de/d17/f4d [0,4194304] 0 2024-03-13T14:49:29.850 INFO:tasks.workunit.client.1.smithi196.stdout:5/258: dread d4/db/d14/d19/d26/f3a [0,4194304] 0 2024-03-13T14:49:29.850 INFO:tasks.workunit.client.1.smithi196.stdout:5/259: readlink d4/l18 0 2024-03-13T14:49:29.855 INFO:tasks.workunit.client.1.smithi196.stdout:5/260: read d4/db/d14/d19/d26/f43 [2953594,108625] 0 2024-03-13T14:49:29.856 INFO:tasks.workunit.client.1.smithi196.stdout:3/130: symlink db/d15/l2a 0 2024-03-13T14:49:29.858 INFO:tasks.workunit.client.1.smithi196.stdout:3/131: dread - db/d19/f28 zero size 2024-03-13T14:49:29.859 INFO:tasks.workunit.client.1.smithi196.stdout:4/170: chown d1/d3/l1f 29706 1 2024-03-13T14:49:29.859 INFO:tasks.workunit.client.1.smithi196.stdout:5/261: truncate d4/db/d14/d19/d1c/f3f 936096 0 2024-03-13T14:49:29.860 INFO:tasks.workunit.client.1.smithi196.stdout:5/262: read d4/db/f24 [1696888,67368] 0 2024-03-13T14:49:29.865 INFO:tasks.workunit.client.1.smithi196.stdout:8/113: creat d0/f27 x:0 0 0 2024-03-13T14:49:29.876 INFO:tasks.workunit.client.1.smithi196.stdout:7/237: mkdir d0/d9/d31/d46/d47 0 2024-03-13T14:49:29.880 INFO:tasks.workunit.client.1.smithi196.stdout:5/263: dread d4/db/d14/d19/d1c/d2d/f4c [0,4194304] 0 2024-03-13T14:49:29.880 INFO:tasks.workunit.client.1.smithi196.stdout:8/114: creat d0/d10/d12/d17/f28 x:0 0 0 2024-03-13T14:49:29.891 INFO:tasks.workunit.client.1.smithi196.stdout:2/104: write d0/d6/f8 [1368533,130741] 0 2024-03-13T14:49:29.893 INFO:tasks.workunit.client.1.smithi196.stdout:3/132: getdents db/d15 0 2024-03-13T14:49:29.894 INFO:tasks.workunit.client.1.smithi196.stdout:1/172: dwrite d1/f31 [0,4194304] 0 2024-03-13T14:49:29.895 INFO:tasks.workunit.client.1.smithi196.stdout:7/238: rmdir d0/d9/d31 39 2024-03-13T14:49:29.895 INFO:tasks.workunit.client.1.smithi196.stdout:4/171: dread d1/d3/d7/f19 [0,4194304] 0 2024-03-13T14:49:29.898 INFO:tasks.workunit.client.1.smithi196.stdout:6/256: dwrite d6/d4b/d50/f51 [0,4194304] 0 2024-03-13T14:49:29.899 INFO:tasks.workunit.client.1.smithi196.stdout:8/115: truncate d0/d10/f26 731085 0 2024-03-13T14:49:29.899 INFO:tasks.workunit.client.1.smithi196.stdout:0/250: getdents d8/da/d1a 0 2024-03-13T14:49:29.901 INFO:tasks.workunit.client.1.smithi196.stdout:5/264: creat d4/db/d14/d37/f4d x:0 0 0 2024-03-13T14:49:29.904 INFO:tasks.workunit.client.1.smithi196.stdout:3/133: unlink f3 0 2024-03-13T14:49:29.904 INFO:tasks.workunit.client.1.smithi196.stdout:6/257: dread - d6/d2a/dc/f3e zero size 2024-03-13T14:49:29.912 INFO:tasks.workunit.client.1.smithi196.stdout:3/134: readlink db/d15/l2a 0 2024-03-13T14:49:29.913 INFO:tasks.workunit.client.1.smithi196.stdout:0/251: dread d8/da/de/d43/f45 [0,4194304] 0 2024-03-13T14:49:29.916 INFO:tasks.workunit.client.1.smithi196.stdout:5/265: creat d4/db/d14/d19/d1c/d2d/f4e x:0 0 0 2024-03-13T14:49:29.916 INFO:tasks.workunit.client.1.smithi196.stdout:8/116: mkdir d0/d1c/d29 0 2024-03-13T14:49:29.920 INFO:tasks.workunit.client.1.smithi196.stdout:2/105: rmdir d0/d7/d15 0 2024-03-13T14:49:29.921 INFO:tasks.workunit.client.1.smithi196.stdout:1/173: rename d1/d9/fd to d1/d28/f35 0 2024-03-13T14:49:29.923 INFO:tasks.workunit.client.1.smithi196.stdout:7/239: symlink d0/d9/d31/d46/d47/l48 0 2024-03-13T14:49:29.923 INFO:tasks.workunit.client.1.smithi196.stdout:0/252: symlink d8/da/d25/d26/d35/d3b/d50/l56 0 2024-03-13T14:49:29.924 INFO:tasks.workunit.client.1.smithi196.stdout:1/174: fdatasync d1/d9/d27/f17 0 2024-03-13T14:49:29.925 INFO:tasks.workunit.client.1.smithi196.stdout:5/266: mknod d4/d1b/c4f 0 2024-03-13T14:49:29.926 INFO:tasks.workunit.client.1.smithi196.stdout:4/172: dwrite d1/d3/fa [0,4194304] 0 2024-03-13T14:49:29.926 INFO:tasks.workunit.client.1.smithi196.stdout:2/106: symlink d0/d7/d9/l16 0 2024-03-13T14:49:29.928 INFO:tasks.workunit.client.1.smithi196.stdout:0/253: write d8/da/d25/f2f [1580836,5584] 0 2024-03-13T14:49:29.928 INFO:tasks.workunit.client.1.smithi196.stdout:6/258: rename d6/d2a/dc/d1c/c3c to d6/d7/df/d27/c56 0 2024-03-13T14:49:29.932 INFO:tasks.workunit.client.1.smithi196.stdout:3/135: getdents db/d15 0 2024-03-13T14:49:29.933 INFO:tasks.workunit.client.1.smithi196.stdout:6/259: chown d6/d2a/dc/f3e 8512399 1 2024-03-13T14:49:29.934 INFO:tasks.workunit.client.1.smithi196.stdout:8/117: rename d0/c6 to d0/d16/c2a 0 2024-03-13T14:49:29.934 INFO:tasks.workunit.client.1.smithi196.stdout:1/175: write d1/d9/f20 [4306504,45993] 0 2024-03-13T14:49:29.936 INFO:tasks.workunit.client.1.smithi196.stdout:0/254: creat d8/da/d1a/f57 x:0 0 0 2024-03-13T14:49:29.940 INFO:tasks.workunit.client.1.smithi196.stdout:4/173: symlink d1/l2d 0 2024-03-13T14:49:29.941 INFO:tasks.workunit.client.1.smithi196.stdout:4/174: chown d1/l2d 1 1 2024-03-13T14:49:29.945 INFO:tasks.workunit.client.1.smithi196.stdout:5/267: rename d4/f13 to d4/db/d14/d19/f50 0 2024-03-13T14:49:29.949 INFO:tasks.workunit.client.1.smithi196.stdout:6/260: unlink d6/d9/d11/f31 0 2024-03-13T14:49:29.952 INFO:tasks.workunit.client.1.smithi196.stdout:8/118: mknod d0/d10/d14/c2b 0 2024-03-13T14:49:29.953 INFO:tasks.workunit.client.1.smithi196.stdout:8/119: stat d0/c18 0 2024-03-13T14:49:29.954 INFO:tasks.workunit.client.1.smithi196.stdout:3/136: dwrite db/fe [0,4194304] 0 2024-03-13T14:49:29.957 INFO:tasks.workunit.client.1.smithi196.stdout:8/120: write d0/d10/f15 [398229,41399] 0 2024-03-13T14:49:29.960 INFO:tasks.workunit.client.1.smithi196.stdout:8/121: truncate d0/d10/f19 1311822 0 2024-03-13T14:49:29.962 INFO:tasks.workunit.client.1.smithi196.stdout:8/122: fsync d0/d10/d12/d17/f28 0 2024-03-13T14:49:29.962 INFO:tasks.workunit.client.1.smithi196.stdout:5/268: creat d4/db/d14/d19/d26/f51 x:0 0 0 2024-03-13T14:49:29.964 INFO:tasks.workunit.client.1.smithi196.stdout:0/255: mknod d8/c58 0 2024-03-13T14:49:29.964 INFO:tasks.workunit.client.1.smithi196.stdout:8/123: write d0/d10/f26 [592066,66846] 0 2024-03-13T14:49:29.969 INFO:tasks.workunit.client.1.smithi196.stdout:7/240: rename d0/d6/c28 to d0/d9/d1e/d3f/c49 0 2024-03-13T14:49:29.970 INFO:tasks.workunit.client.1.smithi196.stdout:6/261: mkdir d6/d57 0 2024-03-13T14:49:29.971 INFO:tasks.workunit.client.1.smithi196.stdout:7/241: write d0/f11 [4296214,10946] 0 2024-03-13T14:49:29.973 INFO:tasks.workunit.client.1.smithi196.stdout:5/269: creat d4/db/d3c/f52 x:0 0 0 2024-03-13T14:49:29.977 INFO:tasks.workunit.client.1.smithi196.stdout:3/137: chown db/d19 3360765 1 2024-03-13T14:49:29.979 INFO:tasks.workunit.client.1.smithi196.stdout:1/176: creat d1/d9/f36 x:0 0 0 2024-03-13T14:49:29.980 INFO:tasks.workunit.client.1.smithi196.stdout:6/262: creat d6/d4b/f58 x:0 0 0 2024-03-13T14:49:29.984 INFO:tasks.workunit.client.1.smithi196.stdout:5/270: mknod d4/db/d14/d19/c53 0 2024-03-13T14:49:29.989 INFO:tasks.workunit.client.1.smithi196.stdout:6/263: mknod d6/d2a/dc/d47/c59 0 2024-03-13T14:49:29.991 INFO:tasks.workunit.client.1.smithi196.stdout:5/271: readlink d4/db/d14/d19/l34 0 2024-03-13T14:49:29.993 INFO:tasks.workunit.client.1.smithi196.stdout:6/264: write d6/d2a/dc/d1c/f25 [195445,29633] 0 2024-03-13T14:49:29.993 INFO:tasks.workunit.client.1.smithi196.stdout:7/242: creat d0/d6/f4a x:0 0 0 2024-03-13T14:49:29.995 INFO:tasks.workunit.client.1.smithi196.stdout:8/124: truncate d0/f8 4122937 0 2024-03-13T14:49:29.997 INFO:tasks.workunit.client.1.smithi196.stdout:8/125: chown d0/d10 235 1 2024-03-13T14:49:29.997 INFO:tasks.workunit.client.1.smithi196.stdout:1/177: creat d1/f37 x:0 0 0 2024-03-13T14:49:29.999 INFO:tasks.workunit.client.1.smithi196.stdout:3/138: rename db/f20 to db/d19/f2b 0 2024-03-13T14:49:29.999 INFO:tasks.workunit.client.1.smithi196.stdout:5/272: write d4/f11 [753025,82475] 0 2024-03-13T14:49:30.001 INFO:tasks.workunit.client.1.smithi196.stdout:0/256: dwrite d8/da/d1a/f1f [0,4194304] 0 2024-03-13T14:49:30.002 INFO:tasks.workunit.client.1.smithi196.stdout:7/243: readlink d0/l1f 0 2024-03-13T14:49:30.006 INFO:tasks.workunit.client.1.smithi196.stdout:7/244: write d0/d9/d1e/f29 [3829467,29258] 0 2024-03-13T14:49:30.012 INFO:tasks.workunit.client.1.smithi196.stdout:5/273: dread - d4/db/d3c/f52 zero size 2024-03-13T14:49:30.012 INFO:tasks.workunit.client.1.smithi196.stdout:7/245: chown d0/d6/fc 786169 1 2024-03-13T14:49:30.012 INFO:tasks.workunit.client.1.smithi196.stdout:8/126: rename d0/ce to d0/d1c/d29/c2c 0 2024-03-13T14:49:30.014 INFO:tasks.workunit.client.1.smithi196.stdout:5/274: chown d4/d1b/f29 14961 1 2024-03-13T14:49:30.023 INFO:tasks.workunit.client.1.smithi196.stdout:3/139: rename db/d15/f21 to db/f2c 0 2024-03-13T14:49:30.024 INFO:tasks.workunit.client.1.smithi196.stdout:7/246: readlink d0/d9/d31/d46/d47/l48 0 2024-03-13T14:49:30.025 INFO:tasks.workunit.client.1.smithi196.stdout:8/127: write d0/d10/d14/f1f [569641,50431] 0 2024-03-13T14:49:30.027 INFO:tasks.workunit.client.1.smithi196.stdout:5/275: symlink d4/db/d14/d19/d1c/l54 0 2024-03-13T14:49:30.031 INFO:tasks.workunit.client.1.smithi196.stdout:8/128: chown d0/d1c/d29 296016020 1 2024-03-13T14:49:30.033 INFO:tasks.workunit.client.1.smithi196.stdout:7/247: creat d0/d9/d1e/f4b x:0 0 0 2024-03-13T14:49:30.042 INFO:tasks.workunit.client.1.smithi196.stdout:7/248: stat d0/d9/d1e/d3f 0 2024-03-13T14:49:30.043 INFO:tasks.workunit.client.1.smithi196.stdout:7/249: readlink d0/d6/l3e 0 2024-03-13T14:49:30.045 INFO:tasks.workunit.client.1.smithi196.stdout:1/178: dwrite d1/d9/d27/f16 [0,4194304] 0 2024-03-13T14:49:30.052 INFO:tasks.workunit.client.1.smithi196.stdout:8/129: mkdir d0/d10/d12/d1e/d2d 0 2024-03-13T14:49:30.052 INFO:tasks.workunit.client.1.smithi196.stdout:0/257: dwrite d8/da/d25/d26/f42 [0,4194304] 0 2024-03-13T14:49:30.056 INFO:tasks.workunit.client.1.smithi196.stdout:3/140: rename db/fd to db/f2d 0 2024-03-13T14:49:30.057 INFO:tasks.workunit.client.1.smithi196.stdout:5/276: mknod d4/c55 0 2024-03-13T14:49:30.057 INFO:tasks.workunit.client.1.smithi196.stdout:1/179: readlink d1/l6 0 2024-03-13T14:49:30.061 INFO:tasks.workunit.client.1.smithi196.stdout:1/180: write d1/d28/f2b [76765,76027] 0 2024-03-13T14:49:30.066 INFO:tasks.workunit.client.1.smithi196.stdout:7/250: creat d0/d6/d33/f4c x:0 0 0 2024-03-13T14:49:30.067 INFO:tasks.workunit.client.1.smithi196.stdout:9/171: sync 2024-03-13T14:49:30.074 INFO:tasks.workunit.client.1.smithi196.stdout:0/258: mkdir d8/da/d25/d26/d35/d3b/d59 0 2024-03-13T14:49:30.076 INFO:tasks.workunit.client.1.smithi196.stdout:6/265: dread d6/d9/d17/d19/f2d [0,4194304] 0 2024-03-13T14:49:30.076 INFO:tasks.workunit.client.1.smithi196.stdout:9/172: write d7/de/f1f [156263,116794] 0 2024-03-13T14:49:30.077 INFO:tasks.workunit.client.1.smithi196.stdout:0/259: chown d8/da/de/f4a 497224 1 2024-03-13T14:49:30.079 INFO:tasks.workunit.client.1.smithi196.stdout:6/266: write d6/d7/dd/f2c [2729204,30994] 0 2024-03-13T14:49:30.081 INFO:tasks.workunit.client.1.smithi196.stdout:0/260: chown d8/da/de/d43 704803 1 2024-03-13T14:49:30.082 INFO:tasks.workunit.client.1.smithi196.stdout:6/267: stat d6/d9/d17/d19/f2b 0 2024-03-13T14:49:30.082 INFO:tasks.workunit.client.1.smithi196.stdout:1/181: write d1/d29/f1b [1380196,38628] 0 2024-03-13T14:49:30.085 INFO:tasks.workunit.client.1.smithi196.stdout:0/261: chown d8/da/de/d17/f51 1 1 2024-03-13T14:49:30.088 INFO:tasks.workunit.client.1.smithi196.stdout:0/262: chown d8/da/f3e 1488 1 2024-03-13T14:49:30.089 INFO:tasks.workunit.client.1.smithi196.stdout:3/141: fsync db/d19/f1d 0 2024-03-13T14:49:30.100 INFO:tasks.workunit.client.1.smithi196.stdout:9/173: dread d7/d9/df/d1d/f23 [0,4194304] 0 2024-03-13T14:49:30.103 INFO:tasks.workunit.client.1.smithi196.stdout:0/263: creat d8/da/d25/f5a x:0 0 0 2024-03-13T14:49:30.105 INFO:tasks.workunit.client.1.smithi196.stdout:5/277: dwrite d4/db/d14/d19/d1c/d25/d38/f3e [0,4194304] 0 2024-03-13T14:49:30.107 INFO:tasks.workunit.client.1.smithi196.stdout:6/268: dread d6/d7/dd/f2c [0,4194304] 0 2024-03-13T14:49:30.108 INFO:tasks.workunit.client.1.smithi196.stdout:7/251: truncate d0/d9/d31/f42 296097 0 2024-03-13T14:49:30.108 INFO:tasks.workunit.client.1.smithi196.stdout:3/142: truncate db/d19/f2b 111323 0 2024-03-13T14:49:30.111 INFO:tasks.workunit.client.1.smithi196.stdout:9/174: creat d7/d9/df/d1d/d22/f2d x:0 0 0 2024-03-13T14:49:30.112 INFO:tasks.workunit.client.1.smithi196.stdout:7/252: write d0/d6/fc [1607225,88281] 0 2024-03-13T14:49:30.112 INFO:tasks.workunit.client.1.smithi196.stdout:5/278: write d4/db/d14/d19/d1c/d2d/f4e [603106,73590] 0 2024-03-13T14:49:30.116 INFO:tasks.workunit.client.1.smithi196.stdout:1/182: rename d1/d9/d27 to d1/d38 0 2024-03-13T14:49:30.122 INFO:tasks.workunit.client.1.smithi196.stdout:2/107: sync 2024-03-13T14:49:30.122 INFO:tasks.workunit.client.1.smithi196.stdout:4/175: sync 2024-03-13T14:49:30.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:29 smithi176 ceph-mon[29064]: pgmap v3: 65 pgs: 65 active+clean; 534 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:49:30.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:29 smithi176 ceph-mon[29064]: mgrmap e28: smithi176.tzfdwd(active, since 2s) 2024-03-13T14:49:30.128 INFO:tasks.workunit.client.1.smithi196.stdout:2/108: readlink d0/d7/l11 0 2024-03-13T14:49:30.129 INFO:tasks.workunit.client.1.smithi196.stdout:9/175: symlink d7/d9/d17/l2e 0 2024-03-13T14:49:30.131 INFO:tasks.workunit.client.1.smithi196.stdout:9/176: readlink d7/d9/d17/l1b 0 2024-03-13T14:49:30.132 INFO:tasks.workunit.client.1.smithi196.stdout:4/176: dread d1/d3/d7/f19 [0,4194304] 0 2024-03-13T14:49:30.137 INFO:tasks.workunit.client.1.smithi196.stdout:0/264: getdents d8/da/de/d33/d44 0 2024-03-13T14:49:30.137 INFO:tasks.workunit.client.1.smithi196.stdout:4/177: write d1/d3/d7/f1d [2152612,98780] 0 2024-03-13T14:49:30.138 INFO:tasks.workunit.client.1.smithi196.stdout:7/253: mkdir d0/d4d 0 2024-03-13T14:49:30.138 INFO:tasks.workunit.client.1.smithi196.stdout:2/109: rmdir d0/d7 39 2024-03-13T14:49:30.140 INFO:tasks.workunit.client.1.smithi196.stdout:5/279: link d4/db/f24 d4/db/d14/d19/d1c/d25/d38/f56 0 2024-03-13T14:49:30.140 INFO:tasks.workunit.client.1.smithi196.stdout:6/269: dwrite d6/d4b/d50/f51 [4194304,4194304] 0 2024-03-13T14:49:30.140 INFO:tasks.workunit.client.1.smithi196.stdout:0/265: readlink l3 0 2024-03-13T14:49:30.140 INFO:tasks.workunit.client.1.smithi196.stdout:1/183: symlink d1/d9/d33/l39 0 2024-03-13T14:49:30.143 INFO:tasks.workunit.client.1.smithi196.stdout:1/184: chown d1/d29/f22 3645515 1 2024-03-13T14:49:30.143 INFO:tasks.workunit.client.1.smithi196.stdout:8/130: dread d0/f8 [0,4194304] 0 2024-03-13T14:49:30.147 INFO:tasks.workunit.client.1.smithi196.stdout:8/131: chown d0/d10/d12 1 1 2024-03-13T14:49:30.153 INFO:tasks.workunit.client.1.smithi196.stdout:7/254: chown d0/d9/f24 3468 1 2024-03-13T14:49:30.156 INFO:tasks.workunit.client.1.smithi196.stdout:6/270: rename d6/f10 to d6/d4b/d50/f5a 0 2024-03-13T14:49:30.156 INFO:tasks.workunit.client.1.smithi196.stdout:9/177: dwrite d7/de/f1f [0,4194304] 0 2024-03-13T14:49:30.159 INFO:tasks.workunit.client.1.smithi196.stdout:6/271: chown d6/d57 195561764 1 2024-03-13T14:49:30.159 INFO:tasks.workunit.client.1.smithi196.stdout:1/185: truncate d1/d29/f1a 1582839 0 2024-03-13T14:49:30.162 INFO:tasks.workunit.client.1.smithi196.stdout:8/132: symlink d0/d1c/l2e 0 2024-03-13T14:49:30.165 INFO:tasks.workunit.client.1.smithi196.stdout:1/186: dread - d1/d38/f2e zero size 2024-03-13T14:49:30.167 INFO:tasks.workunit.client.1.smithi196.stdout:2/110: creat d0/d7/f17 x:0 0 0 2024-03-13T14:49:30.170 INFO:tasks.workunit.client.1.smithi196.stdout:7/255: fdatasync d0/d6/d33/f4c 0 2024-03-13T14:49:30.178 INFO:tasks.workunit.client.1.smithi196.stdout:0/266: rename d8/da/f38 to d8/d37/f5b 0 2024-03-13T14:49:30.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:29 smithi196 ceph-mon[37486]: pgmap v3: 65 pgs: 65 active+clean; 534 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:49:30.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:29 smithi196 ceph-mon[37486]: mgrmap e28: smithi176.tzfdwd(active, since 2s) 2024-03-13T14:49:30.185 INFO:tasks.workunit.client.1.smithi196.stdout:8/133: unlink d0/c7 0 2024-03-13T14:49:30.186 INFO:tasks.workunit.client.1.smithi196.stdout:3/143: getdents db/d19/d22 0 2024-03-13T14:49:30.187 INFO:tasks.workunit.client.1.smithi196.stdout:9/178: dwrite d7/d9/df/d1d/d22/f2d [0,4194304] 0 2024-03-13T14:49:30.190 INFO:tasks.workunit.client.1.smithi196.stdout:5/280: creat d4/db/d3c/f57 x:0 0 0 2024-03-13T14:49:30.191 INFO:tasks.workunit.client.1.smithi196.stdout:6/272: dwrite d6/fa [0,4194304] 0 2024-03-13T14:49:30.191 INFO:tasks.workunit.client.1.smithi196.stdout:4/178: creat d1/f2e x:0 0 0 2024-03-13T14:49:30.195 INFO:tasks.workunit.client.1.smithi196.stdout:1/187: dwrite d1/d29/f22 [0,4194304] 0 2024-03-13T14:49:30.206 INFO:tasks.workunit.client.1.smithi196.stdout:9/179: rmdir d7/d9/d17 39 2024-03-13T14:49:30.207 INFO:tasks.workunit.client.1.smithi196.stdout:2/111: dread d0/fe [0,4194304] 0 2024-03-13T14:49:30.210 INFO:tasks.workunit.client.1.smithi196.stdout:4/179: mkdir d1/d2f 0 2024-03-13T14:49:30.216 INFO:tasks.workunit.client.1.smithi196.stdout:9/180: chown d7/d9/f11 10 1 2024-03-13T14:49:30.226 INFO:tasks.workunit.client.1.smithi196.stdout:7/256: dwrite d0/d6/fc [0,4194304] 0 2024-03-13T14:49:30.233 INFO:tasks.workunit.client.1.smithi196.stdout:7/257: chown d0/d6/fc 1180 1 2024-03-13T14:49:30.235 INFO:tasks.workunit.client.1.smithi196.stdout:8/134: dread d0/d10/f21 [0,4194304] 0 2024-03-13T14:49:30.239 INFO:tasks.workunit.client.1.smithi196.stdout:0/267: dread d8/f1e [0,4194304] 0 2024-03-13T14:49:30.240 INFO:tasks.workunit.client.0.smithi176.stdout:0/10: sync 2024-03-13T14:49:30.241 INFO:tasks.workunit.client.1.smithi196.stdout:9/181: mknod d7/d9/c2f 0 2024-03-13T14:49:30.247 INFO:tasks.workunit.client.0.smithi176.stdout:0/11: truncate d2/f3 305973 0 2024-03-13T14:49:30.248 INFO:tasks.workunit.client.1.smithi196.stdout:5/281: dwrite d4/db/d14/d19/f50 [0,4194304] 0 2024-03-13T14:49:30.248 INFO:tasks.workunit.client.1.smithi196.stdout:7/258: fsync d0/d6/f21 0 2024-03-13T14:49:30.250 INFO:tasks.workunit.client.1.smithi196.stdout:6/273: dread d6/d9/f1f [0,4194304] 0 2024-03-13T14:49:30.253 INFO:tasks.workunit.client.1.smithi196.stdout:8/135: mkdir d0/d10/d12/d2f 0 2024-03-13T14:49:30.255 INFO:tasks.workunit.client.1.smithi196.stdout:5/282: creat d4/db/d14/d19/f58 x:0 0 0 2024-03-13T14:49:30.255 INFO:tasks.workunit.client.0.smithi176.stdout:0/12: dread d2/f3 [0,4194304] 0 2024-03-13T14:49:30.256 INFO:tasks.workunit.client.1.smithi196.stdout:4/180: dread d1/f29 [0,4194304] 0 2024-03-13T14:49:30.257 INFO:tasks.workunit.client.0.smithi176.stdout:0/13: chown d2/f3 2 1 2024-03-13T14:49:30.257 INFO:tasks.workunit.client.1.smithi196.stdout:7/259: write d0/d6/d33/f40 [678532,23765] 0 2024-03-13T14:49:30.259 INFO:tasks.workunit.client.1.smithi196.stdout:3/144: dwrite db/fe [0,4194304] 0 2024-03-13T14:49:30.263 INFO:tasks.workunit.client.0.smithi176.stdout:0/14: write d2/f3 [1139464,114538] 0 2024-03-13T14:49:30.268 INFO:tasks.workunit.client.1.smithi196.stdout:0/268: creat d8/da/f5c x:0 0 0 2024-03-13T14:49:30.270 INFO:tasks.workunit.client.1.smithi196.stdout:6/274: symlink d6/d4b/l5b 0 2024-03-13T14:49:30.271 INFO:tasks.workunit.client.1.smithi196.stdout:3/145: truncate db/d19/f1d 383200 0 2024-03-13T14:49:30.272 INFO:tasks.workunit.client.1.smithi196.stdout:2/112: dwrite d0/d6/f13 [0,4194304] 0 2024-03-13T14:49:30.272 INFO:tasks.workunit.client.1.smithi196.stdout:7/260: fsync d0/fa 0 2024-03-13T14:49:30.273 INFO:tasks.workunit.client.0.smithi176.stdout:0/15: symlink d2/l4 0 2024-03-13T14:49:30.274 INFO:tasks.workunit.client.1.smithi196.stdout:3/146: fdatasync db/d15/f17 0 2024-03-13T14:49:30.275 INFO:tasks.workunit.client.0.smithi176.stdout:0/16: write d2/f3 [1436878,15126] 0 2024-03-13T14:49:30.279 INFO:tasks.workunit.client.1.smithi196.stdout:9/182: rename d7/lc to d7/de/l30 0 2024-03-13T14:49:30.287 INFO:tasks.workunit.client.1.smithi196.stdout:3/147: creat db/d15/f2e x:0 0 0 2024-03-13T14:49:30.288 INFO:tasks.workunit.client.1.smithi196.stdout:2/113: truncate d0/fe 4549670 0 2024-03-13T14:49:30.290 INFO:tasks.workunit.client.1.smithi196.stdout:0/269: rename d8/da/d1a to d8/da/d25/d26/d35/d3b/d50/d5d 0 2024-03-13T14:49:30.293 INFO:tasks.workunit.client.1.smithi196.stdout:6/275: mknod d6/c5c 0 2024-03-13T14:49:30.295 INFO:tasks.workunit.client.1.smithi196.stdout:9/183: creat d7/d24/f31 x:0 0 0 2024-03-13T14:49:30.296 INFO:tasks.workunit.client.0.smithi176.stdout:0/17: dwrite d2/f3 [0,4194304] 0 2024-03-13T14:49:30.297 INFO:tasks.workunit.client.1.smithi196.stdout:6/276: fsync d6/d7/df/f3a 0 2024-03-13T14:49:30.306 INFO:tasks.workunit.client.1.smithi196.stdout:5/283: dread d4/db/ff [0,4194304] 0 2024-03-13T14:49:30.307 INFO:tasks.workunit.client.1.smithi196.stdout:4/181: dwrite d1/d3/d7/f1a [4194304,4194304] 0 2024-03-13T14:49:30.309 INFO:tasks.workunit.client.1.smithi196.stdout:9/184: write d7/de/f1f [4562719,21847] 0 2024-03-13T14:49:30.312 INFO:tasks.workunit.client.1.smithi196.stdout:3/148: write db/d19/f28 [384450,123432] 0 2024-03-13T14:49:30.317 INFO:tasks.workunit.client.1.smithi196.stdout:8/136: dwrite d0/d10/d14/f1f [0,4194304] 0 2024-03-13T14:49:30.329 INFO:tasks.workunit.client.1.smithi196.stdout:0/270: rename l3 to d8/da/l5e 0 2024-03-13T14:49:30.330 INFO:tasks.workunit.client.1.smithi196.stdout:6/277: dread d6/d7/dd/f2c [0,4194304] 0 2024-03-13T14:49:30.331 INFO:tasks.workunit.client.1.smithi196.stdout:4/182: rename d1/d3 to d1/d3/d30 22 2024-03-13T14:49:30.333 INFO:tasks.workunit.client.1.smithi196.stdout:2/114: dwrite d0/d6/f13 [0,4194304] 0 2024-03-13T14:49:30.333 INFO:tasks.workunit.client.1.smithi196.stdout:6/278: dread - d6/d9/d17/d19/f42 zero size 2024-03-13T14:49:30.338 INFO:tasks.workunit.client.0.smithi176.stdout:0/18: fdatasync d2/f3 0 2024-03-13T14:49:30.340 INFO:tasks.workunit.client.0.smithi176.stdout:0/19: write d2/f3 [3010250,23248] 0 2024-03-13T14:49:30.348 INFO:tasks.workunit.client.0.smithi176.stdout:0/20: mkdir d2/d5 0 2024-03-13T14:49:30.350 INFO:tasks.workunit.client.1.smithi196.stdout:1/188: dread d1/d29/f1b [0,4194304] 0 2024-03-13T14:49:30.355 INFO:tasks.workunit.client.1.smithi196.stdout:6/279: mknod d6/d2a/dc/d1c/c5d 0 2024-03-13T14:49:30.356 INFO:tasks.workunit.client.1.smithi196.stdout:2/115: dread d0/fe [0,4194304] 0 2024-03-13T14:49:30.364 INFO:tasks.workunit.client.0.smithi176.stdout:0/21: dread d2/f3 [0,4194304] 0 2024-03-13T14:49:30.370 INFO:tasks.workunit.client.1.smithi196.stdout:3/149: creat db/d19/f2f x:0 0 0 2024-03-13T14:49:30.371 INFO:tasks.workunit.client.1.smithi196.stdout:2/116: dread d0/d6/fb [0,4194304] 0 2024-03-13T14:49:30.375 INFO:tasks.workunit.client.1.smithi196.stdout:7/261: dread d0/f1c [0,4194304] 0 2024-03-13T14:49:30.379 INFO:tasks.workunit.client.1.smithi196.stdout:5/284: rename d4/db/d14/d19/d1c/d2d/f4c to d4/db/d3c/f59 0 2024-03-13T14:49:30.385 INFO:tasks.workunit.client.0.smithi176.stdout:1/10: sync 2024-03-13T14:49:30.385 INFO:tasks.workunit.client.0.smithi176.stdout:5/9: sync 2024-03-13T14:49:30.387 INFO:tasks.workunit.client.1.smithi196.stdout:0/271: truncate d8/da/f4e 2021001 0 2024-03-13T14:49:30.392 INFO:tasks.workunit.client.1.smithi196.stdout:6/280: dread d6/f8 [0,4194304] 0 2024-03-13T14:49:30.397 INFO:tasks.workunit.client.1.smithi196.stdout:1/189: truncate d1/d29/f1b 562098 0 2024-03-13T14:49:30.398 INFO:tasks.workunit.client.0.smithi176.stdout:1/11: dwrite f1 [0,4194304] 0 2024-03-13T14:49:30.399 INFO:tasks.workunit.client.1.smithi196.stdout:8/137: rename d0/f1 to d0/d1c/f30 0 2024-03-13T14:49:30.399 INFO:tasks.workunit.client.0.smithi176.stdout:1/12: readlink l0 0 2024-03-13T14:49:30.400 INFO:tasks.workunit.client.1.smithi196.stdout:1/190: truncate d1/d29/f1d 882033 0 2024-03-13T14:49:30.410 INFO:tasks.workunit.client.1.smithi196.stdout:6/281: dread d6/fa [0,4194304] 0 2024-03-13T14:49:30.412 INFO:tasks.workunit.client.1.smithi196.stdout:2/117: rename d0/f2 to d0/d7/d9/f18 0 2024-03-13T14:49:30.415 INFO:tasks.workunit.client.0.smithi176.stdout:4/31: sync 2024-03-13T14:49:30.415 INFO:tasks.workunit.client.0.smithi176.stdout:6/11: sync 2024-03-13T14:49:30.415 INFO:tasks.workunit.client.0.smithi176.stdout:9/54: sync 2024-03-13T14:49:30.415 INFO:tasks.workunit.client.0.smithi176.stdout:8/14: sync 2024-03-13T14:49:30.415 INFO:tasks.workunit.client.0.smithi176.stdout:2/31: sync 2024-03-13T14:49:30.415 INFO:tasks.workunit.client.0.smithi176.stdout:7/6: sync 2024-03-13T14:49:30.416 INFO:tasks.workunit.client.0.smithi176.stdout:7/7: dwrite - no filename 2024-03-13T14:49:30.416 INFO:tasks.workunit.client.0.smithi176.stdout:3/47: sync 2024-03-13T14:49:30.418 INFO:tasks.workunit.client.0.smithi176.stdout:9/55: dread - d0/f12 zero size 2024-03-13T14:49:30.418 INFO:tasks.workunit.client.0.smithi176.stdout:8/15: write d0/f3 [715330,63753] 0 2024-03-13T14:49:30.420 INFO:tasks.workunit.client.0.smithi176.stdout:8/16: stat d0/f2 0 2024-03-13T14:49:30.421 INFO:tasks.workunit.client.0.smithi176.stdout:8/17: read - d0/f2 zero size 2024-03-13T14:49:30.422 INFO:tasks.workunit.client.0.smithi176.stdout:8/18: write d0/f3 [640571,40877] 0 2024-03-13T14:49:30.422 INFO:tasks.workunit.client.1.smithi196.stdout:8/138: creat d0/d10/f31 x:0 0 0 2024-03-13T14:49:30.428 INFO:tasks.workunit.client.1.smithi196.stdout:0/272: dwrite d8/da/d25/d26/d35/f4f [0,4194304] 0 2024-03-13T14:49:30.430 INFO:tasks.workunit.client.0.smithi176.stdout:3/48: dread d4/fa [0,4194304] 0 2024-03-13T14:49:30.432 INFO:tasks.workunit.client.0.smithi176.stdout:8/19: fsync d0/f3 0 2024-03-13T14:49:30.435 INFO:tasks.workunit.client.0.smithi176.stdout:2/32: dwrite d1/fb [0,4194304] 0 2024-03-13T14:49:30.436 INFO:tasks.workunit.client.0.smithi176.stdout:8/20: truncate d0/f2 411141 0 2024-03-13T14:49:30.439 INFO:tasks.workunit.client.1.smithi196.stdout:5/285: rename d4/db/d14/d19/d1c/d25/f2a to d4/db/d14/d19/f5a 0 2024-03-13T14:49:30.441 INFO:tasks.workunit.client.1.smithi196.stdout:6/282: fdatasync d6/d7/dd/f2c 0 2024-03-13T14:49:30.446 INFO:tasks.workunit.client.1.smithi196.stdout:6/283: dread - d6/d2a/d23/d28/f3f zero size 2024-03-13T14:49:30.446 INFO:tasks.workunit.client.1.smithi196.stdout:7/262: link d0/d6/f19 d0/d9/d31/d46/d47/f4e 0 2024-03-13T14:49:30.450 INFO:tasks.workunit.client.1.smithi196.stdout:1/191: dwrite d1/d29/f1d [0,4194304] 0 2024-03-13T14:49:30.450 INFO:tasks.workunit.client.1.smithi196.stdout:8/139: dwrite d0/d10/f21 [0,4194304] 0 2024-03-13T14:49:30.451 INFO:tasks.workunit.client.1.smithi196.stdout:1/192: write d1/f2a [55060,57410] 0 2024-03-13T14:49:30.454 INFO:tasks.workunit.client.1.smithi196.stdout:1/193: dread - d1/f26 zero size 2024-03-13T14:49:30.454 INFO:tasks.workunit.client.1.smithi196.stdout:0/273: write d8/da/fc [560867,113195] 0 2024-03-13T14:49:30.454 INFO:tasks.workunit.client.1.smithi196.stdout:8/140: write d0/d10/f19 [1132437,34460] 0 2024-03-13T14:49:30.455 INFO:tasks.workunit.client.0.smithi176.stdout:8/21: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:30.457 INFO:tasks.workunit.client.0.smithi176.stdout:8/22: fsync d0/f3 0 2024-03-13T14:49:30.460 INFO:tasks.workunit.client.1.smithi196.stdout:1/194: write d1/d9/f18 [1505301,37893] 0 2024-03-13T14:49:30.460 INFO:tasks.workunit.client.0.smithi176.stdout:8/23: write d0/f3 [20367,100026] 0 2024-03-13T14:49:30.466 INFO:tasks.workunit.client.1.smithi196.stdout:8/141: dread d0/d10/f15 [0,4194304] 0 2024-03-13T14:49:30.469 INFO:tasks.workunit.client.1.smithi196.stdout:8/142: dread - d0/d10/d12/d17/f28 zero size 2024-03-13T14:49:30.471 INFO:tasks.workunit.client.1.smithi196.stdout:3/150: truncate f4 1803780 0 2024-03-13T14:49:30.472 INFO:tasks.workunit.client.1.smithi196.stdout:6/284: creat d6/d9/d11/f5e x:0 0 0 2024-03-13T14:49:30.478 INFO:tasks.workunit.client.1.smithi196.stdout:8/143: write d0/d10/f26 [85370,91881] 0 2024-03-13T14:49:30.484 INFO:tasks.workunit.client.1.smithi196.stdout:7/263: dwrite d0/d6/f45 [0,4194304] 0 2024-03-13T14:49:30.496 INFO:tasks.workunit.client.1.smithi196.stdout:8/144: dread d0/d10/f11 [0,4194304] 0 2024-03-13T14:49:30.497 INFO:tasks.workunit.client.1.smithi196.stdout:2/118: link d0/d7/l11 d0/d7/l19 0 2024-03-13T14:49:30.497 INFO:tasks.workunit.client.1.smithi196.stdout:3/151: mkdir db/d19/d22/d30 0 2024-03-13T14:49:30.500 INFO:tasks.workunit.client.1.smithi196.stdout:3/152: fsync db/d19/f2f 0 2024-03-13T14:49:30.500 INFO:tasks.workunit.client.1.smithi196.stdout:6/285: mkdir d6/d9/d17/d19/d5f 0 2024-03-13T14:49:30.510 INFO:tasks.workunit.client.1.smithi196.stdout:7/264: dread d0/d9/f20 [0,4194304] 0 2024-03-13T14:49:30.512 INFO:tasks.workunit.client.1.smithi196.stdout:1/195: link d1/d29/f1d d1/d28/d34/f3a 0 2024-03-13T14:49:30.514 INFO:tasks.workunit.client.1.smithi196.stdout:8/145: mknod d0/d16/c32 0 2024-03-13T14:49:30.517 INFO:tasks.workunit.client.1.smithi196.stdout:8/146: write d0/d10/f21 [437487,85994] 0 2024-03-13T14:49:30.517 INFO:tasks.workunit.client.1.smithi196.stdout:3/153: creat db/d19/d22/f31 x:0 0 0 2024-03-13T14:49:30.534 INFO:tasks.workunit.client.1.smithi196.stdout:1/196: creat d1/d38/f3b x:0 0 0 2024-03-13T14:49:30.537 INFO:tasks.workunit.client.1.smithi196.stdout:8/147: dwrite d0/d10/d12/d17/f28 [0,4194304] 0 2024-03-13T14:49:30.537 INFO:tasks.workunit.client.1.smithi196.stdout:3/154: creat db/f32 x:0 0 0 2024-03-13T14:49:30.540 INFO:tasks.workunit.client.1.smithi196.stdout:5/286: link d4/db/d14/c22 d4/c5b 0 2024-03-13T14:49:30.547 INFO:tasks.workunit.client.1.smithi196.stdout:0/274: truncate d8/da/f4e 2945567 0 2024-03-13T14:49:30.548 INFO:tasks.workunit.client.1.smithi196.stdout:0/275: readlink d8/da/lf 0 2024-03-13T14:49:30.550 INFO:tasks.workunit.client.1.smithi196.stdout:1/197: symlink d1/d28/l3c 0 2024-03-13T14:49:30.552 INFO:tasks.workunit.client.1.smithi196.stdout:3/155: readlink l6 0 2024-03-13T14:49:30.558 INFO:tasks.workunit.client.1.smithi196.stdout:7/265: creat d0/f4f x:0 0 0 2024-03-13T14:49:30.560 INFO:tasks.workunit.client.1.smithi196.stdout:5/287: creat d4/db/d14/d37/d41/f5c x:0 0 0 2024-03-13T14:49:30.565 INFO:tasks.workunit.client.1.smithi196.stdout:5/288: fdatasync d4/db/d14/d19/d1c/d25/f49 0 2024-03-13T14:49:30.577 INFO:tasks.workunit.client.1.smithi196.stdout:7/266: dread d0/d6/f45 [0,4194304] 0 2024-03-13T14:49:30.581 INFO:tasks.workunit.client.1.smithi196.stdout:6/286: dwrite d6/d9/d17/d19/f2d [0,4194304] 0 2024-03-13T14:49:30.583 INFO:tasks.workunit.client.1.smithi196.stdout:6/287: chown d6/d2a/dc 442 1 2024-03-13T14:49:30.586 INFO:tasks.workunit.client.1.smithi196.stdout:3/156: symlink db/d19/d22/d30/l33 0 2024-03-13T14:49:30.586 INFO:tasks.workunit.client.1.smithi196.stdout:6/288: chown d6/d2a/dc/d1c/d34/l3d 36 1 2024-03-13T14:49:30.587 INFO:tasks.workunit.client.1.smithi196.stdout:0/276: dread d8/da/f32 [0,4194304] 0 2024-03-13T14:49:30.598 INFO:tasks.workunit.client.1.smithi196.stdout:7/267: mkdir d0/d9/d31/d50 0 2024-03-13T14:49:30.603 INFO:tasks.workunit.client.1.smithi196.stdout:8/148: write d0/d10/f15 [61305,9520] 0 2024-03-13T14:49:30.608 INFO:tasks.workunit.client.1.smithi196.stdout:5/289: truncate d4/db/d14/d19/d1c/d25/d38/f56 2952555 0 2024-03-13T14:49:30.610 INFO:tasks.workunit.client.1.smithi196.stdout:5/290: fdatasync d4/db/d14/d19/d1c/f3f 0 2024-03-13T14:49:30.611 INFO:tasks.workunit.client.1.smithi196.stdout:6/289: rmdir d6/d2a/dc/d1c/d34 39 2024-03-13T14:49:30.612 INFO:tasks.workunit.client.1.smithi196.stdout:6/290: stat d6/d9/d17/d19/f2d 0 2024-03-13T14:49:30.614 INFO:tasks.workunit.client.1.smithi196.stdout:6/291: dread - d6/d7/df/f35 zero size 2024-03-13T14:49:30.614 INFO:tasks.workunit.client.1.smithi196.stdout:7/268: creat d0/d9/d1e/f51 x:0 0 0 2024-03-13T14:49:30.615 INFO:tasks.workunit.client.1.smithi196.stdout:7/269: stat d0/d9/f20 0 2024-03-13T14:49:30.616 INFO:tasks.workunit.client.1.smithi196.stdout:3/157: symlink db/d19/l34 0 2024-03-13T14:49:30.617 INFO:tasks.workunit.client.1.smithi196.stdout:7/270: write d0/f4f [544167,34556] 0 2024-03-13T14:49:30.619 INFO:tasks.workunit.client.1.smithi196.stdout:8/149: creat d0/d10/d12/f33 x:0 0 0 2024-03-13T14:49:30.620 INFO:tasks.workunit.client.1.smithi196.stdout:1/198: write d1/d29/f1a [911126,65974] 0 2024-03-13T14:49:30.621 INFO:tasks.workunit.client.1.smithi196.stdout:8/150: write d0/d10/f21 [2364666,71035] 0 2024-03-13T14:49:30.626 INFO:tasks.workunit.client.1.smithi196.stdout:6/292: unlink d6/d9/l36 0 2024-03-13T14:49:30.628 INFO:tasks.workunit.client.1.smithi196.stdout:6/293: write d6/d7/df/f3a [931470,74731] 0 2024-03-13T14:49:30.629 INFO:tasks.workunit.client.1.smithi196.stdout:3/158: creat db/f35 x:0 0 0 2024-03-13T14:49:30.631 INFO:tasks.workunit.client.1.smithi196.stdout:6/294: write d6/d4b/f58 [358493,64775] 0 2024-03-13T14:49:30.641 INFO:tasks.workunit.client.1.smithi196.stdout:3/159: creat db/f36 x:0 0 0 2024-03-13T14:49:30.642 INFO:tasks.workunit.client.1.smithi196.stdout:0/277: write d8/f1e [2871453,124152] 0 2024-03-13T14:49:30.644 INFO:tasks.workunit.client.1.smithi196.stdout:7/271: rename d0/c4 to d0/d6/d35/c52 0 2024-03-13T14:49:30.645 INFO:tasks.workunit.client.1.smithi196.stdout:7/272: chown d0/d9/d1e/d3f 73765987 1 2024-03-13T14:49:30.646 INFO:tasks.workunit.client.1.smithi196.stdout:0/278: fdatasync d8/da/d25/d26/d35/d3b/d50/d5d/f57 0 2024-03-13T14:49:30.646 INFO:tasks.workunit.client.1.smithi196.stdout:6/295: mkdir d6/d7/dd/d60 0 2024-03-13T14:49:30.648 INFO:tasks.workunit.client.0.smithi176.stdout:0/22: creat d2/f6 x:0 0 0 2024-03-13T14:49:30.648 INFO:tasks.workunit.client.1.smithi196.stdout:1/199: mknod d1/d9/c3d 0 2024-03-13T14:49:30.649 INFO:tasks.workunit.client.0.smithi176.stdout:0/23: dread - d2/f6 zero size 2024-03-13T14:49:30.650 INFO:tasks.workunit.client.1.smithi196.stdout:1/200: readlink d1/l6 0 2024-03-13T14:49:30.650 INFO:tasks.workunit.client.0.smithi176.stdout:5/10: creat f2 x:0 0 0 2024-03-13T14:49:30.651 INFO:tasks.workunit.client.1.smithi196.stdout:1/201: stat d1/f31 0 2024-03-13T14:49:30.652 INFO:tasks.workunit.client.0.smithi176.stdout:5/11: chown f2 582 1 2024-03-13T14:49:30.652 INFO:tasks.workunit.client.1.smithi196.stdout:5/291: dread d4/f8 [0,4194304] 0 2024-03-13T14:49:30.652 INFO:tasks.workunit.client.0.smithi176.stdout:0/24: truncate d2/f6 616087 0 2024-03-13T14:49:30.658 INFO:tasks.workunit.client.0.smithi176.stdout:7/8: mkdir d1 0 2024-03-13T14:49:30.659 INFO:tasks.workunit.client.1.smithi196.stdout:3/160: mkdir db/d37 0 2024-03-13T14:49:30.659 INFO:tasks.workunit.client.0.smithi176.stdout:6/12: rename d0/l2 to d0/l5 0 2024-03-13T14:49:30.660 INFO:tasks.workunit.client.1.smithi196.stdout:7/273: mknod d0/d9/d31/d46/d47/c53 0 2024-03-13T14:49:30.664 INFO:tasks.workunit.client.1.smithi196.stdout:0/279: rename d8/da/de to d8/da/d25/d26/d35/d3b/d5f 0 2024-03-13T14:49:30.664 INFO:tasks.workunit.client.1.smithi196.stdout:9/185: sync 2024-03-13T14:49:30.672 INFO:tasks.workunit.client.1.smithi196.stdout:8/151: rmdir d0/d10/d12/d2f 0 2024-03-13T14:49:30.674 INFO:tasks.workunit.client.0.smithi176.stdout:3/49: creat d4/f10 x:0 0 0 2024-03-13T14:49:30.676 INFO:tasks.workunit.client.1.smithi196.stdout:3/161: symlink db/d19/d22/d30/l38 0 2024-03-13T14:49:30.677 INFO:tasks.workunit.client.0.smithi176.stdout:2/33: symlink d1/l10 0 2024-03-13T14:49:30.678 INFO:tasks.workunit.client.0.smithi176.stdout:3/50: write d4/f10 [527780,61919] 0 2024-03-13T14:49:30.678 INFO:tasks.workunit.client.1.smithi196.stdout:8/152: write d0/d10/f19 [668984,2705] 0 2024-03-13T14:49:30.680 INFO:tasks.workunit.client.1.smithi196.stdout:1/202: rename d1/d38/f19 to d1/d28/d34/f3e 0 2024-03-13T14:49:30.681 INFO:tasks.workunit.client.1.smithi196.stdout:7/274: rename d0 to d0/d4d/d54 22 2024-03-13T14:49:30.688 INFO:tasks.workunit.client.1.smithi196.stdout:0/280: rmdir d8/da/d25/d26/d35/d3b/d5f/d17 39 2024-03-13T14:49:30.689 INFO:tasks.workunit.client.1.smithi196.stdout:6/296: dwrite d6/d9/d11/f48 [0,4194304] 0 2024-03-13T14:49:30.692 INFO:tasks.workunit.client.0.smithi176.stdout:8/24: rmdir d0 39 2024-03-13T14:49:30.692 INFO:tasks.workunit.client.0.smithi176.stdout:8/25: readlink - no filename 2024-03-13T14:49:30.707 INFO:tasks.workunit.client.0.smithi176.stdout:5/12: symlink l3 0 2024-03-13T14:49:30.710 INFO:tasks.workunit.client.0.smithi176.stdout:5/13: write f1 [1018758,117162] 0 2024-03-13T14:49:30.710 INFO:tasks.workunit.client.0.smithi176.stdout:5/14: rmdir - no directory 2024-03-13T14:49:30.711 INFO:tasks.workunit.client.0.smithi176.stdout:5/15: rmdir - no directory 2024-03-13T14:49:30.712 INFO:tasks.workunit.client.0.smithi176.stdout:1/13: getdents d2 0 2024-03-13T14:49:30.713 INFO:tasks.workunit.client.1.smithi196.stdout:1/203: dread d1/d29/f1b [0,4194304] 0 2024-03-13T14:49:30.714 INFO:tasks.workunit.client.1.smithi196.stdout:3/162: rename db/d15/f2e to db/d19/d22/f39 0 2024-03-13T14:49:30.715 INFO:tasks.workunit.client.1.smithi196.stdout:1/204: read d1/f31 [3491368,50278] 0 2024-03-13T14:49:30.717 INFO:tasks.workunit.client.0.smithi176.stdout:4/32: creat d7/d8/fd x:0 0 0 2024-03-13T14:49:30.718 INFO:tasks.workunit.client.0.smithi176.stdout:4/33: stat d7/ca 0 2024-03-13T14:49:30.718 INFO:tasks.workunit.client.1.smithi196.stdout:6/297: symlink d6/d7/dd/l61 0 2024-03-13T14:49:30.721 INFO:tasks.workunit.client.0.smithi176.stdout:9/56: creat d0/d13/f16 x:0 0 0 2024-03-13T14:49:30.725 INFO:tasks.workunit.client.1.smithi196.stdout:9/186: dread d7/d9/f11 [0,4194304] 0 2024-03-13T14:49:30.725 INFO:tasks.workunit.client.1.smithi196.stdout:6/298: write d6/d9/d17/d19/f2b [1399903,32791] 0 2024-03-13T14:49:30.727 INFO:tasks.workunit.client.0.smithi176.stdout:2/34: symlink d1/l11 0 2024-03-13T14:49:30.727 INFO:tasks.workunit.client.0.smithi176.stdout:9/57: dread d0/f10 [0,4194304] 0 2024-03-13T14:49:30.734 INFO:tasks.workunit.client.1.smithi196.stdout:7/275: rename d0/d9/d1e/f29 to d0/d6/d35/f55 0 2024-03-13T14:49:30.736 INFO:tasks.workunit.client.1.smithi196.stdout:1/205: fdatasync d1/d29/f1b 0 2024-03-13T14:49:30.739 INFO:tasks.workunit.client.0.smithi176.stdout:5/16: creat f4 x:0 0 0 2024-03-13T14:49:30.740 INFO:tasks.workunit.client.0.smithi176.stdout:6/13: readlink d0/l5 0 2024-03-13T14:49:30.740 INFO:tasks.workunit.client.0.smithi176.stdout:6/14: write - no filename 2024-03-13T14:49:30.740 INFO:tasks.workunit.client.0.smithi176.stdout:6/15: truncate - no filename 2024-03-13T14:49:30.741 INFO:tasks.workunit.client.0.smithi176.stdout:5/17: dread - f2 zero size 2024-03-13T14:49:30.744 INFO:tasks.workunit.client.0.smithi176.stdout:7/9: mkdir d1/d2 0 2024-03-13T14:49:30.746 INFO:tasks.workunit.client.0.smithi176.stdout:4/34: unlink f5 0 2024-03-13T14:49:30.748 INFO:tasks.workunit.client.0.smithi176.stdout:4/35: chown d7/d8 2997301 1 2024-03-13T14:49:30.749 INFO:tasks.workunit.client.0.smithi176.stdout:4/36: stat d7 0 2024-03-13T14:49:30.751 INFO:tasks.workunit.client.0.smithi176.stdout:2/35: creat d1/dc/f12 x:0 0 0 2024-03-13T14:49:30.753 INFO:tasks.workunit.client.0.smithi176.stdout:9/58: readlink d0/la 0 2024-03-13T14:49:30.755 INFO:tasks.workunit.client.1.smithi196.stdout:8/153: rename d0/d10/f15 to d0/d10/d12/d1e/f34 0 2024-03-13T14:49:30.759 INFO:tasks.workunit.client.0.smithi176.stdout:8/26: mknod d0/c4 0 2024-03-13T14:49:30.760 INFO:tasks.workunit.client.0.smithi176.stdout:0/25: truncate d2/f3 2537747 0 2024-03-13T14:49:30.763 INFO:tasks.workunit.client.0.smithi176.stdout:1/14: creat d2/f3 x:0 0 0 2024-03-13T14:49:30.763 INFO:tasks.workunit.client.0.smithi176.stdout:8/27: write d0/f2 [3720201,37819] 0 2024-03-13T14:49:30.764 INFO:tasks.workunit.client.0.smithi176.stdout:6/16: symlink d0/l6 0 2024-03-13T14:49:30.768 INFO:tasks.workunit.client.0.smithi176.stdout:8/28: write d0/f3 [1008504,21767] 0 2024-03-13T14:49:30.771 INFO:tasks.workunit.client.0.smithi176.stdout:8/29: write d0/f2 [3163972,65923] 0 2024-03-13T14:49:30.772 INFO:tasks.workunit.client.1.smithi196.stdout:8/154: dwrite d0/d10/f19 [0,4194304] 0 2024-03-13T14:49:30.773 INFO:tasks.workunit.client.0.smithi176.stdout:5/18: dwrite f4 [0,4194304] 0 2024-03-13T14:49:30.775 INFO:tasks.workunit.client.0.smithi176.stdout:5/19: truncate f2 979271 0 2024-03-13T14:49:30.777 INFO:tasks.workunit.client.0.smithi176.stdout:5/20: write f4 [1982493,115963] 0 2024-03-13T14:49:30.780 INFO:tasks.workunit.client.0.smithi176.stdout:1/15: dread f1 [0,4194304] 0 2024-03-13T14:49:30.782 INFO:tasks.workunit.client.0.smithi176.stdout:4/37: rename f1 to d7/fe 0 2024-03-13T14:49:30.782 INFO:tasks.workunit.client.0.smithi176.stdout:1/16: dread - d2/f3 zero size 2024-03-13T14:49:30.782 INFO:tasks.workunit.client.1.smithi196.stdout:7/276: write d0/f11 [1185342,9844] 0 2024-03-13T14:49:30.784 INFO:tasks.workunit.client.0.smithi176.stdout:2/36: creat d1/f13 x:0 0 0 2024-03-13T14:49:30.786 INFO:tasks.workunit.client.0.smithi176.stdout:6/17: rename d0 to d0/d7 22 2024-03-13T14:49:30.786 INFO:tasks.workunit.client.1.smithi196.stdout:0/281: rename d8/da/d25/d26/d35/f3d to d8/da/d25/d26/f60 0 2024-03-13T14:49:30.787 INFO:tasks.workunit.client.0.smithi176.stdout:2/37: write d1/f4 [5724,51180] 0 2024-03-13T14:49:30.787 INFO:tasks.workunit.client.0.smithi176.stdout:6/18: stat d0 0 2024-03-13T14:49:30.787 INFO:tasks.workunit.client.0.smithi176.stdout:6/19: dread - no filename 2024-03-13T14:49:30.793 INFO:tasks.workunit.client.1.smithi196.stdout:1/206: mkdir d1/d3f 0 2024-03-13T14:49:30.803 INFO:tasks.workunit.client.1.smithi196.stdout:3/163: dwrite db/ff [0,4194304] 0 2024-03-13T14:49:30.803 INFO:tasks.workunit.client.1.smithi196.stdout:6/299: creat d6/d2a/f62 x:0 0 0 2024-03-13T14:49:30.807 INFO:tasks.workunit.client.1.smithi196.stdout:3/164: truncate db/d19/f2b 1098103 0 2024-03-13T14:49:30.808 INFO:tasks.workunit.client.0.smithi176.stdout:9/59: dwrite d0/f10 [0,4194304] 0 2024-03-13T14:49:30.809 INFO:tasks.workunit.client.0.smithi176.stdout:9/60: truncate d0/f10 4905126 0 2024-03-13T14:49:30.810 INFO:tasks.workunit.client.1.smithi196.stdout:3/165: write db/d19/f28 [113617,85563] 0 2024-03-13T14:49:30.811 INFO:tasks.workunit.client.0.smithi176.stdout:9/61: write d0/d13/f16 [600886,1670] 0 2024-03-13T14:49:30.812 INFO:tasks.workunit.client.0.smithi176.stdout:9/62: dread - d0/f12 zero size 2024-03-13T14:49:30.816 INFO:tasks.workunit.client.0.smithi176.stdout:7/10: symlink d1/d2/l3 0 2024-03-13T14:49:30.819 INFO:tasks.workunit.client.0.smithi176.stdout:8/30: mkdir d0/d5 0 2024-03-13T14:49:30.821 INFO:tasks.workunit.client.0.smithi176.stdout:8/31: write d0/f2 [4707511,30292] 0 2024-03-13T14:49:30.825 INFO:tasks.workunit.client.1.smithi196.stdout:9/187: link d7/de/l30 d7/d9/df/d1d/l32 0 2024-03-13T14:49:30.826 INFO:tasks.workunit.client.0.smithi176.stdout:5/21: mkdir d5 0 2024-03-13T14:49:30.826 INFO:tasks.workunit.client.0.smithi176.stdout:5/22: fsync f1 0 2024-03-13T14:49:30.832 INFO:tasks.workunit.client.0.smithi176.stdout:3/51: getdents d4 0 2024-03-13T14:49:30.832 INFO:tasks.workunit.client.0.smithi176.stdout:1/17: creat d2/f4 x:0 0 0 2024-03-13T14:49:30.833 INFO:tasks.workunit.client.0.smithi176.stdout:2/38: creat d1/f14 x:0 0 0 2024-03-13T14:49:30.834 INFO:tasks.workunit.client.0.smithi176.stdout:2/39: readlink d1/l11 0 2024-03-13T14:49:30.834 INFO:tasks.workunit.client.0.smithi176.stdout:6/20: creat d0/f8 x:0 0 0 2024-03-13T14:49:30.835 INFO:tasks.workunit.client.1.smithi196.stdout:0/282: mknod d8/c61 0 2024-03-13T14:49:30.835 INFO:tasks.workunit.client.0.smithi176.stdout:2/40: truncate d1/f14 413580 0 2024-03-13T14:49:30.835 INFO:tasks.workunit.client.0.smithi176.stdout:2/41: fdatasync d1/f8 0 2024-03-13T14:49:30.840 INFO:tasks.workunit.client.1.smithi196.stdout:0/283: write d8/da/d25/d26/d35/d3b/d50/d5d/f1f [4947891,17659] 0 2024-03-13T14:49:30.841 INFO:tasks.workunit.client.1.smithi196.stdout:2/119: sync 2024-03-13T14:49:30.841 INFO:tasks.workunit.client.1.smithi196.stdout:4/183: sync 2024-03-13T14:49:30.849 INFO:tasks.workunit.client.1.smithi196.stdout:7/277: creat d0/f56 x:0 0 0 2024-03-13T14:49:30.851 INFO:tasks.workunit.client.0.smithi176.stdout:9/63: mkdir d0/d17 0 2024-03-13T14:49:30.852 INFO:tasks.workunit.client.1.smithi196.stdout:1/207: write d1/d28/f35 [508606,47505] 0 2024-03-13T14:49:30.853 INFO:tasks.workunit.client.0.smithi176.stdout:9/64: write d0/d13/f16 [697637,129412] 0 2024-03-13T14:49:30.853 INFO:tasks.workunit.client.0.smithi176.stdout:7/11: unlink d1/d2/l3 0 2024-03-13T14:49:30.853 INFO:tasks.workunit.client.0.smithi176.stdout:7/12: dread - no filename 2024-03-13T14:49:30.856 INFO:tasks.workunit.client.1.smithi196.stdout:3/166: dread db/d19/f26 [0,4194304] 0 2024-03-13T14:49:30.858 INFO:tasks.workunit.client.1.smithi196.stdout:9/188: mknod d7/c33 0 2024-03-13T14:49:30.859 INFO:tasks.workunit.client.1.smithi196.stdout:6/300: dwrite d6/d4b/d50/f5a [0,4194304] 0 2024-03-13T14:49:30.860 INFO:tasks.workunit.client.1.smithi196.stdout:4/184: mknod d1/d3/c31 0 2024-03-13T14:49:30.863 INFO:tasks.workunit.client.1.smithi196.stdout:0/284: rmdir d8/da/d25/d26/d35/d3b/d50/d5d 39 2024-03-13T14:49:30.863 INFO:tasks.workunit.client.1.smithi196.stdout:8/155: link d0/d10/l22 d0/l35 0 2024-03-13T14:49:30.865 INFO:tasks.workunit.client.1.smithi196.stdout:7/278: mkdir d0/d6/d33/d57 0 2024-03-13T14:49:30.868 INFO:tasks.workunit.client.0.smithi176.stdout:6/21: mkdir d0/d9 0 2024-03-13T14:49:30.868 INFO:tasks.workunit.client.0.smithi176.stdout:6/22: readlink d0/l6 0 2024-03-13T14:49:30.871 INFO:tasks.workunit.client.1.smithi196.stdout:1/208: write d1/d28/d34/f3e [4850255,94749] 0 2024-03-13T14:49:30.872 INFO:tasks.workunit.client.1.smithi196.stdout:3/167: mkdir db/d19/d22/d3a 0 2024-03-13T14:49:30.879 INFO:tasks.workunit.client.1.smithi196.stdout:6/301: rename d6/f54 to d6/d57/f63 0 2024-03-13T14:49:30.882 INFO:tasks.workunit.client.0.smithi176.stdout:7/13: symlink d1/d2/l4 0 2024-03-13T14:49:30.883 INFO:tasks.workunit.client.0.smithi176.stdout:7/14: chown c0 169 1 2024-03-13T14:49:30.883 INFO:tasks.workunit.client.0.smithi176.stdout:7/15: fsync - no filename 2024-03-13T14:49:30.883 INFO:tasks.workunit.client.0.smithi176.stdout:7/16: dread - no filename 2024-03-13T14:49:30.883 INFO:tasks.workunit.client.1.smithi196.stdout:0/285: creat d8/d37/f62 x:0 0 0 2024-03-13T14:49:30.884 INFO:tasks.workunit.client.0.smithi176.stdout:8/32: symlink d0/d5/l6 0 2024-03-13T14:49:30.886 INFO:tasks.workunit.client.1.smithi196.stdout:7/279: stat d0/l2a 0 2024-03-13T14:49:30.886 INFO:tasks.workunit.client.0.smithi176.stdout:5/23: creat d5/f6 x:0 0 0 2024-03-13T14:49:30.887 INFO:tasks.workunit.client.1.smithi196.stdout:2/120: creat d0/f1a x:0 0 0 2024-03-13T14:49:30.889 INFO:tasks.workunit.client.1.smithi196.stdout:3/168: symlink db/l3b 0 2024-03-13T14:49:30.891 INFO:tasks.workunit.client.1.smithi196.stdout:1/209: rename d1/d38/f17 to d1/d38/f40 0 2024-03-13T14:49:30.893 INFO:tasks.workunit.client.0.smithi176.stdout:1/18: rename d2/f4 to d2/f5 0 2024-03-13T14:49:30.894 INFO:tasks.workunit.client.0.smithi176.stdout:6/23: rename d0/l6 to d0/la 0 2024-03-13T14:49:30.895 INFO:tasks.workunit.client.0.smithi176.stdout:5/24: dread f1 [0,4194304] 0 2024-03-13T14:49:30.896 INFO:tasks.workunit.client.0.smithi176.stdout:0/26: truncate d2/f3 2057285 0 2024-03-13T14:49:30.897 INFO:tasks.workunit.client.0.smithi176.stdout:0/27: chown d2 981652748 1 2024-03-13T14:49:30.899 INFO:tasks.workunit.client.1.smithi196.stdout:9/189: rename d7/d9/df/d1d/l32 to d7/d9/df/d1d/d22/l34 0 2024-03-13T14:49:30.899 INFO:tasks.workunit.client.0.smithi176.stdout:7/17: mknod d1/d2/c5 0 2024-03-13T14:49:30.899 INFO:tasks.workunit.client.0.smithi176.stdout:7/18: dread - no filename 2024-03-13T14:49:30.899 INFO:tasks.workunit.client.0.smithi176.stdout:7/19: truncate - no filename 2024-03-13T14:49:30.899 INFO:tasks.workunit.client.0.smithi176.stdout:7/20: truncate - no filename 2024-03-13T14:49:30.900 INFO:tasks.workunit.client.1.smithi196.stdout:8/156: rename d0 to d0/d1c/d36 22 2024-03-13T14:49:30.900 INFO:tasks.workunit.client.0.smithi176.stdout:0/28: write d2/f6 [695643,34158] 0 2024-03-13T14:49:30.901 INFO:tasks.workunit.client.1.smithi196.stdout:4/185: creat d1/d2f/f32 x:0 0 0 2024-03-13T14:49:30.903 INFO:tasks.workunit.client.1.smithi196.stdout:5/292: sync 2024-03-13T14:49:30.904 INFO:tasks.workunit.client.1.smithi196.stdout:6/302: creat d6/d7/df/d27/f64 x:0 0 0 2024-03-13T14:49:30.904 INFO:tasks.workunit.client.1.smithi196.stdout:5/293: chown d4/db/d3c/f59 86092 1 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:8/33: mkdir d0/d5/d7 0 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:7/21: rename d1/d2 to d1/d2/d6 22 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:7/22: dread - no filename 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:7/23: write - no filename 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:7/24: write - no filename 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:7/25: dwrite - no filename 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:7/26: fsync - no filename 2024-03-13T14:49:30.905 INFO:tasks.workunit.client.0.smithi176.stdout:7/27: dwrite - no filename 2024-03-13T14:49:30.906 INFO:tasks.workunit.client.0.smithi176.stdout:4/38: link c6 d7/cf 0 2024-03-13T14:49:30.906 INFO:tasks.workunit.client.1.smithi196.stdout:5/294: dread - d4/db/d14/d37/f4d zero size 2024-03-13T14:49:30.908 INFO:tasks.workunit.client.0.smithi176.stdout:8/34: readlink d0/d5/l6 0 2024-03-13T14:49:30.908 INFO:tasks.workunit.client.1.smithi196.stdout:5/295: write d4/f8 [4632477,31928] 0 2024-03-13T14:49:30.909 INFO:tasks.workunit.client.1.smithi196.stdout:7/280: mkdir d0/d9/d1e/d58 0 2024-03-13T14:49:30.910 INFO:tasks.workunit.client.0.smithi176.stdout:1/19: creat d2/f6 x:0 0 0 2024-03-13T14:49:30.911 INFO:tasks.workunit.client.0.smithi176.stdout:5/25: creat d5/f7 x:0 0 0 2024-03-13T14:49:30.915 INFO:tasks.workunit.client.0.smithi176.stdout:9/65: dwrite d0/f10 [4194304,4194304] 0 2024-03-13T14:49:30.915 INFO:tasks.workunit.client.0.smithi176.stdout:4/39: mkdir d7/d10 0 2024-03-13T14:49:30.916 INFO:tasks.workunit.client.1.smithi196.stdout:1/210: creat d1/d28/d34/f41 x:0 0 0 2024-03-13T14:49:30.918 INFO:tasks.workunit.client.0.smithi176.stdout:7/28: unlink d1/d2/c5 0 2024-03-13T14:49:30.920 INFO:tasks.workunit.client.1.smithi196.stdout:8/157: symlink d0/d10/d12/d17/l37 0 2024-03-13T14:49:30.923 INFO:tasks.workunit.client.0.smithi176.stdout:7/29: chown d1/d2/l4 418 1 2024-03-13T14:49:30.923 INFO:tasks.workunit.client.0.smithi176.stdout:7/30: dread - no filename 2024-03-13T14:49:30.923 INFO:tasks.workunit.client.0.smithi176.stdout:7/31: dwrite - no filename 2024-03-13T14:49:30.923 INFO:tasks.workunit.client.0.smithi176.stdout:7/32: write - no filename 2024-03-13T14:49:30.924 INFO:tasks.workunit.client.1.smithi196.stdout:8/158: read d0/d10/d12/d17/f28 [1027465,76599] 0 2024-03-13T14:49:30.927 INFO:tasks.workunit.client.1.smithi196.stdout:0/286: mknod d8/da/d25/d26/d35/d3b/d50/c63 0 2024-03-13T14:49:30.928 INFO:tasks.workunit.client.0.smithi176.stdout:1/20: mknod d2/c7 0 2024-03-13T14:49:30.931 INFO:tasks.workunit.client.0.smithi176.stdout:4/40: fsync d7/fe 0 2024-03-13T14:49:30.932 INFO:tasks.workunit.client.0.smithi176.stdout:5/26: rename f2 to d5/f8 0 2024-03-13T14:49:30.935 INFO:tasks.workunit.client.1.smithi196.stdout:6/303: dwrite d6/d4b/d50/f5a [0,4194304] 0 2024-03-13T14:49:30.937 INFO:tasks.workunit.client.1.smithi196.stdout:7/281: unlink d0/d6/cd 0 2024-03-13T14:49:30.937 INFO:tasks.workunit.client.1.smithi196.stdout:2/121: mknod d0/c1b 0 2024-03-13T14:49:30.937 INFO:tasks.workunit.client.0.smithi176.stdout:7/33: mkdir d1/d7 0 2024-03-13T14:49:30.938 INFO:tasks.workunit.client.0.smithi176.stdout:6/24: dwrite d0/f8 [0,4194304] 0 2024-03-13T14:49:30.938 INFO:tasks.workunit.client.1.smithi196.stdout:0/287: fsync d8/da/d25/d26/d35/f4f 0 2024-03-13T14:49:30.939 INFO:tasks.workunit.client.0.smithi176.stdout:4/41: dread - d7/d8/fd zero size 2024-03-13T14:49:30.939 INFO:tasks.workunit.client.0.smithi176.stdout:1/21: write d2/f3 [922525,25649] 0 2024-03-13T14:49:30.940 INFO:tasks.workunit.client.0.smithi176.stdout:5/27: chown d5/f7 409272632 1 2024-03-13T14:49:30.941 INFO:tasks.workunit.client.1.smithi196.stdout:7/282: chown d0/d9/d1e/d2d 486797655 1 2024-03-13T14:49:30.941 INFO:tasks.workunit.client.1.smithi196.stdout:0/288: chown d8 27 1 2024-03-13T14:49:30.944 INFO:tasks.workunit.client.1.smithi196.stdout:5/296: dwrite d4/db/d14/d37/d41/f42 [0,4194304] 0 2024-03-13T14:49:30.947 INFO:tasks.workunit.client.1.smithi196.stdout:4/186: dread d1/d3/d7/f1d [4194304,4194304] 0 2024-03-13T14:49:30.949 INFO:tasks.workunit.client.1.smithi196.stdout:6/304: truncate d6/d7/df/f35 587278 0 2024-03-13T14:49:30.949 INFO:tasks.workunit.client.0.smithi176.stdout:9/66: dread d0/f10 [4194304,4194304] 0 2024-03-13T14:49:30.950 INFO:tasks.workunit.client.1.smithi196.stdout:1/211: rename d1/d38/f2e to d1/d9/d33/f42 0 2024-03-13T14:49:30.950 INFO:tasks.workunit.client.0.smithi176.stdout:8/35: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:49:30.952 INFO:tasks.workunit.client.0.smithi176.stdout:6/25: rename d0/c3 to d0/cb 0 2024-03-13T14:49:30.955 INFO:tasks.workunit.client.1.smithi196.stdout:4/187: fdatasync d1/d2f/f32 0 2024-03-13T14:49:30.957 INFO:tasks.workunit.client.1.smithi196.stdout:7/283: write d0/d6/f2c [36903,16372] 0 2024-03-13T14:49:30.961 INFO:tasks.workunit.client.1.smithi196.stdout:3/169: getdents db/d15 0 2024-03-13T14:49:30.965 INFO:tasks.workunit.client.0.smithi176.stdout:7/34: link c0 d1/d2/c8 0 2024-03-13T14:49:30.965 INFO:tasks.workunit.client.0.smithi176.stdout:9/67: symlink d0/d17/l18 0 2024-03-13T14:49:30.966 INFO:tasks.workunit.client.1.smithi196.stdout:3/170: chown f7 8272 1 2024-03-13T14:49:30.966 INFO:tasks.workunit.client.0.smithi176.stdout:9/68: readlink d0/db/l7 0 2024-03-13T14:49:30.969 INFO:tasks.workunit.client.1.smithi196.stdout:4/188: creat d1/d1e/f33 x:0 0 0 2024-03-13T14:49:30.970 INFO:tasks.workunit.client.0.smithi176.stdout:4/42: dwrite d7/fe [0,4194304] 0 2024-03-13T14:49:30.970 INFO:tasks.workunit.client.1.smithi196.stdout:3/171: write db/f13 [870749,57664] 0 2024-03-13T14:49:30.971 INFO:tasks.workunit.client.1.smithi196.stdout:8/159: rename d0/f8 to d0/d10/d14/f38 0 2024-03-13T14:49:30.971 INFO:tasks.workunit.client.0.smithi176.stdout:8/36: link d0/c4 d0/d5/d7/c8 0 2024-03-13T14:49:30.976 INFO:tasks.workunit.client.1.smithi196.stdout:9/190: getdents d7/d9 0 2024-03-13T14:49:30.977 INFO:tasks.workunit.client.0.smithi176.stdout:9/69: creat d0/d17/f19 x:0 0 0 2024-03-13T14:49:30.978 INFO:tasks.workunit.client.0.smithi176.stdout:7/35: mknod d1/d7/c9 0 2024-03-13T14:49:30.978 INFO:tasks.workunit.client.0.smithi176.stdout:7/36: dread - no filename 2024-03-13T14:49:30.978 INFO:tasks.workunit.client.1.smithi196.stdout:1/212: creat d1/d3f/f43 x:0 0 0 2024-03-13T14:49:30.979 INFO:tasks.workunit.client.0.smithi176.stdout:4/43: creat d7/d8/f11 x:0 0 0 2024-03-13T14:49:30.980 INFO:tasks.workunit.client.1.smithi196.stdout:0/289: creat d8/da/d25/d26/d35/d3b/d50/d55/f64 x:0 0 0 2024-03-13T14:49:30.981 INFO:tasks.workunit.client.1.smithi196.stdout:2/122: link d0/d6/cf d0/d7/d9/c1c 0 2024-03-13T14:49:30.983 INFO:tasks.workunit.client.1.smithi196.stdout:9/191: write d7/fd [4267808,69801] 0 2024-03-13T14:49:30.984 INFO:tasks.workunit.client.1.smithi196.stdout:8/160: read d0/d10/f26 [185026,93554] 0 2024-03-13T14:49:30.984 INFO:tasks.workunit.client.1.smithi196.stdout:3/172: creat db/d19/d22/f3c x:0 0 0 2024-03-13T14:49:30.987 INFO:tasks.workunit.client.0.smithi176.stdout:8/37: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:49:30.988 INFO:tasks.workunit.client.1.smithi196.stdout:5/297: dread d4/db/d14/d19/d26/f43 [0,4194304] 0 2024-03-13T14:49:30.988 INFO:tasks.workunit.client.0.smithi176.stdout:9/70: symlink d0/d17/l1a 0 2024-03-13T14:49:30.989 INFO:tasks.workunit.client.0.smithi176.stdout:9/71: dread - d0/f12 zero size 2024-03-13T14:49:30.990 INFO:tasks.workunit.client.1.smithi196.stdout:0/290: write d8/f1e [1506912,47380] 0 2024-03-13T14:49:30.990 INFO:tasks.workunit.client.1.smithi196.stdout:6/305: dwrite d6/f8 [0,4194304] 0 2024-03-13T14:49:30.990 INFO:tasks.workunit.client.0.smithi176.stdout:9/72: dread - d0/f12 zero size 2024-03-13T14:49:30.992 INFO:tasks.workunit.client.0.smithi176.stdout:9/73: write d0/d13/f16 [533313,97043] 0 2024-03-13T14:49:30.993 INFO:tasks.workunit.client.1.smithi196.stdout:2/123: write d0/d7/f17 [738126,102568] 0 2024-03-13T14:49:31.002 INFO:tasks.workunit.client.1.smithi196.stdout:6/306: fsync d6/d2a/d23/d28/f3f 0 2024-03-13T14:49:31.004 INFO:tasks.workunit.client.1.smithi196.stdout:7/284: dread d0/d9/d31/d46/d47/f4e [0,4194304] 0 2024-03-13T14:49:31.004 INFO:tasks.workunit.client.0.smithi176.stdout:9/74: mkdir d0/d13/d1b 0 2024-03-13T14:49:31.005 INFO:tasks.workunit.client.0.smithi176.stdout:4/44: mknod d7/d10/c12 0 2024-03-13T14:49:31.006 INFO:tasks.workunit.client.0.smithi176.stdout:9/75: write d0/d17/f19 [292296,101783] 0 2024-03-13T14:49:31.007 INFO:tasks.workunit.client.0.smithi176.stdout:6/26: readlink d0/la 0 2024-03-13T14:49:31.008 INFO:tasks.workunit.client.0.smithi176.stdout:9/76: fdatasync d0/f12 0 2024-03-13T14:49:31.012 INFO:tasks.workunit.client.1.smithi196.stdout:4/189: link d1/d3/d7/f1a d1/d1e/f34 0 2024-03-13T14:49:31.015 INFO:tasks.workunit.client.0.smithi176.stdout:0/29: dread d2/f3 [0,4194304] 0 2024-03-13T14:49:31.017 INFO:tasks.workunit.client.0.smithi176.stdout:8/38: unlink d0/d5/d7/c8 0 2024-03-13T14:49:31.018 INFO:tasks.workunit.client.0.smithi176.stdout:8/39: chown d0/d5 117999648 1 2024-03-13T14:49:31.021 INFO:tasks.workunit.client.1.smithi196.stdout:0/291: mknod d8/da/d25/d26/d35/d3b/d50/d55/c65 0 2024-03-13T14:49:31.022 INFO:tasks.workunit.client.0.smithi176.stdout:4/45: symlink d7/d8/l13 0 2024-03-13T14:49:31.026 INFO:tasks.workunit.client.0.smithi176.stdout:6/27: unlink d0/l5 0 2024-03-13T14:49:31.034 INFO:tasks.workunit.client.1.smithi196.stdout:9/192: dwrite d7/f15 [0,4194304] 0 2024-03-13T14:49:31.034 INFO:tasks.workunit.client.0.smithi176.stdout:9/77: rename d0/db/c14 to d0/db/c1c 0 2024-03-13T14:49:31.035 INFO:tasks.workunit.client.0.smithi176.stdout:9/78: fdatasync d0/d13/f16 0 2024-03-13T14:49:31.037 INFO:tasks.workunit.client.0.smithi176.stdout:5/28: getdents d5 0 2024-03-13T14:49:31.037 INFO:tasks.workunit.client.0.smithi176.stdout:9/79: write d0/f12 [331645,80152] 0 2024-03-13T14:49:31.041 INFO:tasks.workunit.client.1.smithi196.stdout:7/285: rmdir d0/d9/d31 39 2024-03-13T14:49:31.043 INFO:tasks.workunit.client.0.smithi176.stdout:0/30: rmdir d2 39 2024-03-13T14:49:31.046 INFO:tasks.workunit.client.1.smithi196.stdout:3/173: dwrite db/f2c [0,4194304] 0 2024-03-13T14:49:31.049 INFO:tasks.workunit.client.1.smithi196.stdout:6/307: unlink d6/d4b/d50/f51 0 2024-03-13T14:49:31.049 INFO:tasks.workunit.client.0.smithi176.stdout:7/37: getdents d1 0 2024-03-13T14:49:31.049 INFO:tasks.workunit.client.0.smithi176.stdout:7/38: dread - no filename 2024-03-13T14:49:31.050 INFO:tasks.workunit.client.0.smithi176.stdout:7/39: chown d1/d7 112 1 2024-03-13T14:49:31.050 INFO:tasks.workunit.client.0.smithi176.stdout:7/40: fsync - no filename 2024-03-13T14:49:31.057 INFO:tasks.workunit.client.0.smithi176.stdout:8/40: rmdir d0/d5 39 2024-03-13T14:49:31.062 INFO:tasks.workunit.client.0.smithi176.stdout:8/41: truncate d0/f2 5266966 0 2024-03-13T14:49:31.066 INFO:tasks.workunit.client.0.smithi176.stdout:5/29: dwrite d5/f6 [0,4194304] 0 2024-03-13T14:49:31.067 INFO:tasks.workunit.client.1.smithi196.stdout:1/213: creat d1/d29/f44 x:0 0 0 2024-03-13T14:49:31.067 INFO:tasks.workunit.client.1.smithi196.stdout:8/161: truncate d0/d10/d12/d17/f28 3845012 0 2024-03-13T14:49:31.067 INFO:tasks.workunit.client.1.smithi196.stdout:8/162: fsync d0/f27 0 2024-03-13T14:49:31.068 INFO:tasks.workunit.client.0.smithi176.stdout:4/46: symlink d7/d10/l14 0 2024-03-13T14:49:31.069 INFO:tasks.workunit.client.0.smithi176.stdout:4/47: dread - d7/d8/fd zero size 2024-03-13T14:49:31.072 INFO:tasks.workunit.client.0.smithi176.stdout:6/28: readlink d0/la 0 2024-03-13T14:49:31.072 INFO:tasks.workunit.client.0.smithi176.stdout:6/29: fsync d0/f8 0 2024-03-13T14:49:31.074 INFO:tasks.workunit.client.1.smithi196.stdout:0/292: mknod d8/d37/c66 0 2024-03-13T14:49:31.077 INFO:tasks.workunit.client.0.smithi176.stdout:8/42: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:31.078 INFO:tasks.workunit.client.1.smithi196.stdout:6/308: dwrite d6/f8 [0,4194304] 0 2024-03-13T14:49:31.082 INFO:tasks.workunit.client.0.smithi176.stdout:0/31: write d2/f6 [1103470,49930] 0 2024-03-13T14:49:31.084 INFO:tasks.workunit.client.1.smithi196.stdout:6/309: write d6/d2a/d23/d28/f2f [244951,108366] 0 2024-03-13T14:49:31.085 INFO:tasks.workunit.client.0.smithi176.stdout:6/30: dread d0/f8 [0,4194304] 0 2024-03-13T14:49:31.085 INFO:tasks.workunit.client.0.smithi176.stdout:6/31: stat d0/la 0 2024-03-13T14:49:31.087 INFO:tasks.workunit.client.0.smithi176.stdout:6/32: truncate d0/f8 4879267 0 2024-03-13T14:49:31.091 INFO:tasks.workunit.client.0.smithi176.stdout:7/41: symlink d1/d2/la 0 2024-03-13T14:49:31.093 INFO:tasks.workunit.client.1.smithi196.stdout:4/190: rmdir d1/d3/d7 39 2024-03-13T14:49:31.101 INFO:tasks.workunit.client.1.smithi196.stdout:3/174: creat db/f3d x:0 0 0 2024-03-13T14:49:31.102 INFO:tasks.workunit.client.0.smithi176.stdout:0/32: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:31.104 INFO:tasks.workunit.client.0.smithi176.stdout:5/30: rename l3 to d5/l9 0 2024-03-13T14:49:31.104 INFO:tasks.workunit.client.1.smithi196.stdout:1/214: dwrite d1/d28/f35 [0,4194304] 0 2024-03-13T14:49:31.105 INFO:tasks.workunit.client.0.smithi176.stdout:5/31: chown d5/f8 3 1 2024-03-13T14:49:31.109 INFO:tasks.workunit.client.0.smithi176.stdout:4/48: mkdir d7/d8/d15 0 2024-03-13T14:49:31.114 INFO:tasks.workunit.client.1.smithi196.stdout:8/163: symlink d0/d10/d12/l39 0 2024-03-13T14:49:31.116 INFO:tasks.workunit.client.1.smithi196.stdout:8/164: write d0/f27 [991809,75105] 0 2024-03-13T14:49:31.119 INFO:tasks.workunit.client.1.smithi196.stdout:8/165: readlink d0/d1c/l2e 0 2024-03-13T14:49:31.120 INFO:tasks.workunit.client.1.smithi196.stdout:0/293: symlink d8/da/d25/d26/d35/d3b/d5f/d43/l67 0 2024-03-13T14:49:31.122 INFO:tasks.workunit.client.0.smithi176.stdout:9/80: link d0/f12 d0/db/f1d 0 2024-03-13T14:49:31.125 INFO:tasks.workunit.client.1.smithi196.stdout:5/298: link d4/db/d14/d19/c48 d4/db/d14/d37/c5d 0 2024-03-13T14:49:31.126 INFO:tasks.workunit.client.1.smithi196.stdout:5/299: stat d4/f8 0 2024-03-13T14:49:31.128 INFO:tasks.workunit.client.0.smithi176.stdout:4/49: dwrite d7/d8/f11 [0,4194304] 0 2024-03-13T14:49:31.133 INFO:tasks.workunit.client.0.smithi176.stdout:6/33: creat d0/fc x:0 0 0 2024-03-13T14:49:31.133 INFO:tasks.workunit.client.0.smithi176.stdout:7/42: symlink d1/d7/lb 0 2024-03-13T14:49:31.133 INFO:tasks.workunit.client.0.smithi176.stdout:7/43: write - no filename 2024-03-13T14:49:31.139 INFO:tasks.workunit.client.0.smithi176.stdout:0/33: rename l0 to d2/l7 0 2024-03-13T14:49:31.142 INFO:tasks.workunit.client.1.smithi196.stdout:8/166: creat d0/d1c/d29/f3a x:0 0 0 2024-03-13T14:49:31.143 INFO:tasks.workunit.client.0.smithi176.stdout:3/52: sync 2024-03-13T14:49:31.143 INFO:tasks.workunit.client.0.smithi176.stdout:2/42: sync 2024-03-13T14:49:31.143 INFO:tasks.workunit.client.0.smithi176.stdout:1/22: sync 2024-03-13T14:49:31.144 INFO:tasks.workunit.client.0.smithi176.stdout:2/43: chown d1/dc/f12 1 1 2024-03-13T14:49:31.150 INFO:tasks.workunit.client.1.smithi196.stdout:2/124: getdents d0/d6 0 2024-03-13T14:49:31.152 INFO:tasks.workunit.client.1.smithi196.stdout:2/125: fdatasync d0/d6/f8 0 2024-03-13T14:49:31.153 INFO:tasks.workunit.client.1.smithi196.stdout:5/300: stat d4/db/d14/d19/d1c/d25/d38/f56 0 2024-03-13T14:49:31.155 INFO:tasks.workunit.client.0.smithi176.stdout:4/50: creat d7/d8/f16 x:0 0 0 2024-03-13T14:49:31.156 INFO:tasks.workunit.client.0.smithi176.stdout:4/51: readlink l3 0 2024-03-13T14:49:31.156 INFO:tasks.workunit.client.1.smithi196.stdout:4/191: mkdir d1/d3/d7/d35 0 2024-03-13T14:49:31.157 INFO:tasks.workunit.client.1.smithi196.stdout:3/175: write f4 [148508,101376] 0 2024-03-13T14:49:31.158 INFO:tasks.workunit.client.1.smithi196.stdout:4/192: chown d1/d1e/f33 50 1 2024-03-13T14:49:31.159 INFO:tasks.workunit.client.0.smithi176.stdout:6/34: rmdir d0 39 2024-03-13T14:49:31.160 INFO:tasks.workunit.client.0.smithi176.stdout:4/52: truncate d7/d8/fd 261983 0 2024-03-13T14:49:31.162 INFO:tasks.workunit.client.1.smithi196.stdout:7/286: creat d0/d9/f59 x:0 0 0 2024-03-13T14:49:31.163 INFO:tasks.workunit.client.1.smithi196.stdout:4/193: read d1/d3/d7/f1a [3761583,7655] 0 2024-03-13T14:49:31.163 INFO:tasks.workunit.client.0.smithi176.stdout:5/32: rename f1 to d5/fa 0 2024-03-13T14:49:31.164 INFO:tasks.workunit.client.0.smithi176.stdout:2/44: dwrite d1/dc/f12 [0,4194304] 0 2024-03-13T14:49:31.165 INFO:tasks.workunit.client.1.smithi196.stdout:9/193: dwrite d7/de/d16/f2a [0,4194304] 0 2024-03-13T14:49:31.165 INFO:tasks.workunit.client.1.smithi196.stdout:4/194: dread - d1/d1e/f33 zero size 2024-03-13T14:49:31.166 INFO:tasks.workunit.client.1.smithi196.stdout:7/287: write d0/d9/d1e/d2d/f3d [132430,44657] 0 2024-03-13T14:49:31.166 INFO:tasks.workunit.client.0.smithi176.stdout:1/23: mknod d2/c8 0 2024-03-13T14:49:31.167 INFO:tasks.workunit.client.1.smithi196.stdout:4/195: readlink d1/l2d 0 2024-03-13T14:49:31.168 INFO:tasks.workunit.client.0.smithi176.stdout:3/53: mknod d4/c11 0 2024-03-13T14:49:31.170 INFO:tasks.workunit.client.1.smithi196.stdout:8/167: stat d0/d1c/d29/c2c 0 2024-03-13T14:49:31.173 INFO:tasks.workunit.client.1.smithi196.stdout:0/294: rename d8/da/d25/d26/d35/d3b/d50/d5d/c3c to d8/da/d25/d26/d35/d3b/c68 0 2024-03-13T14:49:31.175 INFO:tasks.workunit.client.1.smithi196.stdout:2/126: mkdir d0/d6/d1d 0 2024-03-13T14:49:31.176 INFO:tasks.workunit.client.1.smithi196.stdout:5/301: creat d4/db/d14/f5e x:0 0 0 2024-03-13T14:49:31.180 INFO:tasks.workunit.client.0.smithi176.stdout:4/53: mknod d7/d10/c17 0 2024-03-13T14:49:31.182 INFO:tasks.workunit.client.0.smithi176.stdout:5/33: mknod d5/cb 0 2024-03-13T14:49:31.182 INFO:tasks.workunit.client.0.smithi176.stdout:4/54: chown d7/d8/fd 4645 1 2024-03-13T14:49:31.184 INFO:tasks.workunit.client.0.smithi176.stdout:5/34: truncate d5/fa 1814170 0 2024-03-13T14:49:31.184 INFO:tasks.workunit.client.0.smithi176.stdout:7/44: rename d1/d7/c9 to d1/cc 0 2024-03-13T14:49:31.184 INFO:tasks.workunit.client.0.smithi176.stdout:7/45: write - no filename 2024-03-13T14:49:31.184 INFO:tasks.workunit.client.0.smithi176.stdout:7/46: dread - no filename 2024-03-13T14:49:31.184 INFO:tasks.workunit.client.0.smithi176.stdout:7/47: dread - no filename 2024-03-13T14:49:31.184 INFO:tasks.workunit.client.0.smithi176.stdout:7/48: write - no filename 2024-03-13T14:49:31.186 INFO:tasks.workunit.client.1.smithi196.stdout:9/194: mkdir d7/d9/df/d1d/d35 0 2024-03-13T14:49:31.186 INFO:tasks.workunit.client.0.smithi176.stdout:0/34: creat d2/d5/f8 x:0 0 0 2024-03-13T14:49:31.187 INFO:tasks.workunit.client.1.smithi196.stdout:9/195: stat d7/c1e 0 2024-03-13T14:49:31.188 INFO:tasks.workunit.client.0.smithi176.stdout:2/45: creat d1/f15 x:0 0 0 2024-03-13T14:49:31.190 INFO:tasks.workunit.client.0.smithi176.stdout:3/54: mknod d4/c12 0 2024-03-13T14:49:31.190 INFO:tasks.workunit.client.1.smithi196.stdout:9/196: write d7/de/d16/f2a [1828710,81795] 0 2024-03-13T14:49:31.191 INFO:tasks.workunit.client.0.smithi176.stdout:3/55: readlink d4/ld 0 2024-03-13T14:49:31.191 INFO:tasks.workunit.client.1.smithi196.stdout:8/168: write d0/d10/d12/d1e/f34 [1398507,77083] 0 2024-03-13T14:49:31.192 INFO:tasks.workunit.client.0.smithi176.stdout:8/43: rmdir d0/d5/d7 0 2024-03-13T14:49:31.194 INFO:tasks.workunit.client.0.smithi176.stdout:4/55: dread d7/d8/f11 [0,4194304] 0 2024-03-13T14:49:31.195 INFO:tasks.workunit.client.1.smithi196.stdout:9/197: chown f5 0 1 2024-03-13T14:49:31.196 INFO:tasks.workunit.client.0.smithi176.stdout:4/56: chown c0 187960 1 2024-03-13T14:49:31.196 INFO:tasks.workunit.client.0.smithi176.stdout:9/81: rmdir d0/d13/d1b 0 2024-03-13T14:49:31.197 INFO:tasks.workunit.client.1.smithi196.stdout:7/288: dwrite d0/d9/d1e/d2d/f3b [0,4194304] 0 2024-03-13T14:49:31.198 INFO:tasks.workunit.client.0.smithi176.stdout:0/35: dread d2/f6 [0,4194304] 0 2024-03-13T14:49:31.200 INFO:tasks.workunit.client.1.smithi196.stdout:9/198: write d7/d9/df/f19 [137248,123649] 0 2024-03-13T14:49:31.202 INFO:tasks.workunit.client.1.smithi196.stdout:5/302: creat d4/db/d4a/f5f x:0 0 0 2024-03-13T14:49:31.203 INFO:tasks.workunit.client.0.smithi176.stdout:6/35: creat d0/fd x:0 0 0 2024-03-13T14:49:31.207 INFO:tasks.workunit.client.0.smithi176.stdout:6/36: write d0/fc [835896,56467] 0 2024-03-13T14:49:31.210 INFO:tasks.workunit.client.0.smithi176.stdout:5/35: symlink d5/lc 0 2024-03-13T14:49:31.212 INFO:tasks.workunit.client.1.smithi196.stdout:3/176: truncate db/d19/d22/f23 1873129 0 2024-03-13T14:49:31.213 INFO:tasks.workunit.client.0.smithi176.stdout:3/56: dwrite d4/f10 [0,4194304] 0 2024-03-13T14:49:31.214 INFO:tasks.workunit.client.1.smithi196.stdout:3/177: chown db/f2c 19780 1 2024-03-13T14:49:31.215 INFO:tasks.workunit.client.0.smithi176.stdout:7/49: mknod d1/d7/cd 0 2024-03-13T14:49:31.215 INFO:tasks.workunit.client.0.smithi176.stdout:7/50: write - no filename 2024-03-13T14:49:31.224 INFO:tasks.workunit.client.0.smithi176.stdout:1/24: link f1 d2/f9 0 2024-03-13T14:49:31.225 INFO:tasks.workunit.client.1.smithi196.stdout:8/169: unlink d0/d10/d14/l1d 0 2024-03-13T14:49:31.229 INFO:tasks.workunit.client.1.smithi196.stdout:0/295: truncate d8/da/d25/d26/d35/d3b/d50/f53 1140108 0 2024-03-13T14:49:31.230 INFO:tasks.workunit.client.0.smithi176.stdout:5/36: dwrite d5/f6 [0,4194304] 0 2024-03-13T14:49:31.230 INFO:tasks.workunit.client.0.smithi176.stdout:9/82: mkdir d0/d1e 0 2024-03-13T14:49:31.231 INFO:tasks.workunit.client.0.smithi176.stdout:5/37: fsync d5/f7 0 2024-03-13T14:49:31.233 INFO:tasks.workunit.client.0.smithi176.stdout:9/83: write d0/d17/f19 [210204,8300] 0 2024-03-13T14:49:31.234 INFO:tasks.workunit.client.0.smithi176.stdout:4/57: creat d7/d8/f18 x:0 0 0 2024-03-13T14:49:31.235 INFO:tasks.workunit.client.1.smithi196.stdout:9/199: rmdir d7/d9/df/d1d/d22 39 2024-03-13T14:49:31.236 INFO:tasks.workunit.client.0.smithi176.stdout:4/58: write d7/d8/f18 [782522,125926] 0 2024-03-13T14:49:31.237 INFO:tasks.workunit.client.0.smithi176.stdout:0/36: truncate d2/f3 2268172 0 2024-03-13T14:49:31.243 INFO:tasks.workunit.client.1.smithi196.stdout:3/178: dwrite db/d19/f2f [0,4194304] 0 2024-03-13T14:49:31.245 INFO:tasks.workunit.client.0.smithi176.stdout:6/37: mkdir d0/de 0 2024-03-13T14:49:31.245 INFO:tasks.workunit.client.0.smithi176.stdout:6/38: readlink d0/la 0 2024-03-13T14:49:31.252 INFO:tasks.workunit.client.1.smithi196.stdout:4/196: creat d1/d3/f36 x:0 0 0 2024-03-13T14:49:31.255 INFO:tasks.workunit.client.1.smithi196.stdout:4/197: dread - d1/d3/f36 zero size 2024-03-13T14:49:31.260 INFO:tasks.workunit.client.0.smithi176.stdout:9/84: dread d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:31.261 INFO:tasks.workunit.client.1.smithi196.stdout:8/170: rename d0/d10/d14 to d0/d1c/d3b 0 2024-03-13T14:49:31.262 INFO:tasks.workunit.client.0.smithi176.stdout:7/51: mkdir d1/d2/de 0 2024-03-13T14:49:31.262 INFO:tasks.workunit.client.0.smithi176.stdout:7/52: dread - no filename 2024-03-13T14:49:31.262 INFO:tasks.workunit.client.0.smithi176.stdout:7/53: write - no filename 2024-03-13T14:49:31.263 INFO:tasks.workunit.client.0.smithi176.stdout:9/85: chown d0/db/cd 345748 1 2024-03-13T14:49:31.265 INFO:tasks.workunit.client.0.smithi176.stdout:7/54: stat d1/d2/la 0 2024-03-13T14:49:31.271 INFO:tasks.workunit.client.1.smithi196.stdout:2/127: dread d0/d6/f8 [0,4194304] 0 2024-03-13T14:49:31.271 INFO:tasks.workunit.client.1.smithi196.stdout:5/303: mkdir d4/d60 0 2024-03-13T14:49:31.271 INFO:tasks.workunit.client.1.smithi196.stdout:5/304: readlink d4/l9 0 2024-03-13T14:49:31.271 INFO:tasks.workunit.client.0.smithi176.stdout:7/55: readlink d1/d7/lb 0 2024-03-13T14:49:31.271 INFO:tasks.workunit.client.0.smithi176.stdout:6/39: dwrite d0/f8 [4194304,4194304] 0 2024-03-13T14:49:31.271 INFO:tasks.workunit.client.0.smithi176.stdout:7/56: chown d1/d2/la 18089178 1 2024-03-13T14:49:31.271 INFO:tasks.workunit.client.0.smithi176.stdout:1/25: rename d2/c8 to d2/ca 0 2024-03-13T14:49:31.272 INFO:tasks.workunit.client.0.smithi176.stdout:1/26: truncate d2/f3 1320031 0 2024-03-13T14:49:31.274 INFO:tasks.workunit.client.0.smithi176.stdout:1/27: chown d2/f5 20163533 1 2024-03-13T14:49:31.274 INFO:tasks.workunit.client.0.smithi176.stdout:8/44: unlink d0/c4 0 2024-03-13T14:49:31.275 INFO:tasks.workunit.client.0.smithi176.stdout:1/28: truncate d2/f6 69888 0 2024-03-13T14:49:31.276 INFO:tasks.workunit.client.0.smithi176.stdout:1/29: chown d2/f3 3 1 2024-03-13T14:49:31.282 INFO:tasks.workunit.client.1.smithi196.stdout:5/305: dwrite d4/db/d14/d37/d41/f42 [0,4194304] 0 2024-03-13T14:49:31.282 INFO:tasks.workunit.client.1.smithi196.stdout:3/179: unlink db/l10 0 2024-03-13T14:49:31.286 INFO:tasks.workunit.client.0.smithi176.stdout:0/37: mknod d2/c9 0 2024-03-13T14:49:31.288 INFO:tasks.workunit.client.1.smithi196.stdout:7/289: creat d0/d9/f5a x:0 0 0 2024-03-13T14:49:31.289 INFO:tasks.workunit.client.1.smithi196.stdout:7/290: read - d0/d6/d33/f4c zero size 2024-03-13T14:49:31.292 INFO:tasks.workunit.client.1.smithi196.stdout:0/296: mknod d8/c69 0 2024-03-13T14:49:31.293 INFO:tasks.workunit.client.0.smithi176.stdout:6/40: mkdir d0/df 0 2024-03-13T14:49:31.294 INFO:tasks.workunit.client.0.smithi176.stdout:6/41: chown d0/de 12662253 1 2024-03-13T14:49:31.295 INFO:tasks.workunit.client.1.smithi196.stdout:2/128: symlink d0/d7/l1e 0 2024-03-13T14:49:31.295 INFO:tasks.workunit.client.0.smithi176.stdout:7/57: symlink d1/d7/lf 0 2024-03-13T14:49:31.298 INFO:tasks.workunit.client.0.smithi176.stdout:1/30: symlink d2/lb 0 2024-03-13T14:49:31.298 INFO:tasks.workunit.client.1.smithi196.stdout:9/200: readlink d7/d9/df/d1d/d22/l34 0 2024-03-13T14:49:31.300 INFO:tasks.workunit.client.0.smithi176.stdout:5/38: rename d5/l9 to d5/ld 0 2024-03-13T14:49:31.301 INFO:tasks.workunit.client.0.smithi176.stdout:5/39: fdatasync d5/f7 0 2024-03-13T14:49:31.301 INFO:tasks.workunit.client.0.smithi176.stdout:5/40: chown f4 5571118 1 2024-03-13T14:49:31.302 INFO:tasks.workunit.client.1.smithi196.stdout:3/180: creat db/f3e x:0 0 0 2024-03-13T14:49:31.302 INFO:tasks.workunit.client.0.smithi176.stdout:5/41: chown f0 80456281 1 2024-03-13T14:49:31.304 INFO:tasks.workunit.client.0.smithi176.stdout:0/38: mkdir d2/d5/da 0 2024-03-13T14:49:31.305 INFO:tasks.workunit.client.0.smithi176.stdout:0/39: chown d2/f6 89759865 1 2024-03-13T14:49:31.306 INFO:tasks.workunit.client.1.smithi196.stdout:7/291: unlink d0/d6/le 0 2024-03-13T14:49:31.306 INFO:tasks.workunit.client.1.smithi196.stdout:8/171: getdents d0/d10/d12/d1e/d2d 0 2024-03-13T14:49:31.308 INFO:tasks.workunit.client.1.smithi196.stdout:5/306: dwrite d4/db/d14/d19/d1c/f3f [0,4194304] 0 2024-03-13T14:49:31.308 INFO:tasks.workunit.client.1.smithi196.stdout:8/172: fdatasync d0/f27 0 2024-03-13T14:49:31.312 INFO:tasks.workunit.client.1.smithi196.stdout:0/297: rmdir d8/da/d25/d26/d35/d3b/d5f/d33 39 2024-03-13T14:49:31.314 INFO:tasks.workunit.client.1.smithi196.stdout:0/298: dread - d8/d37/f62 zero size 2024-03-13T14:49:31.316 INFO:tasks.workunit.client.0.smithi176.stdout:6/42: mkdir d0/d10 0 2024-03-13T14:49:31.320 INFO:tasks.workunit.client.0.smithi176.stdout:7/58: creat d1/d7/f10 x:0 0 0 2024-03-13T14:49:31.325 INFO:tasks.workunit.client.0.smithi176.stdout:1/31: rename d2/f3 to d2/fc 0 2024-03-13T14:49:31.329 INFO:tasks.workunit.client.0.smithi176.stdout:5/42: mkdir d5/de 0 2024-03-13T14:49:31.331 INFO:tasks.workunit.client.0.smithi176.stdout:5/43: write f4 [4111555,37423] 0 2024-03-13T14:49:31.336 INFO:tasks.workunit.client.1.smithi196.stdout:9/201: rename d7/d9/d17/l2e to d7/d24/d28/l36 0 2024-03-13T14:49:31.337 INFO:tasks.workunit.client.0.smithi176.stdout:0/40: readlink d2/l7 0 2024-03-13T14:49:31.337 INFO:tasks.workunit.client.1.smithi196.stdout:0/299: dwrite d8/da/f10 [0,4194304] 0 2024-03-13T14:49:31.339 INFO:tasks.workunit.client.0.smithi176.stdout:3/57: getdents d4 0 2024-03-13T14:49:31.340 INFO:tasks.workunit.client.0.smithi176.stdout:0/41: fsync d2/d5/f8 0 2024-03-13T14:49:31.344 INFO:tasks.workunit.client.1.smithi196.stdout:2/129: dread d0/d7/d9/f18 [0,4194304] 0 2024-03-13T14:49:31.345 INFO:tasks.workunit.client.0.smithi176.stdout:1/32: dread d2/f9 [0,4194304] 0 2024-03-13T14:49:31.346 INFO:tasks.workunit.client.1.smithi196.stdout:8/173: creat d0/d1c/d29/f3c x:0 0 0 2024-03-13T14:49:31.347 INFO:tasks.workunit.client.0.smithi176.stdout:1/33: chown d2/f9 487307212 1 2024-03-13T14:49:31.350 INFO:tasks.workunit.client.0.smithi176.stdout:1/34: read d2/fc [1267520,84803] 0 2024-03-13T14:49:31.350 INFO:tasks.workunit.client.0.smithi176.stdout:7/59: rmdir d1/d2 39 2024-03-13T14:49:31.353 INFO:tasks.workunit.client.1.smithi196.stdout:4/198: getdents d1/d3 0 2024-03-13T14:49:31.355 INFO:tasks.workunit.client.1.smithi196.stdout:4/199: fsync d1/d2f/f32 0 2024-03-13T14:49:31.357 INFO:tasks.workunit.client.1.smithi196.stdout:4/200: truncate d1/d3/d7/f21 1656854 0 2024-03-13T14:49:31.358 INFO:tasks.workunit.client.1.smithi196.stdout:4/201: write d1/d3/f12 [2164565,93139] 0 2024-03-13T14:49:31.360 INFO:tasks.workunit.client.1.smithi196.stdout:4/202: stat d1/d3/d7/cc 0 2024-03-13T14:49:31.363 INFO:tasks.workunit.client.1.smithi196.stdout:0/300: write d8/da/d25/d26/d35/d3b/d5f/d33/f34 [1466845,129895] 0 2024-03-13T14:49:31.367 INFO:tasks.workunit.client.1.smithi196.stdout:0/301: truncate d8/da/d25/f5a 352776 0 2024-03-13T14:49:31.369 INFO:tasks.workunit.client.1.smithi196.stdout:7/292: creat d0/d4d/f5b x:0 0 0 2024-03-13T14:49:31.372 INFO:tasks.workunit.client.0.smithi176.stdout:5/44: write d5/f8 [938158,123226] 0 2024-03-13T14:49:31.373 INFO:tasks.workunit.client.1.smithi196.stdout:2/130: rmdir d0 39 2024-03-13T14:49:31.375 INFO:tasks.workunit.client.0.smithi176.stdout:3/58: mkdir d4/d13 0 2024-03-13T14:49:31.377 INFO:tasks.workunit.client.1.smithi196.stdout:5/307: mkdir d4/db/d14/d19/d26/d33/d61 0 2024-03-13T14:49:31.378 INFO:tasks.workunit.client.0.smithi176.stdout:0/42: fsync d2/f6 0 2024-03-13T14:49:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:31 smithi176 ceph-mon[29064]: [13/Mar/2024:14:49:29] ENGINE Bus STARTING 2024-03-13T14:49:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:31 smithi176 ceph-mon[29064]: [13/Mar/2024:14:49:29] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:49:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:31 smithi176 ceph-mon[29064]: [13/Mar/2024:14:49:30] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:49:31.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:31 smithi176 ceph-mon[29064]: [13/Mar/2024:14:49:30] ENGINE Bus STARTED 2024-03-13T14:49:31.379 INFO:tasks.workunit.client.1.smithi196.stdout:8/174: mkdir d0/d16/d3d 0 2024-03-13T14:49:31.380 INFO:tasks.workunit.client.0.smithi176.stdout:9/86: getdents d0/d13 0 2024-03-13T14:49:31.380 INFO:tasks.workunit.client.0.smithi176.stdout:0/43: fdatasync d2/d5/f8 0 2024-03-13T14:49:31.380 INFO:tasks.workunit.client.1.smithi196.stdout:3/181: creat db/d19/f3f x:0 0 0 2024-03-13T14:49:31.382 INFO:tasks.workunit.client.0.smithi176.stdout:6/43: getdents d0/de 0 2024-03-13T14:49:31.384 INFO:tasks.workunit.client.0.smithi176.stdout:6/44: write d0/f8 [2149699,21810] 0 2024-03-13T14:49:31.384 INFO:tasks.workunit.client.0.smithi176.stdout:1/35: mkdir d2/dd 0 2024-03-13T14:49:31.384 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:31 smithi196 ceph-mon[37486]: [13/Mar/2024:14:49:29] ENGINE Bus STARTING 2024-03-13T14:49:31.384 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:31 smithi196 ceph-mon[37486]: [13/Mar/2024:14:49:29] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:49:31.384 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:31 smithi196 ceph-mon[37486]: [13/Mar/2024:14:49:30] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:49:31.384 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:31 smithi196 ceph-mon[37486]: [13/Mar/2024:14:49:30] ENGINE Bus STARTED 2024-03-13T14:49:31.385 INFO:tasks.workunit.client.0.smithi176.stdout:1/36: fsync d2/f6 0 2024-03-13T14:49:31.396 INFO:tasks.workunit.client.1.smithi196.stdout:8/175: creat d0/d10/d12/f3e x:0 0 0 2024-03-13T14:49:31.396 INFO:tasks.workunit.client.0.smithi176.stdout:2/46: sync 2024-03-13T14:49:31.398 INFO:tasks.workunit.client.1.smithi196.stdout:3/182: mknod db/d19/d22/d30/c40 0 2024-03-13T14:49:31.401 INFO:tasks.workunit.client.1.smithi196.stdout:0/302: mkdir d8/da/d25/d26/d35/d3b/d59/d6a 0 2024-03-13T14:49:31.408 INFO:tasks.workunit.client.1.smithi196.stdout:8/176: mkdir d0/d1c/d3b/d3f 0 2024-03-13T14:49:31.413 INFO:tasks.workunit.client.1.smithi196.stdout:4/203: dwrite d1/d1e/f34 [8388608,4194304] 0 2024-03-13T14:49:31.415 INFO:tasks.workunit.client.1.smithi196.stdout:8/177: truncate d0/d1c/d29/f3a 877181 0 2024-03-13T14:49:31.419 INFO:tasks.workunit.client.1.smithi196.stdout:2/131: unlink d0/d6/cf 0 2024-03-13T14:49:31.419 INFO:tasks.workunit.client.1.smithi196.stdout:3/183: link db/d19/d22/f3c db/d15/f41 0 2024-03-13T14:49:31.420 INFO:tasks.workunit.client.1.smithi196.stdout:3/184: dread - db/f35 zero size 2024-03-13T14:49:31.421 INFO:tasks.workunit.client.1.smithi196.stdout:7/293: getdents d0/d6/d35 0 2024-03-13T14:49:31.422 INFO:tasks.workunit.client.1.smithi196.stdout:0/303: dread d8/f1e [0,4194304] 0 2024-03-13T14:49:31.422 INFO:tasks.workunit.client.1.smithi196.stdout:3/185: chown db/d15 406 1 2024-03-13T14:49:31.423 INFO:tasks.workunit.client.1.smithi196.stdout:4/204: rmdir d1/d3/d7 39 2024-03-13T14:49:31.425 INFO:tasks.workunit.client.0.smithi176.stdout:2/47: dwrite d1/f15 [0,4194304] 0 2024-03-13T14:49:31.434 INFO:tasks.workunit.client.1.smithi196.stdout:5/308: getdents d4/db 0 2024-03-13T14:49:31.438 INFO:tasks.workunit.client.1.smithi196.stdout:3/186: unlink db/l12 0 2024-03-13T14:49:31.438 INFO:tasks.workunit.client.1.smithi196.stdout:3/187: read - db/f29 zero size 2024-03-13T14:49:31.440 INFO:tasks.workunit.client.0.smithi176.stdout:4/59: sync 2024-03-13T14:49:31.440 INFO:tasks.workunit.client.0.smithi176.stdout:8/45: sync 2024-03-13T14:49:31.446 INFO:tasks.workunit.client.1.smithi196.stdout:2/132: unlink d0/d6/fb 0 2024-03-13T14:49:31.446 INFO:tasks.workunit.client.1.smithi196.stdout:3/188: dread db/d19/f28 [0,4194304] 0 2024-03-13T14:49:31.451 INFO:tasks.workunit.client.1.smithi196.stdout:0/304: mknod d8/c6b 0 2024-03-13T14:49:31.458 INFO:tasks.workunit.client.0.smithi176.stdout:4/60: dread d7/fe [0,4194304] 0 2024-03-13T14:49:31.463 INFO:tasks.workunit.client.1.smithi196.stdout:8/178: rmdir d0 39 2024-03-13T14:49:31.463 INFO:tasks.workunit.client.1.smithi196.stdout:3/189: rmdir db/d15 39 2024-03-13T14:49:31.464 INFO:tasks.workunit.client.1.smithi196.stdout:7/294: dread d0/fa [0,4194304] 0 2024-03-13T14:49:31.464 INFO:tasks.workunit.client.1.smithi196.stdout:3/190: dread - db/f29 zero size 2024-03-13T14:49:31.468 INFO:tasks.workunit.client.1.smithi196.stdout:0/305: mkdir d8/da/d25/d26/d35/d3b/d6c 0 2024-03-13T14:49:31.477 INFO:tasks.workunit.client.1.smithi196.stdout:4/205: unlink d1/d3/f12 0 2024-03-13T14:49:31.478 INFO:tasks.workunit.client.1.smithi196.stdout:3/191: dread db/f2c [0,4194304] 0 2024-03-13T14:49:31.479 INFO:tasks.workunit.client.1.smithi196.stdout:3/192: stat db/f2c 0 2024-03-13T14:49:31.480 INFO:tasks.workunit.client.1.smithi196.stdout:8/179: readlink d0/d10/d12/l39 0 2024-03-13T14:49:31.481 INFO:tasks.workunit.client.1.smithi196.stdout:3/193: fsync db/f3e 0 2024-03-13T14:49:31.482 INFO:tasks.workunit.client.1.smithi196.stdout:8/180: dread - d0/d10/d12/f33 zero size 2024-03-13T14:49:31.483 INFO:tasks.workunit.client.1.smithi196.stdout:2/133: mknod d0/c1f 0 2024-03-13T14:49:31.484 INFO:tasks.workunit.client.1.smithi196.stdout:7/295: dwrite d0/d6/d33/f3c [0,4194304] 0 2024-03-13T14:49:31.499 INFO:tasks.workunit.client.1.smithi196.stdout:5/309: write d4/db/d14/d19/d1c/d25/d38/f56 [2020681,118302] 0 2024-03-13T14:49:31.503 INFO:tasks.workunit.client.0.smithi176.stdout:4/61: dwrite d7/d8/f18 [0,4194304] 0 2024-03-13T14:49:31.503 INFO:tasks.workunit.client.1.smithi196.stdout:5/310: chown d4/db/d14/d19/d1c/d25/f49 423659 1 2024-03-13T14:49:31.505 INFO:tasks.workunit.client.1.smithi196.stdout:3/194: dwrite db/d19/f26 [0,4194304] 0 2024-03-13T14:49:31.506 INFO:tasks.workunit.client.0.smithi176.stdout:4/62: dread - d7/fc zero size 2024-03-13T14:49:31.521 INFO:tasks.workunit.client.1.smithi196.stdout:8/181: mkdir d0/d10/d12/d17/d40 0 2024-03-13T14:49:31.529 INFO:tasks.workunit.client.1.smithi196.stdout:4/206: unlink d1/d3/f17 0 2024-03-13T14:49:31.531 INFO:tasks.workunit.client.1.smithi196.stdout:3/195: creat db/d19/d22/d30/f42 x:0 0 0 2024-03-13T14:49:31.538 INFO:tasks.workunit.client.1.smithi196.stdout:0/306: creat d8/da/d25/d26/d35/d3b/d50/f6d x:0 0 0 2024-03-13T14:49:31.549 INFO:tasks.workunit.client.1.smithi196.stdout:7/296: getdents d0/d9/d31/d50 0 2024-03-13T14:49:31.551 INFO:tasks.workunit.client.1.smithi196.stdout:8/182: mknod d0/d16/d3d/c41 0 2024-03-13T14:49:31.555 INFO:tasks.workunit.client.1.smithi196.stdout:1/215: sync 2024-03-13T14:49:31.555 INFO:tasks.workunit.client.1.smithi196.stdout:6/310: sync 2024-03-13T14:49:31.559 INFO:tasks.workunit.client.1.smithi196.stdout:3/196: mknod db/d19/c43 0 2024-03-13T14:49:31.561 INFO:tasks.workunit.client.1.smithi196.stdout:3/197: write db/d19/f1d [1068282,25720] 0 2024-03-13T14:49:31.564 INFO:tasks.workunit.client.1.smithi196.stdout:7/297: symlink d0/d9/d31/d46/d47/l5c 0 2024-03-13T14:49:31.566 INFO:tasks.workunit.client.1.smithi196.stdout:8/183: mkdir d0/d10/d12/d17/d42 0 2024-03-13T14:49:31.567 INFO:tasks.workunit.client.1.smithi196.stdout:7/298: chown d0/d6/f19 53417651 1 2024-03-13T14:49:31.571 INFO:tasks.workunit.client.1.smithi196.stdout:0/307: creat d8/da/d25/d26/d35/d3b/d6c/f6e x:0 0 0 2024-03-13T14:49:31.576 INFO:tasks.workunit.client.1.smithi196.stdout:1/216: rename d1/d38/c2f to d1/d3f/c45 0 2024-03-13T14:49:31.577 INFO:tasks.workunit.client.1.smithi196.stdout:6/311: creat d6/d4b/f65 x:0 0 0 2024-03-13T14:49:31.582 INFO:tasks.workunit.client.1.smithi196.stdout:4/207: dwrite d1/f29 [0,4194304] 0 2024-03-13T14:49:31.582 INFO:tasks.workunit.client.1.smithi196.stdout:2/134: creat d0/d7/d9/f20 x:0 0 0 2024-03-13T14:49:31.587 INFO:tasks.workunit.client.1.smithi196.stdout:5/311: getdents d4/db 0 2024-03-13T14:49:31.591 INFO:tasks.workunit.client.1.smithi196.stdout:8/184: dwrite d0/d10/f31 [0,4194304] 0 2024-03-13T14:49:31.599 INFO:tasks.workunit.client.1.smithi196.stdout:7/299: dwrite d0/d6/f2c [0,4194304] 0 2024-03-13T14:49:31.607 INFO:tasks.workunit.client.1.smithi196.stdout:2/135: creat d0/d7/f21 x:0 0 0 2024-03-13T14:49:31.607 INFO:tasks.workunit.client.1.smithi196.stdout:4/208: dread - d1/d3/f36 zero size 2024-03-13T14:49:31.608 INFO:tasks.workunit.client.1.smithi196.stdout:3/198: mkdir db/d15/d44 0 2024-03-13T14:49:31.608 INFO:tasks.workunit.client.1.smithi196.stdout:2/136: stat d0/fe 0 2024-03-13T14:49:31.611 INFO:tasks.workunit.client.1.smithi196.stdout:6/312: dread d6/d4b/f58 [0,4194304] 0 2024-03-13T14:49:31.621 INFO:tasks.workunit.client.1.smithi196.stdout:5/312: mknod d4/db/d14/c62 0 2024-03-13T14:49:31.624 INFO:tasks.workunit.client.1.smithi196.stdout:7/300: symlink d0/d4d/l5d 0 2024-03-13T14:49:31.624 INFO:tasks.workunit.client.1.smithi196.stdout:5/313: chown d4/db/d14/d19/f58 5358 1 2024-03-13T14:49:31.642 INFO:tasks.workunit.client.1.smithi196.stdout:5/314: mkdir d4/db/d14/d37/d41/d63 0 2024-03-13T14:49:31.648 INFO:tasks.workunit.client.1.smithi196.stdout:7/301: rename d0/d6/l36 to d0/d9/d31/d50/l5e 0 2024-03-13T14:49:31.649 INFO:tasks.workunit.client.1.smithi196.stdout:3/199: symlink db/d19/l45 0 2024-03-13T14:49:31.649 INFO:tasks.workunit.client.1.smithi196.stdout:6/313: mkdir d6/d9/d17/d19/d4e/d66 0 2024-03-13T14:49:31.651 INFO:tasks.workunit.client.1.smithi196.stdout:3/200: dread - db/d19/f3f zero size 2024-03-13T14:49:31.652 INFO:tasks.workunit.client.1.smithi196.stdout:2/137: chown d0/d7/d9/f20 31 1 2024-03-13T14:49:31.653 INFO:tasks.workunit.client.1.smithi196.stdout:3/201: fsync db/d19/f1d 0 2024-03-13T14:49:31.660 INFO:tasks.workunit.client.1.smithi196.stdout:4/209: rename d1/d3/d7/f19 to d1/d3/d7/d35/f37 0 2024-03-13T14:49:31.668 INFO:tasks.workunit.client.1.smithi196.stdout:5/315: dwrite d4/db/d14/d19/d1c/d2d/f4e [0,4194304] 0 2024-03-13T14:49:31.673 INFO:tasks.workunit.client.1.smithi196.stdout:6/314: dwrite d6/d7/df/f2e [4194304,4194304] 0 2024-03-13T14:49:31.686 INFO:tasks.workunit.client.1.smithi196.stdout:3/202: dwrite db/ff [4194304,4194304] 0 2024-03-13T14:49:31.701 INFO:tasks.workunit.client.1.smithi196.stdout:7/302: getdents d0/d6/d35 0 2024-03-13T14:49:31.708 INFO:tasks.workunit.client.1.smithi196.stdout:1/217: truncate d1/d38/f40 3366768 0 2024-03-13T14:49:31.710 INFO:tasks.workunit.client.1.smithi196.stdout:4/210: dread d1/f6 [0,4194304] 0 2024-03-13T14:49:31.713 INFO:tasks.workunit.client.1.smithi196.stdout:6/315: mknod d6/d7/df/c67 0 2024-03-13T14:49:31.719 INFO:tasks.workunit.client.1.smithi196.stdout:8/185: truncate d0/d10/f31 2707451 0 2024-03-13T14:49:31.723 INFO:tasks.workunit.client.1.smithi196.stdout:1/218: dread d1/d28/d34/f3a [0,4194304] 0 2024-03-13T14:49:31.728 INFO:tasks.workunit.client.1.smithi196.stdout:5/316: dread d4/db/d14/d19/d1c/d25/d38/f56 [0,4194304] 0 2024-03-13T14:49:31.771 INFO:tasks.workunit.client.1.smithi196.stdout:4/211: creat d1/d2f/f38 x:0 0 0 2024-03-13T14:49:31.772 INFO:tasks.workunit.client.1.smithi196.stdout:4/212: chown d1/d2f/f38 3707 1 2024-03-13T14:49:31.772 INFO:tasks.workunit.client.1.smithi196.stdout:6/316: mknod d6/d2a/d23/c68 0 2024-03-13T14:49:31.774 INFO:tasks.workunit.client.1.smithi196.stdout:3/203: mknod db/c46 0 2024-03-13T14:49:31.774 INFO:tasks.workunit.client.1.smithi196.stdout:9/202: sync 2024-03-13T14:49:31.775 INFO:tasks.workunit.client.1.smithi196.stdout:9/203: stat d7/l13 0 2024-03-13T14:49:31.779 INFO:tasks.workunit.client.1.smithi196.stdout:1/219: creat d1/d9/d33/f46 x:0 0 0 2024-03-13T14:49:31.787 INFO:tasks.workunit.client.0.smithi176.stdout:5/45: mknod d5/cf 0 2024-03-13T14:49:31.789 INFO:tasks.workunit.client.1.smithi196.stdout:3/204: mkdir db/d19/d22/d47 0 2024-03-13T14:49:31.790 INFO:tasks.workunit.client.1.smithi196.stdout:4/213: dwrite d1/f22 [0,4194304] 0 2024-03-13T14:49:31.792 INFO:tasks.workunit.client.0.smithi176.stdout:3/59: rename d4/ce to d4/c14 0 2024-03-13T14:49:31.793 INFO:tasks.workunit.client.0.smithi176.stdout:3/60: readlink d4/ld 0 2024-03-13T14:49:31.794 INFO:tasks.workunit.client.0.smithi176.stdout:3/61: write f2 [4299232,4898] 0 2024-03-13T14:49:31.796 INFO:tasks.workunit.client.0.smithi176.stdout:3/62: write f1 [916112,55426] 0 2024-03-13T14:49:31.808 INFO:tasks.workunit.client.1.smithi196.stdout:1/220: fdatasync d1/d29/f1d 0 2024-03-13T14:49:31.808 INFO:tasks.workunit.client.1.smithi196.stdout:3/205: dread db/d19/f2f [0,4194304] 0 2024-03-13T14:49:31.810 INFO:tasks.workunit.client.1.smithi196.stdout:3/206: dread - db/d19/d22/f31 zero size 2024-03-13T14:49:31.816 INFO:tasks.workunit.client.1.smithi196.stdout:9/204: dread d7/de/f1f [0,4194304] 0 2024-03-13T14:49:31.820 INFO:tasks.workunit.client.1.smithi196.stdout:2/138: rename d0/d6/f13 to d0/d7/d9/f22 0 2024-03-13T14:49:31.821 INFO:tasks.workunit.client.0.smithi176.stdout:1/37: mkdir d2/de 0 2024-03-13T14:49:31.822 INFO:tasks.workunit.client.1.smithi196.stdout:9/205: truncate d7/de/f29 470267 0 2024-03-13T14:49:31.827 INFO:tasks.workunit.client.1.smithi196.stdout:9/206: read - d7/d24/f2b zero size 2024-03-13T14:49:31.836 INFO:tasks.workunit.client.1.smithi196.stdout:0/308: sync 2024-03-13T14:49:31.836 INFO:tasks.workunit.client.1.smithi196.stdout:0/309: write d8/da/f3e [1499461,106435] 0 2024-03-13T14:49:31.836 INFO:tasks.workunit.client.1.smithi196.stdout:3/207: dwrite db/f13 [0,4194304] 0 2024-03-13T14:49:31.845 INFO:tasks.workunit.client.1.smithi196.stdout:2/139: dread d0/fe [0,4194304] 0 2024-03-13T14:49:31.846 INFO:tasks.workunit.client.1.smithi196.stdout:3/208: write db/d19/d22/f31 [716453,122168] 0 2024-03-13T14:49:31.847 INFO:tasks.workunit.client.1.smithi196.stdout:3/209: write db/d19/f1d [1854997,16540] 0 2024-03-13T14:49:31.852 INFO:tasks.workunit.client.1.smithi196.stdout:9/207: dwrite d7/f15 [0,4194304] 0 2024-03-13T14:49:31.863 INFO:tasks.workunit.client.0.smithi176.stdout:4/63: mknod d7/c19 0 2024-03-13T14:49:31.888 INFO:tasks.workunit.client.1.smithi196.stdout:9/208: symlink d7/d24/d28/l37 0 2024-03-13T14:49:31.889 INFO:tasks.workunit.client.0.smithi176.stdout:6/45: symlink d0/d10/l11 0 2024-03-13T14:49:31.893 INFO:tasks.workunit.client.1.smithi196.stdout:0/310: creat d8/da/d25/d26/d35/d3b/d59/d6a/f6f x:0 0 0 2024-03-13T14:49:31.896 INFO:tasks.workunit.client.0.smithi176.stdout:2/48: link d1/ff d1/dc/f16 0 2024-03-13T14:49:31.902 INFO:tasks.workunit.client.1.smithi196.stdout:3/210: creat db/d19/d22/d47/f48 x:0 0 0 2024-03-13T14:49:31.904 INFO:tasks.workunit.client.1.smithi196.stdout:9/209: creat d7/d24/f38 x:0 0 0 2024-03-13T14:49:31.904 INFO:tasks.workunit.client.1.smithi196.stdout:0/311: unlink d8/da/d25/d26/d35/d3b/d5f/d43/l67 0 2024-03-13T14:49:31.909 INFO:tasks.workunit.client.0.smithi176.stdout:5/46: creat d5/de/f10 x:0 0 0 2024-03-13T14:49:31.911 INFO:tasks.workunit.client.1.smithi196.stdout:9/210: unlink d7/d9/df/d1d/d22/f2d 0 2024-03-13T14:49:31.913 INFO:tasks.workunit.client.0.smithi176.stdout:5/47: chown d5/f7 7223 1 2024-03-13T14:49:31.915 INFO:tasks.workunit.client.1.smithi196.stdout:7/303: rename d0/d9/d31/d46/d47/l48 to d0/d9/d1e/l5f 0 2024-03-13T14:49:31.920 INFO:tasks.workunit.client.1.smithi196.stdout:6/317: getdents d6/d2a/d23 0 2024-03-13T14:49:31.923 INFO:tasks.workunit.client.1.smithi196.stdout:9/211: creat d7/d9/df/d1d/d22/f39 x:0 0 0 2024-03-13T14:49:31.923 INFO:tasks.workunit.client.1.smithi196.stdout:6/318: truncate d6/d2a/dc/d1c/f3b 707452 0 2024-03-13T14:49:31.926 INFO:tasks.workunit.client.1.smithi196.stdout:3/211: creat db/f49 x:0 0 0 2024-03-13T14:49:31.926 INFO:tasks.workunit.client.1.smithi196.stdout:9/212: write d7/d9/df/d1d/d22/f39 [913547,44559] 0 2024-03-13T14:49:31.926 INFO:tasks.workunit.client.0.smithi176.stdout:7/60: link d1/d7/cd d1/d2/de/c11 0 2024-03-13T14:49:31.928 INFO:tasks.workunit.client.1.smithi196.stdout:9/213: chown d7/de/d16 5536292 1 2024-03-13T14:49:31.928 INFO:tasks.workunit.client.1.smithi196.stdout:3/212: chown db/d19/f26 647416 1 2024-03-13T14:49:31.929 INFO:tasks.workunit.client.1.smithi196.stdout:7/304: rename d0/d6/f4a to d0/d9/d1e/d3f/f60 0 2024-03-13T14:49:31.929 INFO:tasks.workunit.client.0.smithi176.stdout:6/46: dwrite d0/f8 [8388608,4194304] 0 2024-03-13T14:49:31.931 INFO:tasks.workunit.client.1.smithi196.stdout:7/305: readlink d0/d9/d31/d46/d47/l5c 0 2024-03-13T14:49:31.932 INFO:tasks.workunit.client.0.smithi176.stdout:2/49: creat d1/dc/f17 x:0 0 0 2024-03-13T14:49:31.933 INFO:tasks.workunit.client.0.smithi176.stdout:6/47: chown d0/de 224403397 1 2024-03-13T14:49:31.934 INFO:tasks.workunit.client.0.smithi176.stdout:8/46: creat d0/f9 x:0 0 0 2024-03-13T14:49:31.936 INFO:tasks.workunit.client.1.smithi196.stdout:9/214: creat d7/d24/f3a x:0 0 0 2024-03-13T14:49:31.937 INFO:tasks.workunit.client.0.smithi176.stdout:4/64: symlink d7/d8/d15/l1a 0 2024-03-13T14:49:31.939 INFO:tasks.workunit.client.0.smithi176.stdout:5/48: mknod d5/c11 0 2024-03-13T14:49:31.939 INFO:tasks.workunit.client.1.smithi196.stdout:3/213: dread db/d19/f28 [0,4194304] 0 2024-03-13T14:49:31.942 INFO:tasks.workunit.client.0.smithi176.stdout:4/65: stat d7/d8/l13 0 2024-03-13T14:49:31.943 INFO:tasks.workunit.client.1.smithi196.stdout:0/312: rename d8/d37/f62 to d8/da/d25/d26/d35/d3b/d5f/d17/f70 0 2024-03-13T14:49:31.946 INFO:tasks.workunit.client.0.smithi176.stdout:2/50: mknod d1/c18 0 2024-03-13T14:49:31.946 INFO:tasks.workunit.client.1.smithi196.stdout:0/313: write d8/da/d25/d26/d35/d3b/d5f/f4a [215750,83061] 0 2024-03-13T14:49:31.947 INFO:tasks.workunit.client.1.smithi196.stdout:8/186: truncate d0/d1c/d3b/f1f 1354864 0 2024-03-13T14:49:31.948 INFO:tasks.workunit.client.1.smithi196.stdout:7/306: stat d0/l1f 0 2024-03-13T14:49:31.948 INFO:tasks.workunit.client.0.smithi176.stdout:2/51: fsync d1/dc/f12 0 2024-03-13T14:49:31.950 INFO:tasks.workunit.client.0.smithi176.stdout:2/52: stat d1/f15 0 2024-03-13T14:49:31.955 INFO:tasks.workunit.client.0.smithi176.stdout:2/53: chown d1/dc/f16 17867233 1 2024-03-13T14:49:31.955 INFO:tasks.workunit.client.1.smithi196.stdout:3/214: dread f7 [4194304,4194304] 0 2024-03-13T14:49:31.957 INFO:tasks.workunit.client.1.smithi196.stdout:3/215: fdatasync f4 0 2024-03-13T14:49:31.964 INFO:tasks.workunit.client.1.smithi196.stdout:5/317: write d4/db/d3c/f30 [1173268,18432] 0 2024-03-13T14:49:31.966 INFO:tasks.workunit.client.0.smithi176.stdout:5/49: symlink d5/de/l12 0 2024-03-13T14:49:31.967 INFO:tasks.workunit.client.1.smithi196.stdout:5/318: chown d4/db/d3c/f30 1 1 2024-03-13T14:49:31.970 INFO:tasks.workunit.client.1.smithi196.stdout:7/307: dwrite d0/d9/d1e/f4b [0,4194304] 0 2024-03-13T14:49:31.971 INFO:tasks.workunit.client.0.smithi176.stdout:2/54: dread d1/f14 [0,4194304] 0 2024-03-13T14:49:31.972 INFO:tasks.workunit.client.1.smithi196.stdout:7/308: chown d0/d6/l3e 776518925 1 2024-03-13T14:49:31.974 INFO:tasks.workunit.client.0.smithi176.stdout:5/50: chown d5/lc 620 1 2024-03-13T14:49:31.976 INFO:tasks.workunit.client.0.smithi176.stdout:1/38: getdents d2 0 2024-03-13T14:49:31.987 INFO:tasks.workunit.client.0.smithi176.stdout:4/66: dwrite d7/d8/fd [0,4194304] 0 2024-03-13T14:49:31.991 INFO:tasks.workunit.client.0.smithi176.stdout:6/48: dwrite d0/f8 [8388608,4194304] 0 2024-03-13T14:49:31.992 INFO:tasks.workunit.client.0.smithi176.stdout:8/47: dwrite d0/f9 [0,4194304] 0 2024-03-13T14:49:31.996 INFO:tasks.workunit.client.0.smithi176.stdout:2/55: unlink d1/f4 0 2024-03-13T14:49:32.004 INFO:tasks.workunit.client.0.smithi176.stdout:7/61: link d1/d7/lf d1/d2/l12 0 2024-03-13T14:49:32.006 INFO:tasks.workunit.client.0.smithi176.stdout:7/62: chown d1/d2/l4 27 1 2024-03-13T14:49:32.010 INFO:tasks.workunit.client.0.smithi176.stdout:5/51: creat d5/de/f13 x:0 0 0 2024-03-13T14:49:32.011 INFO:tasks.workunit.client.1.smithi196.stdout:7/309: chown d0/l1 28 1 2024-03-13T14:49:32.015 INFO:tasks.workunit.client.1.smithi196.stdout:7/310: fsync d0/f11 0 2024-03-13T14:49:32.021 INFO:tasks.workunit.client.0.smithi176.stdout:9/87: truncate d0/f10 5681567 0 2024-03-13T14:49:32.024 INFO:tasks.workunit.client.0.smithi176.stdout:5/52: dread d5/f8 [0,4194304] 0 2024-03-13T14:49:32.027 INFO:tasks.workunit.client.0.smithi176.stdout:0/44: dwrite d2/f3 [0,4194304] 0 2024-03-13T14:49:32.027 INFO:tasks.workunit.client.0.smithi176.stdout:0/45: read - d2/d5/f8 zero size 2024-03-13T14:49:32.028 INFO:tasks.workunit.client.0.smithi176.stdout:6/49: chown d0/cb 2019 1 2024-03-13T14:49:32.033 INFO:tasks.workunit.client.1.smithi196.stdout:3/216: mkdir db/d19/d4a 0 2024-03-13T14:49:32.034 INFO:tasks.workunit.client.0.smithi176.stdout:8/48: unlink d0/d5/l6 0 2024-03-13T14:49:32.035 INFO:tasks.workunit.client.0.smithi176.stdout:8/49: chown d0/f2 1341510 1 2024-03-13T14:49:32.035 INFO:tasks.workunit.client.1.smithi196.stdout:3/217: read db/d19/f26 [235695,117364] 0 2024-03-13T14:49:32.039 INFO:tasks.workunit.client.1.smithi196.stdout:9/215: creat d7/d9/df/d1d/f3b x:0 0 0 2024-03-13T14:49:32.044 INFO:tasks.workunit.client.1.smithi196.stdout:5/319: creat d4/db/d14/d19/d1c/d25/d38/d47/f64 x:0 0 0 2024-03-13T14:49:32.045 INFO:tasks.workunit.client.1.smithi196.stdout:0/314: mkdir d8/da/d71 0 2024-03-13T14:49:32.048 INFO:tasks.workunit.client.0.smithi176.stdout:2/56: mknod d1/c19 0 2024-03-13T14:49:32.052 INFO:tasks.workunit.client.0.smithi176.stdout:7/63: chown d1/d2/de/c11 876 1 2024-03-13T14:49:32.053 INFO:tasks.workunit.client.0.smithi176.stdout:7/64: chown d1/d7/lb 413260930 1 2024-03-13T14:49:32.054 INFO:tasks.workunit.client.0.smithi176.stdout:7/65: stat d1/d2/l4 0 2024-03-13T14:49:32.056 INFO:tasks.workunit.client.0.smithi176.stdout:4/67: fsync d7/d8/fd 0 2024-03-13T14:49:32.058 INFO:tasks.workunit.client.1.smithi196.stdout:7/311: chown d0/d9/d1e/d3f/c49 6619718 1 2024-03-13T14:49:32.061 INFO:tasks.workunit.client.0.smithi176.stdout:1/39: mknod d2/de/cf 0 2024-03-13T14:49:32.062 INFO:tasks.workunit.client.1.smithi196.stdout:9/216: fsync f5 0 2024-03-13T14:49:32.063 INFO:tasks.workunit.client.0.smithi176.stdout:1/40: chown d2/ca 1 1 2024-03-13T14:49:32.066 INFO:tasks.workunit.client.0.smithi176.stdout:3/63: truncate f2 3887428 0 2024-03-13T14:49:32.069 INFO:tasks.workunit.client.1.smithi196.stdout:7/312: dread d0/d9/d1e/f4b [0,4194304] 0 2024-03-13T14:49:32.070 INFO:tasks.workunit.client.0.smithi176.stdout:7/66: dwrite d1/d7/f10 [0,4194304] 0 2024-03-13T14:49:32.074 INFO:tasks.workunit.client.1.smithi196.stdout:3/218: mkdir db/d37/d4b 0 2024-03-13T14:49:32.077 INFO:tasks.workunit.client.1.smithi196.stdout:3/219: truncate db/d19/d22/f3c 199038 0 2024-03-13T14:49:32.078 INFO:tasks.workunit.client.1.smithi196.stdout:9/217: rmdir d7/d9/df/d1d/d22 39 2024-03-13T14:49:32.082 INFO:tasks.workunit.client.0.smithi176.stdout:2/57: mknod d1/dc/c1a 0 2024-03-13T14:49:32.086 INFO:tasks.workunit.client.0.smithi176.stdout:2/58: write d1/dc/f12 [401206,90823] 0 2024-03-13T14:49:32.087 INFO:tasks.workunit.client.0.smithi176.stdout:2/59: truncate d1/dc/f16 957024 0 2024-03-13T14:49:32.088 INFO:tasks.workunit.client.0.smithi176.stdout:2/60: dread - d1/f13 zero size 2024-03-13T14:49:32.090 INFO:tasks.workunit.client.0.smithi176.stdout:8/50: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:32.090 INFO:tasks.workunit.client.0.smithi176.stdout:2/61: write d1/fb [522454,56730] 0 2024-03-13T14:49:32.094 INFO:tasks.workunit.client.1.smithi196.stdout:6/319: truncate d6/d9/d11/f48 191408 0 2024-03-13T14:49:32.095 INFO:tasks.workunit.client.1.smithi196.stdout:6/320: fdatasync d6/f8 0 2024-03-13T14:49:32.096 INFO:tasks.workunit.client.0.smithi176.stdout:1/41: rmdir d2 39 2024-03-13T14:49:32.101 INFO:tasks.workunit.client.1.smithi196.stdout:7/313: rmdir d0/d9 39 2024-03-13T14:49:32.103 INFO:tasks.workunit.client.0.smithi176.stdout:9/88: truncate d0/f12 163823 0 2024-03-13T14:49:32.107 INFO:tasks.workunit.client.0.smithi176.stdout:3/64: write d4/fa [1315605,115345] 0 2024-03-13T14:49:32.108 INFO:tasks.workunit.client.1.smithi196.stdout:9/218: mkdir d7/de/d16/d3c 0 2024-03-13T14:49:32.110 INFO:tasks.workunit.client.0.smithi176.stdout:5/53: link f0 d5/f14 0 2024-03-13T14:49:32.113 INFO:tasks.workunit.client.0.smithi176.stdout:0/46: getdents d2/d5/da 0 2024-03-13T14:49:32.114 INFO:tasks.workunit.client.0.smithi176.stdout:0/47: fdatasync d2/d5/f8 0 2024-03-13T14:49:32.118 INFO:tasks.workunit.client.0.smithi176.stdout:6/50: creat d0/de/f12 x:0 0 0 2024-03-13T14:49:32.119 INFO:tasks.workunit.client.1.smithi196.stdout:6/321: stat d6/d9/d17/d19/f22 0 2024-03-13T14:49:32.122 INFO:tasks.workunit.client.1.smithi196.stdout:6/322: stat c1 0 2024-03-13T14:49:32.122 INFO:tasks.workunit.client.1.smithi196.stdout:3/220: dwrite db/d19/d22/f39 [0,4194304] 0 2024-03-13T14:49:32.122 INFO:tasks.workunit.client.1.smithi196.stdout:6/323: chown d6/d2a/d23 275014 1 2024-03-13T14:49:32.123 INFO:tasks.workunit.client.1.smithi196.stdout:3/221: stat db/d19/d4a 0 2024-03-13T14:49:32.124 INFO:tasks.workunit.client.1.smithi196.stdout:3/222: fdatasync db/f32 0 2024-03-13T14:49:32.133 INFO:tasks.workunit.client.1.smithi196.stdout:5/320: getdents d4/db/d14 0 2024-03-13T14:49:32.135 INFO:tasks.workunit.client.0.smithi176.stdout:2/62: symlink d1/l1b 0 2024-03-13T14:49:32.135 INFO:tasks.workunit.client.1.smithi196.stdout:5/321: chown d4/db/d4a/f5f 111057859 1 2024-03-13T14:49:32.136 INFO:tasks.workunit.client.1.smithi196.stdout:4/214: sync 2024-03-13T14:49:32.137 INFO:tasks.workunit.client.0.smithi176.stdout:2/63: fsync d1/f8 0 2024-03-13T14:49:32.137 INFO:tasks.workunit.client.1.smithi196.stdout:4/215: read d1/f22 [1302416,1923] 0 2024-03-13T14:49:32.138 INFO:tasks.workunit.client.0.smithi176.stdout:5/54: dwrite d5/f7 [0,4194304] 0 2024-03-13T14:49:32.139 INFO:tasks.workunit.client.1.smithi196.stdout:4/216: dread - d1/d3/f36 zero size 2024-03-13T14:49:32.140 INFO:tasks.workunit.client.0.smithi176.stdout:2/64: chown d1/l10 137869 1 2024-03-13T14:49:32.141 INFO:tasks.workunit.client.1.smithi196.stdout:6/324: creat d6/d2a/d23/d28/f69 x:0 0 0 2024-03-13T14:49:32.142 INFO:tasks.workunit.client.0.smithi176.stdout:2/65: chown d1/fb 179503936 1 2024-03-13T14:49:32.146 INFO:tasks.workunit.client.0.smithi176.stdout:3/65: symlink d4/l15 0 2024-03-13T14:49:32.148 INFO:tasks.workunit.client.0.smithi176.stdout:3/66: write d4/fa [1506991,7722] 0 2024-03-13T14:49:32.149 INFO:tasks.workunit.client.1.smithi196.stdout:5/322: dread d4/db/f24 [0,4194304] 0 2024-03-13T14:49:32.150 INFO:tasks.workunit.client.1.smithi196.stdout:9/219: mknod d7/de/d16/d3c/c3d 0 2024-03-13T14:49:32.153 INFO:tasks.workunit.client.1.smithi196.stdout:0/315: truncate d8/da/d25/d26/d35/d3b/d5f/f4a 242062 0 2024-03-13T14:49:32.157 INFO:tasks.workunit.client.1.smithi196.stdout:5/323: fsync d4/db/d14/d19/d1c/d25/f49 0 2024-03-13T14:49:32.168 INFO:tasks.workunit.client.0.smithi176.stdout:0/48: mkdir d2/db 0 2024-03-13T14:49:32.169 INFO:tasks.workunit.client.0.smithi176.stdout:6/51: rmdir d0 39 2024-03-13T14:49:32.170 INFO:tasks.workunit.client.1.smithi196.stdout:3/223: mkdir db/d4c 0 2024-03-13T14:49:32.170 INFO:tasks.workunit.client.0.smithi176.stdout:4/68: truncate d7/d8/f11 408091 0 2024-03-13T14:49:32.170 INFO:tasks.workunit.client.0.smithi176.stdout:0/49: fdatasync d2/f3 0 2024-03-13T14:49:32.170 INFO:tasks.workunit.client.0.smithi176.stdout:0/50: chown d2/db 107617883 1 2024-03-13T14:49:32.171 INFO:tasks.workunit.client.0.smithi176.stdout:0/51: fdatasync d2/f6 0 2024-03-13T14:49:32.172 INFO:tasks.workunit.client.0.smithi176.stdout:0/52: readlink d2/l4 0 2024-03-13T14:49:32.173 INFO:tasks.workunit.client.1.smithi196.stdout:4/217: rename d1/d2f to d1/d1e/d39 0 2024-03-13T14:49:32.174 INFO:tasks.workunit.client.1.smithi196.stdout:8/187: dwrite d0/d1c/d3b/f1f [0,4194304] 0 2024-03-13T14:49:32.180 INFO:tasks.workunit.client.0.smithi176.stdout:8/51: symlink d0/la 0 2024-03-13T14:49:32.182 INFO:tasks.workunit.client.1.smithi196.stdout:6/325: fsync d6/d9/d17/d19/f20 0 2024-03-13T14:49:32.184 INFO:tasks.workunit.client.1.smithi196.stdout:6/326: readlink d6/d9/d17/d19/l43 0 2024-03-13T14:49:32.186 INFO:tasks.workunit.client.1.smithi196.stdout:0/316: dwrite d8/da/d25/d26/d35/d3b/d6c/f6e [0,4194304] 0 2024-03-13T14:49:32.189 INFO:tasks.workunit.client.1.smithi196.stdout:6/327: dread - d6/d2a/dc/d47/f55 zero size 2024-03-13T14:49:32.193 INFO:tasks.workunit.client.0.smithi176.stdout:0/53: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:32.196 INFO:tasks.workunit.client.1.smithi196.stdout:2/140: sync 2024-03-13T14:49:32.196 INFO:tasks.workunit.client.1.smithi196.stdout:1/221: sync 2024-03-13T14:49:32.199 INFO:tasks.workunit.client.0.smithi176.stdout:8/52: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:32.200 INFO:tasks.workunit.client.0.smithi176.stdout:7/67: rename d1/d2/de to d1/d13 0 2024-03-13T14:49:32.200 INFO:tasks.workunit.client.1.smithi196.stdout:1/222: read - d1/f26 zero size 2024-03-13T14:49:32.201 INFO:tasks.workunit.client.1.smithi196.stdout:4/218: dwrite d1/d3/d7/f1a [4194304,4194304] 0 2024-03-13T14:49:32.202 INFO:tasks.workunit.client.0.smithi176.stdout:7/68: readlink d1/d7/lb 0 2024-03-13T14:49:32.203 INFO:tasks.workunit.client.0.smithi176.stdout:0/54: write d2/f3 [3490446,87544] 0 2024-03-13T14:49:32.205 INFO:tasks.workunit.client.0.smithi176.stdout:8/53: read d0/f3 [478575,30445] 0 2024-03-13T14:49:32.206 INFO:tasks.workunit.client.0.smithi176.stdout:0/55: write d2/f6 [2431519,33874] 0 2024-03-13T14:49:32.210 INFO:tasks.workunit.client.0.smithi176.stdout:1/42: dwrite d2/f9 [0,4194304] 0 2024-03-13T14:49:32.212 INFO:tasks.workunit.client.0.smithi176.stdout:1/43: chown d2/de/cf 0 1 2024-03-13T14:49:32.221 INFO:tasks.workunit.client.1.smithi196.stdout:8/188: write d0/d10/f31 [3622611,9205] 0 2024-03-13T14:49:32.225 INFO:tasks.workunit.client.1.smithi196.stdout:0/317: creat d8/da/d25/d26/d35/d3b/d50/d55/f72 x:0 0 0 2024-03-13T14:49:32.226 INFO:tasks.workunit.client.1.smithi196.stdout:6/328: mknod d6/d9/d17/d19/d4e/c6a 0 2024-03-13T14:49:32.229 INFO:tasks.workunit.client.1.smithi196.stdout:6/329: chown d6/c45 291457 1 2024-03-13T14:49:32.229 INFO:tasks.workunit.client.0.smithi176.stdout:6/52: dwrite d0/de/f12 [0,4194304] 0 2024-03-13T14:49:32.245 INFO:tasks.workunit.client.1.smithi196.stdout:1/223: unlink d1/d28/d34/f3a 0 2024-03-13T14:49:32.247 INFO:tasks.workunit.client.0.smithi176.stdout:7/69: rmdir d1/d2 39 2024-03-13T14:49:32.255 INFO:tasks.workunit.client.1.smithi196.stdout:2/141: rename d0/d7/d9/f22 to d0/d6/d1d/f23 0 2024-03-13T14:49:32.255 INFO:tasks.workunit.client.0.smithi176.stdout:9/89: dwrite d0/f12 [0,4194304] 0 2024-03-13T14:49:32.258 INFO:tasks.workunit.client.0.smithi176.stdout:6/53: dwrite d0/fc [0,4194304] 0 2024-03-13T14:49:32.259 INFO:tasks.workunit.client.1.smithi196.stdout:7/314: truncate d0/d9/d1e/d2d/f3b 1862209 0 2024-03-13T14:49:32.267 INFO:tasks.workunit.client.1.smithi196.stdout:5/324: link d4/l18 d4/db/d14/d19/d26/l65 0 2024-03-13T14:49:32.269 INFO:tasks.workunit.client.1.smithi196.stdout:5/325: chown d4/lc 0 1 2024-03-13T14:49:32.271 INFO:tasks.workunit.client.0.smithi176.stdout:0/56: mkdir d2/d5/dc 0 2024-03-13T14:49:32.271 INFO:tasks.workunit.client.1.smithi196.stdout:5/326: write d4/d1b/f29 [170766,129330] 0 2024-03-13T14:49:32.272 INFO:tasks.workunit.client.0.smithi176.stdout:0/57: write d2/f6 [4358183,67246] 0 2024-03-13T14:49:32.275 INFO:tasks.workunit.client.0.smithi176.stdout:4/69: write d7/fe [2618930,14683] 0 2024-03-13T14:49:32.275 INFO:tasks.workunit.client.0.smithi176.stdout:1/44: mkdir d2/de/d10 0 2024-03-13T14:49:32.276 INFO:tasks.workunit.client.0.smithi176.stdout:0/58: write d2/f3 [4658887,69892] 0 2024-03-13T14:49:32.276 INFO:tasks.workunit.client.0.smithi176.stdout:1/45: dread - d2/f5 zero size 2024-03-13T14:49:32.277 INFO:tasks.workunit.client.0.smithi176.stdout:0/59: chown d2/db 248 1 2024-03-13T14:49:32.280 INFO:tasks.workunit.client.1.smithi196.stdout:1/224: stat d1/d28/f2d 0 2024-03-13T14:49:32.281 INFO:tasks.workunit.client.0.smithi176.stdout:0/60: read d2/f3 [2763871,82771] 0 2024-03-13T14:49:32.282 INFO:tasks.workunit.client.0.smithi176.stdout:0/61: dread - d2/d5/f8 zero size 2024-03-13T14:49:32.283 INFO:tasks.workunit.client.1.smithi196.stdout:1/225: write d1/f2a [40179,25525] 0 2024-03-13T14:49:32.291 INFO:tasks.workunit.client.1.smithi196.stdout:6/330: rename d6/c45 to d6/d2a/dc/c6b 0 2024-03-13T14:49:32.302 INFO:tasks.workunit.client.1.smithi196.stdout:1/226: dwrite d1/d29/f44 [0,4194304] 0 2024-03-13T14:49:32.303 INFO:tasks.workunit.client.1.smithi196.stdout:3/224: truncate db/d19/f2b 988201 0 2024-03-13T14:49:32.306 INFO:tasks.workunit.client.1.smithi196.stdout:8/189: creat d0/d10/d12/d17/d40/f43 x:0 0 0 2024-03-13T14:49:32.307 INFO:tasks.workunit.client.1.smithi196.stdout:5/327: unlink d4/db/d14/d19/d1c/d25/d38/f46 0 2024-03-13T14:49:32.309 INFO:tasks.workunit.client.1.smithi196.stdout:0/318: getdents d8/da/d25/d26/d35/d3b/d5f/d33/d44 0 2024-03-13T14:49:32.312 INFO:tasks.workunit.client.1.smithi196.stdout:0/319: write d8/da/f10 [5034808,454] 0 2024-03-13T14:49:32.316 INFO:tasks.workunit.client.1.smithi196.stdout:4/219: creat d1/d3/f3a x:0 0 0 2024-03-13T14:49:32.330 INFO:tasks.workunit.client.1.smithi196.stdout:7/315: mkdir d0/d9/d1e/d58/d61 0 2024-03-13T14:49:32.331 INFO:tasks.workunit.client.1.smithi196.stdout:5/328: dwrite d4/db/d3c/f52 [0,4194304] 0 2024-03-13T14:49:32.333 INFO:tasks.workunit.client.1.smithi196.stdout:7/316: stat d0/d9/d1e 0 2024-03-13T14:49:32.334 INFO:tasks.workunit.client.1.smithi196.stdout:6/331: dread d6/d2a/d23/f4c [0,4194304] 0 2024-03-13T14:49:32.343 INFO:tasks.workunit.client.1.smithi196.stdout:8/190: chown d0/d10/d12/c20 25 1 2024-03-13T14:49:32.349 INFO:tasks.workunit.client.1.smithi196.stdout:8/191: write d0/d1c/d3b/f1f [1577578,111183] 0 2024-03-13T14:49:32.349 INFO:tasks.workunit.client.1.smithi196.stdout:4/220: symlink d1/d1e/d39/l3b 0 2024-03-13T14:49:32.350 INFO:tasks.workunit.client.1.smithi196.stdout:2/142: creat d0/d7/d9/f24 x:0 0 0 2024-03-13T14:49:32.354 INFO:tasks.workunit.client.0.smithi176.stdout:5/55: truncate d5/f7 538556 0 2024-03-13T14:49:32.356 INFO:tasks.workunit.client.0.smithi176.stdout:5/56: readlink d5/lc 0 2024-03-13T14:49:32.356 INFO:tasks.workunit.client.1.smithi196.stdout:2/143: write d0/d7/d9/f24 [275106,99177] 0 2024-03-13T14:49:32.357 INFO:tasks.workunit.client.0.smithi176.stdout:0/62: readlink d2/l4 0 2024-03-13T14:49:32.359 INFO:tasks.workunit.client.0.smithi176.stdout:5/57: chown d5/f8 18420 1 2024-03-13T14:49:32.359 INFO:tasks.workunit.client.0.smithi176.stdout:7/70: chown c0 3187 1 2024-03-13T14:49:32.360 INFO:tasks.workunit.client.0.smithi176.stdout:6/54: creat d0/f13 x:0 0 0 2024-03-13T14:49:32.361 INFO:tasks.workunit.client.0.smithi176.stdout:8/54: mkdir d0/db 0 2024-03-13T14:49:32.362 INFO:tasks.workunit.client.1.smithi196.stdout:6/332: dread d6/d4b/d50/f5a [0,4194304] 0 2024-03-13T14:49:32.362 INFO:tasks.workunit.client.1.smithi196.stdout:5/329: dwrite d4/db/d14/f5e [0,4194304] 0 2024-03-13T14:49:32.362 INFO:tasks.workunit.client.1.smithi196.stdout:1/227: creat d1/f47 x:0 0 0 2024-03-13T14:49:32.368 INFO:tasks.workunit.client.0.smithi176.stdout:1/46: dread d2/fc [0,4194304] 0 2024-03-13T14:49:32.369 INFO:tasks.workunit.client.1.smithi196.stdout:3/225: write db/d19/f28 [903831,37549] 0 2024-03-13T14:49:32.373 INFO:tasks.workunit.client.1.smithi196.stdout:0/320: dwrite d8/da/d25/d26/f60 [0,4194304] 0 2024-03-13T14:49:32.378 INFO:tasks.workunit.client.1.smithi196.stdout:6/333: creat d6/d9/d17/f6c x:0 0 0 2024-03-13T14:49:32.379 INFO:tasks.workunit.client.0.smithi176.stdout:1/47: readlink d2/lb 0 2024-03-13T14:49:32.381 INFO:tasks.workunit.client.1.smithi196.stdout:6/334: read d6/d7/df/f2e [2772108,15919] 0 2024-03-13T14:49:32.382 INFO:tasks.workunit.client.1.smithi196.stdout:5/330: creat d4/db/f66 x:0 0 0 2024-03-13T14:49:32.383 INFO:tasks.workunit.client.1.smithi196.stdout:6/335: fdatasync d6/d2a/dc/d1c/f25 0 2024-03-13T14:49:32.385 INFO:tasks.workunit.client.1.smithi196.stdout:6/336: read - d6/d7/df/d27/f64 zero size 2024-03-13T14:49:32.387 INFO:tasks.workunit.client.0.smithi176.stdout:8/55: dread d0/f9 [0,4194304] 0 2024-03-13T14:49:32.387 INFO:tasks.workunit.client.0.smithi176.stdout:7/71: dread d1/d7/f10 [0,4194304] 0 2024-03-13T14:49:32.388 INFO:tasks.workunit.client.1.smithi196.stdout:6/337: fdatasync d6/d2a/dc/f3e 0 2024-03-13T14:49:32.388 INFO:tasks.workunit.client.1.smithi196.stdout:3/226: mknod db/d19/d22/d30/c4d 0 2024-03-13T14:49:32.390 INFO:tasks.workunit.client.1.smithi196.stdout:6/338: chown d6/d2a/f62 29881110 1 2024-03-13T14:49:32.392 INFO:tasks.workunit.client.1.smithi196.stdout:6/339: chown d6/d9/f30 3 1 2024-03-13T14:49:32.392 INFO:tasks.workunit.client.1.smithi196.stdout:7/317: dread d0/d6/f45 [0,4194304] 0 2024-03-13T14:49:32.399 INFO:tasks.workunit.client.1.smithi196.stdout:5/331: write d4/db/d14/d19/d26/f43 [5070860,20456] 0 2024-03-13T14:49:32.401 INFO:tasks.workunit.client.0.smithi176.stdout:2/66: sync 2024-03-13T14:49:32.402 INFO:tasks.workunit.client.1.smithi196.stdout:5/332: chown d4/db/d14/d19/c53 81316023 1 2024-03-13T14:49:32.404 INFO:tasks.workunit.client.1.smithi196.stdout:5/333: write d4/db/d3c/f52 [2589448,57038] 0 2024-03-13T14:49:32.405 INFO:tasks.workunit.client.1.smithi196.stdout:2/144: creat d0/f25 x:0 0 0 2024-03-13T14:49:32.410 INFO:tasks.workunit.client.1.smithi196.stdout:0/321: unlink d8/da/d25/c40 0 2024-03-13T14:49:32.412 INFO:tasks.workunit.client.1.smithi196.stdout:6/340: dread d6/d7/df/f3a [0,4194304] 0 2024-03-13T14:49:32.413 INFO:tasks.workunit.client.1.smithi196.stdout:0/322: write d8/da/fc [2430777,98748] 0 2024-03-13T14:49:32.417 INFO:tasks.workunit.client.1.smithi196.stdout:6/341: write d6/d2a/d23/d28/f3f [556798,14915] 0 2024-03-13T14:49:32.419 INFO:tasks.workunit.client.1.smithi196.stdout:6/342: write d6/d7/df/f35 [1207097,110722] 0 2024-03-13T14:49:32.421 INFO:tasks.workunit.client.1.smithi196.stdout:6/343: chown d6/d7/df/f3a 8 1 2024-03-13T14:49:32.421 INFO:tasks.workunit.client.1.smithi196.stdout:7/318: dread d0/f37 [0,4194304] 0 2024-03-13T14:49:32.431 INFO:tasks.workunit.client.1.smithi196.stdout:5/334: creat d4/db/d14/d19/f67 x:0 0 0 2024-03-13T14:49:32.443 INFO:tasks.workunit.client.1.smithi196.stdout:7/319: creat d0/d6/d33/f62 x:0 0 0 2024-03-13T14:49:32.445 INFO:tasks.workunit.client.1.smithi196.stdout:6/344: creat d6/d2a/d23/d28/f6d x:0 0 0 2024-03-13T14:49:32.446 INFO:tasks.workunit.client.1.smithi196.stdout:3/227: creat db/f4e x:0 0 0 2024-03-13T14:49:32.447 INFO:tasks.workunit.client.0.smithi176.stdout:3/67: sync 2024-03-13T14:49:32.448 INFO:tasks.workunit.client.1.smithi196.stdout:2/145: mknod d0/d7/d9/c26 0 2024-03-13T14:49:32.449 INFO:tasks.workunit.client.1.smithi196.stdout:5/335: mkdir d4/db/d14/d19/d26/d68 0 2024-03-13T14:49:32.452 INFO:tasks.workunit.client.1.smithi196.stdout:0/323: truncate d8/da/f4e 3848471 0 2024-03-13T14:49:32.453 INFO:tasks.workunit.client.1.smithi196.stdout:7/320: creat d0/d4d/f63 x:0 0 0 2024-03-13T14:49:32.454 INFO:tasks.workunit.client.1.smithi196.stdout:7/321: chown d0/f11 0 1 2024-03-13T14:49:32.455 INFO:tasks.workunit.client.1.smithi196.stdout:6/345: unlink d6/d2a/d23/l26 0 2024-03-13T14:49:32.456 INFO:tasks.workunit.client.1.smithi196.stdout:3/228: fsync db/d19/f2f 0 2024-03-13T14:49:32.462 INFO:tasks.workunit.client.1.smithi196.stdout:5/336: truncate d4/db/d14/d19/d1c/f23 2019441 0 2024-03-13T14:49:32.462 INFO:tasks.workunit.client.1.smithi196.stdout:2/146: write d0/fe [2655472,58308] 0 2024-03-13T14:49:32.466 INFO:tasks.workunit.client.1.smithi196.stdout:5/337: read d4/db/d3c/f59 [4016271,23414] 0 2024-03-13T14:49:32.468 INFO:tasks.workunit.client.0.smithi176.stdout:3/68: dread d4/f5 [0,4194304] 0 2024-03-13T14:49:32.472 INFO:tasks.workunit.client.1.smithi196.stdout:6/346: creat d6/d9/d17/d19/d4e/f6e x:0 0 0 2024-03-13T14:49:32.473 INFO:tasks.workunit.client.1.smithi196.stdout:6/347: chown d6/f8 5246178 1 2024-03-13T14:49:32.475 INFO:tasks.workunit.client.1.smithi196.stdout:7/322: dwrite d0/d4d/f63 [0,4194304] 0 2024-03-13T14:49:32.476 INFO:tasks.workunit.client.1.smithi196.stdout:3/229: rename db/d19/d22/d47 to db/d4c/d4f 0 2024-03-13T14:49:32.494 INFO:tasks.workunit.client.1.smithi196.stdout:7/323: dread d0/fa [0,4194304] 0 2024-03-13T14:49:32.495 INFO:tasks.workunit.client.1.smithi196.stdout:2/147: link d0/d7/d9/f24 d0/d6/d1d/f27 0 2024-03-13T14:49:32.498 INFO:tasks.workunit.client.1.smithi196.stdout:5/338: symlink d4/db/d3c/l69 0 2024-03-13T14:49:32.502 INFO:tasks.workunit.client.1.smithi196.stdout:7/324: symlink d0/d9/d1e/l64 0 2024-03-13T14:49:32.504 INFO:tasks.workunit.client.1.smithi196.stdout:3/230: write db/d19/f2f [3551742,59975] 0 2024-03-13T14:49:32.510 INFO:tasks.workunit.client.1.smithi196.stdout:0/324: write d8/da/d25/d26/d35/d3b/d5f/d43/f45 [1303446,41259] 0 2024-03-13T14:49:32.512 INFO:tasks.workunit.client.1.smithi196.stdout:6/348: dwrite d6/fa [0,4194304] 0 2024-03-13T14:49:32.512 INFO:tasks.workunit.client.1.smithi196.stdout:2/148: mkdir d0/d6/d1d/d28 0 2024-03-13T14:49:32.520 INFO:tasks.workunit.client.1.smithi196.stdout:0/325: truncate d8/da/d25/f2f 2287902 0 2024-03-13T14:49:32.524 INFO:tasks.workunit.client.1.smithi196.stdout:7/325: rename d0/d9/d31/d46/d47/l5c to d0/d6/d33/l65 0 2024-03-13T14:49:32.530 INFO:tasks.workunit.client.1.smithi196.stdout:5/339: mkdir d4/db/d3c/d6a 0 2024-03-13T14:49:32.533 INFO:tasks.workunit.client.1.smithi196.stdout:0/326: rename d8/da/d25/d26/d35/d3b/d59/d6a/f6f to d8/d37/f73 0 2024-03-13T14:49:32.534 INFO:tasks.workunit.client.1.smithi196.stdout:2/149: dwrite d0/f1a [0,4194304] 0 2024-03-13T14:49:32.535 INFO:tasks.workunit.client.0.smithi176.stdout:9/90: sync 2024-03-13T14:49:32.538 INFO:tasks.workunit.client.1.smithi196.stdout:3/231: mkdir db/d19/d50 0 2024-03-13T14:49:32.541 INFO:tasks.workunit.client.1.smithi196.stdout:7/326: creat d0/d9/d1e/d2d/f66 x:0 0 0 2024-03-13T14:49:32.556 INFO:tasks.workunit.client.1.smithi196.stdout:5/340: creat d4/db/d14/d19/d1c/f6b x:0 0 0 2024-03-13T14:49:32.557 INFO:tasks.workunit.client.1.smithi196.stdout:2/150: fdatasync d0/d6/f14 0 2024-03-13T14:49:32.558 INFO:tasks.workunit.client.1.smithi196.stdout:5/341: stat d4/db/d14/d19/f5a 0 2024-03-13T14:49:32.558 INFO:tasks.workunit.client.1.smithi196.stdout:7/327: dread d0/d4d/f63 [0,4194304] 0 2024-03-13T14:49:32.560 INFO:tasks.workunit.client.1.smithi196.stdout:5/342: chown d4/db 1028363 1 2024-03-13T14:49:32.562 INFO:tasks.workunit.client.1.smithi196.stdout:5/343: dread - d4/db/f66 zero size 2024-03-13T14:49:32.566 INFO:tasks.workunit.client.1.smithi196.stdout:0/327: mkdir d8/da/d74 0 2024-03-13T14:49:32.568 INFO:tasks.workunit.client.1.smithi196.stdout:3/232: mkdir db/d15/d44/d51 0 2024-03-13T14:49:32.568 INFO:tasks.workunit.client.1.smithi196.stdout:2/151: dread d0/d6/f14 [0,4194304] 0 2024-03-13T14:49:32.574 INFO:tasks.workunit.client.1.smithi196.stdout:2/152: dread d0/d7/d9/f24 [0,4194304] 0 2024-03-13T14:49:32.587 INFO:tasks.workunit.client.1.smithi196.stdout:0/328: rmdir d8/da/d25/d26/d35/d3b/d5f/d17 39 2024-03-13T14:49:32.590 INFO:tasks.workunit.client.1.smithi196.stdout:6/349: truncate f4 356982 0 2024-03-13T14:49:32.593 INFO:tasks.workunit.client.1.smithi196.stdout:6/350: fsync d6/d2a/d23/d28/f2f 0 2024-03-13T14:49:32.596 INFO:tasks.workunit.client.1.smithi196.stdout:6/351: chown d6/d4b/l5b 131069 1 2024-03-13T14:49:32.600 INFO:tasks.workunit.client.1.smithi196.stdout:5/344: symlink d4/l6c 0 2024-03-13T14:49:32.602 INFO:tasks.workunit.client.0.smithi176.stdout:4/70: link d7/d10/l14 d7/d8/l1b 0 2024-03-13T14:49:32.606 INFO:tasks.workunit.client.1.smithi196.stdout:0/329: write d8/d37/f5b [3317958,9420] 0 2024-03-13T14:49:32.607 INFO:tasks.workunit.client.1.smithi196.stdout:0/330: fdatasync d8/da/f5c 0 2024-03-13T14:49:32.608 INFO:tasks.workunit.client.1.smithi196.stdout:2/153: symlink d0/d6/l29 0 2024-03-13T14:49:32.611 INFO:tasks.workunit.client.0.smithi176.stdout:7/72: creat d1/d13/f14 x:0 0 0 2024-03-13T14:49:32.612 INFO:tasks.workunit.client.1.smithi196.stdout:0/331: write d8/da/d25/d26/d35/d3b/d5f/d33/f34 [373471,129208] 0 2024-03-13T14:49:32.615 INFO:tasks.workunit.client.1.smithi196.stdout:0/332: stat d8/da/d25/d26/d35/d3b/d59 0 2024-03-13T14:49:32.617 INFO:tasks.workunit.client.0.smithi176.stdout:6/55: creat d0/d9/f14 x:0 0 0 2024-03-13T14:49:32.617 INFO:tasks.workunit.client.1.smithi196.stdout:2/154: symlink d0/d6/d1d/l2a 0 2024-03-13T14:49:32.618 INFO:tasks.workunit.client.1.smithi196.stdout:2/155: stat d0/d6/d1d/l2a 0 2024-03-13T14:49:32.619 INFO:tasks.workunit.client.0.smithi176.stdout:0/63: rmdir d2/d5/da 0 2024-03-13T14:49:32.619 INFO:tasks.workunit.client.0.smithi176.stdout:6/56: write d0/f13 [73472,108215] 0 2024-03-13T14:49:32.620 INFO:tasks.workunit.client.1.smithi196.stdout:0/333: truncate d8/da/d25/d26/d35/d3b/d50/d5d/f57 171024 0 2024-03-13T14:49:32.622 INFO:tasks.workunit.client.0.smithi176.stdout:2/67: mkdir d1/dc/d1c 0 2024-03-13T14:49:32.623 INFO:tasks.workunit.client.0.smithi176.stdout:6/57: stat d0/d10/l11 0 2024-03-13T14:49:32.623 INFO:tasks.workunit.client.1.smithi196.stdout:7/328: getdents d0/d9/d31/d50 0 2024-03-13T14:49:32.625 INFO:tasks.workunit.client.1.smithi196.stdout:3/233: write db/d19/d22/f3c [470855,30691] 0 2024-03-13T14:49:32.626 INFO:tasks.workunit.client.1.smithi196.stdout:6/352: mkdir d6/d6f 0 2024-03-13T14:49:32.627 INFO:tasks.workunit.client.0.smithi176.stdout:6/58: write d0/f13 [57301,107924] 0 2024-03-13T14:49:32.628 INFO:tasks.workunit.client.1.smithi196.stdout:6/353: write d6/d2a/f62 [358528,15778] 0 2024-03-13T14:49:32.628 INFO:tasks.workunit.client.1.smithi196.stdout:5/345: mknod d4/db/d14/d19/d1c/d25/c6d 0 2024-03-13T14:49:32.632 INFO:tasks.workunit.client.1.smithi196.stdout:0/334: fsync d8/f9 0 2024-03-13T14:49:32.632 INFO:tasks.workunit.client.1.smithi196.stdout:6/354: fsync d6/d7/df/f35 0 2024-03-13T14:49:32.632 INFO:tasks.workunit.client.0.smithi176.stdout:9/91: rmdir d0/d13 39 2024-03-13T14:49:32.634 INFO:tasks.workunit.client.0.smithi176.stdout:6/59: truncate d0/f13 1152636 0 2024-03-13T14:49:32.635 INFO:tasks.workunit.client.1.smithi196.stdout:7/329: unlink d0/d6/d35/f55 0 2024-03-13T14:49:32.636 INFO:tasks.workunit.client.0.smithi176.stdout:7/73: symlink d1/d13/l15 0 2024-03-13T14:49:32.636 INFO:tasks.workunit.client.1.smithi196.stdout:9/220: sync 2024-03-13T14:49:32.638 INFO:tasks.workunit.client.0.smithi176.stdout:6/60: write d0/d9/f14 [649337,114796] 0 2024-03-13T14:49:32.638 INFO:tasks.workunit.client.0.smithi176.stdout:1/48: dwrite d2/fc [0,4194304] 0 2024-03-13T14:49:32.638 INFO:tasks.workunit.client.1.smithi196.stdout:2/156: symlink d0/d7/d9/l2b 0 2024-03-13T14:49:32.639 INFO:tasks.workunit.client.0.smithi176.stdout:1/49: write d2/f6 [677071,15665] 0 2024-03-13T14:49:32.641 INFO:tasks.workunit.client.0.smithi176.stdout:8/56: mkdir d0/db/dc 0 2024-03-13T14:49:32.643 INFO:tasks.workunit.client.0.smithi176.stdout:0/64: mknod d2/d5/cd 0 2024-03-13T14:49:32.647 INFO:tasks.workunit.client.1.smithi196.stdout:3/234: dwrite db/f32 [0,4194304] 0 2024-03-13T14:49:32.648 INFO:tasks.workunit.client.1.smithi196.stdout:5/346: mknod d4/db/d14/d19/d1c/d2d/c6e 0 2024-03-13T14:49:32.652 INFO:tasks.workunit.client.1.smithi196.stdout:5/347: dread - d4/db/d14/d37/f4d zero size 2024-03-13T14:49:32.652 INFO:tasks.workunit.client.1.smithi196.stdout:6/355: write d6/d57/f63 [464598,39776] 0 2024-03-13T14:49:32.655 INFO:tasks.workunit.client.1.smithi196.stdout:7/330: chown d0/d9/d1e/d2d/f3b 7806526 1 2024-03-13T14:49:32.661 INFO:tasks.workunit.client.1.smithi196.stdout:0/335: rename d8/da/d25/d26/d35/f4f to d8/da/d25/d26/d35/d3b/d59/d6a/f75 0 2024-03-13T14:49:32.663 INFO:tasks.workunit.client.1.smithi196.stdout:9/221: creat d7/d24/d28/f3e x:0 0 0 2024-03-13T14:49:32.672 INFO:tasks.workunit.client.0.smithi176.stdout:9/92: dread d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:32.674 INFO:tasks.workunit.client.0.smithi176.stdout:5/58: getdents d5/de 0 2024-03-13T14:49:32.677 INFO:tasks.workunit.client.1.smithi196.stdout:3/235: symlink db/d37/l52 0 2024-03-13T14:49:32.680 INFO:tasks.workunit.client.0.smithi176.stdout:7/74: unlink d1/d13/c11 0 2024-03-13T14:49:32.681 INFO:tasks.workunit.client.1.smithi196.stdout:8/192: sync 2024-03-13T14:49:32.681 INFO:tasks.workunit.client.1.smithi196.stdout:1/228: sync 2024-03-13T14:49:32.681 INFO:tasks.workunit.client.1.smithi196.stdout:4/221: sync 2024-03-13T14:49:32.682 INFO:tasks.workunit.client.0.smithi176.stdout:5/59: dread d5/fa [0,4194304] 0 2024-03-13T14:49:32.687 INFO:tasks.workunit.client.1.smithi196.stdout:0/336: dwrite d8/da/d25/d26/d35/d3b/f47 [0,4194304] 0 2024-03-13T14:49:32.687 INFO:tasks.workunit.client.1.smithi196.stdout:6/356: creat d6/d9/d11/f70 x:0 0 0 2024-03-13T14:49:32.692 INFO:tasks.workunit.client.1.smithi196.stdout:7/331: fdatasync d0/f1c 0 2024-03-13T14:49:32.692 INFO:tasks.workunit.client.1.smithi196.stdout:6/357: fdatasync d6/d9/f30 0 2024-03-13T14:49:32.693 INFO:tasks.workunit.client.0.smithi176.stdout:1/50: mknod d2/de/c11 0 2024-03-13T14:49:32.693 INFO:tasks.workunit.client.1.smithi196.stdout:0/337: fsync d8/da/d25/d26/d35/d3b/d6c/f6e 0 2024-03-13T14:49:32.697 INFO:tasks.workunit.client.0.smithi176.stdout:8/57: creat d0/db/fd x:0 0 0 2024-03-13T14:49:32.704 INFO:tasks.workunit.client.1.smithi196.stdout:3/236: dwrite db/d4c/d4f/f48 [0,4194304] 0 2024-03-13T14:49:32.706 INFO:tasks.workunit.client.0.smithi176.stdout:2/68: creat d1/dc/d1c/f1d x:0 0 0 2024-03-13T14:49:32.711 INFO:tasks.workunit.client.0.smithi176.stdout:3/69: write f2 [1980418,116079] 0 2024-03-13T14:49:32.716 INFO:tasks.workunit.client.1.smithi196.stdout:3/237: dread db/d19/f26 [0,4194304] 0 2024-03-13T14:49:32.716 INFO:tasks.workunit.client.1.smithi196.stdout:9/222: creat d7/d24/f3f x:0 0 0 2024-03-13T14:49:32.717 INFO:tasks.workunit.client.1.smithi196.stdout:2/157: mknod d0/d6/c2c 0 2024-03-13T14:49:32.717 INFO:tasks.workunit.client.0.smithi176.stdout:6/61: symlink d0/df/l15 0 2024-03-13T14:49:32.719 INFO:tasks.workunit.client.1.smithi196.stdout:8/193: rmdir d0/d10/d12/d17 39 2024-03-13T14:49:32.731 INFO:tasks.workunit.client.0.smithi176.stdout:2/69: symlink d1/dc/l1e 0 2024-03-13T14:49:32.736 INFO:tasks.workunit.client.1.smithi196.stdout:6/358: mkdir d6/d2a/dc/d71 0 2024-03-13T14:49:32.739 INFO:tasks.workunit.client.1.smithi196.stdout:2/158: dwrite d0/d7/d9/f20 [0,4194304] 0 2024-03-13T14:49:32.741 INFO:tasks.workunit.client.0.smithi176.stdout:4/71: getdents d7/d8 0 2024-03-13T14:49:32.743 INFO:tasks.workunit.client.1.smithi196.stdout:0/338: rmdir d8/da/d25/d26/d35/d3b/d59 39 2024-03-13T14:49:32.745 INFO:tasks.workunit.client.1.smithi196.stdout:8/194: dwrite d0/d10/d12/f3e [0,4194304] 0 2024-03-13T14:49:32.746 INFO:tasks.workunit.client.0.smithi176.stdout:9/93: mkdir d0/d1e/d1f 0 2024-03-13T14:49:32.755 INFO:tasks.workunit.client.0.smithi176.stdout:0/65: truncate d2/f6 3216256 0 2024-03-13T14:49:32.761 INFO:tasks.workunit.client.0.smithi176.stdout:1/51: mknod d2/de/d10/c12 0 2024-03-13T14:49:32.761 INFO:tasks.workunit.client.0.smithi176.stdout:6/62: symlink d0/d10/l16 0 2024-03-13T14:49:32.762 INFO:tasks.workunit.client.1.smithi196.stdout:5/348: creat d4/db/d14/d19/d26/d33/d61/f6f x:0 0 0 2024-03-13T14:49:32.764 INFO:tasks.workunit.client.1.smithi196.stdout:5/349: write d4/f8 [6130399,85144] 0 2024-03-13T14:49:32.765 INFO:tasks.workunit.client.0.smithi176.stdout:6/63: write d0/f13 [2062104,60185] 0 2024-03-13T14:49:32.768 INFO:tasks.workunit.client.0.smithi176.stdout:8/58: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:49:32.769 INFO:tasks.workunit.client.1.smithi196.stdout:3/238: rename db/d4c/d4f to db/d19/d22/d3a/d53 0 2024-03-13T14:49:32.769 INFO:tasks.workunit.client.1.smithi196.stdout:5/350: fdatasync d4/db/d14/d19/d1c/d25/d38/d47/f64 0 2024-03-13T14:49:32.770 INFO:tasks.workunit.client.0.smithi176.stdout:4/72: dread d7/fe [0,4194304] 0 2024-03-13T14:49:32.770 INFO:tasks.workunit.client.0.smithi176.stdout:8/59: truncate d0/f9 4423549 0 2024-03-13T14:49:32.774 INFO:tasks.workunit.client.0.smithi176.stdout:3/70: mkdir d4/d13/d16 0 2024-03-13T14:49:32.777 INFO:tasks.workunit.client.1.smithi196.stdout:7/332: dwrite d0/d9/d1e/f4b [0,4194304] 0 2024-03-13T14:49:32.778 INFO:tasks.workunit.client.1.smithi196.stdout:3/239: dread - db/d19/d22/d30/f42 zero size 2024-03-13T14:49:32.778 INFO:tasks.workunit.client.0.smithi176.stdout:2/70: rmdir d1 39 2024-03-13T14:49:32.783 INFO:tasks.workunit.client.0.smithi176.stdout:1/52: symlink d2/de/l13 0 2024-03-13T14:49:32.784 INFO:tasks.workunit.client.0.smithi176.stdout:1/53: chown d2/c7 183355 1 2024-03-13T14:49:32.786 INFO:tasks.workunit.client.0.smithi176.stdout:6/64: symlink d0/df/l17 0 2024-03-13T14:49:32.786 INFO:tasks.workunit.client.1.smithi196.stdout:3/240: dread db/d15/f41 [0,4194304] 0 2024-03-13T14:49:32.788 INFO:tasks.workunit.client.0.smithi176.stdout:1/54: write d2/fc [5082625,28044] 0 2024-03-13T14:49:32.790 INFO:tasks.workunit.client.1.smithi196.stdout:2/159: mknod d0/d6/d1d/c2d 0 2024-03-13T14:49:32.790 INFO:tasks.workunit.client.0.smithi176.stdout:8/60: mkdir d0/db/de 0 2024-03-13T14:49:32.796 INFO:tasks.workunit.client.1.smithi196.stdout:1/229: creat d1/d29/f48 x:0 0 0 2024-03-13T14:49:32.797 INFO:tasks.workunit.client.0.smithi176.stdout:6/65: creat d0/f18 x:0 0 0 2024-03-13T14:49:32.800 INFO:tasks.workunit.client.1.smithi196.stdout:8/195: rename d0/d10/l22 to d0/l44 0 2024-03-13T14:49:32.801 INFO:tasks.workunit.client.0.smithi176.stdout:1/55: mkdir d2/de/d10/d14 0 2024-03-13T14:49:32.804 INFO:tasks.workunit.client.0.smithi176.stdout:4/73: link d7/fc d7/f1c 0 2024-03-13T14:49:32.805 INFO:tasks.workunit.client.0.smithi176.stdout:4/74: dread - d7/fc zero size 2024-03-13T14:49:32.808 INFO:tasks.workunit.client.0.smithi176.stdout:8/61: creat d0/d5/ff x:0 0 0 2024-03-13T14:49:32.809 INFO:tasks.workunit.client.0.smithi176.stdout:8/62: dread - d0/d5/ff zero size 2024-03-13T14:49:32.809 INFO:tasks.workunit.client.1.smithi196.stdout:7/333: mkdir d0/d9/d31/d67 0 2024-03-13T14:49:32.814 INFO:tasks.workunit.client.1.smithi196.stdout:0/339: fdatasync d8/da/f4e 0 2024-03-13T14:49:32.815 INFO:tasks.workunit.client.1.smithi196.stdout:5/351: dwrite d4/db/d14/d19/f50 [0,4194304] 0 2024-03-13T14:49:32.815 INFO:tasks.workunit.client.1.smithi196.stdout:6/359: dwrite d6/d4b/f58 [0,4194304] 0 2024-03-13T14:49:32.821 INFO:tasks.workunit.client.1.smithi196.stdout:9/223: creat d7/de/f40 x:0 0 0 2024-03-13T14:49:32.823 INFO:tasks.workunit.client.1.smithi196.stdout:0/340: write d8/da/d25/d26/d35/d3b/d50/d55/f72 [941618,13040] 0 2024-03-13T14:49:32.828 INFO:tasks.workunit.client.1.smithi196.stdout:0/341: write d8/da/d25/d26/f60 [2821019,16886] 0 2024-03-13T14:49:32.829 INFO:tasks.workunit.client.0.smithi176.stdout:1/56: dwrite d2/f9 [0,4194304] 0 2024-03-13T14:49:32.829 INFO:tasks.workunit.client.0.smithi176.stdout:6/66: dwrite d0/de/f12 [0,4194304] 0 2024-03-13T14:49:32.836 INFO:tasks.workunit.client.0.smithi176.stdout:4/75: mknod d7/d8/c1d 0 2024-03-13T14:49:32.842 INFO:tasks.workunit.client.0.smithi176.stdout:4/76: chown d7/d8/d15 1 1 2024-03-13T14:49:32.844 INFO:tasks.workunit.client.0.smithi176.stdout:9/94: dread d0/d13/f16 [0,4194304] 0 2024-03-13T14:49:32.844 INFO:tasks.workunit.client.0.smithi176.stdout:4/77: write d7/d8/fd [1957175,25287] 0 2024-03-13T14:49:32.846 INFO:tasks.workunit.client.1.smithi196.stdout:4/222: truncate d1/d3/d7/f1d 1520356 0 2024-03-13T14:49:32.849 INFO:tasks.workunit.client.1.smithi196.stdout:3/241: mknod db/d19/d50/c54 0 2024-03-13T14:49:32.854 INFO:tasks.workunit.client.0.smithi176.stdout:6/67: dread d0/de/f12 [0,4194304] 0 2024-03-13T14:49:32.857 INFO:tasks.workunit.client.0.smithi176.stdout:8/63: getdents d0/db/de 0 2024-03-13T14:49:32.859 INFO:tasks.workunit.client.0.smithi176.stdout:8/64: write d0/d5/ff [782157,104007] 0 2024-03-13T14:49:32.861 INFO:tasks.workunit.client.0.smithi176.stdout:8/65: chown d0/f9 64642248 1 2024-03-13T14:49:32.866 INFO:tasks.workunit.client.0.smithi176.stdout:4/78: symlink d7/d8/l1e 0 2024-03-13T14:49:32.869 INFO:tasks.workunit.client.1.smithi196.stdout:2/160: mknod d0/c2e 0 2024-03-13T14:49:32.870 INFO:tasks.workunit.client.1.smithi196.stdout:2/161: fdatasync d0/fe 0 2024-03-13T14:49:32.871 INFO:tasks.workunit.client.1.smithi196.stdout:2/162: readlink d0/d7/l1e 0 2024-03-13T14:49:32.874 INFO:tasks.workunit.client.0.smithi176.stdout:8/66: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:49:32.874 INFO:tasks.workunit.client.0.smithi176.stdout:6/68: mkdir d0/d10/d19 0 2024-03-13T14:49:32.876 INFO:tasks.workunit.client.0.smithi176.stdout:6/69: write d0/f13 [2965248,41256] 0 2024-03-13T14:49:32.881 INFO:tasks.workunit.client.0.smithi176.stdout:9/95: symlink d0/d1e/d1f/l20 0 2024-03-13T14:49:32.887 INFO:tasks.workunit.client.1.smithi196.stdout:5/352: fdatasync d4/db/f24 0 2024-03-13T14:49:32.890 INFO:tasks.workunit.client.1.smithi196.stdout:5/353: stat d4/db/d14/c1e 0 2024-03-13T14:49:32.892 INFO:tasks.workunit.client.1.smithi196.stdout:5/354: fdatasync d4/db/d14/f5e 0 2024-03-13T14:49:32.894 INFO:tasks.workunit.client.1.smithi196.stdout:0/342: write d8/d37/f73 [91710,128800] 0 2024-03-13T14:49:32.896 INFO:tasks.workunit.client.0.smithi176.stdout:9/96: dwrite d0/d13/f16 [0,4194304] 0 2024-03-13T14:49:32.899 INFO:tasks.workunit.client.1.smithi196.stdout:8/196: link d0/d1c/d29/f3a d0/d1c/d3b/d3f/f45 0 2024-03-13T14:49:32.906 INFO:tasks.workunit.client.0.smithi176.stdout:6/70: rmdir d0/df 39 2024-03-13T14:49:32.910 INFO:tasks.workunit.client.0.smithi176.stdout:9/97: dread d0/f12 [0,4194304] 0 2024-03-13T14:49:32.915 INFO:tasks.workunit.client.1.smithi196.stdout:1/230: write d1/f31 [1748414,91182] 0 2024-03-13T14:49:32.918 INFO:tasks.workunit.client.1.smithi196.stdout:7/334: mknod d0/d9/c68 0 2024-03-13T14:49:32.924 INFO:tasks.workunit.client.1.smithi196.stdout:6/360: mknod d6/d9/d17/d19/d4e/d66/c72 0 2024-03-13T14:49:32.925 INFO:tasks.workunit.client.0.smithi176.stdout:2/71: fsync d1/ff 0 2024-03-13T14:49:32.927 INFO:tasks.workunit.client.0.smithi176.stdout:6/71: dwrite d0/f18 [0,4194304] 0 2024-03-13T14:49:32.928 INFO:tasks.workunit.client.0.smithi176.stdout:2/72: chown d1/c18 1016 1 2024-03-13T14:49:32.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:32 smithi196 ceph-mon[37486]: pgmap v5: 65 pgs: 65 active+clean; 534 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:49:32.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:32 smithi196 ceph-mon[37486]: mgrmap e29: smithi176.tzfdwd(active, since 4s) 2024-03-13T14:49:32.941 INFO:tasks.workunit.client.1.smithi196.stdout:6/361: dread d6/d2a/d23/f4c [0,4194304] 0 2024-03-13T14:49:32.943 INFO:tasks.workunit.client.0.smithi176.stdout:9/98: symlink d0/l21 0 2024-03-13T14:49:32.944 INFO:tasks.workunit.client.0.smithi176.stdout:9/99: chown d0/db/l11 39 1 2024-03-13T14:49:32.946 INFO:tasks.workunit.client.0.smithi176.stdout:2/73: symlink d1/dc/l1f 0 2024-03-13T14:49:32.949 INFO:tasks.workunit.client.1.smithi196.stdout:1/231: symlink d1/d28/l49 0 2024-03-13T14:49:32.953 INFO:tasks.workunit.client.0.smithi176.stdout:8/67: truncate d0/d5/ff 864937 0 2024-03-13T14:49:32.956 INFO:tasks.workunit.client.0.smithi176.stdout:4/79: dwrite d7/d8/f11 [0,4194304] 0 2024-03-13T14:49:32.956 INFO:tasks.workunit.client.1.smithi196.stdout:2/163: symlink d0/l2f 0 2024-03-13T14:49:32.958 INFO:tasks.workunit.client.0.smithi176.stdout:8/68: dread d0/f2 [4194304,4194304] 0 2024-03-13T14:49:32.960 INFO:tasks.workunit.client.0.smithi176.stdout:2/74: mknod d1/dc/d1c/c20 0 2024-03-13T14:49:32.961 INFO:tasks.workunit.client.0.smithi176.stdout:2/75: fdatasync d1/ff 0 2024-03-13T14:49:32.964 INFO:tasks.workunit.client.1.smithi196.stdout:5/355: rename d4/c5b to d4/db/d14/d19/d26/d33/c70 0 2024-03-13T14:49:32.973 INFO:tasks.workunit.client.0.smithi176.stdout:7/75: sync 2024-03-13T14:49:32.973 INFO:tasks.workunit.client.0.smithi176.stdout:6/72: truncate d0/fc 357132 0 2024-03-13T14:49:32.974 INFO:tasks.workunit.client.0.smithi176.stdout:7/76: readlink d1/d7/lb 0 2024-03-13T14:49:32.976 INFO:tasks.workunit.client.0.smithi176.stdout:7/77: read d1/d7/f10 [2799406,68607] 0 2024-03-13T14:49:32.977 INFO:tasks.workunit.client.0.smithi176.stdout:7/78: chown d1/d7 0 1 2024-03-13T14:49:32.978 INFO:tasks.workunit.client.0.smithi176.stdout:7/79: dread - d1/d13/f14 zero size 2024-03-13T14:49:32.981 INFO:tasks.workunit.client.0.smithi176.stdout:7/80: write d1/d13/f14 [1033762,52381] 0 2024-03-13T14:49:32.982 INFO:tasks.workunit.client.1.smithi196.stdout:3/242: creat db/d19/f55 x:0 0 0 2024-03-13T14:49:32.984 INFO:tasks.workunit.client.1.smithi196.stdout:3/243: chown db/lc 0 1 2024-03-13T14:49:32.984 INFO:tasks.workunit.client.1.smithi196.stdout:1/232: mknod d1/d3f/c4a 0 2024-03-13T14:49:32.986 INFO:tasks.workunit.client.0.smithi176.stdout:4/80: symlink d7/d10/l1f 0 2024-03-13T14:49:32.987 INFO:tasks.workunit.client.1.smithi196.stdout:1/233: truncate d1/f37 256141 0 2024-03-13T14:49:32.987 INFO:tasks.workunit.client.0.smithi176.stdout:8/69: mknod d0/d5/c10 0 2024-03-13T14:49:32.987 INFO:tasks.workunit.client.0.smithi176.stdout:2/76: symlink d1/l21 0 2024-03-13T14:49:32.991 INFO:tasks.workunit.client.1.smithi196.stdout:7/335: mknod d0/d6/d33/d57/c69 0 2024-03-13T14:49:32.996 INFO:tasks.workunit.client.1.smithi196.stdout:2/164: unlink d0/d7/d9/f20 0 2024-03-13T14:49:32.998 INFO:tasks.workunit.client.0.smithi176.stdout:9/100: truncate d0/d13/f16 3110844 0 2024-03-13T14:49:32.999 INFO:tasks.workunit.client.0.smithi176.stdout:4/81: creat d7/d10/f20 x:0 0 0 2024-03-13T14:49:33.000 INFO:tasks.workunit.client.0.smithi176.stdout:6/73: creat d0/d10/d19/f1a x:0 0 0 2024-03-13T14:49:33.002 INFO:tasks.workunit.client.1.smithi196.stdout:9/224: getdents d7/d24 0 2024-03-13T14:49:33.003 INFO:tasks.workunit.client.1.smithi196.stdout:0/343: rename d8/da/d25/d26/d35/d3b/d50/d5d/f31 to d8/da/d25/d26/f76 0 2024-03-13T14:49:33.004 INFO:tasks.workunit.client.0.smithi176.stdout:2/77: symlink d1/dc/d1c/l22 0 2024-03-13T14:49:33.006 INFO:tasks.workunit.client.0.smithi176.stdout:8/70: rename d0/db/de to d0/d5/d11 0 2024-03-13T14:49:33.008 INFO:tasks.workunit.client.1.smithi196.stdout:9/225: chown d7/d9/c2f 1 1 2024-03-13T14:49:33.011 INFO:tasks.workunit.client.0.smithi176.stdout:7/81: dwrite d1/d13/f14 [0,4194304] 0 2024-03-13T14:49:33.011 INFO:tasks.workunit.client.1.smithi196.stdout:0/344: write d8/da/d25/d26/d35/d3b/d6c/f6e [294716,33482] 0 2024-03-13T14:49:33.013 INFO:tasks.workunit.client.1.smithi196.stdout:6/362: mknod d6/c73 0 2024-03-13T14:49:33.021 INFO:tasks.workunit.client.1.smithi196.stdout:7/336: dwrite d0/d4d/f5b [0,4194304] 0 2024-03-13T14:49:33.022 INFO:tasks.workunit.client.0.smithi176.stdout:4/82: dread d7/d8/f18 [0,4194304] 0 2024-03-13T14:49:33.024 INFO:tasks.workunit.client.1.smithi196.stdout:7/337: dread - d0/d6/d33/f4c zero size 2024-03-13T14:49:33.024 INFO:tasks.workunit.client.1.smithi196.stdout:3/244: rmdir db/d15 39 2024-03-13T14:49:33.025 INFO:tasks.workunit.client.0.smithi176.stdout:4/83: chown d7/d8/d15/l1a 353399 1 2024-03-13T14:49:33.030 INFO:tasks.workunit.client.0.smithi176.stdout:2/78: dwrite d1/f8 [0,4194304] 0 2024-03-13T14:49:33.031 INFO:tasks.workunit.client.1.smithi196.stdout:1/234: dread d1/d28/f35 [0,4194304] 0 2024-03-13T14:49:33.033 INFO:tasks.workunit.client.1.smithi196.stdout:7/338: dread - d0/d9/d1e/d2d/f66 zero size 2024-03-13T14:49:33.034 INFO:tasks.workunit.client.1.smithi196.stdout:1/235: read - d1/d9/d33/f42 zero size 2024-03-13T14:49:33.036 INFO:tasks.workunit.client.1.smithi196.stdout:7/339: read d0/fa [3060228,13456] 0 2024-03-13T14:49:33.038 INFO:tasks.workunit.client.1.smithi196.stdout:7/340: truncate d0/f56 712078 0 2024-03-13T14:49:33.039 INFO:tasks.workunit.client.0.smithi176.stdout:3/71: sync 2024-03-13T14:49:33.039 INFO:tasks.workunit.client.0.smithi176.stdout:0/66: sync 2024-03-13T14:49:33.039 INFO:tasks.workunit.client.0.smithi176.stdout:5/60: sync 2024-03-13T14:49:33.039 INFO:tasks.workunit.client.0.smithi176.stdout:1/57: sync 2024-03-13T14:49:33.040 INFO:tasks.workunit.client.1.smithi196.stdout:7/341: fdatasync d0/d6/f1d 0 2024-03-13T14:49:33.041 INFO:tasks.workunit.client.0.smithi176.stdout:0/67: read - d2/d5/f8 zero size 2024-03-13T14:49:33.050 INFO:tasks.workunit.client.0.smithi176.stdout:6/74: symlink d0/df/l1b 0 2024-03-13T14:49:33.054 INFO:tasks.workunit.client.0.smithi176.stdout:6/75: write d0/f8 [6577997,92022] 0 2024-03-13T14:49:33.057 INFO:tasks.workunit.client.1.smithi196.stdout:2/165: unlink d0/c1b 0 2024-03-13T14:49:33.059 INFO:tasks.workunit.client.1.smithi196.stdout:3/245: dwrite db/f13 [0,4194304] 0 2024-03-13T14:49:33.061 INFO:tasks.workunit.client.1.smithi196.stdout:5/356: mknod d4/db/d14/d37/d41/d63/c71 0 2024-03-13T14:49:33.062 INFO:tasks.workunit.client.1.smithi196.stdout:8/197: getdents d0/d10/d12 0 2024-03-13T14:49:33.063 INFO:tasks.workunit.client.1.smithi196.stdout:2/166: write d0/d7/f17 [1371383,106126] 0 2024-03-13T14:49:33.065 INFO:tasks.workunit.client.1.smithi196.stdout:3/246: readlink db/d19/d22/d30/l38 0 2024-03-13T14:49:33.066 INFO:tasks.workunit.client.1.smithi196.stdout:3/247: read f7 [4335760,26521] 0 2024-03-13T14:49:33.070 INFO:tasks.workunit.client.1.smithi196.stdout:3/248: truncate db/f36 521622 0 2024-03-13T14:49:33.071 INFO:tasks.workunit.client.1.smithi196.stdout:6/363: dread d6/d9/d17/d19/f2d [0,4194304] 0 2024-03-13T14:49:33.086 INFO:tasks.workunit.client.0.smithi176.stdout:5/61: write d5/fa [1504009,72889] 0 2024-03-13T14:49:33.091 INFO:tasks.workunit.client.0.smithi176.stdout:9/101: link d0/l21 d0/d13/l22 0 2024-03-13T14:49:33.095 INFO:tasks.workunit.client.0.smithi176.stdout:8/71: write d0/f2 [5885560,10827] 0 2024-03-13T14:49:33.102 INFO:tasks.workunit.client.1.smithi196.stdout:1/236: rename d1/d29 to d1/d9/d33/d4b 0 2024-03-13T14:49:33.102 INFO:tasks.workunit.client.0.smithi176.stdout:0/68: symlink d2/d5/le 0 2024-03-13T14:49:33.103 INFO:tasks.workunit.client.0.smithi176.stdout:0/69: stat d2 0 2024-03-13T14:49:33.105 INFO:tasks.workunit.client.1.smithi196.stdout:9/226: write d7/f20 [262698,68039] 0 2024-03-13T14:49:33.108 INFO:tasks.workunit.client.0.smithi176.stdout:3/72: write d4/f5 [6720243,121501] 0 2024-03-13T14:49:33.114 INFO:tasks.workunit.client.0.smithi176.stdout:6/76: symlink d0/d9/l1c 0 2024-03-13T14:49:33.117 INFO:tasks.workunit.client.1.smithi196.stdout:2/167: creat d0/d6/d1d/f30 x:0 0 0 2024-03-13T14:49:33.119 INFO:tasks.workunit.client.1.smithi196.stdout:0/345: symlink d8/da/d25/d26/d35/d3b/d59/l77 0 2024-03-13T14:49:33.121 INFO:tasks.workunit.client.1.smithi196.stdout:2/168: chown d0/d6/d1d/f23 30 1 2024-03-13T14:49:33.123 INFO:tasks.workunit.client.1.smithi196.stdout:3/249: unlink db/d37/l52 0 2024-03-13T14:49:33.125 INFO:tasks.workunit.client.1.smithi196.stdout:0/346: write d8/da/d25/d26/d35/d3b/d50/d5d/f57 [1189977,61619] 0 2024-03-13T14:49:33.126 INFO:tasks.workunit.client.0.smithi176.stdout:5/62: mkdir d5/de/d15 0 2024-03-13T14:49:33.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:32 smithi176 ceph-mon[29064]: pgmap v5: 65 pgs: 65 active+clean; 534 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:49:33.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:32 smithi176 ceph-mon[29064]: mgrmap e29: smithi176.tzfdwd(active, since 4s) 2024-03-13T14:49:33.129 INFO:tasks.workunit.client.1.smithi196.stdout:6/364: mkdir d6/d7/df/d27/d39/d44/d74 0 2024-03-13T14:49:33.129 INFO:tasks.workunit.client.1.smithi196.stdout:0/347: chown d8/da/d25/d26/d35/d3b/d50 11887047 1 2024-03-13T14:49:33.130 INFO:tasks.workunit.client.1.smithi196.stdout:3/250: write db/d19/f28 [946308,73767] 0 2024-03-13T14:49:33.131 INFO:tasks.workunit.client.0.smithi176.stdout:7/82: truncate d1/d7/f10 1609865 0 2024-03-13T14:49:33.132 INFO:tasks.workunit.client.1.smithi196.stdout:3/251: chown db/d4c 241 1 2024-03-13T14:49:33.133 INFO:tasks.workunit.client.1.smithi196.stdout:5/357: dwrite d4/db/d3c/f59 [0,4194304] 0 2024-03-13T14:49:33.134 INFO:tasks.workunit.client.0.smithi176.stdout:1/58: mknod d2/de/d10/d14/c15 0 2024-03-13T14:49:33.134 INFO:tasks.workunit.client.0.smithi176.stdout:9/102: rmdir d0/d1e 39 2024-03-13T14:49:33.135 INFO:tasks.workunit.client.1.smithi196.stdout:0/348: readlink d8/da/d25/d26/d35/d3b/d50/d5d/l4c 0 2024-03-13T14:49:33.136 INFO:tasks.workunit.client.1.smithi196.stdout:4/223: truncate d1/d3/d7/f1d 130955 0 2024-03-13T14:49:33.137 INFO:tasks.workunit.client.1.smithi196.stdout:5/358: chown d4/d60 3688540 1 2024-03-13T14:49:33.139 INFO:tasks.workunit.client.1.smithi196.stdout:5/359: chown d4/d1b/f29 19061 1 2024-03-13T14:49:33.140 INFO:tasks.workunit.client.0.smithi176.stdout:8/72: creat d0/db/f12 x:0 0 0 2024-03-13T14:49:33.146 INFO:tasks.workunit.client.1.smithi196.stdout:0/349: write d8/d37/f73 [1170202,33066] 0 2024-03-13T14:49:33.147 INFO:tasks.workunit.client.1.smithi196.stdout:8/198: fsync d0/d10/d12/d17/f28 0 2024-03-13T14:49:33.147 INFO:tasks.workunit.client.0.smithi176.stdout:3/73: rename d4/cc to d4/d13/c17 0 2024-03-13T14:49:33.149 INFO:tasks.workunit.client.1.smithi196.stdout:8/199: stat d0/d10/f26 0 2024-03-13T14:49:33.150 INFO:tasks.workunit.client.1.smithi196.stdout:0/350: chown d8/da/d25/d26/d35/d3b/d5f/c16 780136 1 2024-03-13T14:49:33.150 INFO:tasks.workunit.client.0.smithi176.stdout:1/59: dread d2/f6 [0,4194304] 0 2024-03-13T14:49:33.153 INFO:tasks.workunit.client.0.smithi176.stdout:0/70: dread d2/f6 [0,4194304] 0 2024-03-13T14:49:33.154 INFO:tasks.workunit.client.0.smithi176.stdout:2/79: link d1/l21 d1/dc/l23 0 2024-03-13T14:49:33.155 INFO:tasks.workunit.client.0.smithi176.stdout:6/77: dread d0/fc [0,4194304] 0 2024-03-13T14:49:33.155 INFO:tasks.workunit.client.1.smithi196.stdout:0/351: chown d8/da/d25/d26/d35/d3b/d5f/l30 58423 1 2024-03-13T14:49:33.157 INFO:tasks.workunit.client.0.smithi176.stdout:2/80: truncate d1/dc/f17 754307 0 2024-03-13T14:49:33.157 INFO:tasks.workunit.client.0.smithi176.stdout:6/78: write d0/d9/f14 [1122235,77348] 0 2024-03-13T14:49:33.161 INFO:tasks.workunit.client.0.smithi176.stdout:5/63: rmdir d5/de 39 2024-03-13T14:49:33.164 INFO:tasks.workunit.client.1.smithi196.stdout:6/365: write d6/d2a/dc/d47/f55 [979984,38650] 0 2024-03-13T14:49:33.174 INFO:tasks.workunit.client.0.smithi176.stdout:8/73: fdatasync d0/f9 0 2024-03-13T14:49:33.178 INFO:tasks.workunit.client.0.smithi176.stdout:4/84: getdents d7/d8/d15 0 2024-03-13T14:49:33.179 INFO:tasks.workunit.client.0.smithi176.stdout:5/64: dwrite d5/fa [0,4194304] 0 2024-03-13T14:49:33.184 INFO:tasks.workunit.client.1.smithi196.stdout:6/366: dread d6/d4b/d50/f5a [0,4194304] 0 2024-03-13T14:49:33.184 INFO:tasks.workunit.client.1.smithi196.stdout:9/227: symlink d7/l41 0 2024-03-13T14:49:33.186 INFO:tasks.workunit.client.1.smithi196.stdout:5/360: rename d4/db/d14/d19/f5a to d4/db/d14/d19/d26/d68/f72 0 2024-03-13T14:49:33.187 INFO:tasks.workunit.client.0.smithi176.stdout:1/60: truncate d2/f5 728061 0 2024-03-13T14:49:33.187 INFO:tasks.workunit.client.0.smithi176.stdout:1/61: stat d2/lb 0 2024-03-13T14:49:33.189 INFO:tasks.workunit.client.1.smithi196.stdout:4/224: dread d1/d3/d7/f1a [0,4194304] 0 2024-03-13T14:49:33.196 INFO:tasks.workunit.client.1.smithi196.stdout:6/367: dread d6/d2a/d23/d28/f3f [0,4194304] 0 2024-03-13T14:49:33.198 INFO:tasks.workunit.client.1.smithi196.stdout:8/200: symlink d0/d10/d12/d17/d40/l46 0 2024-03-13T14:49:33.199 INFO:tasks.workunit.client.0.smithi176.stdout:0/71: mknod d2/cf 0 2024-03-13T14:49:33.201 INFO:tasks.workunit.client.1.smithi196.stdout:2/169: mkdir d0/d31 0 2024-03-13T14:49:33.201 INFO:tasks.workunit.client.1.smithi196.stdout:8/201: read - d0/d1c/d29/f3c zero size 2024-03-13T14:49:33.204 INFO:tasks.workunit.client.0.smithi176.stdout:1/62: dwrite d2/fc [0,4194304] 0 2024-03-13T14:49:33.208 INFO:tasks.workunit.client.1.smithi196.stdout:3/252: symlink db/d19/l56 0 2024-03-13T14:49:33.208 INFO:tasks.workunit.client.1.smithi196.stdout:3/253: fsync f4 0 2024-03-13T14:49:33.215 INFO:tasks.workunit.client.0.smithi176.stdout:2/81: mknod d1/c24 0 2024-03-13T14:49:33.217 INFO:tasks.workunit.client.0.smithi176.stdout:1/63: dread d2/f9 [0,4194304] 0 2024-03-13T14:49:33.219 INFO:tasks.workunit.client.0.smithi176.stdout:9/103: rename d0/db/l11 to d0/d1e/d1f/l23 0 2024-03-13T14:49:33.222 INFO:tasks.workunit.client.1.smithi196.stdout:9/228: unlink d7/d24/f27 0 2024-03-13T14:49:33.226 INFO:tasks.workunit.client.1.smithi196.stdout:6/368: dwrite d6/d2a/d23/d28/f6d [0,4194304] 0 2024-03-13T14:49:33.227 INFO:tasks.workunit.client.1.smithi196.stdout:5/361: mknod d4/db/c73 0 2024-03-13T14:49:33.230 INFO:tasks.workunit.client.0.smithi176.stdout:5/65: stat d5/de/f13 0 2024-03-13T14:49:33.230 INFO:tasks.workunit.client.0.smithi176.stdout:5/66: chown d5 454 1 2024-03-13T14:49:33.230 INFO:tasks.workunit.client.0.smithi176.stdout:5/67: stat d5/f6 0 2024-03-13T14:49:33.232 INFO:tasks.workunit.client.1.smithi196.stdout:1/237: dwrite d1/f5 [0,4194304] 0 2024-03-13T14:49:33.235 INFO:tasks.workunit.client.1.smithi196.stdout:8/202: readlink d0/l44 0 2024-03-13T14:49:33.236 INFO:tasks.workunit.client.0.smithi176.stdout:0/72: unlink d2/c9 0 2024-03-13T14:49:33.237 INFO:tasks.workunit.client.1.smithi196.stdout:5/362: dread - d4/db/d14/d19/d1c/d25/d38/d47/f64 zero size 2024-03-13T14:49:33.244 INFO:tasks.workunit.client.1.smithi196.stdout:9/229: symlink d7/d9/df/l42 0 2024-03-13T14:49:33.244 INFO:tasks.workunit.client.1.smithi196.stdout:3/254: dread f4 [0,4194304] 0 2024-03-13T14:49:33.246 INFO:tasks.workunit.client.0.smithi176.stdout:7/83: write d1/d7/f10 [1418016,10646] 0 2024-03-13T14:49:33.247 INFO:tasks.workunit.client.0.smithi176.stdout:4/85: dwrite d7/d8/f18 [0,4194304] 0 2024-03-13T14:49:33.248 INFO:tasks.workunit.client.1.smithi196.stdout:5/363: read - d4/db/d14/d19/f58 zero size 2024-03-13T14:49:33.249 INFO:tasks.workunit.client.0.smithi176.stdout:2/82: mknod d1/dc/c25 0 2024-03-13T14:49:33.249 INFO:tasks.workunit.client.0.smithi176.stdout:4/86: truncate d7/d10/f20 201970 0 2024-03-13T14:49:33.250 INFO:tasks.workunit.client.1.smithi196.stdout:3/255: chown db/f29 589096196 1 2024-03-13T14:49:33.250 INFO:tasks.workunit.client.0.smithi176.stdout:2/83: write d1/fa [958704,83161] 0 2024-03-13T14:49:33.251 INFO:tasks.workunit.client.0.smithi176.stdout:7/84: chown d1/d7/f10 11 1 2024-03-13T14:49:33.253 INFO:tasks.workunit.client.1.smithi196.stdout:9/230: write d7/de/f1f [3404588,125848] 0 2024-03-13T14:49:33.254 INFO:tasks.workunit.client.1.smithi196.stdout:5/364: write d4/db/f66 [200945,3616] 0 2024-03-13T14:49:33.256 INFO:tasks.workunit.client.1.smithi196.stdout:3/256: stat db/d19/d22/d30/f42 0 2024-03-13T14:49:33.257 INFO:tasks.workunit.client.1.smithi196.stdout:9/231: fsync d7/fd 0 2024-03-13T14:49:33.258 INFO:tasks.workunit.client.0.smithi176.stdout:1/64: fsync d2/f6 0 2024-03-13T14:49:33.258 INFO:tasks.workunit.client.1.smithi196.stdout:3/257: chown db/f4e 3 1 2024-03-13T14:49:33.260 INFO:tasks.workunit.client.0.smithi176.stdout:9/104: truncate d0/db/f1d 633448 0 2024-03-13T14:49:33.260 INFO:tasks.workunit.client.1.smithi196.stdout:3/258: dread - db/f35 zero size 2024-03-13T14:49:33.262 INFO:tasks.workunit.client.0.smithi176.stdout:8/74: creat d0/db/dc/f13 x:0 0 0 2024-03-13T14:49:33.265 INFO:tasks.workunit.client.1.smithi196.stdout:8/203: dread d0/f27 [0,4194304] 0 2024-03-13T14:49:33.277 INFO:tasks.workunit.client.1.smithi196.stdout:2/170: creat d0/f32 x:0 0 0 2024-03-13T14:49:33.280 INFO:tasks.workunit.client.0.smithi176.stdout:8/75: dwrite d0/f9 [0,4194304] 0 2024-03-13T14:49:33.282 INFO:tasks.workunit.client.1.smithi196.stdout:4/225: getdents d1/d3/d7/d35 0 2024-03-13T14:49:33.287 INFO:tasks.workunit.client.1.smithi196.stdout:9/232: dwrite d7/d24/f31 [0,4194304] 0 2024-03-13T14:49:33.287 INFO:tasks.workunit.client.0.smithi176.stdout:2/84: creat d1/f26 x:0 0 0 2024-03-13T14:49:33.288 INFO:tasks.workunit.client.1.smithi196.stdout:5/365: mknod d4/d60/c74 0 2024-03-13T14:49:33.293 INFO:tasks.workunit.client.1.smithi196.stdout:5/366: write d4/d1b/f29 [2929424,58120] 0 2024-03-13T14:49:33.294 INFO:tasks.workunit.client.1.smithi196.stdout:7/342: sync 2024-03-13T14:49:33.295 INFO:tasks.workunit.client.1.smithi196.stdout:8/204: mkdir d0/d1c/d29/d47 0 2024-03-13T14:49:33.301 INFO:tasks.workunit.client.0.smithi176.stdout:1/65: mkdir d2/de/d10/d16 0 2024-03-13T14:49:33.301 INFO:tasks.workunit.client.0.smithi176.stdout:9/105: symlink d0/d1e/l24 0 2024-03-13T14:49:33.302 INFO:tasks.workunit.client.0.smithi176.stdout:9/106: chown d0/d1e/d1f 255277 1 2024-03-13T14:49:33.304 INFO:tasks.workunit.client.1.smithi196.stdout:1/238: creat d1/f4c x:0 0 0 2024-03-13T14:49:33.304 INFO:tasks.workunit.client.1.smithi196.stdout:1/239: chown d1 1681 1 2024-03-13T14:49:33.304 INFO:tasks.workunit.client.1.smithi196.stdout:0/352: write d8/da/d25/d26/d35/d3b/f47 [4540748,124130] 0 2024-03-13T14:49:33.305 INFO:tasks.workunit.client.1.smithi196.stdout:4/226: dread d1/d3/d7/f21 [0,4194304] 0 2024-03-13T14:49:33.305 INFO:tasks.workunit.client.1.smithi196.stdout:2/171: write d0/d6/f8 [16772,81681] 0 2024-03-13T14:49:33.309 INFO:tasks.workunit.client.1.smithi196.stdout:0/353: readlink d8/da/d25/d26/d35/d3b/d50/d5d/l15 0 2024-03-13T14:49:33.310 INFO:tasks.workunit.client.1.smithi196.stdout:2/172: write d0/d6/f14 [1652241,58360] 0 2024-03-13T14:49:33.314 INFO:tasks.workunit.client.1.smithi196.stdout:2/173: chown d0/d7/l1e 1 1 2024-03-13T14:49:33.314 INFO:tasks.workunit.client.0.smithi176.stdout:6/79: getdents d0/df 0 2024-03-13T14:49:33.317 INFO:tasks.workunit.client.0.smithi176.stdout:4/87: unlink d7/cf 0 2024-03-13T14:49:33.321 INFO:tasks.workunit.client.1.smithi196.stdout:9/233: creat d7/d9/df/f43 x:0 0 0 2024-03-13T14:49:33.321 INFO:tasks.workunit.client.0.smithi176.stdout:2/85: symlink d1/dc/d1c/l27 0 2024-03-13T14:49:33.324 INFO:tasks.workunit.client.1.smithi196.stdout:8/205: mkdir d0/d1c/d3b/d48 0 2024-03-13T14:49:33.325 INFO:tasks.workunit.client.0.smithi176.stdout:1/66: fsync f1 0 2024-03-13T14:49:33.330 INFO:tasks.workunit.client.1.smithi196.stdout:1/240: symlink d1/d9/d33/l4d 0 2024-03-13T14:49:33.335 INFO:tasks.workunit.client.1.smithi196.stdout:4/227: symlink d1/d1e/l3c 0 2024-03-13T14:49:33.337 INFO:tasks.workunit.client.0.smithi176.stdout:2/86: dwrite d1/dc/d1c/f1d [0,4194304] 0 2024-03-13T14:49:33.345 INFO:tasks.workunit.client.0.smithi176.stdout:3/74: truncate f2 2283403 0 2024-03-13T14:49:33.348 INFO:tasks.workunit.client.0.smithi176.stdout:3/75: write d4/f10 [3627453,68733] 0 2024-03-13T14:49:33.352 INFO:tasks.workunit.client.0.smithi176.stdout:1/67: creat d2/de/f17 x:0 0 0 2024-03-13T14:49:33.354 INFO:tasks.workunit.client.1.smithi196.stdout:4/228: dread d1/f29 [0,4194304] 0 2024-03-13T14:49:33.357 INFO:tasks.workunit.client.0.smithi176.stdout:2/87: dwrite d1/dc/f17 [0,4194304] 0 2024-03-13T14:49:33.358 INFO:tasks.workunit.client.0.smithi176.stdout:2/88: dread - d1/f26 zero size 2024-03-13T14:49:33.361 INFO:tasks.workunit.client.0.smithi176.stdout:9/107: unlink d0/f12 0 2024-03-13T14:49:33.366 INFO:tasks.workunit.client.1.smithi196.stdout:6/369: dread d6/d9/f1f [4194304,4194304] 0 2024-03-13T14:49:33.376 INFO:tasks.workunit.client.1.smithi196.stdout:9/234: fsync d7/d9/df/d1d/d22/f39 0 2024-03-13T14:49:33.379 INFO:tasks.workunit.client.1.smithi196.stdout:4/229: dread d1/d1e/f34 [4194304,4194304] 0 2024-03-13T14:49:33.383 INFO:tasks.workunit.client.0.smithi176.stdout:0/73: write d2/f3 [2093919,52598] 0 2024-03-13T14:49:33.385 INFO:tasks.workunit.client.0.smithi176.stdout:0/74: rename d2 to d2/db/d10 22 2024-03-13T14:49:33.386 INFO:tasks.workunit.client.0.smithi176.stdout:7/85: write d1/d13/f14 [4187924,105730] 0 2024-03-13T14:49:33.387 INFO:tasks.workunit.client.0.smithi176.stdout:0/75: fsync d2/d5/f8 0 2024-03-13T14:49:33.388 INFO:tasks.workunit.client.0.smithi176.stdout:5/68: dwrite f0 [0,4194304] 0 2024-03-13T14:49:33.390 INFO:tasks.workunit.client.0.smithi176.stdout:5/69: dread - d5/de/f10 zero size 2024-03-13T14:49:33.390 INFO:tasks.workunit.client.0.smithi176.stdout:8/76: link d0/f9 d0/f14 0 2024-03-13T14:49:33.398 INFO:tasks.workunit.client.1.smithi196.stdout:0/354: symlink d8/da/d25/d26/d35/d3b/d5f/d17/l78 0 2024-03-13T14:49:33.400 INFO:tasks.workunit.client.1.smithi196.stdout:7/343: rename d0/d9/f20 to d0/f6a 0 2024-03-13T14:49:33.405 INFO:tasks.workunit.client.1.smithi196.stdout:1/241: rmdir d1/d9 39 2024-03-13T14:49:33.407 INFO:tasks.workunit.client.1.smithi196.stdout:3/259: getdents db/d15/d44 0 2024-03-13T14:49:33.407 INFO:tasks.workunit.client.1.smithi196.stdout:3/260: readlink db/l25 0 2024-03-13T14:49:33.409 INFO:tasks.workunit.client.1.smithi196.stdout:3/261: truncate db/f35 859472 0 2024-03-13T14:49:33.411 INFO:tasks.workunit.client.0.smithi176.stdout:4/88: write d7/fc [141092,46557] 0 2024-03-13T14:49:33.413 INFO:tasks.workunit.client.0.smithi176.stdout:9/108: creat d0/d1e/f25 x:0 0 0 2024-03-13T14:49:33.413 INFO:tasks.workunit.client.1.smithi196.stdout:4/230: mknod d1/d3/d7/c3d 0 2024-03-13T14:49:33.415 INFO:tasks.workunit.client.1.smithi196.stdout:4/231: write d1/d1e/d39/f32 [264582,71798] 0 2024-03-13T14:49:33.417 INFO:tasks.workunit.client.0.smithi176.stdout:5/70: dwrite f0 [0,4194304] 0 2024-03-13T14:49:33.418 INFO:tasks.workunit.client.0.smithi176.stdout:1/68: fsync d2/de/f17 0 2024-03-13T14:49:33.419 INFO:tasks.workunit.client.0.smithi176.stdout:0/76: dwrite d2/d5/f8 [0,4194304] 0 2024-03-13T14:49:33.420 INFO:tasks.workunit.client.0.smithi176.stdout:9/109: truncate d0/d1e/f25 899279 0 2024-03-13T14:49:33.421 INFO:tasks.workunit.client.1.smithi196.stdout:8/206: mknod d0/d1c/d3b/d3f/c49 0 2024-03-13T14:49:33.421 INFO:tasks.workunit.client.1.smithi196.stdout:5/367: write d4/db/d14/d19/d1c/f23 [608936,65893] 0 2024-03-13T14:49:33.421 INFO:tasks.workunit.client.0.smithi176.stdout:5/71: fsync f4 0 2024-03-13T14:49:33.422 INFO:tasks.workunit.client.0.smithi176.stdout:1/69: chown d2/f9 21124 1 2024-03-13T14:49:33.428 INFO:tasks.workunit.client.0.smithi176.stdout:6/80: truncate d0/f18 3241215 0 2024-03-13T14:49:33.431 INFO:tasks.workunit.client.0.smithi176.stdout:6/81: chown d0/df/l17 18 1 2024-03-13T14:49:33.435 INFO:tasks.workunit.client.1.smithi196.stdout:2/174: rename d0/d6/f8 to d0/d6/d1d/d28/f33 0 2024-03-13T14:49:33.440 INFO:tasks.workunit.client.1.smithi196.stdout:4/232: dread d1/d3/d7/d35/f37 [0,4194304] 0 2024-03-13T14:49:33.442 INFO:tasks.workunit.client.1.smithi196.stdout:6/370: mknod d6/d7/dd/d60/c75 0 2024-03-13T14:49:33.448 INFO:tasks.workunit.client.1.smithi196.stdout:0/355: symlink d8/da/d25/d26/d35/l79 0 2024-03-13T14:49:33.452 INFO:tasks.workunit.client.0.smithi176.stdout:4/89: mknod d7/d8/d15/c21 0 2024-03-13T14:49:33.455 INFO:tasks.workunit.client.1.smithi196.stdout:8/207: unlink d0/d10/f31 0 2024-03-13T14:49:33.456 INFO:tasks.workunit.client.1.smithi196.stdout:8/208: chown d0/ca 1 1 2024-03-13T14:49:33.457 INFO:tasks.workunit.client.0.smithi176.stdout:2/89: dwrite d1/f14 [0,4194304] 0 2024-03-13T14:49:33.461 INFO:tasks.workunit.client.0.smithi176.stdout:1/70: chown d2/f5 27 1 2024-03-13T14:49:33.461 INFO:tasks.workunit.client.0.smithi176.stdout:9/110: symlink d0/d13/l26 0 2024-03-13T14:49:33.465 INFO:tasks.workunit.client.1.smithi196.stdout:5/368: write d4/db/d14/d19/d26/f3a [637607,90047] 0 2024-03-13T14:49:33.471 INFO:tasks.workunit.client.1.smithi196.stdout:9/235: rename d7/de/f1f to d7/d9/f44 0 2024-03-13T14:49:33.474 INFO:tasks.workunit.client.1.smithi196.stdout:2/175: chown d0/d7/d9/l16 231384 1 2024-03-13T14:49:33.477 INFO:tasks.workunit.client.1.smithi196.stdout:4/233: mkdir d1/d3/d7/d35/d3e 0 2024-03-13T14:49:33.478 INFO:tasks.workunit.client.0.smithi176.stdout:4/90: dwrite d7/d8/f16 [0,4194304] 0 2024-03-13T14:49:33.479 INFO:tasks.workunit.client.1.smithi196.stdout:6/371: symlink d6/d7/dd/d60/l76 0 2024-03-13T14:49:33.481 INFO:tasks.workunit.client.0.smithi176.stdout:3/76: dwrite f2 [0,4194304] 0 2024-03-13T14:49:33.486 INFO:tasks.workunit.client.1.smithi196.stdout:3/262: fsync db/d19/f2b 0 2024-03-13T14:49:33.488 INFO:tasks.workunit.client.1.smithi196.stdout:3/263: fdatasync db/d19/f3f 0 2024-03-13T14:49:33.501 INFO:tasks.workunit.client.0.smithi176.stdout:5/72: creat d5/de/d15/f16 x:0 0 0 2024-03-13T14:49:33.502 INFO:tasks.workunit.client.1.smithi196.stdout:8/209: creat d0/d1c/d3b/f4a x:0 0 0 2024-03-13T14:49:33.503 INFO:tasks.workunit.client.0.smithi176.stdout:7/86: sync 2024-03-13T14:49:33.504 INFO:tasks.workunit.client.0.smithi176.stdout:5/73: write d5/fa [1105931,78381] 0 2024-03-13T14:49:33.506 INFO:tasks.workunit.client.0.smithi176.stdout:8/77: link d0/d5/c10 d0/db/c15 0 2024-03-13T14:49:33.509 INFO:tasks.workunit.client.1.smithi196.stdout:7/344: creat d0/f6b x:0 0 0 2024-03-13T14:49:33.510 INFO:tasks.workunit.client.1.smithi196.stdout:3/264: dwrite db/f3d [0,4194304] 0 2024-03-13T14:49:33.511 INFO:tasks.workunit.client.0.smithi176.stdout:3/77: dwrite f2 [0,4194304] 0 2024-03-13T14:49:33.512 INFO:tasks.workunit.client.1.smithi196.stdout:5/369: unlink d4/db/d14/d19/d26/c27 0 2024-03-13T14:49:33.512 INFO:tasks.workunit.client.1.smithi196.stdout:3/265: dread - db/d19/f55 zero size 2024-03-13T14:49:33.517 INFO:tasks.workunit.client.0.smithi176.stdout:3/78: truncate d4/f5 8683385 0 2024-03-13T14:49:33.518 INFO:tasks.workunit.client.1.smithi196.stdout:9/236: truncate d7/f12 105326 0 2024-03-13T14:49:33.518 INFO:tasks.workunit.client.0.smithi176.stdout:4/91: unlink d7/d8/fd 0 2024-03-13T14:49:33.521 INFO:tasks.workunit.client.0.smithi176.stdout:3/79: write d4/f10 [2508152,21910] 0 2024-03-13T14:49:33.525 INFO:tasks.workunit.client.0.smithi176.stdout:9/111: rename d0/d1e/d1f/l23 to d0/d1e/l27 0 2024-03-13T14:49:33.528 INFO:tasks.workunit.client.1.smithi196.stdout:1/242: mkdir d1/d9/d4e 0 2024-03-13T14:49:33.529 INFO:tasks.workunit.client.0.smithi176.stdout:7/87: dread d1/d13/f14 [0,4194304] 0 2024-03-13T14:49:33.530 INFO:tasks.workunit.client.0.smithi176.stdout:1/71: dwrite d2/f5 [0,4194304] 0 2024-03-13T14:49:33.533 INFO:tasks.workunit.client.0.smithi176.stdout:5/74: rmdir d5/de 39 2024-03-13T14:49:33.533 INFO:tasks.workunit.client.0.smithi176.stdout:1/72: write f1 [944655,85319] 0 2024-03-13T14:49:33.537 INFO:tasks.workunit.client.0.smithi176.stdout:6/82: link d0/d10/l11 d0/d10/l1d 0 2024-03-13T14:49:33.538 INFO:tasks.workunit.client.1.smithi196.stdout:7/345: dwrite d0/d9/f59 [0,4194304] 0 2024-03-13T14:49:33.540 INFO:tasks.workunit.client.0.smithi176.stdout:3/80: dread d4/fa [0,4194304] 0 2024-03-13T14:49:33.541 INFO:tasks.workunit.client.1.smithi196.stdout:3/266: dread db/d19/f2f [0,4194304] 0 2024-03-13T14:49:33.542 INFO:tasks.workunit.client.1.smithi196.stdout:7/346: fsync d0/d9/d1e/f4b 0 2024-03-13T14:49:33.543 INFO:tasks.workunit.client.0.smithi176.stdout:3/81: read d4/f10 [2841054,80095] 0 2024-03-13T14:49:33.546 INFO:tasks.workunit.client.0.smithi176.stdout:3/82: write d4/f10 [4923467,56786] 0 2024-03-13T14:49:33.550 INFO:tasks.workunit.client.1.smithi196.stdout:2/176: mkdir d0/d6/d34 0 2024-03-13T14:49:33.551 INFO:tasks.workunit.client.1.smithi196.stdout:7/347: fsync d0/d6/f2c 0 2024-03-13T14:49:33.553 INFO:tasks.workunit.client.1.smithi196.stdout:6/372: mknod d6/d7/c77 0 2024-03-13T14:49:33.555 INFO:tasks.workunit.client.1.smithi196.stdout:4/234: dwrite d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:33.556 INFO:tasks.workunit.client.0.smithi176.stdout:9/112: mknod d0/d17/c28 0 2024-03-13T14:49:33.558 INFO:tasks.workunit.client.0.smithi176.stdout:3/83: dread d4/f5 [0,4194304] 0 2024-03-13T14:49:33.564 INFO:tasks.workunit.client.0.smithi176.stdout:7/88: mknod d1/d13/c16 0 2024-03-13T14:49:33.576 INFO:tasks.workunit.client.1.smithi196.stdout:8/210: link d0/d1c/f30 d0/d1c/d3b/f4b 0 2024-03-13T14:49:33.581 INFO:tasks.workunit.client.0.smithi176.stdout:0/77: sync 2024-03-13T14:49:33.581 INFO:tasks.workunit.client.0.smithi176.stdout:6/83: fsync d0/fc 0 2024-03-13T14:49:33.582 INFO:tasks.workunit.client.0.smithi176.stdout:6/84: chown d0/la 14567 1 2024-03-13T14:49:33.583 INFO:tasks.workunit.client.0.smithi176.stdout:0/78: read d2/f6 [949965,104743] 0 2024-03-13T14:49:33.584 INFO:tasks.workunit.client.0.smithi176.stdout:0/79: write d2/d5/f8 [2688261,62870] 0 2024-03-13T14:49:33.596 INFO:tasks.workunit.client.0.smithi176.stdout:2/90: truncate d1/dc/d1c/f1d 3143040 0 2024-03-13T14:49:33.597 INFO:tasks.workunit.client.1.smithi196.stdout:3/267: fdatasync db/d19/d22/f3c 0 2024-03-13T14:49:33.603 INFO:tasks.workunit.client.1.smithi196.stdout:8/211: dread d0/d1c/f30 [0,4194304] 0 2024-03-13T14:49:33.606 INFO:tasks.workunit.client.0.smithi176.stdout:6/85: dwrite d0/f13 [0,4194304] 0 2024-03-13T14:49:33.611 INFO:tasks.workunit.client.1.smithi196.stdout:8/212: dread d0/d1c/d29/f3a [0,4194304] 0 2024-03-13T14:49:33.615 INFO:tasks.workunit.client.1.smithi196.stdout:7/348: mkdir d0/d9/d1e/d3f/d6c 0 2024-03-13T14:49:33.616 INFO:tasks.workunit.client.0.smithi176.stdout:3/84: mknod d4/d13/c18 0 2024-03-13T14:49:33.620 INFO:tasks.workunit.client.1.smithi196.stdout:6/373: rename d6/d7/dd/d60/c75 to d6/d6f/c78 0 2024-03-13T14:49:33.621 INFO:tasks.workunit.client.1.smithi196.stdout:6/374: chown d6/d4b/l5b 4415 1 2024-03-13T14:49:33.624 INFO:tasks.workunit.client.0.smithi176.stdout:8/78: dwrite d0/f9 [0,4194304] 0 2024-03-13T14:49:33.625 INFO:tasks.workunit.client.1.smithi196.stdout:4/235: rename d1/d3/d7/c10 to d1/d3/d7/c3f 0 2024-03-13T14:49:33.629 INFO:tasks.workunit.client.0.smithi176.stdout:3/85: dread f1 [0,4194304] 0 2024-03-13T14:49:33.631 INFO:tasks.workunit.client.0.smithi176.stdout:6/86: dwrite d0/f13 [0,4194304] 0 2024-03-13T14:49:33.639 INFO:tasks.workunit.client.1.smithi196.stdout:1/243: dwrite d1/d28/f2d [0,4194304] 0 2024-03-13T14:49:33.647 INFO:tasks.workunit.client.1.smithi196.stdout:5/370: link d4/lc d4/db/d14/d19/d1c/d25/d38/l75 0 2024-03-13T14:49:33.650 INFO:tasks.workunit.client.1.smithi196.stdout:9/237: creat d7/f45 x:0 0 0 2024-03-13T14:49:33.658 INFO:tasks.workunit.client.0.smithi176.stdout:9/113: dwrite d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:33.660 INFO:tasks.workunit.client.0.smithi176.stdout:7/89: readlink d1/d2/l12 0 2024-03-13T14:49:33.661 INFO:tasks.workunit.client.0.smithi176.stdout:7/90: chown d1/d13 0 1 2024-03-13T14:49:33.665 INFO:tasks.workunit.client.1.smithi196.stdout:7/349: mknod d0/d9/d1e/c6d 0 2024-03-13T14:49:33.667 INFO:tasks.workunit.client.0.smithi176.stdout:5/75: rename f4 to d5/f17 0 2024-03-13T14:49:33.667 INFO:tasks.workunit.client.1.smithi196.stdout:7/350: read - d0/d9/d1e/d3f/f60 zero size 2024-03-13T14:49:33.668 INFO:tasks.workunit.client.0.smithi176.stdout:1/73: link l0 d2/dd/l18 0 2024-03-13T14:49:33.669 INFO:tasks.workunit.client.1.smithi196.stdout:6/375: stat d6/d2a/dc/d1c/d34/l3d 0 2024-03-13T14:49:33.670 INFO:tasks.workunit.client.1.smithi196.stdout:0/356: link d8/da/d25/d26/d35/d3b/d5f/f4a d8/da/d25/d26/d35/d3b/d5f/f7a 0 2024-03-13T14:49:33.673 INFO:tasks.workunit.client.1.smithi196.stdout:7/351: dread - d0/d9/d1e/f51 zero size 2024-03-13T14:49:33.674 INFO:tasks.workunit.client.1.smithi196.stdout:7/352: dread - d0/d9/f5a zero size 2024-03-13T14:49:33.679 INFO:tasks.workunit.client.1.smithi196.stdout:3/268: creat db/d37/d4b/f57 x:0 0 0 2024-03-13T14:49:33.679 INFO:tasks.workunit.client.0.smithi176.stdout:3/86: fsync d4/f10 0 2024-03-13T14:49:33.680 INFO:tasks.workunit.client.0.smithi176.stdout:3/87: readlink l0 0 2024-03-13T14:49:33.681 INFO:tasks.workunit.client.1.smithi196.stdout:2/177: truncate d0/d7/f17 412660 0 2024-03-13T14:49:33.685 INFO:tasks.workunit.client.1.smithi196.stdout:2/178: write d0/fe [602658,51223] 0 2024-03-13T14:49:33.688 INFO:tasks.workunit.client.1.smithi196.stdout:3/269: write f4 [4343447,36667] 0 2024-03-13T14:49:33.689 INFO:tasks.workunit.client.1.smithi196.stdout:6/376: dread d6/d9/f1f [4194304,4194304] 0 2024-03-13T14:49:33.694 INFO:tasks.workunit.client.1.smithi196.stdout:9/238: write f5 [5454236,49369] 0 2024-03-13T14:49:33.697 INFO:tasks.workunit.client.1.smithi196.stdout:8/213: creat d0/d10/d12/d17/d42/f4c x:0 0 0 2024-03-13T14:49:33.699 INFO:tasks.workunit.client.0.smithi176.stdout:4/92: getdents d7/d8/d15 0 2024-03-13T14:49:33.702 INFO:tasks.workunit.client.0.smithi176.stdout:4/93: write d7/fc [130110,119617] 0 2024-03-13T14:49:33.703 INFO:tasks.workunit.client.0.smithi176.stdout:9/114: chown d0/l21 0 1 2024-03-13T14:49:33.704 INFO:tasks.workunit.client.0.smithi176.stdout:0/80: truncate d2/d5/f8 222349 0 2024-03-13T14:49:33.706 INFO:tasks.workunit.client.0.smithi176.stdout:9/115: truncate d0/d1e/f25 1560433 0 2024-03-13T14:49:33.708 INFO:tasks.workunit.client.0.smithi176.stdout:9/116: truncate d0/d17/f19 4323918 0 2024-03-13T14:49:33.709 INFO:tasks.workunit.client.0.smithi176.stdout:5/76: read d5/f8 [419900,92544] 0 2024-03-13T14:49:33.711 INFO:tasks.workunit.client.0.smithi176.stdout:6/87: rename d0/df/l17 to d0/d9/l1e 0 2024-03-13T14:49:33.711 INFO:tasks.workunit.client.1.smithi196.stdout:4/236: fsync d1/d3/fa 0 2024-03-13T14:49:33.711 INFO:tasks.workunit.client.1.smithi196.stdout:7/353: mkdir d0/d9/d1e/d2d/d6e 0 2024-03-13T14:49:33.712 INFO:tasks.workunit.client.0.smithi176.stdout:1/74: unlink d2/de/cf 0 2024-03-13T14:49:33.713 INFO:tasks.workunit.client.0.smithi176.stdout:1/75: readlink d2/de/l13 0 2024-03-13T14:49:33.714 INFO:tasks.workunit.client.1.smithi196.stdout:5/371: dwrite d4/db/d14/d19/d26/d68/f72 [0,4194304] 0 2024-03-13T14:49:33.714 INFO:tasks.workunit.client.1.smithi196.stdout:1/244: mknod d1/d9/d4e/c4f 0 2024-03-13T14:49:33.715 INFO:tasks.workunit.client.1.smithi196.stdout:1/245: write d1/f4c [260154,125998] 0 2024-03-13T14:49:33.716 INFO:tasks.workunit.client.1.smithi196.stdout:3/270: creat db/d19/d50/f58 x:0 0 0 2024-03-13T14:49:33.719 INFO:tasks.workunit.client.1.smithi196.stdout:6/377: creat d6/d7/dd/d60/f79 x:0 0 0 2024-03-13T14:49:33.721 INFO:tasks.workunit.client.1.smithi196.stdout:5/372: fdatasync d4/db/d14/d19/d26/f51 0 2024-03-13T14:49:33.722 INFO:tasks.workunit.client.1.smithi196.stdout:9/239: mknod d7/de/d16/d3c/c46 0 2024-03-13T14:49:33.722 INFO:tasks.workunit.client.0.smithi176.stdout:6/88: dread d0/f13 [0,4194304] 0 2024-03-13T14:49:33.727 INFO:tasks.workunit.client.0.smithi176.stdout:4/94: symlink d7/d8/d15/l22 0 2024-03-13T14:49:33.727 INFO:tasks.workunit.client.1.smithi196.stdout:6/378: chown d6/d9/f1f 9 1 2024-03-13T14:49:33.729 INFO:tasks.workunit.client.1.smithi196.stdout:4/237: creat d1/d3/d7/d35/f40 x:0 0 0 2024-03-13T14:49:33.730 INFO:tasks.workunit.client.1.smithi196.stdout:2/179: mknod d0/c35 0 2024-03-13T14:49:33.731 INFO:tasks.workunit.client.1.smithi196.stdout:5/373: chown d4/db/d14/d19/d1c/d25/c6d 0 1 2024-03-13T14:49:33.731 INFO:tasks.workunit.client.1.smithi196.stdout:1/246: mkdir d1/d3f/d50 0 2024-03-13T14:49:33.736 INFO:tasks.workunit.client.1.smithi196.stdout:1/247: read - d1/d28/d34/f41 zero size 2024-03-13T14:49:33.736 INFO:tasks.workunit.client.1.smithi196.stdout:7/354: mknod d0/d6/c6f 0 2024-03-13T14:49:33.740 INFO:tasks.workunit.client.0.smithi176.stdout:7/91: write d1/d13/f14 [4741450,60795] 0 2024-03-13T14:49:33.750 INFO:tasks.workunit.client.0.smithi176.stdout:8/79: creat d0/f16 x:0 0 0 2024-03-13T14:49:33.750 INFO:tasks.workunit.client.0.smithi176.stdout:1/76: unlink d2/de/d10/d14/c15 0 2024-03-13T14:49:33.751 INFO:tasks.workunit.client.1.smithi196.stdout:8/214: link d0/d10/d12/d17/d40/l46 d0/d1c/d29/l4d 0 2024-03-13T14:49:33.751 INFO:tasks.workunit.client.1.smithi196.stdout:6/379: dread d6/d9/d11/f48 [0,4194304] 0 2024-03-13T14:49:33.751 INFO:tasks.workunit.client.1.smithi196.stdout:3/271: dwrite db/f29 [0,4194304] 0 2024-03-13T14:49:33.752 INFO:tasks.workunit.client.0.smithi176.stdout:3/88: write d4/f5 [5910744,121165] 0 2024-03-13T14:49:33.752 INFO:tasks.workunit.client.1.smithi196.stdout:8/215: chown d0/d1c/l2e 274 1 2024-03-13T14:49:33.753 INFO:tasks.workunit.client.0.smithi176.stdout:1/77: write f1 [4697026,33157] 0 2024-03-13T14:49:33.753 INFO:tasks.workunit.client.0.smithi176.stdout:8/80: chown d0/db/fd 3109856 1 2024-03-13T14:49:33.754 INFO:tasks.workunit.client.0.smithi176.stdout:1/78: chown d2/de 2708 1 2024-03-13T14:49:33.756 INFO:tasks.workunit.client.1.smithi196.stdout:8/216: write d0/d10/d12/d1e/f34 [701821,69334] 0 2024-03-13T14:49:33.756 INFO:tasks.workunit.client.0.smithi176.stdout:1/79: dread - d2/de/f17 zero size 2024-03-13T14:49:33.757 INFO:tasks.workunit.client.1.smithi196.stdout:2/180: rename d0/d6/d1d/d28/f33 to d0/d31/f36 0 2024-03-13T14:49:33.758 INFO:tasks.workunit.client.0.smithi176.stdout:1/80: dread - d2/de/f17 zero size 2024-03-13T14:49:33.761 INFO:tasks.workunit.client.0.smithi176.stdout:6/89: creat d0/d9/f1f x:0 0 0 2024-03-13T14:49:33.763 INFO:tasks.workunit.client.1.smithi196.stdout:7/355: write d0/d9/d1e/d3f/f60 [17737,34824] 0 2024-03-13T14:49:33.767 INFO:tasks.workunit.client.1.smithi196.stdout:9/240: chown d7/d9/f11 2 1 2024-03-13T14:49:33.768 INFO:tasks.workunit.client.0.smithi176.stdout:7/92: mkdir d1/d2/d17 0 2024-03-13T14:49:33.769 INFO:tasks.workunit.client.1.smithi196.stdout:4/238: mknod d1/d3/c41 0 2024-03-13T14:49:33.773 INFO:tasks.workunit.client.1.smithi196.stdout:2/181: dread d0/f1a [0,4194304] 0 2024-03-13T14:49:33.775 INFO:tasks.workunit.client.0.smithi176.stdout:7/93: dread d1/d7/f10 [0,4194304] 0 2024-03-13T14:49:33.780 INFO:tasks.workunit.client.1.smithi196.stdout:5/374: symlink d4/db/d3c/d6a/l76 0 2024-03-13T14:49:33.782 INFO:tasks.workunit.client.0.smithi176.stdout:2/91: dread d1/dc/d1c/f1d [0,4194304] 0 2024-03-13T14:49:33.783 INFO:tasks.workunit.client.1.smithi196.stdout:8/217: symlink d0/d1c/d29/l4e 0 2024-03-13T14:49:33.785 INFO:tasks.workunit.client.0.smithi176.stdout:8/81: rmdir d0 39 2024-03-13T14:49:33.790 INFO:tasks.workunit.client.1.smithi196.stdout:9/241: symlink d7/de/d16/l47 0 2024-03-13T14:49:33.794 INFO:tasks.workunit.client.0.smithi176.stdout:7/94: stat d1/d7/cd 0 2024-03-13T14:49:33.796 INFO:tasks.workunit.client.0.smithi176.stdout:1/81: mkdir d2/dd/d19 0 2024-03-13T14:49:33.797 INFO:tasks.workunit.client.0.smithi176.stdout:1/82: chown d2/lb 670987 1 2024-03-13T14:49:33.798 INFO:tasks.workunit.client.1.smithi196.stdout:8/218: dwrite d0/d10/f19 [0,4194304] 0 2024-03-13T14:49:33.798 INFO:tasks.workunit.client.0.smithi176.stdout:6/90: dread d0/f18 [0,4194304] 0 2024-03-13T14:49:33.799 INFO:tasks.workunit.client.1.smithi196.stdout:8/219: stat d0/d10/d12/c20 0 2024-03-13T14:49:33.800 INFO:tasks.workunit.client.0.smithi176.stdout:6/91: readlink d0/la 0 2024-03-13T14:49:33.802 INFO:tasks.workunit.client.0.smithi176.stdout:4/95: truncate d7/d8/f16 2578949 0 2024-03-13T14:49:33.804 INFO:tasks.workunit.client.0.smithi176.stdout:2/92: truncate d1/dc/d1c/f1d 3604545 0 2024-03-13T14:49:33.806 INFO:tasks.workunit.client.1.smithi196.stdout:5/375: creat d4/db/d14/d19/d26/d68/f77 x:0 0 0 2024-03-13T14:49:33.810 INFO:tasks.workunit.client.1.smithi196.stdout:6/380: truncate d6/d57/f63 15192 0 2024-03-13T14:49:33.810 INFO:tasks.workunit.client.1.smithi196.stdout:1/248: link d1/l6 d1/d28/l51 0 2024-03-13T14:49:33.811 INFO:tasks.workunit.client.0.smithi176.stdout:0/81: dwrite d2/d5/f8 [0,4194304] 0 2024-03-13T14:49:33.812 INFO:tasks.workunit.client.1.smithi196.stdout:7/356: creat d0/d9/d1e/d2d/d6e/f70 x:0 0 0 2024-03-13T14:49:33.812 INFO:tasks.workunit.client.0.smithi176.stdout:0/82: chown d2/db 58071 1 2024-03-13T14:49:33.813 INFO:tasks.workunit.client.0.smithi176.stdout:1/83: mknod d2/c1a 0 2024-03-13T14:49:33.816 INFO:tasks.workunit.client.1.smithi196.stdout:6/381: read d6/d2a/d23/d28/f2f [129992,59036] 0 2024-03-13T14:49:33.818 INFO:tasks.workunit.client.0.smithi176.stdout:1/84: read d2/f5 [816269,86842] 0 2024-03-13T14:49:33.818 INFO:tasks.workunit.client.1.smithi196.stdout:4/239: truncate d1/f6 1811898 0 2024-03-13T14:49:33.821 INFO:tasks.workunit.client.0.smithi176.stdout:6/92: symlink d0/d9/l20 0 2024-03-13T14:49:33.821 INFO:tasks.workunit.client.0.smithi176.stdout:1/85: stat d2/de/d10/d16 0 2024-03-13T14:49:33.822 INFO:tasks.workunit.client.1.smithi196.stdout:4/240: write d1/d3/f36 [735526,31311] 0 2024-03-13T14:49:33.823 INFO:tasks.workunit.client.1.smithi196.stdout:8/220: dwrite d0/d10/d12/f3e [0,4194304] 0 2024-03-13T14:49:33.826 INFO:tasks.workunit.client.0.smithi176.stdout:5/77: getdents d5/de 0 2024-03-13T14:49:33.828 INFO:tasks.workunit.client.1.smithi196.stdout:5/376: rmdir d4/db/d14/d19/d26 39 2024-03-13T14:49:33.832 INFO:tasks.workunit.client.0.smithi176.stdout:7/95: mkdir d1/d2/d17/d18 0 2024-03-13T14:49:33.834 INFO:tasks.workunit.client.1.smithi196.stdout:1/249: rename d1/f31 to d1/d28/d34/f52 0 2024-03-13T14:49:33.836 INFO:tasks.workunit.client.0.smithi176.stdout:4/96: dwrite d7/d10/f20 [0,4194304] 0 2024-03-13T14:49:33.837 INFO:tasks.workunit.client.0.smithi176.stdout:0/83: unlink d2/d5/f8 0 2024-03-13T14:49:33.839 INFO:tasks.workunit.client.1.smithi196.stdout:7/357: unlink d0/d4d/l5d 0 2024-03-13T14:49:33.839 INFO:tasks.workunit.client.0.smithi176.stdout:4/97: read d7/fe [3584282,113821] 0 2024-03-13T14:49:33.841 INFO:tasks.workunit.client.1.smithi196.stdout:7/358: chown d0/d6/f1d 4128303 1 2024-03-13T14:49:33.844 INFO:tasks.workunit.client.0.smithi176.stdout:2/93: dwrite d1/f15 [4194304,4194304] 0 2024-03-13T14:49:33.844 INFO:tasks.workunit.client.1.smithi196.stdout:2/182: rmdir d0/d6/d1d/d28 0 2024-03-13T14:49:33.848 INFO:tasks.workunit.client.1.smithi196.stdout:8/221: truncate d0/d10/d12/d17/f28 4465701 0 2024-03-13T14:49:33.854 INFO:tasks.workunit.client.0.smithi176.stdout:2/94: read d1/dc/f16 [505757,76127] 0 2024-03-13T14:49:33.856 INFO:tasks.workunit.client.0.smithi176.stdout:2/95: write d1/ff [1438145,25581] 0 2024-03-13T14:49:33.857 INFO:tasks.workunit.client.0.smithi176.stdout:2/96: chown d1/ff 166299 1 2024-03-13T14:49:33.858 INFO:tasks.workunit.client.0.smithi176.stdout:2/97: readlink d1/dc/l1f 0 2024-03-13T14:49:33.859 INFO:tasks.workunit.client.0.smithi176.stdout:6/93: dread d0/f13 [0,4194304] 0 2024-03-13T14:49:33.934 INFO:tasks.workunit.client.0.smithi176.stdout:9/117: sync 2024-03-13T14:49:33.934 INFO:tasks.workunit.client.0.smithi176.stdout:3/89: sync 2024-03-13T14:49:33.998 INFO:tasks.workunit.client.1.smithi196.stdout:5/377: creat d4/db/d14/d37/f78 x:0 0 0 2024-03-13T14:49:34.005 INFO:tasks.workunit.client.0.smithi176.stdout:5/78: symlink d5/de/l18 0 2024-03-13T14:49:34.009 INFO:tasks.workunit.client.0.smithi176.stdout:5/79: rename d5/de to d5/de/d15/d19 22 2024-03-13T14:49:34.010 INFO:tasks.workunit.client.0.smithi176.stdout:0/84: sync 2024-03-13T14:49:34.016 INFO:tasks.workunit.client.1.smithi196.stdout:5/378: dread d4/db/d3c/f52 [0,4194304] 0 2024-03-13T14:49:34.019 INFO:tasks.workunit.client.0.smithi176.stdout:4/98: mkdir d7/d8/d15/d23 0 2024-03-13T14:49:34.023 INFO:tasks.workunit.client.1.smithi196.stdout:2/183: stat d0/d6/cc 0 2024-03-13T14:49:34.025 INFO:tasks.workunit.client.1.smithi196.stdout:2/184: dread - d0/f25 zero size 2024-03-13T14:49:34.026 INFO:tasks.workunit.client.1.smithi196.stdout:7/359: dread d0/f37 [0,4194304] 0 2024-03-13T14:49:34.029 INFO:tasks.workunit.client.1.smithi196.stdout:4/241: rename d1/d3/d7/c3f to d1/d3/d7/c42 0 2024-03-13T14:49:34.030 INFO:tasks.workunit.client.0.smithi176.stdout:6/94: mknod d0/d9/c21 0 2024-03-13T14:49:34.031 INFO:tasks.workunit.client.1.smithi196.stdout:8/222: symlink d0/l4f 0 2024-03-13T14:49:34.033 INFO:tasks.workunit.client.1.smithi196.stdout:6/382: dwrite f4 [0,4194304] 0 2024-03-13T14:49:34.033 INFO:tasks.workunit.client.1.smithi196.stdout:4/242: truncate d1/d1e/f33 768304 0 2024-03-13T14:49:34.039 INFO:tasks.workunit.client.0.smithi176.stdout:6/95: write d0/d9/f14 [785503,51652] 0 2024-03-13T14:49:34.040 INFO:tasks.workunit.client.0.smithi176.stdout:9/118: mknod d0/d1e/c29 0 2024-03-13T14:49:34.040 INFO:tasks.workunit.client.1.smithi196.stdout:6/383: readlink d6/d2a/dc/d1c/d34/l4f 0 2024-03-13T14:49:34.043 INFO:tasks.workunit.client.0.smithi176.stdout:3/90: creat d4/d13/f19 x:0 0 0 2024-03-13T14:49:34.044 INFO:tasks.workunit.client.0.smithi176.stdout:1/86: mkdir d2/de/d10/d16/d1b 0 2024-03-13T14:49:34.045 INFO:tasks.workunit.client.1.smithi196.stdout:1/250: mknod d1/d9/c53 0 2024-03-13T14:49:34.045 INFO:tasks.workunit.client.1.smithi196.stdout:5/379: mknod d4/db/d14/d19/d1c/d25/d38/c79 0 2024-03-13T14:49:34.047 INFO:tasks.workunit.client.1.smithi196.stdout:1/251: fdatasync d1/f26 0 2024-03-13T14:49:34.048 INFO:tasks.workunit.client.0.smithi176.stdout:0/85: creat d2/f11 x:0 0 0 2024-03-13T14:49:34.049 INFO:tasks.workunit.client.1.smithi196.stdout:1/252: fsync d1/d38/f3b 0 2024-03-13T14:49:34.051 INFO:tasks.workunit.client.0.smithi176.stdout:4/99: rename d7/d10/f20 to d7/f24 0 2024-03-13T14:49:34.051 INFO:tasks.workunit.client.1.smithi196.stdout:4/243: dread d1/d3/f36 [0,4194304] 0 2024-03-13T14:49:34.053 INFO:tasks.workunit.client.0.smithi176.stdout:8/82: getdents d0/d5 0 2024-03-13T14:49:34.055 INFO:tasks.workunit.client.1.smithi196.stdout:4/244: fdatasync d1/f2e 0 2024-03-13T14:49:34.060 INFO:tasks.workunit.client.0.smithi176.stdout:7/96: creat d1/f19 x:0 0 0 2024-03-13T14:49:34.061 INFO:tasks.workunit.client.0.smithi176.stdout:3/91: mkdir d4/d1a 0 2024-03-13T14:49:34.063 INFO:tasks.workunit.client.1.smithi196.stdout:5/380: rename d4/d1b/c2e to d4/db/d14/d37/d41/c7a 0 2024-03-13T14:49:34.063 INFO:tasks.workunit.client.1.smithi196.stdout:6/384: dwrite d6/d7/dd/f2c [0,4194304] 0 2024-03-13T14:49:34.066 INFO:tasks.workunit.client.1.smithi196.stdout:5/381: stat d4/db/d14/d37/d41 0 2024-03-13T14:49:34.066 INFO:tasks.workunit.client.0.smithi176.stdout:4/100: dread d7/fc [0,4194304] 0 2024-03-13T14:49:34.072 INFO:tasks.workunit.client.1.smithi196.stdout:2/185: creat d0/f37 x:0 0 0 2024-03-13T14:49:34.073 INFO:tasks.workunit.client.1.smithi196.stdout:7/360: dwrite d0/d6/f26 [0,4194304] 0 2024-03-13T14:49:34.077 INFO:tasks.workunit.client.0.smithi176.stdout:6/96: rename d0/df/l1b to d0/d9/l22 0 2024-03-13T14:49:34.083 INFO:tasks.workunit.client.0.smithi176.stdout:9/119: stat d0/db/f1d 0 2024-03-13T14:49:34.083 INFO:tasks.workunit.client.1.smithi196.stdout:4/245: dread d1/d3/d7/f21 [0,4194304] 0 2024-03-13T14:49:34.084 INFO:tasks.workunit.client.1.smithi196.stdout:6/385: mkdir d6/d57/d7a 0 2024-03-13T14:49:34.086 INFO:tasks.workunit.client.0.smithi176.stdout:2/98: sync 2024-03-13T14:49:34.087 INFO:tasks.workunit.client.1.smithi196.stdout:1/253: unlink d1/d3f/c45 0 2024-03-13T14:49:34.089 INFO:tasks.workunit.client.0.smithi176.stdout:1/87: mknod d2/dd/d19/c1c 0 2024-03-13T14:49:34.092 INFO:tasks.workunit.client.0.smithi176.stdout:1/88: write f1 [3191231,1861] 0 2024-03-13T14:49:34.093 INFO:tasks.workunit.client.1.smithi196.stdout:7/361: mknod d0/d9/d1e/d2d/c71 0 2024-03-13T14:49:34.097 INFO:tasks.workunit.client.1.smithi196.stdout:4/246: mkdir d1/d1e/d43 0 2024-03-13T14:49:34.101 INFO:tasks.workunit.client.0.smithi176.stdout:7/97: creat d1/d7/f1a x:0 0 0 2024-03-13T14:49:34.103 INFO:tasks.workunit.client.1.smithi196.stdout:2/186: mkdir d0/d6/d34/d38 0 2024-03-13T14:49:34.107 INFO:tasks.workunit.client.1.smithi196.stdout:5/382: rename d4/l18 to d4/db/d14/d19/d26/d33/d61/l7b 0 2024-03-13T14:49:34.108 INFO:tasks.workunit.client.0.smithi176.stdout:6/97: truncate d0/fc 240040 0 2024-03-13T14:49:34.109 INFO:tasks.workunit.client.0.smithi176.stdout:0/86: creat d2/db/f12 x:0 0 0 2024-03-13T14:49:34.112 INFO:tasks.workunit.client.0.smithi176.stdout:8/83: link d0/f16 d0/d5/f17 0 2024-03-13T14:49:34.115 INFO:tasks.workunit.client.0.smithi176.stdout:0/87: write d2/db/f12 [240361,32218] 0 2024-03-13T14:49:34.117 INFO:tasks.workunit.client.0.smithi176.stdout:8/84: read - d0/d5/f17 zero size 2024-03-13T14:49:34.120 INFO:tasks.workunit.client.0.smithi176.stdout:0/88: dread - d2/f11 zero size 2024-03-13T14:49:34.122 INFO:tasks.workunit.client.0.smithi176.stdout:2/99: dread d1/f8 [0,4194304] 0 2024-03-13T14:49:34.130 INFO:tasks.workunit.client.0.smithi176.stdout:1/89: dread d2/f9 [0,4194304] 0 2024-03-13T14:49:34.132 INFO:tasks.workunit.client.0.smithi176.stdout:9/120: dread d0/d13/f16 [0,4194304] 0 2024-03-13T14:49:34.139 INFO:tasks.workunit.client.0.smithi176.stdout:0/89: dread d2/f6 [0,4194304] 0 2024-03-13T14:49:34.142 INFO:tasks.workunit.client.0.smithi176.stdout:7/98: dwrite d1/d7/f1a [0,4194304] 0 2024-03-13T14:49:34.237 INFO:tasks.workunit.client.0.smithi176.stdout:0/90: sync 2024-03-13T14:49:34.240 INFO:tasks.workunit.client.1.smithi196.stdout:4/247: write d1/d3/d7/f1a [12834947,50026] 0 2024-03-13T14:49:34.242 INFO:tasks.workunit.client.0.smithi176.stdout:0/91: write d2/db/f12 [488349,78216] 0 2024-03-13T14:49:34.242 INFO:tasks.workunit.client.1.smithi196.stdout:4/248: fsync d1/d3/f13 0 2024-03-13T14:49:34.244 INFO:tasks.workunit.client.1.smithi196.stdout:1/254: creat d1/d3f/d50/f54 x:0 0 0 2024-03-13T14:49:34.246 INFO:tasks.workunit.client.1.smithi196.stdout:1/255: readlink d1/d28/l3c 0 2024-03-13T14:49:34.246 INFO:tasks.workunit.client.0.smithi176.stdout:3/92: creat d4/d1a/f1b x:0 0 0 2024-03-13T14:49:34.248 INFO:tasks.workunit.client.0.smithi176.stdout:3/93: chown d4/d13/c18 2771173 1 2024-03-13T14:49:34.248 INFO:tasks.workunit.client.1.smithi196.stdout:5/383: symlink d4/db/d14/d19/d26/d33/d61/l7c 0 2024-03-13T14:49:34.249 INFO:tasks.workunit.client.0.smithi176.stdout:3/94: truncate d4/d1a/f1b 561256 0 2024-03-13T14:49:34.254 INFO:tasks.workunit.client.0.smithi176.stdout:5/80: link d5/ld d5/de/d15/l1a 0 2024-03-13T14:49:34.257 INFO:tasks.workunit.client.0.smithi176.stdout:6/98: chown d0/d9/l1e 275045 1 2024-03-13T14:49:34.262 INFO:tasks.workunit.client.1.smithi196.stdout:1/256: mkdir d1/d9/d4e/d55 0 2024-03-13T14:49:34.265 INFO:tasks.workunit.client.1.smithi196.stdout:5/384: mkdir d4/db/d14/d37/d41/d7d 0 2024-03-13T14:49:34.267 INFO:tasks.workunit.client.0.smithi176.stdout:8/85: symlink d0/db/dc/l18 0 2024-03-13T14:49:34.268 INFO:tasks.workunit.client.1.smithi196.stdout:6/386: write d6/d57/f63 [313092,87460] 0 2024-03-13T14:49:34.268 INFO:tasks.workunit.client.1.smithi196.stdout:4/249: symlink d1/d1e/d43/l44 0 2024-03-13T14:49:34.269 INFO:tasks.workunit.client.0.smithi176.stdout:2/100: symlink d1/dc/l28 0 2024-03-13T14:49:34.270 INFO:tasks.workunit.client.1.smithi196.stdout:6/387: readlink d6/d7/dd/l12 0 2024-03-13T14:49:34.271 INFO:tasks.workunit.client.1.smithi196.stdout:5/385: write d4/db/d14/d19/d26/d68/f77 [893944,129263] 0 2024-03-13T14:49:34.271 INFO:tasks.workunit.client.1.smithi196.stdout:7/362: dwrite d0/d6/d33/f39 [0,4194304] 0 2024-03-13T14:49:34.272 INFO:tasks.workunit.client.0.smithi176.stdout:1/90: mknod d2/dd/c1d 0 2024-03-13T14:49:34.274 INFO:tasks.workunit.client.0.smithi176.stdout:1/91: truncate d2/f6 824302 0 2024-03-13T14:49:34.274 INFO:tasks.workunit.client.1.smithi196.stdout:1/257: creat d1/d28/f56 x:0 0 0 2024-03-13T14:49:34.276 INFO:tasks.workunit.client.1.smithi196.stdout:4/250: fsync d1/d3/f13 0 2024-03-13T14:49:34.281 INFO:tasks.workunit.client.0.smithi176.stdout:4/101: write d7/d8/f16 [3285006,123881] 0 2024-03-13T14:49:34.282 INFO:tasks.workunit.client.1.smithi196.stdout:5/386: rename d4/l6 to d4/db/d4a/l7e 0 2024-03-13T14:49:34.284 INFO:tasks.workunit.client.1.smithi196.stdout:1/258: mknod d1/d3f/d50/c57 0 2024-03-13T14:49:34.287 INFO:tasks.workunit.client.1.smithi196.stdout:1/259: chown d1/f37 11828 1 2024-03-13T14:49:34.289 INFO:tasks.workunit.client.1.smithi196.stdout:5/387: chown d4/db/d14/d37/d41/f42 21201 1 2024-03-13T14:49:34.290 INFO:tasks.workunit.client.1.smithi196.stdout:4/251: creat d1/d1e/d39/f45 x:0 0 0 2024-03-13T14:49:34.293 INFO:tasks.workunit.client.0.smithi176.stdout:4/102: dread d7/d8/f18 [0,4194304] 0 2024-03-13T14:49:34.294 INFO:tasks.workunit.client.1.smithi196.stdout:7/363: rename d0/d9/d1e/d2d to d0/d9/d1e/d58/d72 0 2024-03-13T14:49:34.294 INFO:tasks.workunit.client.1.smithi196.stdout:4/252: write d1/d1e/f33 [851942,108217] 0 2024-03-13T14:49:34.298 INFO:tasks.workunit.client.1.smithi196.stdout:1/260: mkdir d1/d9/d4e/d58 0 2024-03-13T14:49:34.300 INFO:tasks.workunit.client.0.smithi176.stdout:3/95: symlink d4/d13/l1c 0 2024-03-13T14:49:34.300 INFO:tasks.workunit.client.1.smithi196.stdout:6/388: dwrite d6/d2a/d23/d28/f69 [0,4194304] 0 2024-03-13T14:49:34.301 INFO:tasks.workunit.client.0.smithi176.stdout:3/96: chown d4/d13 4 1 2024-03-13T14:49:34.307 INFO:tasks.workunit.client.0.smithi176.stdout:5/81: symlink d5/de/l1b 0 2024-03-13T14:49:34.307 INFO:tasks.workunit.client.1.smithi196.stdout:7/364: chown d0/l1a 96588 1 2024-03-13T14:49:34.309 INFO:tasks.workunit.client.1.smithi196.stdout:5/388: symlink d4/db/d14/d19/d1c/d25/d38/d47/l7f 0 2024-03-13T14:49:34.309 INFO:tasks.workunit.client.0.smithi176.stdout:5/82: readlink d5/de/l1b 0 2024-03-13T14:49:34.311 INFO:tasks.workunit.client.1.smithi196.stdout:5/389: truncate d4/f11 5872225 0 2024-03-13T14:49:34.316 INFO:tasks.workunit.client.1.smithi196.stdout:5/390: chown d4/db/d14/d19/c53 9990 1 2024-03-13T14:49:34.318 INFO:tasks.workunit.client.0.smithi176.stdout:8/86: rename d0/f16 to d0/db/f19 0 2024-03-13T14:49:34.319 INFO:tasks.workunit.client.1.smithi196.stdout:0/357: sync 2024-03-13T14:49:34.319 INFO:tasks.workunit.client.1.smithi196.stdout:7/365: mknod d0/d9/d31/c73 0 2024-03-13T14:49:34.323 INFO:tasks.workunit.client.1.smithi196.stdout:1/261: symlink d1/d9/d4e/d58/l59 0 2024-03-13T14:49:34.325 INFO:tasks.workunit.client.0.smithi176.stdout:9/121: dwrite d0/d13/f16 [0,4194304] 0 2024-03-13T14:49:34.326 INFO:tasks.workunit.client.1.smithi196.stdout:5/391: mkdir d4/db/d14/d37/d41/d80 0 2024-03-13T14:49:34.327 INFO:tasks.workunit.client.1.smithi196.stdout:7/366: fsync d0/f6a 0 2024-03-13T14:49:34.329 INFO:tasks.workunit.client.1.smithi196.stdout:7/367: readlink d0/l2a 0 2024-03-13T14:49:34.330 INFO:tasks.workunit.client.0.smithi176.stdout:7/99: dwrite d1/d7/f10 [0,4194304] 0 2024-03-13T14:49:34.335 INFO:tasks.workunit.client.1.smithi196.stdout:1/262: creat d1/d28/f5a x:0 0 0 2024-03-13T14:49:34.335 INFO:tasks.workunit.client.1.smithi196.stdout:5/392: creat d4/db/f81 x:0 0 0 2024-03-13T14:49:34.336 INFO:tasks.workunit.client.0.smithi176.stdout:0/92: creat d2/d5/dc/f13 x:0 0 0 2024-03-13T14:49:34.336 INFO:tasks.workunit.client.0.smithi176.stdout:3/97: creat d4/f1d x:0 0 0 2024-03-13T14:49:34.338 INFO:tasks.workunit.client.1.smithi196.stdout:7/368: mkdir d0/d4d/d74 0 2024-03-13T14:49:34.339 INFO:tasks.workunit.client.0.smithi176.stdout:8/87: creat d0/db/f1a x:0 0 0 2024-03-13T14:49:34.341 INFO:tasks.workunit.client.1.smithi196.stdout:1/263: fsync d1/d28/d34/f52 0 2024-03-13T14:49:34.342 INFO:tasks.workunit.client.1.smithi196.stdout:6/389: dread d6/d9/d17/d19/f20 [0,4194304] 0 2024-03-13T14:49:34.342 INFO:tasks.workunit.client.0.smithi176.stdout:5/83: dwrite d5/f14 [0,4194304] 0 2024-03-13T14:49:34.343 INFO:tasks.workunit.client.1.smithi196.stdout:4/253: dread d1/d3/fa [0,4194304] 0 2024-03-13T14:49:34.344 INFO:tasks.workunit.client.1.smithi196.stdout:7/369: dread - d0/d9/d1e/d58/d72/d6e/f70 zero size 2024-03-13T14:49:34.346 INFO:tasks.workunit.client.1.smithi196.stdout:5/393: rmdir d4/db/d14/d19/d1c/d25/d38 39 2024-03-13T14:49:34.346 INFO:tasks.workunit.client.1.smithi196.stdout:0/358: getdents d8/da/d25/d26/d35/d3b/d5f/d43 0 2024-03-13T14:49:34.347 INFO:tasks.workunit.client.0.smithi176.stdout:1/92: dread d2/f9 [0,4194304] 0 2024-03-13T14:49:34.349 INFO:tasks.workunit.client.1.smithi196.stdout:1/264: write d1/d3f/d50/f54 [16295,18041] 0 2024-03-13T14:49:34.353 INFO:tasks.workunit.client.0.smithi176.stdout:8/88: write d0/f9 [3939000,55525] 0 2024-03-13T14:49:34.356 INFO:tasks.workunit.client.1.smithi196.stdout:4/254: mkdir d1/d1e/d39/d46 0 2024-03-13T14:49:34.358 INFO:tasks.workunit.client.0.smithi176.stdout:8/89: write d0/f14 [1505634,128549] 0 2024-03-13T14:49:34.358 INFO:tasks.workunit.client.1.smithi196.stdout:4/255: truncate d1/f2e 666780 0 2024-03-13T14:49:34.359 INFO:tasks.workunit.client.0.smithi176.stdout:6/99: dwrite d0/f13 [0,4194304] 0 2024-03-13T14:49:34.360 INFO:tasks.workunit.client.0.smithi176.stdout:3/98: rename d4/c11 to d4/c1e 0 2024-03-13T14:49:34.363 INFO:tasks.workunit.client.1.smithi196.stdout:5/394: mkdir d4/db/d14/d19/d1c/d2d/d82 0 2024-03-13T14:49:34.363 INFO:tasks.workunit.client.0.smithi176.stdout:2/101: sync 2024-03-13T14:49:34.364 INFO:tasks.workunit.client.0.smithi176.stdout:6/100: chown d0/df/l15 2475299 1 2024-03-13T14:49:34.365 INFO:tasks.workunit.client.1.smithi196.stdout:7/370: truncate d0/d9/f24 4689565 0 2024-03-13T14:49:34.367 INFO:tasks.workunit.client.0.smithi176.stdout:2/102: chown d1/dc/cd 127 1 2024-03-13T14:49:34.367 INFO:tasks.workunit.client.0.smithi176.stdout:7/100: dwrite d1/d7/f10 [0,4194304] 0 2024-03-13T14:49:34.372 INFO:tasks.workunit.client.1.smithi196.stdout:0/359: rmdir d8/da/d25/d26/d35/d3b/d5f/d43 39 2024-03-13T14:49:34.374 INFO:tasks.workunit.client.0.smithi176.stdout:5/84: symlink d5/de/d15/l1c 0 2024-03-13T14:49:34.376 INFO:tasks.workunit.client.0.smithi176.stdout:8/90: fsync d0/f9 0 2024-03-13T14:49:34.377 INFO:tasks.workunit.client.0.smithi176.stdout:5/85: write d5/f14 [2646777,41088] 0 2024-03-13T14:49:34.377 INFO:tasks.workunit.client.1.smithi196.stdout:1/265: rename d1/d3f/d50 to d1/d3f/d5b 0 2024-03-13T14:49:34.379 INFO:tasks.workunit.client.0.smithi176.stdout:8/91: chown d0/db/f1a 313267 1 2024-03-13T14:49:34.380 INFO:tasks.workunit.client.1.smithi196.stdout:4/256: mknod d1/d3/d7/c47 0 2024-03-13T14:49:34.382 INFO:tasks.workunit.client.1.smithi196.stdout:4/257: fdatasync d1/d3/f3a 0 2024-03-13T14:49:34.385 INFO:tasks.workunit.client.1.smithi196.stdout:6/390: creat d6/d9/f7b x:0 0 0 2024-03-13T14:49:34.386 INFO:tasks.workunit.client.1.smithi196.stdout:5/395: dwrite d4/db/f81 [0,4194304] 0 2024-03-13T14:49:34.389 INFO:tasks.workunit.client.1.smithi196.stdout:0/360: creat d8/da/d25/f7b x:0 0 0 2024-03-13T14:49:34.393 INFO:tasks.workunit.client.0.smithi176.stdout:6/101: creat d0/de/f23 x:0 0 0 2024-03-13T14:49:34.400 INFO:tasks.workunit.client.1.smithi196.stdout:7/371: truncate d0/f11 1039944 0 2024-03-13T14:49:34.402 INFO:tasks.workunit.client.0.smithi176.stdout:2/103: mknod d1/dc/d1c/c29 0 2024-03-13T14:49:34.404 INFO:tasks.workunit.client.1.smithi196.stdout:4/258: readlink d1/d3/l1f 0 2024-03-13T14:49:34.405 INFO:tasks.workunit.client.0.smithi176.stdout:6/102: dread d0/f8 [4194304,4194304] 0 2024-03-13T14:49:34.408 INFO:tasks.workunit.client.0.smithi176.stdout:7/101: rename d1/d2/d17 to d1/d13/d1b 0 2024-03-13T14:49:34.410 INFO:tasks.workunit.client.1.smithi196.stdout:6/391: creat d6/d9/d17/d19/d4e/f7c x:0 0 0 2024-03-13T14:49:34.415 INFO:tasks.workunit.client.1.smithi196.stdout:5/396: dwrite d4/d1b/f29 [0,4194304] 0 2024-03-13T14:49:34.415 INFO:tasks.workunit.client.1.smithi196.stdout:6/392: dread - d6/d9/f7b zero size 2024-03-13T14:49:34.416 INFO:tasks.workunit.client.0.smithi176.stdout:9/122: write d0/f10 [2240906,104518] 0 2024-03-13T14:49:34.417 INFO:tasks.workunit.client.1.smithi196.stdout:5/397: stat d4/db/d14/d19/l35 0 2024-03-13T14:49:34.417 INFO:tasks.workunit.client.1.smithi196.stdout:9/242: sync 2024-03-13T14:49:34.417 INFO:tasks.workunit.client.1.smithi196.stdout:2/187: sync 2024-03-13T14:49:34.418 INFO:tasks.workunit.client.1.smithi196.stdout:3/272: sync 2024-03-13T14:49:34.418 INFO:tasks.workunit.client.1.smithi196.stdout:8/223: sync 2024-03-13T14:49:34.420 INFO:tasks.workunit.client.1.smithi196.stdout:3/273: stat db/d15/f17 0 2024-03-13T14:49:34.421 INFO:tasks.workunit.client.1.smithi196.stdout:0/361: rename d8/da/d25/d26/f42 to d8/da/d25/d26/d35/d3b/d6c/f7c 0 2024-03-13T14:49:34.424 INFO:tasks.workunit.client.1.smithi196.stdout:3/274: chown db/d19/d22/d30/f42 230405 1 2024-03-13T14:49:34.426 INFO:tasks.workunit.client.0.smithi176.stdout:8/92: creat d0/d5/d11/f1b x:0 0 0 2024-03-13T14:49:34.426 INFO:tasks.workunit.client.1.smithi196.stdout:3/275: stat db/d19/d22/d30/c4d 0 2024-03-13T14:49:34.426 INFO:tasks.workunit.client.0.smithi176.stdout:1/93: dwrite d2/f9 [0,4194304] 0 2024-03-13T14:49:34.429 INFO:tasks.workunit.client.0.smithi176.stdout:8/93: stat d0/la 0 2024-03-13T14:49:34.430 INFO:tasks.workunit.client.0.smithi176.stdout:4/103: getdents d7/d8 0 2024-03-13T14:49:34.431 INFO:tasks.workunit.client.1.smithi196.stdout:7/372: creat d0/d9/d31/d50/f75 x:0 0 0 2024-03-13T14:49:34.431 INFO:tasks.workunit.client.0.smithi176.stdout:4/104: readlink d7/d8/d15/l22 0 2024-03-13T14:49:34.432 INFO:tasks.workunit.client.0.smithi176.stdout:4/105: fsync d7/d8/f16 0 2024-03-13T14:49:34.433 INFO:tasks.workunit.client.1.smithi196.stdout:1/266: link d1/f4c d1/d9/f5c 0 2024-03-13T14:49:34.440 INFO:tasks.workunit.client.0.smithi176.stdout:2/104: rmdir d1 39 2024-03-13T14:49:34.441 INFO:tasks.workunit.client.0.smithi176.stdout:6/103: truncate d0/f18 1633016 0 2024-03-13T14:49:34.442 INFO:tasks.workunit.client.1.smithi196.stdout:5/398: dread d4/f11 [4194304,4194304] 0 2024-03-13T14:49:34.443 INFO:tasks.workunit.client.0.smithi176.stdout:6/104: write d0/d9/f14 [1959532,80426] 0 2024-03-13T14:49:34.444 INFO:tasks.workunit.client.1.smithi196.stdout:9/243: write d7/d9/df/d1d/f3b [393884,113265] 0 2024-03-13T14:49:34.444 INFO:tasks.workunit.client.0.smithi176.stdout:9/123: rmdir d0/db 39 2024-03-13T14:49:34.447 INFO:tasks.workunit.client.1.smithi196.stdout:8/224: creat d0/d10/d12/f50 x:0 0 0 2024-03-13T14:49:34.448 INFO:tasks.workunit.client.0.smithi176.stdout:1/94: symlink d2/dd/l1e 0 2024-03-13T14:49:34.452 INFO:tasks.workunit.client.1.smithi196.stdout:0/362: dwrite d8/da/fc [4194304,4194304] 0 2024-03-13T14:49:34.453 INFO:tasks.workunit.client.0.smithi176.stdout:0/93: getdents d2/db 0 2024-03-13T14:49:34.454 INFO:tasks.workunit.client.0.smithi176.stdout:0/94: readlink d2/l4 0 2024-03-13T14:49:34.459 INFO:tasks.workunit.client.0.smithi176.stdout:3/99: truncate d4/f5 4585110 0 2024-03-13T14:49:34.460 INFO:tasks.workunit.client.1.smithi196.stdout:3/276: creat db/d19/d22/d3a/d53/f59 x:0 0 0 2024-03-13T14:49:34.467 INFO:tasks.workunit.client.1.smithi196.stdout:1/267: creat d1/d3f/f5d x:0 0 0 2024-03-13T14:49:34.468 INFO:tasks.workunit.client.1.smithi196.stdout:2/188: mknod d0/c39 0 2024-03-13T14:49:34.468 INFO:tasks.workunit.client.0.smithi176.stdout:2/105: write d1/dc/f12 [1507198,3954] 0 2024-03-13T14:49:34.471 INFO:tasks.workunit.client.1.smithi196.stdout:9/244: symlink d7/d24/l48 0 2024-03-13T14:49:34.476 INFO:tasks.workunit.client.0.smithi176.stdout:9/124: stat d0/l3 0 2024-03-13T14:49:34.478 INFO:tasks.workunit.client.0.smithi176.stdout:5/86: creat d5/f1d x:0 0 0 2024-03-13T14:49:34.479 INFO:tasks.workunit.client.1.smithi196.stdout:4/259: link d1/d3/d7/c18 d1/d1e/c48 0 2024-03-13T14:49:34.480 INFO:tasks.workunit.client.0.smithi176.stdout:0/95: symlink d2/l14 0 2024-03-13T14:49:34.484 INFO:tasks.workunit.client.0.smithi176.stdout:3/100: unlink d4/d1a/f1b 0 2024-03-13T14:49:34.487 INFO:tasks.workunit.client.1.smithi196.stdout:1/268: symlink d1/d3f/d5b/l5e 0 2024-03-13T14:49:34.489 INFO:tasks.workunit.client.1.smithi196.stdout:1/269: chown d1/f37 3 1 2024-03-13T14:49:34.490 INFO:tasks.workunit.client.1.smithi196.stdout:2/189: symlink d0/d31/l3a 0 2024-03-13T14:49:34.494 INFO:tasks.workunit.client.0.smithi176.stdout:2/106: truncate d1/f8 959915 0 2024-03-13T14:49:34.495 INFO:tasks.workunit.client.0.smithi176.stdout:1/95: dwrite d2/f5 [0,4194304] 0 2024-03-13T14:49:34.496 INFO:tasks.workunit.client.1.smithi196.stdout:5/399: creat d4/db/d14/d19/d1c/d2d/d82/f83 x:0 0 0 2024-03-13T14:49:34.498 INFO:tasks.workunit.client.0.smithi176.stdout:3/101: dread f2 [0,4194304] 0 2024-03-13T14:49:34.500 INFO:tasks.workunit.client.1.smithi196.stdout:9/245: creat d7/de/d16/f49 x:0 0 0 2024-03-13T14:49:34.504 INFO:tasks.workunit.client.0.smithi176.stdout:5/87: creat d5/de/f1e x:0 0 0 2024-03-13T14:49:34.506 INFO:tasks.workunit.client.0.smithi176.stdout:7/102: truncate d1/d13/f14 1282280 0 2024-03-13T14:49:34.507 INFO:tasks.workunit.client.1.smithi196.stdout:6/393: getdents d6/d9 0 2024-03-13T14:49:34.508 INFO:tasks.workunit.client.0.smithi176.stdout:0/96: creat d2/d5/dc/f15 x:0 0 0 2024-03-13T14:49:34.508 INFO:tasks.workunit.client.0.smithi176.stdout:4/106: dwrite d7/fc [0,4194304] 0 2024-03-13T14:49:34.514 INFO:tasks.workunit.client.1.smithi196.stdout:4/260: truncate d1/d3/d7/f21 1937028 0 2024-03-13T14:49:34.517 INFO:tasks.workunit.client.1.smithi196.stdout:9/246: dwrite d7/f45 [0,4194304] 0 2024-03-13T14:49:34.519 INFO:tasks.workunit.client.0.smithi176.stdout:3/102: dwrite d4/d13/f19 [0,4194304] 0 2024-03-13T14:49:34.521 INFO:tasks.workunit.client.1.smithi196.stdout:9/247: truncate d7/d24/d28/f3e 368460 0 2024-03-13T14:49:34.532 INFO:tasks.workunit.client.0.smithi176.stdout:2/107: chown d1/l3 1 1 2024-03-13T14:49:34.533 INFO:tasks.workunit.client.1.smithi196.stdout:2/190: dread d0/d6/d1d/f23 [0,4194304] 0 2024-03-13T14:49:34.535 INFO:tasks.workunit.client.0.smithi176.stdout:1/96: symlink d2/de/d10/d16/l1f 0 2024-03-13T14:49:34.536 INFO:tasks.workunit.client.1.smithi196.stdout:2/191: fdatasync d0/fe 0 2024-03-13T14:49:34.542 INFO:tasks.workunit.client.1.smithi196.stdout:2/192: write d0/f25 [821643,62983] 0 2024-03-13T14:49:34.543 INFO:tasks.workunit.client.0.smithi176.stdout:8/94: sync 2024-03-13T14:49:34.552 INFO:tasks.workunit.client.0.smithi176.stdout:5/88: unlink d5/cf 0 2024-03-13T14:49:34.554 INFO:tasks.workunit.client.0.smithi176.stdout:4/107: fsync d7/fe 0 2024-03-13T14:49:34.556 INFO:tasks.workunit.client.0.smithi176.stdout:0/97: creat d2/d5/dc/f16 x:0 0 0 2024-03-13T14:49:34.557 INFO:tasks.workunit.client.0.smithi176.stdout:0/98: chown d2/f6 67 1 2024-03-13T14:49:34.561 INFO:tasks.workunit.client.0.smithi176.stdout:2/108: mkdir d1/d2a 0 2024-03-13T14:49:34.562 INFO:tasks.workunit.client.0.smithi176.stdout:6/105: getdents d0/d10/d19 0 2024-03-13T14:49:34.564 INFO:tasks.workunit.client.0.smithi176.stdout:9/125: link d0/d17/l1a d0/d1e/d1f/l2a 0 2024-03-13T14:49:34.565 INFO:tasks.workunit.client.0.smithi176.stdout:5/89: symlink d5/de/l1f 0 2024-03-13T14:49:34.566 INFO:tasks.workunit.client.0.smithi176.stdout:7/103: mkdir d1/d1c 0 2024-03-13T14:49:34.567 INFO:tasks.workunit.client.0.smithi176.stdout:7/104: dread - d1/f19 zero size 2024-03-13T14:49:34.567 INFO:tasks.workunit.client.0.smithi176.stdout:4/108: symlink d7/d8/l25 0 2024-03-13T14:49:34.568 INFO:tasks.workunit.client.0.smithi176.stdout:4/109: stat d7 0 2024-03-13T14:49:34.571 INFO:tasks.workunit.client.0.smithi176.stdout:5/90: mknod d5/de/d15/c20 0 2024-03-13T14:49:34.572 INFO:tasks.workunit.client.0.smithi176.stdout:5/91: chown d5 60450 1 2024-03-13T14:49:34.601 INFO:tasks.workunit.client.0.smithi176.stdout:3/103: write d4/f10 [353968,111530] 0 2024-03-13T14:49:34.602 INFO:tasks.workunit.client.0.smithi176.stdout:8/95: creat d0/f1c x:0 0 0 2024-03-13T14:49:34.605 INFO:tasks.workunit.client.0.smithi176.stdout:4/110: creat d7/f26 x:0 0 0 2024-03-13T14:49:34.607 INFO:tasks.workunit.client.0.smithi176.stdout:1/97: getdents d2 0 2024-03-13T14:49:34.610 INFO:tasks.workunit.client.0.smithi176.stdout:2/109: write d1/dc/d1c/f1d [3167936,127535] 0 2024-03-13T14:49:34.610 INFO:tasks.workunit.client.0.smithi176.stdout:9/126: getdents d0/d1e/d1f 0 2024-03-13T14:49:34.616 INFO:tasks.workunit.client.1.smithi196.stdout:0/363: creat d8/da/d25/d26/d35/d3b/f7d x:0 0 0 2024-03-13T14:49:34.619 INFO:tasks.workunit.client.0.smithi176.stdout:2/110: fdatasync d1/dc/d1c/f1d 0 2024-03-13T14:49:34.620 INFO:tasks.workunit.client.0.smithi176.stdout:8/96: symlink d0/db/dc/l1d 0 2024-03-13T14:49:34.621 INFO:tasks.workunit.client.1.smithi196.stdout:1/270: write d1/d9/d33/d4b/f1d [4114900,21043] 0 2024-03-13T14:49:34.621 INFO:tasks.workunit.client.1.smithi196.stdout:6/394: symlink d6/d2a/dc/d1c/l7d 0 2024-03-13T14:49:34.622 INFO:tasks.workunit.client.1.smithi196.stdout:9/248: dwrite f5 [4194304,4194304] 0 2024-03-13T14:49:34.623 INFO:tasks.workunit.client.0.smithi176.stdout:5/92: rename d5/de/f10 to d5/de/d15/f21 0 2024-03-13T14:49:34.623 INFO:tasks.workunit.client.0.smithi176.stdout:0/99: dwrite d2/d5/dc/f13 [0,4194304] 0 2024-03-13T14:49:34.630 INFO:tasks.workunit.client.1.smithi196.stdout:3/277: getdents db/d4c 0 2024-03-13T14:49:34.630 INFO:tasks.workunit.client.1.smithi196.stdout:5/400: creat d4/db/d3c/f84 x:0 0 0 2024-03-13T14:49:34.631 INFO:tasks.workunit.client.0.smithi176.stdout:8/97: symlink d0/d5/l1e 0 2024-03-13T14:49:34.631 INFO:tasks.workunit.client.0.smithi176.stdout:0/100: write d2/db/f12 [139664,63693] 0 2024-03-13T14:49:34.631 INFO:tasks.workunit.client.0.smithi176.stdout:9/127: symlink d0/l2b 0 2024-03-13T14:49:34.631 INFO:tasks.workunit.client.0.smithi176.stdout:7/105: dread d1/d7/f1a [0,4194304] 0 2024-03-13T14:49:34.631 INFO:tasks.workunit.client.0.smithi176.stdout:8/98: readlink d0/db/dc/l1d 0 2024-03-13T14:49:34.632 INFO:tasks.workunit.client.0.smithi176.stdout:4/111: truncate d7/d8/f16 2399207 0 2024-03-13T14:49:34.632 INFO:tasks.workunit.client.1.smithi196.stdout:3/278: chown db/f32 56697089 1 2024-03-13T14:49:34.634 INFO:tasks.workunit.client.0.smithi176.stdout:3/104: dread d4/fa [0,4194304] 0 2024-03-13T14:49:34.634 INFO:tasks.workunit.client.0.smithi176.stdout:1/98: dread d2/f9 [0,4194304] 0 2024-03-13T14:49:34.636 INFO:tasks.workunit.client.1.smithi196.stdout:2/193: creat d0/d6/d34/d38/f3b x:0 0 0 2024-03-13T14:49:34.636 INFO:tasks.workunit.client.0.smithi176.stdout:2/111: symlink d1/d2a/l2b 0 2024-03-13T14:49:34.637 INFO:tasks.workunit.client.0.smithi176.stdout:2/112: fsync d1/f14 0 2024-03-13T14:49:34.641 INFO:tasks.workunit.client.1.smithi196.stdout:8/225: dwrite d0/d10/d12/d17/d42/f4c [0,4194304] 0 2024-03-13T14:49:34.647 INFO:tasks.workunit.client.0.smithi176.stdout:0/101: rmdir d2 39 2024-03-13T14:49:34.652 INFO:tasks.workunit.client.1.smithi196.stdout:6/395: rename d6/d2a/dc/d1c/f3b to d6/d7/dd/f7e 0 2024-03-13T14:49:34.652 INFO:tasks.workunit.client.1.smithi196.stdout:0/364: dwrite d8/da/d25/d26/d35/d3b/f47 [0,4194304] 0 2024-03-13T14:49:34.653 INFO:tasks.workunit.client.0.smithi176.stdout:7/106: symlink d1/d13/d1b/l1d 0 2024-03-13T14:49:34.655 INFO:tasks.workunit.client.1.smithi196.stdout:6/396: readlink d6/d2a/dc/l1e 0 2024-03-13T14:49:34.658 INFO:tasks.workunit.client.0.smithi176.stdout:8/99: creat d0/d5/f1f x:0 0 0 2024-03-13T14:49:34.661 INFO:tasks.workunit.client.1.smithi196.stdout:6/397: chown d6/d9/d11/f13 9 1 2024-03-13T14:49:34.661 INFO:tasks.workunit.client.1.smithi196.stdout:0/365: stat d8/da/d25/d26/d35/d3b/d5f/d33/f34 0 2024-03-13T14:49:34.662 INFO:tasks.workunit.client.0.smithi176.stdout:4/112: creat d7/d10/f27 x:0 0 0 2024-03-13T14:49:34.662 INFO:tasks.workunit.client.1.smithi196.stdout:9/249: symlink d7/d9/d17/l4a 0 2024-03-13T14:49:34.665 INFO:tasks.workunit.client.0.smithi176.stdout:6/106: sync 2024-03-13T14:49:34.671 INFO:tasks.workunit.client.1.smithi196.stdout:3/279: dwrite db/d19/d22/d3a/d53/f48 [0,4194304] 0 2024-03-13T14:49:34.674 INFO:tasks.workunit.client.1.smithi196.stdout:5/401: dwrite d4/db/d14/d19/d1c/f6b [0,4194304] 0 2024-03-13T14:49:34.676 INFO:tasks.workunit.client.1.smithi196.stdout:2/194: rename d0/d6/d1d/c2d to d0/d31/c3c 0 2024-03-13T14:49:34.677 INFO:tasks.workunit.client.0.smithi176.stdout:1/99: unlink d2/dd/l1e 0 2024-03-13T14:49:34.678 INFO:tasks.workunit.client.1.smithi196.stdout:2/195: stat d0/d6/cc 0 2024-03-13T14:49:34.678 INFO:tasks.workunit.client.0.smithi176.stdout:1/100: chown d2/dd/d19 2 1 2024-03-13T14:49:34.680 INFO:tasks.workunit.client.1.smithi196.stdout:1/271: creat d1/d9/d33/d4b/f5f x:0 0 0 2024-03-13T14:49:34.684 INFO:tasks.workunit.client.0.smithi176.stdout:2/113: creat d1/dc/f2c x:0 0 0 2024-03-13T14:49:34.692 INFO:tasks.workunit.client.1.smithi196.stdout:9/250: dwrite d7/d9/df/f43 [0,4194304] 0 2024-03-13T14:49:34.693 INFO:tasks.workunit.client.0.smithi176.stdout:0/102: write d2/f3 [1505825,97769] 0 2024-03-13T14:49:34.693 INFO:tasks.workunit.client.1.smithi196.stdout:4/261: sync 2024-03-13T14:49:34.694 INFO:tasks.workunit.client.1.smithi196.stdout:7/373: sync 2024-03-13T14:49:34.697 INFO:tasks.workunit.client.0.smithi176.stdout:5/93: rename d5/de to d5/d22 0 2024-03-13T14:49:34.698 INFO:tasks.workunit.client.0.smithi176.stdout:0/103: write d2/d5/dc/f15 [815776,103100] 0 2024-03-13T14:49:34.700 INFO:tasks.workunit.client.0.smithi176.stdout:5/94: readlink d5/d22/d15/l1c 0 2024-03-13T14:49:34.701 INFO:tasks.workunit.client.0.smithi176.stdout:5/95: fdatasync f0 0 2024-03-13T14:49:34.704 INFO:tasks.workunit.client.0.smithi176.stdout:8/100: creat d0/d5/f20 x:0 0 0 2024-03-13T14:49:34.704 INFO:tasks.workunit.client.0.smithi176.stdout:4/113: rename d7/f26 to d7/d8/f28 0 2024-03-13T14:49:34.709 INFO:tasks.workunit.client.0.smithi176.stdout:1/101: mknod d2/c20 0 2024-03-13T14:49:34.712 INFO:tasks.workunit.client.1.smithi196.stdout:6/398: fdatasync d6/d2a/d23/d28/f3f 0 2024-03-13T14:49:34.712 INFO:tasks.workunit.client.0.smithi176.stdout:2/114: dread d1/fa [0,4194304] 0 2024-03-13T14:49:34.712 INFO:tasks.workunit.client.0.smithi176.stdout:3/105: dwrite f2 [0,4194304] 0 2024-03-13T14:49:34.713 INFO:tasks.workunit.client.1.smithi196.stdout:0/366: unlink d8/da/d25/d26/d35/d3b/d6c/f7c 0 2024-03-13T14:49:34.717 INFO:tasks.workunit.client.0.smithi176.stdout:7/107: creat d1/d13/d1b/d18/f1e x:0 0 0 2024-03-13T14:49:34.718 INFO:tasks.workunit.client.1.smithi196.stdout:5/402: truncate d4/f11 3708508 0 2024-03-13T14:49:34.720 INFO:tasks.workunit.client.1.smithi196.stdout:3/280: mknod db/d19/d22/c5a 0 2024-03-13T14:49:34.721 INFO:tasks.workunit.client.1.smithi196.stdout:2/196: mknod d0/d6/d34/c3d 0 2024-03-13T14:49:34.722 INFO:tasks.workunit.client.1.smithi196.stdout:3/281: write f7 [5880825,2204] 0 2024-03-13T14:49:34.723 INFO:tasks.workunit.client.0.smithi176.stdout:4/114: dwrite d7/f1c [0,4194304] 0 2024-03-13T14:49:34.723 INFO:tasks.workunit.client.0.smithi176.stdout:0/104: rename d2/d5/le to d2/d5/l17 0 2024-03-13T14:49:34.725 INFO:tasks.workunit.client.1.smithi196.stdout:1/272: creat d1/d3f/d5b/f60 x:0 0 0 2024-03-13T14:49:34.725 INFO:tasks.workunit.client.1.smithi196.stdout:3/282: write db/f29 [3132590,105825] 0 2024-03-13T14:49:34.728 INFO:tasks.workunit.client.1.smithi196.stdout:8/226: write d0/d10/d12/d17/f28 [1339544,57787] 0 2024-03-13T14:49:34.731 INFO:tasks.workunit.client.1.smithi196.stdout:3/283: write db/d19/d22/d30/f42 [924864,17228] 0 2024-03-13T14:49:34.735 INFO:tasks.workunit.client.1.smithi196.stdout:4/262: chown d1/l20 10 1 2024-03-13T14:49:34.735 INFO:tasks.workunit.client.0.smithi176.stdout:5/96: mknod d5/d22/d15/c23 0 2024-03-13T14:49:34.739 INFO:tasks.workunit.client.0.smithi176.stdout:9/128: dwrite d0/db/f1d [0,4194304] 0 2024-03-13T14:49:34.739 INFO:tasks.workunit.client.0.smithi176.stdout:6/107: mkdir d0/d24 0 2024-03-13T14:49:34.741 INFO:tasks.workunit.client.0.smithi176.stdout:8/101: symlink d0/db/dc/l21 0 2024-03-13T14:49:34.746 INFO:tasks.workunit.client.1.smithi196.stdout:6/399: truncate d6/d9/d11/f13 1127700 0 2024-03-13T14:49:34.750 INFO:tasks.workunit.client.1.smithi196.stdout:5/403: symlink d4/db/d4a/l85 0 2024-03-13T14:49:34.755 INFO:tasks.workunit.client.0.smithi176.stdout:2/115: mkdir d1/dc/d2d 0 2024-03-13T14:49:34.755 INFO:tasks.workunit.client.0.smithi176.stdout:5/97: dwrite f0 [0,4194304] 0 2024-03-13T14:49:34.756 INFO:tasks.workunit.client.1.smithi196.stdout:2/197: symlink d0/d31/l3e 0 2024-03-13T14:49:34.758 INFO:tasks.workunit.client.0.smithi176.stdout:3/106: truncate d4/fa 2382747 0 2024-03-13T14:49:34.765 INFO:tasks.workunit.client.1.smithi196.stdout:1/273: symlink d1/d3f/d5b/l61 0 2024-03-13T14:49:34.769 INFO:tasks.workunit.client.0.smithi176.stdout:4/115: rmdir d7 39 2024-03-13T14:49:34.773 INFO:tasks.workunit.client.1.smithi196.stdout:8/227: chown d0/d10/d12/d1e 362100826 1 2024-03-13T14:49:34.774 INFO:tasks.workunit.client.0.smithi176.stdout:2/116: dwrite d1/dc/f16 [0,4194304] 0 2024-03-13T14:49:34.780 INFO:tasks.workunit.client.1.smithi196.stdout:3/284: symlink db/d37/l5b 0 2024-03-13T14:49:34.781 INFO:tasks.workunit.client.0.smithi176.stdout:9/129: creat d0/d1e/d1f/f2c x:0 0 0 2024-03-13T14:49:34.781 INFO:tasks.workunit.client.1.smithi196.stdout:2/198: dwrite d0/d6/f14 [0,4194304] 0 2024-03-13T14:49:34.787 INFO:tasks.workunit.client.1.smithi196.stdout:4/263: dread d1/d3/f36 [0,4194304] 0 2024-03-13T14:49:34.794 INFO:tasks.workunit.client.0.smithi176.stdout:8/102: mkdir d0/d5/d11/d22 0 2024-03-13T14:49:34.794 INFO:tasks.workunit.client.0.smithi176.stdout:9/130: dread d0/db/f1d [0,4194304] 0 2024-03-13T14:49:34.801 INFO:tasks.workunit.client.1.smithi196.stdout:6/400: dread d6/f8 [0,4194304] 0 2024-03-13T14:49:34.801 INFO:tasks.workunit.client.1.smithi196.stdout:8/228: dread d0/d10/d12/d1e/f34 [0,4194304] 0 2024-03-13T14:49:34.801 INFO:tasks.workunit.client.1.smithi196.stdout:3/285: dread db/d19/d22/d3a/d53/f48 [0,4194304] 0 2024-03-13T14:49:34.805 INFO:tasks.workunit.client.1.smithi196.stdout:8/229: write d0/d10/f19 [2836776,42914] 0 2024-03-13T14:49:34.807 INFO:tasks.workunit.client.1.smithi196.stdout:8/230: dread - d0/d1c/d3b/f4a zero size 2024-03-13T14:49:34.809 INFO:tasks.workunit.client.0.smithi176.stdout:3/107: symlink d4/d13/l1f 0 2024-03-13T14:49:34.811 INFO:tasks.workunit.client.0.smithi176.stdout:8/103: dwrite d0/f1c [0,4194304] 0 2024-03-13T14:49:34.813 INFO:tasks.workunit.client.0.smithi176.stdout:8/104: read d0/f2 [1885701,106763] 0 2024-03-13T14:49:34.814 INFO:tasks.workunit.client.0.smithi176.stdout:8/105: dread - d0/db/fd zero size 2024-03-13T14:49:34.815 INFO:tasks.workunit.client.1.smithi196.stdout:7/374: creat d0/f76 x:0 0 0 2024-03-13T14:49:34.815 INFO:tasks.workunit.client.0.smithi176.stdout:8/106: write d0/f9 [5111509,45152] 0 2024-03-13T14:49:34.816 INFO:tasks.workunit.client.0.smithi176.stdout:0/105: link d2/d5/dc/f13 d2/d5/dc/f18 0 2024-03-13T14:49:34.817 INFO:tasks.workunit.client.1.smithi196.stdout:6/401: dread d6/d2a/d23/f4c [0,4194304] 0 2024-03-13T14:49:34.817 INFO:tasks.workunit.client.0.smithi176.stdout:0/106: truncate d2/db/f12 949588 0 2024-03-13T14:49:34.818 INFO:tasks.workunit.client.0.smithi176.stdout:2/117: fdatasync d1/ff 0 2024-03-13T14:49:34.818 INFO:tasks.workunit.client.1.smithi196.stdout:2/199: creat d0/d7/f3f x:0 0 0 2024-03-13T14:49:34.820 INFO:tasks.workunit.client.0.smithi176.stdout:2/118: write d1/f15 [7493403,110490] 0 2024-03-13T14:49:34.821 INFO:tasks.workunit.client.0.smithi176.stdout:2/119: chown d1/f26 106951 1 2024-03-13T14:49:34.822 INFO:tasks.workunit.client.1.smithi196.stdout:3/286: dread db/d19/d22/d3a/d53/f48 [0,4194304] 0 2024-03-13T14:49:34.822 INFO:tasks.workunit.client.0.smithi176.stdout:6/108: mknod d0/d24/c25 0 2024-03-13T14:49:34.823 INFO:tasks.workunit.client.0.smithi176.stdout:6/109: chown d0/d9/c21 219 1 2024-03-13T14:49:34.829 INFO:tasks.workunit.client.1.smithi196.stdout:9/251: rename d7/d9/df to d7/de/d4b 0 2024-03-13T14:49:34.830 INFO:tasks.workunit.client.0.smithi176.stdout:1/102: truncate f1 2232120 0 2024-03-13T14:49:34.833 INFO:tasks.workunit.client.0.smithi176.stdout:3/108: unlink d4/ld 0 2024-03-13T14:49:34.836 INFO:tasks.workunit.client.1.smithi196.stdout:0/367: truncate d8/da/d25/d26/d35/d3b/f47 4477783 0 2024-03-13T14:49:34.838 INFO:tasks.workunit.client.1.smithi196.stdout:8/231: creat d0/f51 x:0 0 0 2024-03-13T14:49:34.847 INFO:tasks.workunit.client.1.smithi196.stdout:7/375: mkdir d0/d9/d31/d77 0 2024-03-13T14:49:34.850 INFO:tasks.workunit.client.1.smithi196.stdout:1/274: dwrite d1/f2c [0,4194304] 0 2024-03-13T14:49:34.851 INFO:tasks.workunit.client.1.smithi196.stdout:6/402: mknod d6/d7/df/d27/d39/c7f 0 2024-03-13T14:49:34.854 INFO:tasks.workunit.client.0.smithi176.stdout:2/120: rmdir d1/dc 39 2024-03-13T14:49:34.857 INFO:tasks.workunit.client.0.smithi176.stdout:6/110: mknod d0/d9/c26 0 2024-03-13T14:49:34.860 INFO:tasks.workunit.client.0.smithi176.stdout:5/98: rename d5/fa to d5/f24 0 2024-03-13T14:49:34.866 INFO:tasks.workunit.client.0.smithi176.stdout:5/99: dread d5/f8 [0,4194304] 0 2024-03-13T14:49:34.866 INFO:tasks.workunit.client.1.smithi196.stdout:2/200: rename d0/d31 to d0/d7/d40 0 2024-03-13T14:49:34.870 INFO:tasks.workunit.client.1.smithi196.stdout:9/252: write d7/d9/f44 [5182616,8889] 0 2024-03-13T14:49:34.873 INFO:tasks.workunit.client.0.smithi176.stdout:7/108: getdents d1/d13/d1b 0 2024-03-13T14:49:34.874 INFO:tasks.workunit.client.0.smithi176.stdout:3/109: dread f1 [0,4194304] 0 2024-03-13T14:49:34.875 INFO:tasks.workunit.client.1.smithi196.stdout:0/368: unlink d8/da/d25/d26/d35/d3b/d50/d5d/f1f 0 2024-03-13T14:49:34.876 INFO:tasks.workunit.client.0.smithi176.stdout:7/109: chown d1/d13/c16 110 1 2024-03-13T14:49:34.876 INFO:tasks.workunit.client.0.smithi176.stdout:2/121: dwrite d1/f14 [4194304,4194304] 0 2024-03-13T14:49:34.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:34 smithi176 ceph-mon[29064]: pgmap v6: 65 pgs: 65 active+clean; 534 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:49:34.878 INFO:tasks.workunit.client.1.smithi196.stdout:3/287: write db/d19/d22/f3c [212373,11579] 0 2024-03-13T14:49:34.885 INFO:tasks.workunit.client.1.smithi196.stdout:6/403: creat d6/d2a/dc/f80 x:0 0 0 2024-03-13T14:49:34.886 INFO:tasks.workunit.client.1.smithi196.stdout:6/404: chown l5 236424911 1 2024-03-13T14:49:34.887 INFO:tasks.workunit.client.1.smithi196.stdout:1/275: truncate d1/d28/f35 206261 0 2024-03-13T14:49:34.889 INFO:tasks.workunit.client.0.smithi176.stdout:6/111: chown d0/d10/l1d 40372 1 2024-03-13T14:49:34.893 INFO:tasks.workunit.client.0.smithi176.stdout:5/100: rmdir d5/d22 39 2024-03-13T14:49:34.895 INFO:tasks.workunit.client.1.smithi196.stdout:2/201: write d0/d6/d1d/f23 [1347973,80265] 0 2024-03-13T14:49:34.899 INFO:tasks.workunit.client.1.smithi196.stdout:9/253: rmdir d7 39 2024-03-13T14:49:34.901 INFO:tasks.workunit.client.1.smithi196.stdout:5/404: link d4/d1b/c3d d4/db/d14/d19/d26/d33/c86 0 2024-03-13T14:49:34.906 INFO:tasks.workunit.client.1.smithi196.stdout:8/232: creat d0/d1c/d29/d47/f52 x:0 0 0 2024-03-13T14:49:34.907 INFO:tasks.workunit.client.0.smithi176.stdout:4/116: link l3 d7/d8/d15/d23/l29 0 2024-03-13T14:49:34.907 INFO:tasks.workunit.client.1.smithi196.stdout:8/233: readlink d0/d10/d12/d17/l37 0 2024-03-13T14:49:34.908 INFO:tasks.workunit.client.1.smithi196.stdout:0/369: mkdir d8/da/d25/d26/d35/d3b/d50/d5d/d7e 0 2024-03-13T14:49:34.910 INFO:tasks.workunit.client.1.smithi196.stdout:7/376: mkdir d0/d78 0 2024-03-13T14:49:34.911 INFO:tasks.workunit.client.0.smithi176.stdout:8/107: dwrite d0/db/f19 [0,4194304] 0 2024-03-13T14:49:34.912 INFO:tasks.workunit.client.1.smithi196.stdout:3/288: creat db/d19/d22/d3a/d53/f5c x:0 0 0 2024-03-13T14:49:34.917 INFO:tasks.workunit.client.0.smithi176.stdout:6/112: rename d0/df to d0/d10/d19/d27 0 2024-03-13T14:49:34.918 INFO:tasks.workunit.client.1.smithi196.stdout:4/264: getdents d1/d1e 0 2024-03-13T14:49:34.923 INFO:tasks.workunit.client.1.smithi196.stdout:2/202: unlink d0/d7/d40/l3a 0 2024-03-13T14:49:34.923 INFO:tasks.workunit.client.1.smithi196.stdout:8/234: dread d0/d1c/d3b/f4b [0,4194304] 0 2024-03-13T14:49:34.925 INFO:tasks.workunit.client.0.smithi176.stdout:2/122: creat d1/dc/f2e x:0 0 0 2024-03-13T14:49:34.926 INFO:tasks.workunit.client.1.smithi196.stdout:5/405: mknod d4/db/d14/d37/d41/c87 0 2024-03-13T14:49:34.926 INFO:tasks.workunit.client.1.smithi196.stdout:9/254: write d7/de/d16/f2a [1461993,70830] 0 2024-03-13T14:49:34.926 INFO:tasks.workunit.client.1.smithi196.stdout:2/203: dread - d0/f37 zero size 2024-03-13T14:49:34.927 INFO:tasks.workunit.client.0.smithi176.stdout:4/117: creat d7/d10/f2a x:0 0 0 2024-03-13T14:49:34.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:34 smithi196 ceph-mon[37486]: pgmap v6: 65 pgs: 65 active+clean; 534 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:49:34.929 INFO:tasks.workunit.client.0.smithi176.stdout:0/107: getdents d2/d5 0 2024-03-13T14:49:34.930 INFO:tasks.workunit.client.0.smithi176.stdout:4/118: stat d7/d8/d15/c21 0 2024-03-13T14:49:34.931 INFO:tasks.workunit.client.1.smithi196.stdout:1/276: link d1/d28/f2d d1/d9/d33/f62 0 2024-03-13T14:49:34.932 INFO:tasks.workunit.client.0.smithi176.stdout:4/119: chown d7/d8/d15/c21 7208 1 2024-03-13T14:49:34.934 INFO:tasks.workunit.client.0.smithi176.stdout:5/101: symlink d5/l25 0 2024-03-13T14:49:34.934 INFO:tasks.workunit.client.1.smithi196.stdout:3/289: rename db/d19/c43 to db/d19/d22/d3a/d53/c5d 0 2024-03-13T14:49:34.935 INFO:tasks.workunit.client.0.smithi176.stdout:4/120: write d7/fc [3022974,18218] 0 2024-03-13T14:49:34.937 INFO:tasks.workunit.client.0.smithi176.stdout:6/113: mknod d0/de/c28 0 2024-03-13T14:49:34.937 INFO:tasks.workunit.client.1.smithi196.stdout:1/277: dread d1/d3f/d5b/f54 [0,4194304] 0 2024-03-13T14:49:34.940 INFO:tasks.workunit.client.0.smithi176.stdout:7/110: creat d1/f1f x:0 0 0 2024-03-13T14:49:34.940 INFO:tasks.workunit.client.0.smithi176.stdout:6/114: write d0/d9/f1f [866872,34693] 0 2024-03-13T14:49:34.940 INFO:tasks.workunit.client.0.smithi176.stdout:9/131: sync 2024-03-13T14:49:34.942 INFO:tasks.workunit.client.0.smithi176.stdout:7/111: chown d1/f1f 11 1 2024-03-13T14:49:34.943 INFO:tasks.workunit.client.0.smithi176.stdout:0/108: creat d2/d5/dc/f19 x:0 0 0 2024-03-13T14:49:34.943 INFO:tasks.workunit.client.1.smithi196.stdout:1/278: read d1/d9/d33/d4b/f1b [326515,12031] 0 2024-03-13T14:49:34.944 INFO:tasks.workunit.client.0.smithi176.stdout:9/132: write d0/d1e/d1f/f2c [284786,101691] 0 2024-03-13T14:49:34.944 INFO:tasks.workunit.client.0.smithi176.stdout:8/108: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:34.946 INFO:tasks.workunit.client.0.smithi176.stdout:8/109: chown d0/f3 1 1 2024-03-13T14:49:34.946 INFO:tasks.workunit.client.0.smithi176.stdout:9/133: chown d0/f10 878298 1 2024-03-13T14:49:34.946 INFO:tasks.workunit.client.1.smithi196.stdout:1/279: chown d1/d38/c14 890035 1 2024-03-13T14:49:34.948 INFO:tasks.workunit.client.1.smithi196.stdout:5/406: readlink d4/db/d14/d19/d1c/d25/d38/l75 0 2024-03-13T14:49:34.954 INFO:tasks.workunit.client.1.smithi196.stdout:9/255: dwrite d7/f45 [0,4194304] 0 2024-03-13T14:49:34.954 INFO:tasks.workunit.client.1.smithi196.stdout:2/204: rename d0/d7/d9/f24 to d0/d6/d34/f41 0 2024-03-13T14:49:34.957 INFO:tasks.workunit.client.0.smithi176.stdout:4/121: creat d7/d8/d15/f2b x:0 0 0 2024-03-13T14:49:34.958 INFO:tasks.workunit.client.0.smithi176.stdout:2/123: creat d1/dc/d2d/f2f x:0 0 0 2024-03-13T14:49:34.960 INFO:tasks.workunit.client.0.smithi176.stdout:3/110: write d4/fa [3030415,2626] 0 2024-03-13T14:49:34.961 INFO:tasks.workunit.client.0.smithi176.stdout:6/115: rename d0/d9/l1e to d0/d9/l29 0 2024-03-13T14:49:34.961 INFO:tasks.workunit.client.1.smithi196.stdout:6/405: link d6/c16 d6/d2a/c81 0 2024-03-13T14:49:34.961 INFO:tasks.workunit.client.1.smithi196.stdout:3/290: mknod db/d19/d22/d3a/c5e 0 2024-03-13T14:49:34.961 INFO:tasks.workunit.client.1.smithi196.stdout:9/256: write d7/de/d4b/f19 [34087,91957] 0 2024-03-13T14:49:34.964 INFO:tasks.workunit.client.0.smithi176.stdout:0/109: mkdir d2/d5/d1a 0 2024-03-13T14:49:34.965 INFO:tasks.workunit.client.1.smithi196.stdout:3/291: write db/d19/d22/f31 [1336430,75051] 0 2024-03-13T14:49:34.970 INFO:tasks.workunit.client.0.smithi176.stdout:1/103: dread f1 [0,4194304] 0 2024-03-13T14:49:34.970 INFO:tasks.workunit.client.0.smithi176.stdout:0/110: rename d2/d5 to d2/d5/d1b 22 2024-03-13T14:49:34.970 INFO:tasks.workunit.client.1.smithi196.stdout:2/205: unlink d0/c35 0 2024-03-13T14:49:34.971 INFO:tasks.workunit.client.0.smithi176.stdout:8/110: dwrite d0/db/f19 [0,4194304] 0 2024-03-13T14:49:34.971 INFO:tasks.workunit.client.1.smithi196.stdout:4/265: rmdir d1/d1e/d39/d46 0 2024-03-13T14:49:34.971 INFO:tasks.workunit.client.0.smithi176.stdout:5/102: symlink d5/l26 0 2024-03-13T14:49:34.971 INFO:tasks.workunit.client.0.smithi176.stdout:9/134: dwrite d0/d13/f16 [0,4194304] 0 2024-03-13T14:49:34.973 INFO:tasks.workunit.client.0.smithi176.stdout:9/135: chown d0 3642535 1 2024-03-13T14:49:34.973 INFO:tasks.workunit.client.1.smithi196.stdout:0/370: truncate d8/f1e 2955433 0 2024-03-13T14:49:34.973 INFO:tasks.workunit.client.0.smithi176.stdout:4/122: mkdir d7/d10/d2c 0 2024-03-13T14:49:34.974 INFO:tasks.workunit.client.0.smithi176.stdout:8/111: write d0/d5/f20 [1014200,110592] 0 2024-03-13T14:49:34.974 INFO:tasks.workunit.client.1.smithi196.stdout:3/292: readlink db/d37/l5b 0 2024-03-13T14:49:34.975 INFO:tasks.workunit.client.1.smithi196.stdout:2/206: dread - d0/d6/d1d/f30 zero size 2024-03-13T14:49:34.976 INFO:tasks.workunit.client.0.smithi176.stdout:9/136: chown d0/d17/c28 0 1 2024-03-13T14:49:34.977 INFO:tasks.workunit.client.1.smithi196.stdout:6/406: mkdir d6/d57/d82 0 2024-03-13T14:49:34.977 INFO:tasks.workunit.client.0.smithi176.stdout:7/112: truncate d1/d13/f14 2111731 0 2024-03-13T14:49:34.977 INFO:tasks.workunit.client.1.smithi196.stdout:8/235: dread d0/d1c/d3b/f38 [0,4194304] 0 2024-03-13T14:49:34.978 INFO:tasks.workunit.client.0.smithi176.stdout:1/104: creat d2/f21 x:0 0 0 2024-03-13T14:49:34.979 INFO:tasks.workunit.client.0.smithi176.stdout:7/113: write d1/d7/f10 [4931339,49577] 0 2024-03-13T14:49:34.980 INFO:tasks.workunit.client.0.smithi176.stdout:2/124: dread d1/ff [0,4194304] 0 2024-03-13T14:49:34.981 INFO:tasks.workunit.client.0.smithi176.stdout:0/111: creat d2/db/f1c x:0 0 0 2024-03-13T14:49:34.983 INFO:tasks.workunit.client.0.smithi176.stdout:4/123: mkdir d7/d2d 0 2024-03-13T14:49:34.989 INFO:tasks.workunit.client.0.smithi176.stdout:9/137: dread d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:34.991 INFO:tasks.workunit.client.0.smithi176.stdout:8/112: symlink d0/db/dc/l23 0 2024-03-13T14:49:34.993 INFO:tasks.workunit.client.0.smithi176.stdout:8/113: dread - d0/d5/f1f zero size 2024-03-13T14:49:34.995 INFO:tasks.workunit.client.0.smithi176.stdout:1/105: truncate d2/f9 2720496 0 2024-03-13T14:49:34.997 INFO:tasks.workunit.client.0.smithi176.stdout:7/114: chown d1/cc 12360021 1 2024-03-13T14:49:34.998 INFO:tasks.workunit.client.0.smithi176.stdout:1/106: write d2/f6 [498852,17242] 0 2024-03-13T14:49:35.000 INFO:tasks.workunit.client.0.smithi176.stdout:5/103: dwrite d5/f7 [0,4194304] 0 2024-03-13T14:49:35.000 INFO:tasks.workunit.client.0.smithi176.stdout:7/115: read d1/d7/f1a [1196585,29754] 0 2024-03-13T14:49:35.001 INFO:tasks.workunit.client.0.smithi176.stdout:1/107: write d2/de/f17 [113434,57683] 0 2024-03-13T14:49:35.005 INFO:tasks.workunit.client.0.smithi176.stdout:2/125: rmdir d1/dc/d2d 39 2024-03-13T14:49:35.009 INFO:tasks.workunit.client.0.smithi176.stdout:0/112: mknod d2/d5/d1a/c1d 0 2024-03-13T14:49:35.010 INFO:tasks.workunit.client.0.smithi176.stdout:8/114: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:35.010 INFO:tasks.workunit.client.0.smithi176.stdout:4/124: creat d7/d2d/f2e x:0 0 0 2024-03-13T14:49:35.012 INFO:tasks.workunit.client.0.smithi176.stdout:0/113: chown d2/cf 1705329 1 2024-03-13T14:49:35.013 INFO:tasks.workunit.client.0.smithi176.stdout:7/116: creat d1/d13/f20 x:0 0 0 2024-03-13T14:49:35.013 INFO:tasks.workunit.client.0.smithi176.stdout:8/115: write d0/f14 [4756817,18375] 0 2024-03-13T14:49:35.014 INFO:tasks.workunit.client.0.smithi176.stdout:1/108: rename d2/de/d10/d16/d1b to d2/de/d10/d14/d22 0 2024-03-13T14:49:35.016 INFO:tasks.workunit.client.0.smithi176.stdout:7/117: readlink d1/d2/la 0 2024-03-13T14:49:35.017 INFO:tasks.workunit.client.0.smithi176.stdout:0/114: read d2/d5/dc/f15 [662831,207] 0 2024-03-13T14:49:35.019 INFO:tasks.workunit.client.0.smithi176.stdout:5/104: rmdir d5/d22/d15 39 2024-03-13T14:49:35.022 INFO:tasks.workunit.client.0.smithi176.stdout:4/125: mkdir d7/d8/d2f 0 2024-03-13T14:49:35.024 INFO:tasks.workunit.client.0.smithi176.stdout:2/126: dread d1/f14 [0,4194304] 0 2024-03-13T14:49:35.025 INFO:tasks.workunit.client.0.smithi176.stdout:0/115: rmdir d2 39 2024-03-13T14:49:35.025 INFO:tasks.workunit.client.0.smithi176.stdout:3/111: write d4/f5 [675306,35946] 0 2024-03-13T14:49:35.027 INFO:tasks.workunit.client.0.smithi176.stdout:4/126: write d7/d8/d15/f2b [31260,1780] 0 2024-03-13T14:49:35.032 INFO:tasks.workunit.client.0.smithi176.stdout:5/105: dread d5/f7 [0,4194304] 0 2024-03-13T14:49:35.034 INFO:tasks.workunit.client.0.smithi176.stdout:7/118: mknod d1/d1c/c21 0 2024-03-13T14:49:35.036 INFO:tasks.workunit.client.0.smithi176.stdout:7/119: chown d1/d13/l15 13 1 2024-03-13T14:49:35.038 INFO:tasks.workunit.client.0.smithi176.stdout:3/112: dread d4/f5 [0,4194304] 0 2024-03-13T14:49:35.039 INFO:tasks.workunit.client.0.smithi176.stdout:8/116: rmdir d0/d5/d11/d22 0 2024-03-13T14:49:35.039 INFO:tasks.workunit.client.0.smithi176.stdout:9/138: dwrite d0/db/f1d [0,4194304] 0 2024-03-13T14:49:35.041 INFO:tasks.workunit.client.0.smithi176.stdout:1/109: write d2/f9 [2161375,91751] 0 2024-03-13T14:49:35.044 INFO:tasks.workunit.client.0.smithi176.stdout:2/127: link d1/l10 d1/dc/d1c/l30 0 2024-03-13T14:49:35.044 INFO:tasks.workunit.client.0.smithi176.stdout:8/117: write d0/f3 [5046285,95656] 0 2024-03-13T14:49:35.050 INFO:tasks.workunit.client.0.smithi176.stdout:1/110: symlink d2/de/d10/l23 0 2024-03-13T14:49:35.052 INFO:tasks.workunit.client.0.smithi176.stdout:9/139: rename d0/db/cd to d0/d13/c2d 0 2024-03-13T14:49:35.053 INFO:tasks.workunit.client.0.smithi176.stdout:3/113: getdents d4/d13 0 2024-03-13T14:49:35.065 INFO:tasks.workunit.client.0.smithi176.stdout:8/118: write d0/d5/ff [1044355,82489] 0 2024-03-13T14:49:35.067 INFO:tasks.workunit.client.0.smithi176.stdout:2/128: rename d1/f26 to d1/f31 0 2024-03-13T14:49:35.074 INFO:tasks.workunit.client.0.smithi176.stdout:0/116: dwrite d2/d5/dc/f16 [0,4194304] 0 2024-03-13T14:49:35.078 INFO:tasks.workunit.client.0.smithi176.stdout:5/106: dwrite d5/d22/d15/f16 [0,4194304] 0 2024-03-13T14:49:35.085 INFO:tasks.workunit.client.0.smithi176.stdout:7/120: dwrite d1/d13/d1b/d18/f1e [0,4194304] 0 2024-03-13T14:49:35.085 INFO:tasks.workunit.client.0.smithi176.stdout:2/129: rmdir d1/dc/d2d 39 2024-03-13T14:49:35.096 INFO:tasks.workunit.client.0.smithi176.stdout:2/130: symlink d1/dc/l32 0 2024-03-13T14:49:35.097 INFO:tasks.workunit.client.0.smithi176.stdout:1/111: dwrite f1 [0,4194304] 0 2024-03-13T14:49:35.098 INFO:tasks.workunit.client.0.smithi176.stdout:3/114: dread f2 [0,4194304] 0 2024-03-13T14:49:35.100 INFO:tasks.workunit.client.0.smithi176.stdout:7/121: rename d1/d2/la to d1/d7/l22 0 2024-03-13T14:49:35.111 INFO:tasks.workunit.client.0.smithi176.stdout:9/140: dwrite d0/d1e/f25 [0,4194304] 0 2024-03-13T14:49:35.115 INFO:tasks.workunit.client.0.smithi176.stdout:7/122: read d1/d7/f1a [2909479,122847] 0 2024-03-13T14:49:35.115 INFO:tasks.workunit.client.0.smithi176.stdout:3/115: mknod d4/d13/c20 0 2024-03-13T14:49:35.115 INFO:tasks.workunit.client.0.smithi176.stdout:2/131: chown d1/dc/d2d/f2f 352488117 1 2024-03-13T14:49:35.116 INFO:tasks.workunit.client.0.smithi176.stdout:1/112: write d2/fc [2713803,83356] 0 2024-03-13T14:49:35.118 INFO:tasks.workunit.client.0.smithi176.stdout:9/141: write d0/db/f1d [4569539,37219] 0 2024-03-13T14:49:35.120 INFO:tasks.workunit.client.0.smithi176.stdout:3/116: chown d4/d13/f19 266161590 1 2024-03-13T14:49:35.121 INFO:tasks.workunit.client.0.smithi176.stdout:8/119: dwrite d0/d5/f1f [0,4194304] 0 2024-03-13T14:49:35.126 INFO:tasks.workunit.client.0.smithi176.stdout:2/132: unlink d1/f8 0 2024-03-13T14:49:35.126 INFO:tasks.workunit.client.0.smithi176.stdout:7/123: write d1/d7/f10 [3470969,126649] 0 2024-03-13T14:49:35.126 INFO:tasks.workunit.client.0.smithi176.stdout:0/117: dread d2/d5/dc/f13 [0,4194304] 0 2024-03-13T14:49:35.126 INFO:tasks.workunit.client.0.smithi176.stdout:1/113: symlink d2/de/d10/l24 0 2024-03-13T14:49:35.127 INFO:tasks.workunit.client.0.smithi176.stdout:7/124: mkdir d1/d23 0 2024-03-13T14:49:35.127 INFO:tasks.workunit.client.0.smithi176.stdout:1/114: fsync d2/f9 0 2024-03-13T14:49:35.129 INFO:tasks.workunit.client.0.smithi176.stdout:3/117: symlink d4/d13/d16/l21 0 2024-03-13T14:49:35.130 INFO:tasks.workunit.client.0.smithi176.stdout:7/125: chown d1/d23 0 1 2024-03-13T14:49:35.134 INFO:tasks.workunit.client.0.smithi176.stdout:0/118: rename d2/d5/d1a to d2/d5/d1e 0 2024-03-13T14:49:35.138 INFO:tasks.workunit.client.0.smithi176.stdout:7/126: write d1/d13/f20 [65752,88335] 0 2024-03-13T14:49:35.138 INFO:tasks.workunit.client.0.smithi176.stdout:0/119: chown d2/l14 480384 1 2024-03-13T14:49:35.140 INFO:tasks.workunit.client.0.smithi176.stdout:0/120: readlink d2/l4 0 2024-03-13T14:49:35.145 INFO:tasks.workunit.client.0.smithi176.stdout:9/142: dwrite d0/f10 [4194304,4194304] 0 2024-03-13T14:49:35.146 INFO:tasks.workunit.client.0.smithi176.stdout:2/133: dread d1/f14 [4194304,4194304] 0 2024-03-13T14:49:35.148 INFO:tasks.workunit.client.0.smithi176.stdout:8/120: dread d0/f14 [0,4194304] 0 2024-03-13T14:49:35.149 INFO:tasks.workunit.client.0.smithi176.stdout:7/127: write d1/d13/f14 [2386566,112359] 0 2024-03-13T14:49:35.150 INFO:tasks.workunit.client.0.smithi176.stdout:2/134: mkdir d1/d2a/d33 0 2024-03-13T14:49:35.153 INFO:tasks.workunit.client.0.smithi176.stdout:2/135: rmdir d1/d2a 39 2024-03-13T14:49:35.154 INFO:tasks.workunit.client.0.smithi176.stdout:7/128: write d1/d13/f20 [1091195,94282] 0 2024-03-13T14:49:35.156 INFO:tasks.workunit.client.0.smithi176.stdout:3/118: dwrite d4/fa [0,4194304] 0 2024-03-13T14:49:35.157 INFO:tasks.workunit.client.0.smithi176.stdout:3/119: readlink d4/d13/l1c 0 2024-03-13T14:49:35.157 INFO:tasks.workunit.client.0.smithi176.stdout:8/121: getdents d0/db/dc 0 2024-03-13T14:49:35.158 INFO:tasks.workunit.client.0.smithi176.stdout:7/129: symlink d1/d7/l24 0 2024-03-13T14:49:35.159 INFO:tasks.workunit.client.0.smithi176.stdout:7/130: stat d1/d7 0 2024-03-13T14:49:35.160 INFO:tasks.workunit.client.0.smithi176.stdout:2/136: creat d1/dc/f34 x:0 0 0 2024-03-13T14:49:35.161 INFO:tasks.workunit.client.0.smithi176.stdout:8/122: unlink d0/d5/ff 0 2024-03-13T14:49:35.161 INFO:tasks.workunit.client.0.smithi176.stdout:7/131: write d1/f19 [503898,51278] 0 2024-03-13T14:49:35.162 INFO:tasks.workunit.client.0.smithi176.stdout:2/137: read d1/ff [2251777,87770] 0 2024-03-13T14:49:35.162 INFO:tasks.workunit.client.0.smithi176.stdout:3/120: rename d4/d13/l1f to d4/l22 0 2024-03-13T14:49:35.166 INFO:tasks.workunit.client.0.smithi176.stdout:2/138: truncate d1/f13 73208 0 2024-03-13T14:49:35.167 INFO:tasks.workunit.client.0.smithi176.stdout:8/123: mkdir d0/d24 0 2024-03-13T14:49:35.168 INFO:tasks.workunit.client.1.smithi196.stdout:1/280: dwrite d1/d28/d34/f41 [0,4194304] 0 2024-03-13T14:49:35.168 INFO:tasks.workunit.client.1.smithi196.stdout:5/407: getdents d4/db/d14/d19/d1c/d25/d38 0 2024-03-13T14:49:35.169 INFO:tasks.workunit.client.0.smithi176.stdout:3/121: mknod d4/d13/d16/c23 0 2024-03-13T14:49:35.170 INFO:tasks.workunit.client.0.smithi176.stdout:8/124: chown d0/db/dc/f13 0 1 2024-03-13T14:49:35.173 INFO:tasks.workunit.client.0.smithi176.stdout:3/122: mknod d4/d13/d16/c24 0 2024-03-13T14:49:35.175 INFO:tasks.workunit.client.0.smithi176.stdout:2/139: creat d1/f35 x:0 0 0 2024-03-13T14:49:35.176 INFO:tasks.workunit.client.1.smithi196.stdout:0/371: creat d8/da/d25/d26/d54/f7f x:0 0 0 2024-03-13T14:49:35.178 INFO:tasks.workunit.client.1.smithi196.stdout:3/293: link db/l16 db/d15/d44/d51/l5f 0 2024-03-13T14:49:35.178 INFO:tasks.workunit.client.1.smithi196.stdout:6/407: getdents d6/d9/d17/d19/d5f 0 2024-03-13T14:49:35.179 INFO:tasks.workunit.client.0.smithi176.stdout:3/123: creat d4/f25 x:0 0 0 2024-03-13T14:49:35.180 INFO:tasks.workunit.client.1.smithi196.stdout:1/281: mkdir d1/d9/d4e/d55/d63 0 2024-03-13T14:49:35.181 INFO:tasks.workunit.client.0.smithi176.stdout:3/124: chown d4/d13/d16/c24 0 1 2024-03-13T14:49:35.182 INFO:tasks.workunit.client.0.smithi176.stdout:7/132: dread d1/d7/f1a [0,4194304] 0 2024-03-13T14:49:35.182 INFO:tasks.workunit.client.0.smithi176.stdout:3/125: dread - d4/f25 zero size 2024-03-13T14:49:35.183 INFO:tasks.workunit.client.0.smithi176.stdout:2/140: creat d1/dc/d1c/f36 x:0 0 0 2024-03-13T14:49:35.184 INFO:tasks.workunit.client.1.smithi196.stdout:8/236: creat d0/d10/d12/f53 x:0 0 0 2024-03-13T14:49:35.184 INFO:tasks.workunit.client.0.smithi176.stdout:9/143: fsync d0/f10 0 2024-03-13T14:49:35.185 INFO:tasks.workunit.client.0.smithi176.stdout:7/133: write d1/d13/f14 [2519918,110814] 0 2024-03-13T14:49:35.186 INFO:tasks.workunit.client.1.smithi196.stdout:8/237: dread - d0/d10/d12/f53 zero size 2024-03-13T14:49:35.187 INFO:tasks.workunit.client.1.smithi196.stdout:5/408: symlink d4/db/d14/d19/l88 0 2024-03-13T14:49:35.188 INFO:tasks.workunit.client.1.smithi196.stdout:6/408: rmdir d6/d9/d17/d19/d4e/d66 39 2024-03-13T14:49:35.188 INFO:tasks.workunit.client.0.smithi176.stdout:8/125: dread d0/f14 [0,4194304] 0 2024-03-13T14:49:35.189 INFO:tasks.workunit.client.1.smithi196.stdout:6/409: stat l5 0 2024-03-13T14:49:35.190 INFO:tasks.workunit.client.0.smithi176.stdout:3/126: symlink d4/d13/l26 0 2024-03-13T14:49:35.190 INFO:tasks.workunit.client.0.smithi176.stdout:8/126: write d0/db/fd [34569,16896] 0 2024-03-13T14:49:35.191 INFO:tasks.workunit.client.1.smithi196.stdout:5/409: chown d4/db/d3c/f57 24357 1 2024-03-13T14:49:35.191 INFO:tasks.workunit.client.0.smithi176.stdout:3/127: stat d4/d13/c18 0 2024-03-13T14:49:35.192 INFO:tasks.workunit.client.0.smithi176.stdout:8/127: fsync d0/f3 0 2024-03-13T14:49:35.193 INFO:tasks.workunit.client.0.smithi176.stdout:9/144: readlink d0/d1e/l27 0 2024-03-13T14:49:35.195 INFO:tasks.workunit.client.0.smithi176.stdout:7/134: creat d1/d2/f25 x:0 0 0 2024-03-13T14:49:35.196 INFO:tasks.workunit.client.1.smithi196.stdout:9/257: dread d7/de/d4b/d1d/f23 [0,4194304] 0 2024-03-13T14:49:35.199 INFO:tasks.workunit.client.0.smithi176.stdout:3/128: creat d4/d1a/f27 x:0 0 0 2024-03-13T14:49:35.201 INFO:tasks.workunit.client.0.smithi176.stdout:3/129: write d4/d13/f19 [2037666,34900] 0 2024-03-13T14:49:35.201 INFO:tasks.workunit.client.0.smithi176.stdout:2/141: dwrite d1/fb [0,4194304] 0 2024-03-13T14:49:35.202 INFO:tasks.workunit.client.1.smithi196.stdout:0/372: rename d8/da/d25/d26/d35/d3b/c68 to d8/da/d25/d26/d35/d3b/d5f/d43/c80 0 2024-03-13T14:49:35.202 INFO:tasks.workunit.client.0.smithi176.stdout:9/145: creat d0/d1e/f2e x:0 0 0 2024-03-13T14:49:35.203 INFO:tasks.workunit.client.0.smithi176.stdout:3/130: dread - d4/f1d zero size 2024-03-13T14:49:35.203 INFO:tasks.workunit.client.0.smithi176.stdout:2/142: stat d1/l9 0 2024-03-13T14:49:35.204 INFO:tasks.workunit.client.1.smithi196.stdout:3/294: link db/d15/f17 db/d19/f60 0 2024-03-13T14:49:35.206 INFO:tasks.workunit.client.0.smithi176.stdout:7/135: mkdir d1/d1c/d26 0 2024-03-13T14:49:35.207 INFO:tasks.workunit.client.0.smithi176.stdout:3/131: chown d4/d13/l1c 216 1 2024-03-13T14:49:35.208 INFO:tasks.workunit.client.1.smithi196.stdout:3/295: dread db/f36 [0,4194304] 0 2024-03-13T14:49:35.209 INFO:tasks.workunit.client.1.smithi196.stdout:2/207: dread d0/fe [0,4194304] 0 2024-03-13T14:49:35.211 INFO:tasks.workunit.client.0.smithi176.stdout:8/128: dwrite d0/d5/f1f [0,4194304] 0 2024-03-13T14:49:35.213 INFO:tasks.workunit.client.1.smithi196.stdout:3/296: write db/f36 [1138241,36204] 0 2024-03-13T14:49:35.214 INFO:tasks.workunit.client.0.smithi176.stdout:2/143: creat d1/dc/d1c/f37 x:0 0 0 2024-03-13T14:49:35.216 INFO:tasks.workunit.client.0.smithi176.stdout:9/146: rename d0/d1e/c29 to d0/d17/c2f 0 2024-03-13T14:49:35.219 INFO:tasks.workunit.client.1.smithi196.stdout:9/258: dwrite d7/d9/f44 [0,4194304] 0 2024-03-13T14:49:35.219 INFO:tasks.workunit.client.1.smithi196.stdout:2/208: truncate d0/d6/d34/d38/f3b 360145 0 2024-03-13T14:49:35.221 INFO:tasks.workunit.client.0.smithi176.stdout:7/136: unlink d1/d2/l4 0 2024-03-13T14:49:35.222 INFO:tasks.workunit.client.0.smithi176.stdout:9/147: dread d0/d1e/d1f/f2c [0,4194304] 0 2024-03-13T14:49:35.224 INFO:tasks.workunit.client.0.smithi176.stdout:3/132: mkdir d4/d28 0 2024-03-13T14:49:35.225 INFO:tasks.workunit.client.0.smithi176.stdout:3/133: dread - d4/f1d zero size 2024-03-13T14:49:35.231 INFO:tasks.workunit.client.0.smithi176.stdout:8/129: getdents d0/db/dc 0 2024-03-13T14:49:35.235 INFO:tasks.workunit.client.1.smithi196.stdout:3/297: creat db/d4c/f61 x:0 0 0 2024-03-13T14:49:35.236 INFO:tasks.workunit.client.0.smithi176.stdout:8/130: chown d0/d5/l1e 6166 1 2024-03-13T14:49:35.238 INFO:tasks.workunit.client.1.smithi196.stdout:3/298: write db/d19/d50/f58 [545901,18525] 0 2024-03-13T14:49:35.239 INFO:tasks.workunit.client.1.smithi196.stdout:6/410: creat d6/d2a/f83 x:0 0 0 2024-03-13T14:49:35.242 INFO:tasks.workunit.client.0.smithi176.stdout:8/131: creat d0/db/f25 x:0 0 0 2024-03-13T14:49:35.246 INFO:tasks.workunit.client.0.smithi176.stdout:8/132: creat d0/d5/d11/f26 x:0 0 0 2024-03-13T14:49:35.247 INFO:tasks.workunit.client.0.smithi176.stdout:3/134: dwrite d4/f5 [4194304,4194304] 0 2024-03-13T14:49:35.247 INFO:tasks.workunit.client.1.smithi196.stdout:8/238: link d0/d1c/d3b/l25 d0/d1c/d3b/d48/l54 0 2024-03-13T14:49:35.248 INFO:tasks.workunit.client.1.smithi196.stdout:6/411: readlink d6/d2a/dc/d1c/d34/l4f 0 2024-03-13T14:49:35.248 INFO:tasks.workunit.client.0.smithi176.stdout:3/135: read d4/fa [2540318,110105] 0 2024-03-13T14:49:35.249 INFO:tasks.workunit.client.1.smithi196.stdout:1/282: dwrite d1/d9/d33/d4b/f1b [0,4194304] 0 2024-03-13T14:49:35.250 INFO:tasks.workunit.client.1.smithi196.stdout:9/259: symlink d7/l4c 0 2024-03-13T14:49:35.263 INFO:tasks.workunit.client.1.smithi196.stdout:3/299: dwrite db/ff [4194304,4194304] 0 2024-03-13T14:49:35.265 INFO:tasks.workunit.client.1.smithi196.stdout:2/209: write d0/d7/d9/f18 [3330382,99857] 0 2024-03-13T14:49:35.268 INFO:tasks.workunit.client.1.smithi196.stdout:3/300: fsync db/d19/d22/f31 0 2024-03-13T14:49:35.269 INFO:tasks.workunit.client.1.smithi196.stdout:2/210: truncate d0/fe 5227919 0 2024-03-13T14:49:35.270 INFO:tasks.workunit.client.1.smithi196.stdout:3/301: read - db/f49 zero size 2024-03-13T14:49:35.275 INFO:tasks.workunit.client.1.smithi196.stdout:5/410: truncate d4/db/d14/d19/d26/f3a 167866 0 2024-03-13T14:49:35.278 INFO:tasks.workunit.client.0.smithi176.stdout:9/148: dwrite d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:35.279 INFO:tasks.workunit.client.0.smithi176.stdout:8/133: getdents d0 0 2024-03-13T14:49:35.279 INFO:tasks.workunit.client.1.smithi196.stdout:9/260: mkdir d7/de/d4b/d1d/d22/d4d 0 2024-03-13T14:49:35.282 INFO:tasks.workunit.client.1.smithi196.stdout:1/283: dwrite d1/f47 [0,4194304] 0 2024-03-13T14:49:35.284 INFO:tasks.workunit.client.1.smithi196.stdout:5/411: write d4/db/d14/d19/d1c/d2d/f4e [1995644,104748] 0 2024-03-13T14:49:35.292 INFO:tasks.workunit.client.1.smithi196.stdout:0/373: getdents d8/da/d25 0 2024-03-13T14:49:35.297 INFO:tasks.workunit.client.1.smithi196.stdout:6/412: symlink d6/d7/l84 0 2024-03-13T14:49:35.297 INFO:tasks.workunit.client.0.smithi176.stdout:9/149: dwrite d0/d1e/f2e [0,4194304] 0 2024-03-13T14:49:35.302 INFO:tasks.workunit.client.1.smithi196.stdout:0/374: fdatasync d8/da/d25/d26/d35/d3b/d50/f6d 0 2024-03-13T14:49:35.303 INFO:tasks.workunit.client.1.smithi196.stdout:8/239: rmdir d0/d10/d12/d17/d40 39 2024-03-13T14:49:35.303 INFO:tasks.workunit.client.1.smithi196.stdout:7/377: sync 2024-03-13T14:49:35.303 INFO:tasks.workunit.client.1.smithi196.stdout:6/413: write d6/d2a/d23/d28/f6d [2043815,125974] 0 2024-03-13T14:49:35.307 INFO:tasks.workunit.client.1.smithi196.stdout:9/261: mknod d7/d24/c4e 0 2024-03-13T14:49:35.307 INFO:tasks.workunit.client.0.smithi176.stdout:9/150: rmdir d0/d1e 39 2024-03-13T14:49:35.308 INFO:tasks.workunit.client.0.smithi176.stdout:9/151: read d0/f10 [4795420,28197] 0 2024-03-13T14:49:35.311 INFO:tasks.workunit.client.1.smithi196.stdout:9/262: write d7/d24/f3a [203921,127584] 0 2024-03-13T14:49:35.314 INFO:tasks.workunit.client.1.smithi196.stdout:5/412: dwrite d4/db/d14/d19/d26/d68/f72 [0,4194304] 0 2024-03-13T14:49:35.320 INFO:tasks.workunit.client.1.smithi196.stdout:5/413: truncate d4/db/d14/d37/d41/f5c 1037156 0 2024-03-13T14:49:35.324 INFO:tasks.workunit.client.1.smithi196.stdout:8/240: fsync d0/d10/d12/f3e 0 2024-03-13T14:49:35.324 INFO:tasks.workunit.client.0.smithi176.stdout:9/152: dread d0/d1e/f25 [0,4194304] 0 2024-03-13T14:49:35.325 INFO:tasks.workunit.client.1.smithi196.stdout:6/414: mknod d6/d57/c85 0 2024-03-13T14:49:35.329 INFO:tasks.workunit.client.1.smithi196.stdout:6/415: chown d6/d9/d17/d19/d4e/f7c 0 1 2024-03-13T14:49:35.334 INFO:tasks.workunit.client.1.smithi196.stdout:1/284: write d1/d38/f25 [3196831,33812] 0 2024-03-13T14:49:35.335 INFO:tasks.workunit.client.1.smithi196.stdout:9/263: fdatasync d7/de/d4b/f14 0 2024-03-13T14:49:35.336 INFO:tasks.workunit.client.0.smithi176.stdout:9/153: fdatasync d0/d1e/f2e 0 2024-03-13T14:49:35.337 INFO:tasks.workunit.client.1.smithi196.stdout:3/302: creat db/d19/f62 x:0 0 0 2024-03-13T14:49:35.339 INFO:tasks.workunit.client.0.smithi176.stdout:9/154: write d0/d17/f19 [127358,32171] 0 2024-03-13T14:49:35.341 INFO:tasks.workunit.client.1.smithi196.stdout:5/414: creat d4/db/d14/d19/d26/f89 x:0 0 0 2024-03-13T14:49:35.341 INFO:tasks.workunit.client.1.smithi196.stdout:7/378: mknod d0/d78/c79 0 2024-03-13T14:49:35.341 INFO:tasks.workunit.client.1.smithi196.stdout:0/375: dread d8/da/d25/d26/d35/d3b/d5f/d43/f45 [0,4194304] 0 2024-03-13T14:49:35.346 INFO:tasks.workunit.client.1.smithi196.stdout:2/211: write d0/d7/f17 [394312,97412] 0 2024-03-13T14:49:35.353 INFO:tasks.workunit.client.1.smithi196.stdout:7/379: mknod d0/d9/d31/c7a 0 2024-03-13T14:49:35.357 INFO:tasks.workunit.client.1.smithi196.stdout:0/376: mknod d8/da/d74/c81 0 2024-03-13T14:49:35.357 INFO:tasks.workunit.client.1.smithi196.stdout:7/380: readlink d0/d9/d1e/l2b 0 2024-03-13T14:49:35.362 INFO:tasks.workunit.client.0.smithi176.stdout:6/116: sync 2024-03-13T14:49:35.363 INFO:tasks.workunit.client.1.smithi196.stdout:9/264: symlink d7/de/d4b/d1d/l4f 0 2024-03-13T14:49:35.363 INFO:tasks.workunit.client.1.smithi196.stdout:5/415: dread d4/db/d14/d19/d26/d68/f72 [0,4194304] 0 2024-03-13T14:49:35.363 INFO:tasks.workunit.client.1.smithi196.stdout:7/381: dread d0/d9/d1e/d3f/f60 [0,4194304] 0 2024-03-13T14:49:35.363 INFO:tasks.workunit.client.1.smithi196.stdout:2/212: mkdir d0/d7/d9/d42 0 2024-03-13T14:49:35.363 INFO:tasks.workunit.client.1.smithi196.stdout:8/241: dread d0/d10/d12/d17/f28 [0,4194304] 0 2024-03-13T14:49:35.366 INFO:tasks.workunit.client.1.smithi196.stdout:1/285: creat d1/d9/f64 x:0 0 0 2024-03-13T14:49:35.368 INFO:tasks.workunit.client.0.smithi176.stdout:6/117: rename d0/d10/l16 to d0/d10/d19/d27/l2a 0 2024-03-13T14:49:35.369 INFO:tasks.workunit.client.1.smithi196.stdout:1/286: fdatasync d1/d9/d33/d4b/f1a 0 2024-03-13T14:49:35.369 INFO:tasks.workunit.client.0.smithi176.stdout:9/155: dwrite d0/f10 [0,4194304] 0 2024-03-13T14:49:35.371 INFO:tasks.workunit.client.1.smithi196.stdout:1/287: fdatasync d1/d9/f36 0 2024-03-13T14:49:35.373 INFO:tasks.workunit.client.1.smithi196.stdout:6/416: write d6/d7/df/f3a [1387365,122006] 0 2024-03-13T14:49:35.373 INFO:tasks.workunit.client.1.smithi196.stdout:2/213: mknod d0/d7/d40/c43 0 2024-03-13T14:49:35.374 INFO:tasks.workunit.client.0.smithi176.stdout:6/118: dread d0/fc [0,4194304] 0 2024-03-13T14:49:35.376 INFO:tasks.workunit.client.1.smithi196.stdout:1/288: unlink d1/f2c 0 2024-03-13T14:49:35.380 INFO:tasks.workunit.client.1.smithi196.stdout:9/265: dread d7/de/d4b/f14 [4194304,4194304] 0 2024-03-13T14:49:35.382 INFO:tasks.workunit.client.1.smithi196.stdout:1/289: unlink d1/f4c 0 2024-03-13T14:49:35.385 INFO:tasks.workunit.client.1.smithi196.stdout:2/214: mknod d0/d7/d40/c44 0 2024-03-13T14:49:35.392 INFO:tasks.workunit.client.0.smithi176.stdout:9/156: dwrite d0/d1e/f2e [0,4194304] 0 2024-03-13T14:49:35.392 INFO:tasks.workunit.client.1.smithi196.stdout:8/242: dread d0/d1c/d3b/f4b [0,4194304] 0 2024-03-13T14:49:35.393 INFO:tasks.workunit.client.1.smithi196.stdout:5/416: dwrite d4/f8 [4194304,4194304] 0 2024-03-13T14:49:35.393 INFO:tasks.workunit.client.1.smithi196.stdout:7/382: dwrite d0/f4f [0,4194304] 0 2024-03-13T14:49:35.394 INFO:tasks.workunit.client.1.smithi196.stdout:8/243: fdatasync d0/d10/d12/f53 0 2024-03-13T14:49:35.395 INFO:tasks.workunit.client.0.smithi176.stdout:9/157: write d0/d1e/f2e [1089442,88711] 0 2024-03-13T14:49:35.398 INFO:tasks.workunit.client.0.smithi176.stdout:9/158: write d0/f10 [2075818,37947] 0 2024-03-13T14:49:35.402 INFO:tasks.workunit.client.1.smithi196.stdout:5/417: dread - d4/db/d3c/f57 zero size 2024-03-13T14:49:35.403 INFO:tasks.workunit.client.0.smithi176.stdout:6/119: write d0/f18 [2444014,92065] 0 2024-03-13T14:49:35.406 INFO:tasks.workunit.client.0.smithi176.stdout:9/159: mknod d0/d13/c30 0 2024-03-13T14:49:35.407 INFO:tasks.workunit.client.1.smithi196.stdout:6/417: truncate d6/d9/f1f 3770431 0 2024-03-13T14:49:35.409 INFO:tasks.workunit.client.1.smithi196.stdout:7/383: chown d0/l2f 75158 1 2024-03-13T14:49:35.411 INFO:tasks.workunit.client.0.smithi176.stdout:6/120: mkdir d0/d10/d19/d27/d2b 0 2024-03-13T14:49:35.411 INFO:tasks.workunit.client.1.smithi196.stdout:6/418: stat d6/d9/d11/f5e 0 2024-03-13T14:49:35.414 INFO:tasks.workunit.client.1.smithi196.stdout:9/266: rename d7/de/d4b/d1d/d22/f39 to d7/de/f50 0 2024-03-13T14:49:35.415 INFO:tasks.workunit.client.0.smithi176.stdout:9/160: chown d0/db/cc 110241 1 2024-03-13T14:49:35.416 INFO:tasks.workunit.client.1.smithi196.stdout:7/384: read d0/d6/f19 [542753,6647] 0 2024-03-13T14:49:35.418 INFO:tasks.workunit.client.1.smithi196.stdout:1/290: dwrite d1/d28/d34/f52 [0,4194304] 0 2024-03-13T14:49:35.423 INFO:tasks.workunit.client.1.smithi196.stdout:1/291: chown d1/d3f/d5b/f60 38103 1 2024-03-13T14:49:35.427 INFO:tasks.workunit.client.0.smithi176.stdout:6/121: creat d0/de/f2c x:0 0 0 2024-03-13T14:49:35.431 INFO:tasks.workunit.client.1.smithi196.stdout:6/419: creat d6/d7/dd/f86 x:0 0 0 2024-03-13T14:49:35.433 INFO:tasks.workunit.client.1.smithi196.stdout:5/418: truncate d4/f11 4578550 0 2024-03-13T14:49:35.435 INFO:tasks.workunit.client.0.smithi176.stdout:9/161: dwrite d0/d1e/f2e [4194304,4194304] 0 2024-03-13T14:49:35.435 INFO:tasks.workunit.client.1.smithi196.stdout:5/419: stat d4/db/d14/d19 0 2024-03-13T14:49:35.436 INFO:tasks.workunit.client.0.smithi176.stdout:9/162: chown d0/db 41568136 1 2024-03-13T14:49:35.437 INFO:tasks.workunit.client.1.smithi196.stdout:5/420: read d4/db/f81 [363619,30365] 0 2024-03-13T14:49:35.437 INFO:tasks.workunit.client.1.smithi196.stdout:8/244: dread d0/d10/f26 [0,4194304] 0 2024-03-13T14:49:35.442 INFO:tasks.workunit.client.0.smithi176.stdout:6/122: getdents d0/d10/d19/d27/d2b 0 2024-03-13T14:49:35.446 INFO:tasks.workunit.client.1.smithi196.stdout:7/385: chown d0/d9/d1e/l5f 6347 1 2024-03-13T14:49:35.450 INFO:tasks.workunit.client.1.smithi196.stdout:9/267: creat d7/d9/d17/f51 x:0 0 0 2024-03-13T14:49:35.452 INFO:tasks.workunit.client.1.smithi196.stdout:9/268: chown d7/de/d4b/d1d/l4f 42533 1 2024-03-13T14:49:35.456 INFO:tasks.workunit.client.1.smithi196.stdout:6/420: write d6/d2a/d23/f4c [3759949,14981] 0 2024-03-13T14:49:35.459 INFO:tasks.workunit.client.1.smithi196.stdout:6/421: read - d6/d2a/f83 zero size 2024-03-13T14:49:35.460 INFO:tasks.workunit.client.1.smithi196.stdout:5/421: rmdir d4/db/d14/d19/d26 39 2024-03-13T14:49:35.461 INFO:tasks.workunit.client.1.smithi196.stdout:8/245: symlink d0/d1c/d3b/d3f/l55 0 2024-03-13T14:49:35.462 INFO:tasks.workunit.client.1.smithi196.stdout:6/422: chown d6/d2a/dc/d1c/d34/l4f 927668 1 2024-03-13T14:49:35.463 INFO:tasks.workunit.client.1.smithi196.stdout:7/386: rmdir d0/d6/d33 39 2024-03-13T14:49:35.463 INFO:tasks.workunit.client.1.smithi196.stdout:6/423: dread - d6/d9/d11/f70 zero size 2024-03-13T14:49:35.464 INFO:tasks.workunit.client.0.smithi176.stdout:4/127: sync 2024-03-13T14:49:35.464 INFO:tasks.workunit.client.0.smithi176.stdout:1/115: sync 2024-03-13T14:49:35.464 INFO:tasks.workunit.client.0.smithi176.stdout:5/107: sync 2024-03-13T14:49:35.464 INFO:tasks.workunit.client.0.smithi176.stdout:6/123: getdents d0/d9 0 2024-03-13T14:49:35.465 INFO:tasks.workunit.client.0.smithi176.stdout:1/116: readlink l0 0 2024-03-13T14:49:35.466 INFO:tasks.workunit.client.0.smithi176.stdout:6/124: chown d0/d10 33 1 2024-03-13T14:49:35.466 INFO:tasks.workunit.client.1.smithi196.stdout:6/424: read d6/d9/d17/d19/f20 [4436606,91238] 0 2024-03-13T14:49:35.467 INFO:tasks.workunit.client.0.smithi176.stdout:4/128: truncate d7/d10/f2a 101593 0 2024-03-13T14:49:35.467 INFO:tasks.workunit.client.0.smithi176.stdout:1/117: dread - d2/f21 zero size 2024-03-13T14:49:35.469 INFO:tasks.workunit.client.0.smithi176.stdout:6/125: dread - d0/d10/d19/f1a zero size 2024-03-13T14:49:35.469 INFO:tasks.workunit.client.0.smithi176.stdout:9/163: truncate d0/d17/f19 383178 0 2024-03-13T14:49:35.472 INFO:tasks.workunit.client.0.smithi176.stdout:1/118: rmdir d2/dd 39 2024-03-13T14:49:35.475 INFO:tasks.workunit.client.1.smithi196.stdout:2/215: rename d0/d6/d34/c3d to d0/d6/c45 0 2024-03-13T14:49:35.479 INFO:tasks.workunit.client.0.smithi176.stdout:6/126: creat d0/d24/f2d x:0 0 0 2024-03-13T14:49:35.480 INFO:tasks.workunit.client.1.smithi196.stdout:7/387: mknod d0/d9/d1e/d58/c7b 0 2024-03-13T14:49:35.481 INFO:tasks.workunit.client.0.smithi176.stdout:6/127: chown d0/d9/c26 1039859340 1 2024-03-13T14:49:35.484 INFO:tasks.workunit.client.0.smithi176.stdout:4/129: creat d7/d10/d2c/f30 x:0 0 0 2024-03-13T14:49:35.485 INFO:tasks.workunit.client.0.smithi176.stdout:1/119: dread d2/f9 [0,4194304] 0 2024-03-13T14:49:35.486 INFO:tasks.workunit.client.1.smithi196.stdout:8/246: rename d0/d1c/d29/f3a to d0/f56 0 2024-03-13T14:49:35.488 INFO:tasks.workunit.client.1.smithi196.stdout:2/216: creat d0/d7/d40/f46 x:0 0 0 2024-03-13T14:49:35.490 INFO:tasks.workunit.client.0.smithi176.stdout:5/108: getdents d5 0 2024-03-13T14:49:35.492 INFO:tasks.workunit.client.0.smithi176.stdout:6/128: rename d0/d24 to d0/d10/d19/d27/d2b/d2e 0 2024-03-13T14:49:35.493 INFO:tasks.workunit.client.0.smithi176.stdout:9/164: creat d0/db/f31 x:0 0 0 2024-03-13T14:49:35.494 INFO:tasks.workunit.client.1.smithi196.stdout:7/388: creat d0/d9/d1e/d58/d72/f7c x:0 0 0 2024-03-13T14:49:35.497 INFO:tasks.workunit.client.1.smithi196.stdout:6/425: creat d6/d7/df/d27/d39/d44/d74/f87 x:0 0 0 2024-03-13T14:49:35.499 INFO:tasks.workunit.client.0.smithi176.stdout:1/120: mkdir d2/de/d10/d14/d22/d25 0 2024-03-13T14:49:35.500 INFO:tasks.workunit.client.1.smithi196.stdout:4/266: sync 2024-03-13T14:49:35.501 INFO:tasks.workunit.client.0.smithi176.stdout:9/165: mkdir d0/d13/d32 0 2024-03-13T14:49:35.504 INFO:tasks.workunit.client.1.smithi196.stdout:9/269: getdents d7/de 0 2024-03-13T14:49:35.507 INFO:tasks.workunit.client.0.smithi176.stdout:4/130: link d7/d8/l1e d7/d8/d15/l31 0 2024-03-13T14:49:35.509 INFO:tasks.workunit.client.1.smithi196.stdout:1/292: link d1/d9/d33/f46 d1/d9/d4e/d58/f65 0 2024-03-13T14:49:35.510 INFO:tasks.workunit.client.0.smithi176.stdout:5/109: write d5/f8 [402028,106449] 0 2024-03-13T14:49:35.511 INFO:tasks.workunit.client.1.smithi196.stdout:1/293: read d1/d38/f40 [3674672,21992] 0 2024-03-13T14:49:35.513 INFO:tasks.workunit.client.0.smithi176.stdout:4/131: creat d7/d10/d2c/f32 x:0 0 0 2024-03-13T14:49:35.515 INFO:tasks.workunit.client.1.smithi196.stdout:5/422: getdents d4/db/d14/d19/d1c/d25/d38/d47 0 2024-03-13T14:49:35.515 INFO:tasks.workunit.client.1.smithi196.stdout:2/217: symlink d0/d7/d9/l47 0 2024-03-13T14:49:35.517 INFO:tasks.workunit.client.1.smithi196.stdout:4/267: dwrite d1/d3/d7/d35/f40 [0,4194304] 0 2024-03-13T14:49:35.521 INFO:tasks.workunit.client.1.smithi196.stdout:7/389: creat d0/d6/d33/d57/f7d x:0 0 0 2024-03-13T14:49:35.523 INFO:tasks.workunit.client.1.smithi196.stdout:5/423: read d4/db/d14/d37/d41/f42 [2386137,127901] 0 2024-03-13T14:49:35.524 INFO:tasks.workunit.client.0.smithi176.stdout:9/166: link d0/d1e/l24 d0/d1e/l33 0 2024-03-13T14:49:35.526 INFO:tasks.workunit.client.0.smithi176.stdout:4/132: mkdir d7/d8/d33 0 2024-03-13T14:49:35.526 INFO:tasks.workunit.client.0.smithi176.stdout:5/110: creat d5/d22/d15/f27 x:0 0 0 2024-03-13T14:49:35.526 INFO:tasks.workunit.client.1.smithi196.stdout:6/426: link d6/d4b/f58 d6/d7/df/d27/d39/d44/d74/f88 0 2024-03-13T14:49:35.528 INFO:tasks.workunit.client.0.smithi176.stdout:9/167: fsync d0/f10 0 2024-03-13T14:49:35.529 INFO:tasks.workunit.client.1.smithi196.stdout:9/270: dwrite d7/de/d16/f2a [0,4194304] 0 2024-03-13T14:49:35.530 INFO:tasks.workunit.client.1.smithi196.stdout:6/427: chown d6/d9/d11/f5e 222492776 1 2024-03-13T14:49:35.531 INFO:tasks.workunit.client.0.smithi176.stdout:1/121: dread d2/f9 [0,4194304] 0 2024-03-13T14:49:35.531 INFO:tasks.workunit.client.1.smithi196.stdout:8/247: creat d0/d10/d12/d17/f57 x:0 0 0 2024-03-13T14:49:35.535 INFO:tasks.workunit.client.0.smithi176.stdout:4/133: fdatasync d7/d8/f28 0 2024-03-13T14:49:35.535 INFO:tasks.workunit.client.1.smithi196.stdout:8/248: write d0/d1c/d3b/f1f [314185,115569] 0 2024-03-13T14:49:35.536 INFO:tasks.workunit.client.1.smithi196.stdout:7/390: dread d0/d6/f1d [0,4194304] 0 2024-03-13T14:49:35.555 INFO:tasks.workunit.client.0.smithi176.stdout:5/111: creat d5/d22/f28 x:0 0 0 2024-03-13T14:49:35.557 INFO:tasks.workunit.client.1.smithi196.stdout:8/249: dwrite d0/d1c/d3b/f1f [0,4194304] 0 2024-03-13T14:49:35.563 INFO:tasks.workunit.client.1.smithi196.stdout:8/250: dread - d0/d1c/d29/f3c zero size 2024-03-13T14:49:35.564 INFO:tasks.workunit.client.0.smithi176.stdout:7/137: sync 2024-03-13T14:49:35.564 INFO:tasks.workunit.client.0.smithi176.stdout:2/144: sync 2024-03-13T14:49:35.565 INFO:tasks.workunit.client.0.smithi176.stdout:0/121: sync 2024-03-13T14:49:35.568 INFO:tasks.workunit.client.1.smithi196.stdout:7/391: read d0/f6a [716971,29538] 0 2024-03-13T14:49:35.571 INFO:tasks.workunit.client.1.smithi196.stdout:8/251: mkdir d0/d16/d58 0 2024-03-13T14:49:35.571 INFO:tasks.workunit.client.1.smithi196.stdout:7/392: dread - d0/d6/d33/d57/f7d zero size 2024-03-13T14:49:35.572 INFO:tasks.workunit.client.1.smithi196.stdout:9/271: dwrite d7/d9/f11 [0,4194304] 0 2024-03-13T14:49:35.572 INFO:tasks.workunit.client.0.smithi176.stdout:1/122: getdents d2/dd/d19 0 2024-03-13T14:49:35.574 INFO:tasks.workunit.client.0.smithi176.stdout:1/123: readlink d2/dd/l18 0 2024-03-13T14:49:35.574 INFO:tasks.workunit.client.0.smithi176.stdout:4/134: write d7/d8/f18 [3391594,28009] 0 2024-03-13T14:49:35.576 INFO:tasks.workunit.client.0.smithi176.stdout:9/168: write d0/d17/f19 [769803,125050] 0 2024-03-13T14:49:35.577 INFO:tasks.workunit.client.0.smithi176.stdout:2/145: mkdir d1/dc/d2d/d38 0 2024-03-13T14:49:35.583 INFO:tasks.workunit.client.1.smithi196.stdout:8/252: creat d0/d1c/d29/f59 x:0 0 0 2024-03-13T14:49:35.585 INFO:tasks.workunit.client.0.smithi176.stdout:5/112: creat d5/d22/f29 x:0 0 0 2024-03-13T14:49:35.587 INFO:tasks.workunit.client.1.smithi196.stdout:2/218: dread d0/d7/d9/f18 [0,4194304] 0 2024-03-13T14:49:35.588 INFO:tasks.workunit.client.1.smithi196.stdout:9/272: creat d7/de/d4b/f52 x:0 0 0 2024-03-13T14:49:35.589 INFO:tasks.workunit.client.0.smithi176.stdout:0/122: dread d2/d5/dc/f13 [0,4194304] 0 2024-03-13T14:49:35.591 INFO:tasks.workunit.client.0.smithi176.stdout:7/138: symlink d1/d13/d1b/d18/l27 0 2024-03-13T14:49:35.594 INFO:tasks.workunit.client.1.smithi196.stdout:9/273: readlink d7/de/d16/l47 0 2024-03-13T14:49:35.594 INFO:tasks.workunit.client.1.smithi196.stdout:1/294: dwrite d1/d9/f5c [0,4194304] 0 2024-03-13T14:49:35.594 INFO:tasks.workunit.client.1.smithi196.stdout:4/268: truncate d1/d1e/f34 11592013 0 2024-03-13T14:49:35.595 INFO:tasks.workunit.client.0.smithi176.stdout:2/146: dwrite d1/f15 [4194304,4194304] 0 2024-03-13T14:49:35.596 INFO:tasks.workunit.client.0.smithi176.stdout:1/124: creat d2/de/f26 x:0 0 0 2024-03-13T14:49:35.599 INFO:tasks.workunit.client.0.smithi176.stdout:1/125: chown d2/fc 205 1 2024-03-13T14:49:35.601 INFO:tasks.workunit.client.0.smithi176.stdout:4/135: rmdir d7/d8/d15/d23 39 2024-03-13T14:49:35.608 INFO:tasks.workunit.client.1.smithi196.stdout:5/424: dwrite d4/db/d14/d19/d26/f3a [0,4194304] 0 2024-03-13T14:49:35.620 INFO:tasks.workunit.client.1.smithi196.stdout:9/274: creat d7/de/d4b/d1d/d22/f53 x:0 0 0 2024-03-13T14:49:35.622 INFO:tasks.workunit.client.0.smithi176.stdout:1/126: creat d2/de/d10/d16/f27 x:0 0 0 2024-03-13T14:49:35.623 INFO:tasks.workunit.client.1.smithi196.stdout:1/295: unlink d1/d38/c14 0 2024-03-13T14:49:35.626 INFO:tasks.workunit.client.0.smithi176.stdout:2/147: dwrite d1/dc/f2e [0,4194304] 0 2024-03-13T14:49:35.627 INFO:tasks.workunit.client.0.smithi176.stdout:1/127: chown f1 1 1 2024-03-13T14:49:35.631 INFO:tasks.workunit.client.0.smithi176.stdout:4/136: dwrite d7/fc [0,4194304] 0 2024-03-13T14:49:35.637 INFO:tasks.workunit.client.1.smithi196.stdout:8/253: dread d0/d10/f21 [0,4194304] 0 2024-03-13T14:49:35.638 INFO:tasks.workunit.client.0.smithi176.stdout:7/139: mkdir d1/d28 0 2024-03-13T14:49:35.639 INFO:tasks.workunit.client.0.smithi176.stdout:7/140: dread - d1/f1f zero size 2024-03-13T14:49:35.643 INFO:tasks.workunit.client.0.smithi176.stdout:1/128: symlink d2/dd/d19/l28 0 2024-03-13T14:49:35.643 INFO:tasks.workunit.client.0.smithi176.stdout:1/129: stat d2/dd/l18 0 2024-03-13T14:49:35.644 INFO:tasks.workunit.client.1.smithi196.stdout:7/393: getdents d0/d9/d1e/d58/d72 0 2024-03-13T14:49:35.645 INFO:tasks.workunit.client.0.smithi176.stdout:1/130: stat d2/de/d10/l23 0 2024-03-13T14:49:35.646 INFO:tasks.workunit.client.0.smithi176.stdout:0/123: link d2/d5/d1e/c1d d2/d5/d1e/c1f 0 2024-03-13T14:49:35.647 INFO:tasks.workunit.client.0.smithi176.stdout:0/124: dread - d2/f11 zero size 2024-03-13T14:49:35.648 INFO:tasks.workunit.client.0.smithi176.stdout:7/141: readlink d1/d7/l22 0 2024-03-13T14:49:35.648 INFO:tasks.workunit.client.0.smithi176.stdout:0/125: readlink d2/l7 0 2024-03-13T14:49:35.651 INFO:tasks.workunit.client.1.smithi196.stdout:9/275: truncate d7/fd 1891302 0 2024-03-13T14:49:35.652 INFO:tasks.workunit.client.1.smithi196.stdout:2/219: getdents d0/d6/d34 0 2024-03-13T14:49:35.653 INFO:tasks.workunit.client.1.smithi196.stdout:1/296: dwrite d1/d9/d33/d4b/f1b [0,4194304] 0 2024-03-13T14:49:35.654 INFO:tasks.workunit.client.0.smithi176.stdout:7/142: rmdir d1/d13 39 2024-03-13T14:49:35.655 INFO:tasks.workunit.client.1.smithi196.stdout:2/220: fsync d0/d7/f3f 0 2024-03-13T14:49:35.655 INFO:tasks.workunit.client.1.smithi196.stdout:1/297: stat d1/d28/d34/f3e 0 2024-03-13T14:49:35.658 INFO:tasks.workunit.client.0.smithi176.stdout:7/143: chown d1/d1c/c21 1275 1 2024-03-13T14:49:35.663 INFO:tasks.workunit.client.1.smithi196.stdout:4/269: getdents d1/d3/d7 0 2024-03-13T14:49:35.663 INFO:tasks.workunit.client.0.smithi176.stdout:3/136: sync 2024-03-13T14:49:35.664 INFO:tasks.workunit.client.0.smithi176.stdout:3/137: stat d4/cf 0 2024-03-13T14:49:35.664 INFO:tasks.workunit.client.0.smithi176.stdout:8/134: sync 2024-03-13T14:49:35.666 INFO:tasks.workunit.client.0.smithi176.stdout:8/135: chown d0/db/f1a 12 1 2024-03-13T14:49:35.667 INFO:tasks.workunit.client.0.smithi176.stdout:4/137: link d7/c19 d7/d10/c34 0 2024-03-13T14:49:35.667 INFO:tasks.workunit.client.0.smithi176.stdout:3/138: write d4/f1d [720359,12750] 0 2024-03-13T14:49:35.669 INFO:tasks.workunit.client.0.smithi176.stdout:0/126: dwrite d2/d5/dc/f16 [0,4194304] 0 2024-03-13T14:49:35.669 INFO:tasks.workunit.client.0.smithi176.stdout:9/169: truncate d0/db/f1d 3619802 0 2024-03-13T14:49:35.670 INFO:tasks.workunit.client.0.smithi176.stdout:8/136: chown d0/d5/f1f 62955172 1 2024-03-13T14:49:35.671 INFO:tasks.workunit.client.1.smithi196.stdout:8/254: creat d0/d1c/d3b/d48/f5a x:0 0 0 2024-03-13T14:49:35.671 INFO:tasks.workunit.client.0.smithi176.stdout:8/137: fdatasync d0/db/fd 0 2024-03-13T14:49:35.672 INFO:tasks.workunit.client.0.smithi176.stdout:0/127: write d2/db/f1c [330944,43186] 0 2024-03-13T14:49:35.673 INFO:tasks.workunit.client.1.smithi196.stdout:4/270: truncate d1/d3/f3a 708081 0 2024-03-13T14:49:35.673 INFO:tasks.workunit.client.0.smithi176.stdout:4/138: truncate d7/d10/d2c/f30 673935 0 2024-03-13T14:49:35.673 INFO:tasks.workunit.client.0.smithi176.stdout:8/138: dread - d0/db/f25 zero size 2024-03-13T14:49:35.675 INFO:tasks.workunit.client.0.smithi176.stdout:7/144: creat d1/d7/f29 x:0 0 0 2024-03-13T14:49:35.675 INFO:tasks.workunit.client.0.smithi176.stdout:8/139: readlink d0/la 0 2024-03-13T14:49:35.677 INFO:tasks.workunit.client.1.smithi196.stdout:4/271: dread d1/d3/d7/f1d [0,4194304] 0 2024-03-13T14:49:35.683 INFO:tasks.workunit.client.0.smithi176.stdout:5/113: sync 2024-03-13T14:49:35.683 INFO:tasks.workunit.client.0.smithi176.stdout:6/129: sync 2024-03-13T14:49:35.686 INFO:tasks.workunit.client.0.smithi176.stdout:1/131: rename d2/de/d10/d14/d22/d25 to d2/d29 0 2024-03-13T14:49:35.687 INFO:tasks.workunit.client.0.smithi176.stdout:5/114: chown d5/d22/d15/c20 257162 1 2024-03-13T14:49:35.691 INFO:tasks.workunit.client.0.smithi176.stdout:1/132: write d2/fc [650786,4253] 0 2024-03-13T14:49:35.693 INFO:tasks.workunit.client.0.smithi176.stdout:5/115: write d5/f6 [4908716,45320] 0 2024-03-13T14:49:35.694 INFO:tasks.workunit.client.0.smithi176.stdout:4/139: dwrite d7/d8/f11 [4194304,4194304] 0 2024-03-13T14:49:35.695 INFO:tasks.workunit.client.0.smithi176.stdout:4/140: readlink d7/l9 0 2024-03-13T14:49:35.696 INFO:tasks.workunit.client.0.smithi176.stdout:0/128: mkdir d2/d5/dc/d20 0 2024-03-13T14:49:35.698 INFO:tasks.workunit.client.0.smithi176.stdout:8/140: mkdir d0/db/d27 0 2024-03-13T14:49:35.698 INFO:tasks.workunit.client.0.smithi176.stdout:5/116: write d5/d22/d15/f27 [623531,123868] 0 2024-03-13T14:49:35.698 INFO:tasks.workunit.client.0.smithi176.stdout:0/129: dread - d2/f11 zero size 2024-03-13T14:49:35.699 INFO:tasks.workunit.client.0.smithi176.stdout:5/117: chown d5/f6 700514269 1 2024-03-13T14:49:35.701 INFO:tasks.workunit.client.0.smithi176.stdout:8/141: write d0/d5/d11/f26 [93339,22311] 0 2024-03-13T14:49:35.703 INFO:tasks.workunit.client.0.smithi176.stdout:5/118: chown d5/f24 20344 1 2024-03-13T14:49:35.706 INFO:tasks.workunit.client.0.smithi176.stdout:7/145: write d1/d7/f1a [3758378,49532] 0 2024-03-13T14:49:35.706 INFO:tasks.workunit.client.0.smithi176.stdout:2/148: link d1/dc/l23 d1/dc/l39 0 2024-03-13T14:49:35.714 INFO:tasks.workunit.client.1.smithi196.stdout:7/394: dwrite d0/d9/f24 [4194304,4194304] 0 2024-03-13T14:49:35.719 INFO:tasks.workunit.client.0.smithi176.stdout:4/141: creat d7/d8/d15/d23/f35 x:0 0 0 2024-03-13T14:49:35.719 INFO:tasks.workunit.client.1.smithi196.stdout:8/255: symlink d0/d1c/d29/d47/l5b 0 2024-03-13T14:49:35.720 INFO:tasks.workunit.client.0.smithi176.stdout:9/170: dwrite d0/d1e/f25 [0,4194304] 0 2024-03-13T14:49:35.722 INFO:tasks.workunit.client.0.smithi176.stdout:1/133: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:35.722 INFO:tasks.workunit.client.0.smithi176.stdout:9/171: fdatasync d0/db/f31 0 2024-03-13T14:49:35.723 INFO:tasks.workunit.client.0.smithi176.stdout:4/142: truncate d7/d10/f2a 336470 0 2024-03-13T14:49:35.726 INFO:tasks.workunit.client.0.smithi176.stdout:7/146: readlink d1/d13/d1b/d18/l27 0 2024-03-13T14:49:35.726 INFO:tasks.workunit.client.0.smithi176.stdout:5/119: mknod d5/d22/d15/c2a 0 2024-03-13T14:49:35.728 INFO:tasks.workunit.client.0.smithi176.stdout:4/143: chown d7/d8/l25 943086 1 2024-03-13T14:49:35.728 INFO:tasks.workunit.client.0.smithi176.stdout:5/120: chown d5/d22 438955480 1 2024-03-13T14:49:35.729 INFO:tasks.workunit.client.1.smithi196.stdout:4/272: fsync d1/f6 0 2024-03-13T14:49:35.729 INFO:tasks.workunit.client.1.smithi196.stdout:1/298: mkdir d1/d9/d33/d4b/d66 0 2024-03-13T14:49:35.732 INFO:tasks.workunit.client.0.smithi176.stdout:3/139: dwrite d4/fa [0,4194304] 0 2024-03-13T14:49:35.736 INFO:tasks.workunit.client.1.smithi196.stdout:7/395: readlink d0/d9/d31/d50/l5e 0 2024-03-13T14:49:35.736 INFO:tasks.workunit.client.0.smithi176.stdout:8/142: dwrite d0/db/f19 [0,4194304] 0 2024-03-13T14:49:35.736 INFO:tasks.workunit.client.1.smithi196.stdout:8/256: fsync d0/d1c/d3b/f38 0 2024-03-13T14:49:35.737 INFO:tasks.workunit.client.0.smithi176.stdout:0/130: link d2/d5/dc/f13 d2/db/f21 0 2024-03-13T14:49:35.739 INFO:tasks.workunit.client.0.smithi176.stdout:6/130: rename d0/d10/d19/d27/d2b/d2e/c25 to d0/c2f 0 2024-03-13T14:49:35.740 INFO:tasks.workunit.client.0.smithi176.stdout:3/140: truncate d4/d1a/f27 986871 0 2024-03-13T14:49:35.740 INFO:tasks.workunit.client.1.smithi196.stdout:8/257: truncate d0/d10/d12/f50 833794 0 2024-03-13T14:49:35.742 INFO:tasks.workunit.client.1.smithi196.stdout:9/276: creat d7/de/d4b/d1d/f54 x:0 0 0 2024-03-13T14:49:35.742 INFO:tasks.workunit.client.0.smithi176.stdout:8/143: write d0/db/f25 [1009328,93055] 0 2024-03-13T14:49:35.746 INFO:tasks.workunit.client.1.smithi196.stdout:2/221: link d0/f32 d0/d7/d9/f48 0 2024-03-13T14:49:35.748 INFO:tasks.workunit.client.1.smithi196.stdout:1/299: mkdir d1/d28/d34/d67 0 2024-03-13T14:49:35.749 INFO:tasks.workunit.client.0.smithi176.stdout:5/121: mkdir d5/d2b 0 2024-03-13T14:49:35.749 INFO:tasks.workunit.client.0.smithi176.stdout:7/147: unlink d1/d13/d1b/d18/l27 0 2024-03-13T14:49:35.750 INFO:tasks.workunit.client.1.smithi196.stdout:4/273: creat d1/d3/d7/f49 x:0 0 0 2024-03-13T14:49:35.751 INFO:tasks.workunit.client.0.smithi176.stdout:3/141: dread f1 [0,4194304] 0 2024-03-13T14:49:35.752 INFO:tasks.workunit.client.1.smithi196.stdout:4/274: chown d1/l2d 0 1 2024-03-13T14:49:35.757 INFO:tasks.workunit.client.1.smithi196.stdout:7/396: creat d0/d9/d1e/d3f/f7e x:0 0 0 2024-03-13T14:49:35.758 INFO:tasks.workunit.client.0.smithi176.stdout:4/144: rename l3 to d7/d2d/l36 0 2024-03-13T14:49:35.759 INFO:tasks.workunit.client.0.smithi176.stdout:9/172: dwrite d0/d1e/f2e [0,4194304] 0 2024-03-13T14:49:35.761 INFO:tasks.workunit.client.0.smithi176.stdout:1/134: dwrite d2/fc [4194304,4194304] 0 2024-03-13T14:49:35.762 INFO:tasks.workunit.client.0.smithi176.stdout:7/148: chown d1/d2/f25 561144367 1 2024-03-13T14:49:35.765 INFO:tasks.workunit.client.0.smithi176.stdout:4/145: write d7/d8/d15/f2b [242287,89622] 0 2024-03-13T14:49:35.765 INFO:tasks.workunit.client.1.smithi196.stdout:1/300: mknod d1/d28/d34/c68 0 2024-03-13T14:49:35.768 INFO:tasks.workunit.client.0.smithi176.stdout:7/149: chown d1/d7 30792996 1 2024-03-13T14:49:35.768 INFO:tasks.workunit.client.1.smithi196.stdout:4/275: mknod d1/d1e/d43/c4a 0 2024-03-13T14:49:35.769 INFO:tasks.workunit.client.0.smithi176.stdout:2/149: creat d1/d2a/d33/f3a x:0 0 0 2024-03-13T14:49:35.770 INFO:tasks.workunit.client.1.smithi196.stdout:7/397: rmdir d0/d6/d35 39 2024-03-13T14:49:35.774 INFO:tasks.workunit.client.1.smithi196.stdout:4/276: write d1/d3/f13 [2688050,31209] 0 2024-03-13T14:49:35.775 INFO:tasks.workunit.client.0.smithi176.stdout:2/150: readlink d1/dc/l39 0 2024-03-13T14:49:35.775 INFO:tasks.workunit.client.0.smithi176.stdout:6/131: symlink d0/d10/d19/d27/d2b/d2e/l30 0 2024-03-13T14:49:35.775 INFO:tasks.workunit.client.0.smithi176.stdout:8/144: rename d0/d5/c10 to d0/c28 0 2024-03-13T14:49:35.775 INFO:tasks.workunit.client.0.smithi176.stdout:0/131: dread d2/d5/dc/f13 [0,4194304] 0 2024-03-13T14:49:35.776 INFO:tasks.workunit.client.1.smithi196.stdout:8/258: rename d0/l2 to d0/d1c/d29/l5c 0 2024-03-13T14:49:35.776 INFO:tasks.workunit.client.0.smithi176.stdout:4/146: chown d7/d10 31 1 2024-03-13T14:49:35.776 INFO:tasks.workunit.client.0.smithi176.stdout:6/132: write d0/d9/f14 [736804,47359] 0 2024-03-13T14:49:35.781 INFO:tasks.workunit.client.1.smithi196.stdout:3/303: sync 2024-03-13T14:49:35.781 INFO:tasks.workunit.client.1.smithi196.stdout:0/377: sync 2024-03-13T14:49:35.782 INFO:tasks.workunit.client.0.smithi176.stdout:9/173: dread d0/d1e/f2e [4194304,4194304] 0 2024-03-13T14:49:35.783 INFO:tasks.workunit.client.0.smithi176.stdout:1/135: creat d2/de/d10/f2a x:0 0 0 2024-03-13T14:49:35.784 INFO:tasks.workunit.client.1.smithi196.stdout:1/301: creat d1/d3f/d5b/f69 x:0 0 0 2024-03-13T14:49:35.786 INFO:tasks.workunit.client.0.smithi176.stdout:1/136: dread - d2/de/f26 zero size 2024-03-13T14:49:35.787 INFO:tasks.workunit.client.0.smithi176.stdout:3/142: creat d4/d28/f29 x:0 0 0 2024-03-13T14:49:35.789 INFO:tasks.workunit.client.0.smithi176.stdout:3/143: chown d4/d13/d16/c23 499316127 1 2024-03-13T14:49:35.790 INFO:tasks.workunit.client.0.smithi176.stdout:3/144: write d4/d28/f29 [252277,117294] 0 2024-03-13T14:49:35.791 INFO:tasks.workunit.client.1.smithi196.stdout:7/398: mkdir d0/d9/d1e/d58/d72/d6e/d7f 0 2024-03-13T14:49:35.792 INFO:tasks.workunit.client.1.smithi196.stdout:9/277: rename d7/c1e to d7/c55 0 2024-03-13T14:49:35.794 INFO:tasks.workunit.client.1.smithi196.stdout:9/278: fdatasync d7/de/d4b/f52 0 2024-03-13T14:49:35.794 INFO:tasks.workunit.client.1.smithi196.stdout:8/259: dread d0/d1c/d3b/f38 [0,4194304] 0 2024-03-13T14:49:35.796 INFO:tasks.workunit.client.1.smithi196.stdout:2/222: link d0/d7/d9/c26 d0/d7/d40/c49 0 2024-03-13T14:49:35.797 INFO:tasks.workunit.client.1.smithi196.stdout:2/223: stat d0/c39 0 2024-03-13T14:49:35.797 INFO:tasks.workunit.client.0.smithi176.stdout:2/151: write d1/f14 [2812815,17054] 0 2024-03-13T14:49:35.799 INFO:tasks.workunit.client.1.smithi196.stdout:2/224: stat d0/d7/d40/f36 0 2024-03-13T14:49:35.802 INFO:tasks.workunit.client.1.smithi196.stdout:3/304: creat db/d19/d50/f63 x:0 0 0 2024-03-13T14:49:35.804 INFO:tasks.workunit.client.1.smithi196.stdout:4/277: mkdir d1/d4b 0 2024-03-13T14:49:35.805 INFO:tasks.workunit.client.0.smithi176.stdout:0/132: unlink d2/db/f12 0 2024-03-13T14:49:35.808 INFO:tasks.workunit.client.1.smithi196.stdout:1/302: creat d1/d38/f6a x:0 0 0 2024-03-13T14:49:35.811 INFO:tasks.workunit.client.1.smithi196.stdout:9/279: dread f5 [4194304,4194304] 0 2024-03-13T14:49:35.813 INFO:tasks.workunit.client.0.smithi176.stdout:6/133: rename d0/fc to d0/d10/d19/d27/d2b/d2e/f31 0 2024-03-13T14:49:35.814 INFO:tasks.workunit.client.0.smithi176.stdout:6/134: fdatasync d0/f18 0 2024-03-13T14:49:35.815 INFO:tasks.workunit.client.1.smithi196.stdout:9/280: chown d7/de/d4b/d1d/f3b 94185 1 2024-03-13T14:49:35.815 INFO:tasks.workunit.client.0.smithi176.stdout:7/150: creat d1/d28/f2a x:0 0 0 2024-03-13T14:49:35.816 INFO:tasks.workunit.client.0.smithi176.stdout:7/151: fdatasync d1/f19 0 2024-03-13T14:49:35.817 INFO:tasks.workunit.client.1.smithi196.stdout:8/260: mknod d0/d16/d3d/c5d 0 2024-03-13T14:49:35.817 INFO:tasks.workunit.client.0.smithi176.stdout:7/152: fdatasync d1/f1f 0 2024-03-13T14:49:35.818 INFO:tasks.workunit.client.1.smithi196.stdout:9/281: read d7/de/d4b/d1d/f23 [3270688,16569] 0 2024-03-13T14:49:35.818 INFO:tasks.workunit.client.1.smithi196.stdout:2/225: dwrite d0/d6/f14 [0,4194304] 0 2024-03-13T14:49:35.819 INFO:tasks.workunit.client.1.smithi196.stdout:8/261: readlink d0/d1c/d3b/d3f/l55 0 2024-03-13T14:49:35.820 INFO:tasks.workunit.client.0.smithi176.stdout:9/174: mknod d0/d1e/c34 0 2024-03-13T14:49:35.821 INFO:tasks.workunit.client.0.smithi176.stdout:9/175: readlink d0/d13/l26 0 2024-03-13T14:49:35.822 INFO:tasks.workunit.client.1.smithi196.stdout:0/378: dread d8/f1e [0,4194304] 0 2024-03-13T14:49:35.826 INFO:tasks.workunit.client.1.smithi196.stdout:0/379: fsync d8/da/d25/d26/d35/d3b/d50/d55/f64 0 2024-03-13T14:49:35.826 INFO:tasks.workunit.client.1.smithi196.stdout:9/282: read d7/de/f29 [165994,30801] 0 2024-03-13T14:49:35.826 INFO:tasks.workunit.client.0.smithi176.stdout:3/145: chown d4/c1e 513632042 1 2024-03-13T14:49:35.827 INFO:tasks.workunit.client.1.smithi196.stdout:9/283: chown d7/de/d4b/d1d 200 1 2024-03-13T14:49:35.833 INFO:tasks.workunit.client.0.smithi176.stdout:5/122: chown d5/d22/d15/f21 29 1 2024-03-13T14:49:35.833 INFO:tasks.workunit.client.0.smithi176.stdout:2/152: symlink d1/dc/d2d/l3b 0 2024-03-13T14:49:35.836 INFO:tasks.workunit.client.0.smithi176.stdout:4/147: creat d7/d8/d2f/f37 x:0 0 0 2024-03-13T14:49:35.838 INFO:tasks.workunit.client.0.smithi176.stdout:4/148: chown d7/d8/d15/d23 99 1 2024-03-13T14:49:35.841 INFO:tasks.workunit.client.0.smithi176.stdout:4/149: write d7/d2d/f2e [347338,63745] 0 2024-03-13T14:49:35.841 INFO:tasks.workunit.client.0.smithi176.stdout:6/135: symlink d0/d10/l32 0 2024-03-13T14:49:35.847 INFO:tasks.workunit.client.0.smithi176.stdout:9/176: dwrite d0/d1e/f25 [0,4194304] 0 2024-03-13T14:49:35.847 INFO:tasks.workunit.client.0.smithi176.stdout:7/153: mkdir d1/d2/d2b 0 2024-03-13T14:49:35.847 INFO:tasks.workunit.client.0.smithi176.stdout:6/136: write d0/d9/f1f [1519497,79975] 0 2024-03-13T14:49:35.848 INFO:tasks.workunit.client.1.smithi196.stdout:2/226: write d0/d7/d9/f18 [3321355,4705] 0 2024-03-13T14:49:35.850 INFO:tasks.workunit.client.0.smithi176.stdout:7/154: chown d1/d13/d1b 0 1 2024-03-13T14:49:35.850 INFO:tasks.workunit.client.0.smithi176.stdout:9/177: chown d0/d1e/c34 7326442 1 2024-03-13T14:49:35.851 INFO:tasks.workunit.client.1.smithi196.stdout:0/380: unlink d8/da/d25/d26/f76 0 2024-03-13T14:49:35.852 INFO:tasks.workunit.client.0.smithi176.stdout:8/145: fsync d0/d5/f1f 0 2024-03-13T14:49:35.853 INFO:tasks.workunit.client.0.smithi176.stdout:7/155: dread - d1/d7/f29 zero size 2024-03-13T14:49:35.854 INFO:tasks.workunit.client.0.smithi176.stdout:3/146: creat d4/d13/f2a x:0 0 0 2024-03-13T14:49:35.855 INFO:tasks.workunit.client.0.smithi176.stdout:4/150: fdatasync d7/d2d/f2e 0 2024-03-13T14:49:35.855 INFO:tasks.workunit.client.1.smithi196.stdout:6/428: sync 2024-03-13T14:49:35.855 INFO:tasks.workunit.client.1.smithi196.stdout:5/425: sync 2024-03-13T14:49:35.856 INFO:tasks.workunit.client.0.smithi176.stdout:9/178: truncate d0/d13/f16 4729165 0 2024-03-13T14:49:35.856 INFO:tasks.workunit.client.0.smithi176.stdout:6/137: truncate d0/de/f2c 942798 0 2024-03-13T14:49:35.857 INFO:tasks.workunit.client.1.smithi196.stdout:4/278: mkdir d1/d3/d4c 0 2024-03-13T14:49:35.858 INFO:tasks.workunit.client.0.smithi176.stdout:6/138: stat d0/f8 0 2024-03-13T14:49:35.858 INFO:tasks.workunit.client.1.smithi196.stdout:7/399: truncate d0/d6/d33/f39 4138184 0 2024-03-13T14:49:35.860 INFO:tasks.workunit.client.0.smithi176.stdout:5/123: chown d5/f7 121 1 2024-03-13T14:49:35.860 INFO:tasks.workunit.client.0.smithi176.stdout:2/153: readlink d1/l10 0 2024-03-13T14:49:35.860 INFO:tasks.workunit.client.1.smithi196.stdout:7/400: fdatasync d0/d9/f59 0 2024-03-13T14:49:35.861 INFO:tasks.workunit.client.1.smithi196.stdout:4/279: read d1/d3/f36 [610349,128353] 0 2024-03-13T14:49:35.861 INFO:tasks.workunit.client.1.smithi196.stdout:8/262: creat d0/d10/d12/d1e/f5e x:0 0 0 2024-03-13T14:49:35.862 INFO:tasks.workunit.client.0.smithi176.stdout:2/154: fsync d1/fb 0 2024-03-13T14:49:35.864 INFO:tasks.workunit.client.0.smithi176.stdout:2/155: write d1/dc/d1c/f1d [4037536,107992] 0 2024-03-13T14:49:35.868 INFO:tasks.workunit.client.0.smithi176.stdout:2/156: truncate d1/d2a/d33/f3a 412985 0 2024-03-13T14:49:35.868 INFO:tasks.workunit.client.1.smithi196.stdout:2/227: mknod d0/d7/d40/c4a 0 2024-03-13T14:49:35.868 INFO:tasks.workunit.client.0.smithi176.stdout:8/146: write d0/db/f25 [1019849,75539] 0 2024-03-13T14:49:35.871 INFO:tasks.workunit.client.0.smithi176.stdout:3/147: dread d4/fa [0,4194304] 0 2024-03-13T14:49:35.872 INFO:tasks.workunit.client.1.smithi196.stdout:0/381: dread d8/da/d25/d26/d35/d3b/d50/d5d/f57 [0,4194304] 0 2024-03-13T14:49:35.873 INFO:tasks.workunit.client.1.smithi196.stdout:2/228: chown d0/f37 101188739 1 2024-03-13T14:49:35.875 INFO:tasks.workunit.client.1.smithi196.stdout:9/284: symlink d7/d9/l56 0 2024-03-13T14:49:35.877 INFO:tasks.workunit.client.1.smithi196.stdout:1/303: dwrite d1/d9/d33/f62 [0,4194304] 0 2024-03-13T14:49:35.877 INFO:tasks.workunit.client.1.smithi196.stdout:5/426: creat d4/db/d4a/f8a x:0 0 0 2024-03-13T14:49:35.879 INFO:tasks.workunit.client.0.smithi176.stdout:9/179: symlink d0/d13/l35 0 2024-03-13T14:49:35.883 INFO:tasks.workunit.client.0.smithi176.stdout:6/139: rename d0/d10/l11 to d0/d10/d19/d27/l33 0 2024-03-13T14:49:35.883 INFO:tasks.workunit.client.0.smithi176.stdout:6/140: chown d0/de/f12 10209525 1 2024-03-13T14:49:35.884 INFO:tasks.workunit.client.1.smithi196.stdout:0/382: dread - d8/da/d25/d26/d54/f7f zero size 2024-03-13T14:49:35.884 INFO:tasks.workunit.client.1.smithi196.stdout:9/285: fsync d7/f20 0 2024-03-13T14:49:35.887 INFO:tasks.workunit.client.1.smithi196.stdout:6/429: dwrite d6/d2a/f83 [0,4194304] 0 2024-03-13T14:49:35.891 INFO:tasks.workunit.client.0.smithi176.stdout:2/157: unlink d1/dc/l1e 0 2024-03-13T14:49:35.891 INFO:tasks.workunit.client.0.smithi176.stdout:1/137: sync 2024-03-13T14:49:35.892 INFO:tasks.workunit.client.0.smithi176.stdout:5/124: dwrite d5/f8 [0,4194304] 0 2024-03-13T14:49:35.892 INFO:tasks.workunit.client.0.smithi176.stdout:2/158: chown d1/dc/cd 158471 1 2024-03-13T14:49:35.894 INFO:tasks.workunit.client.0.smithi176.stdout:2/159: dread - d1/f31 zero size 2024-03-13T14:49:35.894 INFO:tasks.workunit.client.1.smithi196.stdout:5/427: write d4/db/d14/d19/d26/d68/f77 [1124699,59717] 0 2024-03-13T14:49:35.895 INFO:tasks.workunit.client.0.smithi176.stdout:5/125: chown d5/d22/d15/l1c 4273 1 2024-03-13T14:49:35.899 INFO:tasks.workunit.client.0.smithi176.stdout:7/156: mknod d1/c2c 0 2024-03-13T14:49:35.899 INFO:tasks.workunit.client.1.smithi196.stdout:4/280: mknod d1/d1e/d43/c4d 0 2024-03-13T14:49:35.902 INFO:tasks.workunit.client.0.smithi176.stdout:7/157: stat d1/cc 0 2024-03-13T14:49:35.902 INFO:tasks.workunit.client.0.smithi176.stdout:3/148: creat d4/d1a/f2b x:0 0 0 2024-03-13T14:49:35.903 INFO:tasks.workunit.client.1.smithi196.stdout:3/305: getdents db 0 2024-03-13T14:49:35.903 INFO:tasks.workunit.client.0.smithi176.stdout:7/158: chown d1/d2/l12 30638 1 2024-03-13T14:49:35.907 INFO:tasks.workunit.client.1.smithi196.stdout:5/428: chown d4/db/d14/d37/f78 52 1 2024-03-13T14:49:35.910 INFO:tasks.workunit.client.1.smithi196.stdout:0/383: chown d8/da/d25/d26/d35/d3b/d5f/d17/f4d 45876082 1 2024-03-13T14:49:35.916 INFO:tasks.workunit.client.0.smithi176.stdout:2/160: dwrite d1/dc/d1c/f36 [0,4194304] 0 2024-03-13T14:49:35.919 INFO:tasks.workunit.client.0.smithi176.stdout:7/159: dread d1/d7/f1a [0,4194304] 0 2024-03-13T14:49:35.919 INFO:tasks.workunit.client.1.smithi196.stdout:8/263: link d0/d10/f21 d0/d1c/d3b/d48/f5f 0 2024-03-13T14:49:35.920 INFO:tasks.workunit.client.0.smithi176.stdout:7/160: fsync d1/f1f 0 2024-03-13T14:49:35.921 INFO:tasks.workunit.client.0.smithi176.stdout:7/161: stat d1/f19 0 2024-03-13T14:49:35.922 INFO:tasks.workunit.client.0.smithi176.stdout:9/180: mknod d0/d17/c36 0 2024-03-13T14:49:35.928 INFO:tasks.workunit.client.0.smithi176.stdout:6/141: mknod d0/d10/d19/d27/d2b/d2e/c34 0 2024-03-13T14:49:35.934 INFO:tasks.workunit.client.1.smithi196.stdout:6/430: chown d6/d6f/c78 12630 1 2024-03-13T14:49:35.934 INFO:tasks.workunit.client.1.smithi196.stdout:1/304: dwrite d1/d9/d33/d4b/f1a [0,4194304] 0 2024-03-13T14:49:35.935 INFO:tasks.workunit.client.0.smithi176.stdout:1/138: creat d2/de/d10/d14/d22/f2b x:0 0 0 2024-03-13T14:49:35.939 INFO:tasks.workunit.client.1.smithi196.stdout:6/431: chown d6/d9/d11/f70 11045 1 2024-03-13T14:49:35.940 INFO:tasks.workunit.client.1.smithi196.stdout:1/305: write d1/d38/f25 [2256518,42156] 0 2024-03-13T14:49:35.944 INFO:tasks.workunit.client.0.smithi176.stdout:1/139: dread d2/de/f17 [0,4194304] 0 2024-03-13T14:49:35.945 INFO:tasks.workunit.client.0.smithi176.stdout:1/140: read - d2/f21 zero size 2024-03-13T14:49:35.947 INFO:tasks.workunit.client.0.smithi176.stdout:1/141: chown d2/de/d10/d14/d22 167245916 1 2024-03-13T14:49:35.948 INFO:tasks.workunit.client.0.smithi176.stdout:4/151: link d7/d10/f27 d7/d8/d15/f38 0 2024-03-13T14:49:35.950 INFO:tasks.workunit.client.0.smithi176.stdout:4/152: fsync d7/d8/f11 0 2024-03-13T14:49:35.953 INFO:tasks.workunit.client.1.smithi196.stdout:8/264: creat d0/d1c/d3b/f60 x:0 0 0 2024-03-13T14:49:35.960 INFO:tasks.workunit.client.0.smithi176.stdout:3/149: rmdir d4 39 2024-03-13T14:49:35.963 INFO:tasks.workunit.client.0.smithi176.stdout:6/142: mknod d0/d10/d19/d27/d2b/d2e/c35 0 2024-03-13T14:49:35.966 INFO:tasks.workunit.client.1.smithi196.stdout:9/286: write d7/de/d4b/f1a [992803,50214] 0 2024-03-13T14:49:35.969 INFO:tasks.workunit.client.1.smithi196.stdout:9/287: write d7/de/d4b/f1a [2020329,63155] 0 2024-03-13T14:49:35.971 INFO:tasks.workunit.client.1.smithi196.stdout:0/384: mkdir d8/da/d25/d26/d48/d82 0 2024-03-13T14:49:35.972 INFO:tasks.workunit.client.0.smithi176.stdout:5/126: mkdir d5/d2c 0 2024-03-13T14:49:35.973 INFO:tasks.workunit.client.0.smithi176.stdout:8/147: write d0/f2 [4412517,130287] 0 2024-03-13T14:49:35.973 INFO:tasks.workunit.client.1.smithi196.stdout:5/429: getdents d4/db/d14/d19/d1c/d25/d38/d47 0 2024-03-13T14:49:35.974 INFO:tasks.workunit.client.0.smithi176.stdout:1/142: fsync d2/de/d10/d14/d22/f2b 0 2024-03-13T14:49:35.975 INFO:tasks.workunit.client.1.smithi196.stdout:5/430: fsync d4/f8 0 2024-03-13T14:49:35.975 INFO:tasks.workunit.client.1.smithi196.stdout:7/401: dread d0/d6/fc [0,4194304] 0 2024-03-13T14:49:35.976 INFO:tasks.workunit.client.0.smithi176.stdout:1/143: readlink d2/de/d10/d16/l1f 0 2024-03-13T14:49:35.976 INFO:tasks.workunit.client.0.smithi176.stdout:8/148: truncate d0/d5/d11/f1b 828068 0 2024-03-13T14:49:35.980 INFO:tasks.workunit.client.1.smithi196.stdout:7/402: readlink d0/d6/l3e 0 2024-03-13T14:49:35.984 INFO:tasks.workunit.client.0.smithi176.stdout:4/153: creat d7/d8/f39 x:0 0 0 2024-03-13T14:49:35.985 INFO:tasks.workunit.client.1.smithi196.stdout:6/432: symlink d6/d57/d82/l89 0 2024-03-13T14:49:35.988 INFO:tasks.workunit.client.1.smithi196.stdout:1/306: symlink d1/d28/d34/d67/l6b 0 2024-03-13T14:49:35.988 INFO:tasks.workunit.client.0.smithi176.stdout:9/181: symlink d0/d13/d32/l37 0 2024-03-13T14:49:35.992 INFO:tasks.workunit.client.1.smithi196.stdout:1/307: write d1/d9/d33/f62 [4389326,92413] 0 2024-03-13T14:49:35.993 INFO:tasks.workunit.client.1.smithi196.stdout:3/306: dwrite db/d19/f2b [0,4194304] 0 2024-03-13T14:49:35.995 INFO:tasks.workunit.client.0.smithi176.stdout:6/143: symlink d0/de/l36 0 2024-03-13T14:49:35.997 INFO:tasks.workunit.client.1.smithi196.stdout:2/229: dwrite d0/d6/d1d/f27 [0,4194304] 0 2024-03-13T14:49:36.003 INFO:tasks.workunit.client.1.smithi196.stdout:8/265: write d0/d10/f21 [4487323,93270] 0 2024-03-13T14:49:36.004 INFO:tasks.workunit.client.0.smithi176.stdout:1/144: symlink d2/de/l2c 0 2024-03-13T14:49:36.005 INFO:tasks.workunit.client.1.smithi196.stdout:7/403: creat d0/d6/d33/f80 x:0 0 0 2024-03-13T14:49:36.011 INFO:tasks.workunit.client.0.smithi176.stdout:7/162: rmdir d1/d23 0 2024-03-13T14:49:36.011 INFO:tasks.workunit.client.0.smithi176.stdout:9/182: fsync d0/d1e/d1f/f2c 0 2024-03-13T14:49:36.011 INFO:tasks.workunit.client.0.smithi176.stdout:4/154: creat d7/f3a x:0 0 0 2024-03-13T14:49:36.014 INFO:tasks.workunit.client.0.smithi176.stdout:6/144: rename d0/d9/f1f to d0/d9/f37 0 2024-03-13T14:49:36.014 INFO:tasks.workunit.client.0.smithi176.stdout:4/155: stat d7/d10/d2c 0 2024-03-13T14:49:36.015 INFO:tasks.workunit.client.0.smithi176.stdout:9/183: read d0/d1e/f2e [1102527,45112] 0 2024-03-13T14:49:36.015 INFO:tasks.workunit.client.0.smithi176.stdout:5/127: unlink d5/f1d 0 2024-03-13T14:49:36.015 INFO:tasks.workunit.client.0.smithi176.stdout:6/145: fdatasync d0/de/f23 0 2024-03-13T14:49:36.016 INFO:tasks.workunit.client.1.smithi196.stdout:0/385: dread d8/da/d25/d26/d35/d3b/d50/f53 [0,4194304] 0 2024-03-13T14:49:36.018 INFO:tasks.workunit.client.0.smithi176.stdout:9/184: write d0/f10 [8143457,126174] 0 2024-03-13T14:49:36.024 INFO:tasks.workunit.client.0.smithi176.stdout:7/163: rmdir d1/d13/d1b/d18 39 2024-03-13T14:49:36.025 INFO:tasks.workunit.client.1.smithi196.stdout:1/308: mkdir d1/d9/d6c 0 2024-03-13T14:49:36.027 INFO:tasks.workunit.client.0.smithi176.stdout:1/145: symlink d2/de/d10/d14/l2d 0 2024-03-13T14:49:36.028 INFO:tasks.workunit.client.0.smithi176.stdout:8/149: mkdir d0/d24/d29 0 2024-03-13T14:49:36.029 INFO:tasks.workunit.client.1.smithi196.stdout:2/230: mkdir d0/d6/d4b 0 2024-03-13T14:49:36.031 INFO:tasks.workunit.client.1.smithi196.stdout:8/266: dwrite d0/d10/d12/d1e/f5e [0,4194304] 0 2024-03-13T14:49:36.033 INFO:tasks.workunit.client.0.smithi176.stdout:5/128: chown d5/d22/d15/l1a 444 1 2024-03-13T14:49:36.033 INFO:tasks.workunit.client.0.smithi176.stdout:6/146: mkdir d0/de/d38 0 2024-03-13T14:49:36.041 INFO:tasks.workunit.client.0.smithi176.stdout:0/133: sync 2024-03-13T14:49:36.041 INFO:tasks.workunit.client.0.smithi176.stdout:7/164: dwrite d1/d13/f14 [0,4194304] 0 2024-03-13T14:49:36.046 INFO:tasks.workunit.client.0.smithi176.stdout:9/185: creat d0/d1e/d1f/f38 x:0 0 0 2024-03-13T14:49:36.049 INFO:tasks.workunit.client.0.smithi176.stdout:8/150: dwrite d0/f2 [4194304,4194304] 0 2024-03-13T14:49:36.053 INFO:tasks.workunit.client.0.smithi176.stdout:8/151: write d0/db/f12 [960230,59380] 0 2024-03-13T14:49:36.055 INFO:tasks.workunit.client.1.smithi196.stdout:6/433: getdents d6/d57 0 2024-03-13T14:49:36.057 INFO:tasks.workunit.client.0.smithi176.stdout:6/147: dwrite d0/de/f23 [0,4194304] 0 2024-03-13T14:49:36.058 INFO:tasks.workunit.client.0.smithi176.stdout:1/146: mkdir d2/de/d10/d2e 0 2024-03-13T14:49:36.064 INFO:tasks.workunit.client.1.smithi196.stdout:2/231: mknod d0/d7/c4c 0 2024-03-13T14:49:36.065 INFO:tasks.workunit.client.0.smithi176.stdout:9/186: dread d0/d1e/f25 [0,4194304] 0 2024-03-13T14:49:36.067 INFO:tasks.workunit.client.1.smithi196.stdout:3/307: dread db/f2c [0,4194304] 0 2024-03-13T14:49:36.072 INFO:tasks.workunit.client.1.smithi196.stdout:8/267: rmdir d0 39 2024-03-13T14:49:36.074 INFO:tasks.workunit.client.1.smithi196.stdout:0/386: dwrite d8/da/d25/d26/d35/d3b/d5f/d17/f70 [0,4194304] 0 2024-03-13T14:49:36.075 INFO:tasks.workunit.client.0.smithi176.stdout:5/129: write d5/f8 [3806064,61504] 0 2024-03-13T14:49:36.077 INFO:tasks.workunit.client.0.smithi176.stdout:4/156: link d7/d8/f28 d7/d8/f3b 0 2024-03-13T14:49:36.081 INFO:tasks.workunit.client.1.smithi196.stdout:1/309: symlink d1/d9/d33/d4b/d66/l6d 0 2024-03-13T14:49:36.083 INFO:tasks.workunit.client.1.smithi196.stdout:5/431: write d4/d1b/f29 [852992,13459] 0 2024-03-13T14:49:36.084 INFO:tasks.workunit.client.1.smithi196.stdout:6/434: rmdir d6/d9/d17 39 2024-03-13T14:49:36.084 INFO:tasks.workunit.client.0.smithi176.stdout:5/130: write d5/d22/f28 [538722,16305] 0 2024-03-13T14:49:36.084 INFO:tasks.workunit.client.0.smithi176.stdout:1/147: dwrite d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:49:36.085 INFO:tasks.workunit.client.1.smithi196.stdout:1/310: readlink d1/d9/d33/d4b/d66/l6d 0 2024-03-13T14:49:36.088 INFO:tasks.workunit.client.0.smithi176.stdout:3/150: dwrite d4/fa [0,4194304] 0 2024-03-13T14:49:36.089 INFO:tasks.workunit.client.1.smithi196.stdout:1/311: chown d1/d9/d33/d4b/d66 0 1 2024-03-13T14:49:36.093 INFO:tasks.workunit.client.1.smithi196.stdout:1/312: fsync d1/d9/d33/d4b/f5f 0 2024-03-13T14:49:36.094 INFO:tasks.workunit.client.0.smithi176.stdout:8/152: creat d0/db/dc/f2a x:0 0 0 2024-03-13T14:49:36.097 INFO:tasks.workunit.client.0.smithi176.stdout:6/148: creat d0/d10/d19/d27/d2b/f39 x:0 0 0 2024-03-13T14:49:36.098 INFO:tasks.workunit.client.0.smithi176.stdout:8/153: dread d0/db/fd [0,4194304] 0 2024-03-13T14:49:36.099 INFO:tasks.workunit.client.1.smithi196.stdout:3/308: rename db/d37/d4b to db/d37/d64 0 2024-03-13T14:49:36.101 INFO:tasks.workunit.client.1.smithi196.stdout:8/268: write d0/d1c/d3b/f60 [92764,7717] 0 2024-03-13T14:49:36.102 INFO:tasks.workunit.client.1.smithi196.stdout:3/309: fdatasync db/d19/d50/f63 0 2024-03-13T14:49:36.107 INFO:tasks.workunit.client.1.smithi196.stdout:7/404: write d0/d9/d31/f42 [963790,79087] 0 2024-03-13T14:49:36.109 INFO:tasks.workunit.client.0.smithi176.stdout:3/151: dwrite d4/d13/f2a [0,4194304] 0 2024-03-13T14:49:36.109 INFO:tasks.workunit.client.0.smithi176.stdout:2/161: sync 2024-03-13T14:49:36.110 INFO:tasks.workunit.client.1.smithi196.stdout:7/405: chown d0/d9/d1e/d58/d72/d6e/f70 15791170 1 2024-03-13T14:49:36.111 INFO:tasks.workunit.client.1.smithi196.stdout:8/269: chown d0/d10/f21 0 1 2024-03-13T14:49:36.114 INFO:tasks.workunit.client.0.smithi176.stdout:5/131: creat d5/d22/d15/f2d x:0 0 0 2024-03-13T14:49:36.116 INFO:tasks.workunit.client.1.smithi196.stdout:3/310: truncate db/d19/f55 1044473 0 2024-03-13T14:49:36.117 INFO:tasks.workunit.client.1.smithi196.stdout:7/406: truncate d0/d9/f5a 803583 0 2024-03-13T14:49:36.120 INFO:tasks.workunit.client.1.smithi196.stdout:3/311: dread - db/d19/f3f zero size 2024-03-13T14:49:36.123 INFO:tasks.workunit.client.1.smithi196.stdout:8/270: creat d0/f61 x:0 0 0 2024-03-13T14:49:36.123 INFO:tasks.workunit.client.0.smithi176.stdout:3/152: mknod d4/c2c 0 2024-03-13T14:49:36.123 INFO:tasks.workunit.client.0.smithi176.stdout:0/134: link d2/l7 d2/l22 0 2024-03-13T14:49:36.124 INFO:tasks.workunit.client.1.smithi196.stdout:5/432: dread d4/db/d14/d19/d26/d68/f72 [0,4194304] 0 2024-03-13T14:49:36.124 INFO:tasks.workunit.client.0.smithi176.stdout:3/153: chown d4/d13/l26 49 1 2024-03-13T14:49:36.129 INFO:tasks.workunit.client.1.smithi196.stdout:6/435: write d6/d9/d11/f70 [370132,125870] 0 2024-03-13T14:49:36.129 INFO:tasks.workunit.client.1.smithi196.stdout:0/387: rmdir d8/da/d71 0 2024-03-13T14:49:36.130 INFO:tasks.workunit.client.1.smithi196.stdout:3/312: symlink db/d19/d22/l65 0 2024-03-13T14:49:36.131 INFO:tasks.workunit.client.0.smithi176.stdout:3/154: mkdir d4/d13/d2d 0 2024-03-13T14:49:36.131 INFO:tasks.workunit.client.1.smithi196.stdout:2/232: getdents d0/d7/d40 0 2024-03-13T14:49:36.132 INFO:tasks.workunit.client.1.smithi196.stdout:4/281: sync 2024-03-13T14:49:36.135 INFO:tasks.workunit.client.0.smithi176.stdout:1/148: rename d2/c7 to d2/c2f 0 2024-03-13T14:49:36.135 INFO:tasks.workunit.client.0.smithi176.stdout:5/132: mkdir d5/d2e 0 2024-03-13T14:49:36.136 INFO:tasks.workunit.client.0.smithi176.stdout:1/149: read f1 [380293,66099] 0 2024-03-13T14:49:36.137 INFO:tasks.workunit.client.0.smithi176.stdout:5/133: truncate d5/f8 4391478 0 2024-03-13T14:49:36.139 INFO:tasks.workunit.client.0.smithi176.stdout:2/162: creat d1/f3c x:0 0 0 2024-03-13T14:49:36.141 INFO:tasks.workunit.client.1.smithi196.stdout:2/233: fsync d0/d7/d9/f48 0 2024-03-13T14:49:36.142 INFO:tasks.workunit.client.0.smithi176.stdout:6/149: write d0/de/f23 [4301566,86921] 0 2024-03-13T14:49:36.143 INFO:tasks.workunit.client.0.smithi176.stdout:4/157: link d7/d8/d15/l31 d7/l3c 0 2024-03-13T14:49:36.148 INFO:tasks.workunit.client.0.smithi176.stdout:8/154: getdents d0/d5 0 2024-03-13T14:49:36.151 INFO:tasks.workunit.client.1.smithi196.stdout:5/433: mkdir d4/db/d14/d19/d1c/d2d/d82/d8b 0 2024-03-13T14:49:36.152 INFO:tasks.workunit.client.0.smithi176.stdout:8/155: truncate d0/d5/d11/f26 1139406 0 2024-03-13T14:49:36.153 INFO:tasks.workunit.client.1.smithi196.stdout:0/388: symlink d8/da/d25/d26/d35/d3b/d6c/l83 0 2024-03-13T14:49:36.153 INFO:tasks.workunit.client.0.smithi176.stdout:2/163: symlink d1/d2a/d33/l3d 0 2024-03-13T14:49:36.157 INFO:tasks.workunit.client.0.smithi176.stdout:6/150: dwrite d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:36.163 INFO:tasks.workunit.client.0.smithi176.stdout:5/134: mknod d5/d22/c2f 0 2024-03-13T14:49:36.165 INFO:tasks.workunit.client.1.smithi196.stdout:7/407: dwrite d0/d9/d1e/f51 [0,4194304] 0 2024-03-13T14:49:36.167 INFO:tasks.workunit.client.1.smithi196.stdout:8/271: dwrite d0/d10/d12/f50 [0,4194304] 0 2024-03-13T14:49:36.174 INFO:tasks.workunit.client.0.smithi176.stdout:2/164: dread d1/dc/d1c/f1d [0,4194304] 0 2024-03-13T14:49:36.182 INFO:tasks.workunit.client.1.smithi196.stdout:2/234: symlink d0/d6/d1d/l4d 0 2024-03-13T14:49:36.182 INFO:tasks.workunit.client.1.smithi196.stdout:1/313: dwrite d1/d9/d33/f42 [0,4194304] 0 2024-03-13T14:49:36.182 INFO:tasks.workunit.client.1.smithi196.stdout:7/408: fsync d0/d9/d1e/d58/d72/f66 0 2024-03-13T14:49:36.183 INFO:tasks.workunit.client.0.smithi176.stdout:9/187: truncate d0/d17/f19 525633 0 2024-03-13T14:49:36.186 INFO:tasks.workunit.client.1.smithi196.stdout:3/313: dwrite db/d19/f1d [0,4194304] 0 2024-03-13T14:49:36.197 INFO:tasks.workunit.client.0.smithi176.stdout:6/151: creat d0/d10/d19/d27/d2b/f3a x:0 0 0 2024-03-13T14:49:36.198 INFO:tasks.workunit.client.0.smithi176.stdout:8/156: mknod d0/c2b 0 2024-03-13T14:49:36.200 INFO:tasks.workunit.client.1.smithi196.stdout:0/389: creat d8/da/d25/d26/d35/d3b/d5f/d43/f84 x:0 0 0 2024-03-13T14:49:36.201 INFO:tasks.workunit.client.1.smithi196.stdout:1/314: write d1/f37 [804798,3922] 0 2024-03-13T14:49:36.202 INFO:tasks.workunit.client.1.smithi196.stdout:5/434: unlink d4/db/d14/d37/d41/c7a 0 2024-03-13T14:49:36.203 INFO:tasks.workunit.client.1.smithi196.stdout:2/235: write d0/f1a [3708589,20972] 0 2024-03-13T14:49:36.212 INFO:tasks.workunit.client.0.smithi176.stdout:2/165: rename d1/dc/d1c/l27 to d1/l3e 0 2024-03-13T14:49:36.212 INFO:tasks.workunit.client.0.smithi176.stdout:5/135: dwrite d5/f14 [0,4194304] 0 2024-03-13T14:49:36.213 INFO:tasks.workunit.client.1.smithi196.stdout:6/436: link d6/d9/d17/d19/f42 d6/d9/d17/d19/d4e/f8a 0 2024-03-13T14:49:36.215 INFO:tasks.workunit.client.0.smithi176.stdout:5/136: write d5/d22/f13 [578545,37110] 0 2024-03-13T14:49:36.217 INFO:tasks.workunit.client.0.smithi176.stdout:5/137: dread - d5/d22/d15/f21 zero size 2024-03-13T14:49:36.218 INFO:tasks.workunit.client.0.smithi176.stdout:8/157: mknod d0/d24/d29/c2c 0 2024-03-13T14:49:36.223 INFO:tasks.workunit.client.0.smithi176.stdout:2/166: rename d1/fb to d1/dc/d2d/d38/f3f 0 2024-03-13T14:49:36.223 INFO:tasks.workunit.client.0.smithi176.stdout:8/158: stat d0/f2 0 2024-03-13T14:49:36.223 INFO:tasks.workunit.client.0.smithi176.stdout:6/152: dread d0/f8 [4194304,4194304] 0 2024-03-13T14:49:36.225 INFO:tasks.workunit.client.0.smithi176.stdout:5/138: rename d5/d22/f28 to d5/d2b/f30 0 2024-03-13T14:49:36.227 INFO:tasks.workunit.client.0.smithi176.stdout:6/153: mknod d0/de/c3b 0 2024-03-13T14:49:36.228 INFO:tasks.workunit.client.0.smithi176.stdout:2/167: mkdir d1/dc/d40 0 2024-03-13T14:49:36.229 INFO:tasks.workunit.client.0.smithi176.stdout:2/168: chown d1/l11 6 1 2024-03-13T14:49:36.233 INFO:tasks.workunit.client.0.smithi176.stdout:6/154: creat d0/de/f3c x:0 0 0 2024-03-13T14:49:36.234 INFO:tasks.workunit.client.0.smithi176.stdout:2/169: symlink d1/dc/d2d/d38/l41 0 2024-03-13T14:49:36.235 INFO:tasks.workunit.client.0.smithi176.stdout:2/170: readlink d1/dc/l28 0 2024-03-13T14:49:36.235 INFO:tasks.workunit.client.0.smithi176.stdout:5/139: mkdir d5/d31 0 2024-03-13T14:49:36.236 INFO:tasks.workunit.client.0.smithi176.stdout:4/158: write d7/d8/d15/f38 [240293,6653] 0 2024-03-13T14:49:36.239 INFO:tasks.workunit.client.0.smithi176.stdout:1/150: truncate d2/de/d10/d16/f27 3922044 0 2024-03-13T14:49:36.241 INFO:tasks.workunit.client.0.smithi176.stdout:1/151: readlink d2/dd/d19/l28 0 2024-03-13T14:49:36.241 INFO:tasks.workunit.client.0.smithi176.stdout:1/152: dread - d2/f21 zero size 2024-03-13T14:49:36.247 INFO:tasks.workunit.client.0.smithi176.stdout:6/155: dwrite d0/f13 [0,4194304] 0 2024-03-13T14:49:36.249 INFO:tasks.workunit.client.0.smithi176.stdout:2/171: symlink d1/dc/l42 0 2024-03-13T14:49:36.251 INFO:tasks.workunit.client.0.smithi176.stdout:4/159: mkdir d7/d8/d33/d3d 0 2024-03-13T14:49:36.256 INFO:tasks.workunit.client.0.smithi176.stdout:6/156: rename d0/la to d0/d10/d19/d27/l3d 0 2024-03-13T14:49:36.258 INFO:tasks.workunit.client.0.smithi176.stdout:9/188: dread d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:36.259 INFO:tasks.workunit.client.1.smithi196.stdout:2/236: symlink d0/d6/d1d/l4e 0 2024-03-13T14:49:36.259 INFO:tasks.workunit.client.1.smithi196.stdout:1/315: creat d1/d9/d4e/d55/f6e x:0 0 0 2024-03-13T14:49:36.259 INFO:tasks.workunit.client.1.smithi196.stdout:0/390: read d8/da/d25/f2f [1569604,69266] 0 2024-03-13T14:49:36.259 INFO:tasks.workunit.client.1.smithi196.stdout:8/272: link d0/d10/d12/d17/l37 d0/d10/l62 0 2024-03-13T14:49:36.260 INFO:tasks.workunit.client.0.smithi176.stdout:1/153: mkdir d2/d30 0 2024-03-13T14:49:36.262 INFO:tasks.workunit.client.1.smithi196.stdout:4/282: write d1/d1e/f34 [10731235,99073] 0 2024-03-13T14:49:36.263 INFO:tasks.workunit.client.1.smithi196.stdout:2/237: read - d0/f32 zero size 2024-03-13T14:49:36.263 INFO:tasks.workunit.client.0.smithi176.stdout:2/172: rename d1/l10 to d1/d2a/l43 0 2024-03-13T14:49:36.267 INFO:tasks.workunit.client.1.smithi196.stdout:2/238: write d0/d7/d40/f46 [1023576,19588] 0 2024-03-13T14:49:36.269 INFO:tasks.workunit.client.0.smithi176.stdout:4/160: dwrite d7/d10/d2c/f30 [0,4194304] 0 2024-03-13T14:49:36.270 INFO:tasks.workunit.client.1.smithi196.stdout:1/316: write d1/d9/f5c [4500738,95781] 0 2024-03-13T14:49:36.271 INFO:tasks.workunit.client.0.smithi176.stdout:5/140: fsync d5/d2b/f30 0 2024-03-13T14:49:36.271 INFO:tasks.workunit.client.0.smithi176.stdout:4/161: write d7/d8/d2f/f37 [500946,89196] 0 2024-03-13T14:49:36.273 INFO:tasks.workunit.client.0.smithi176.stdout:9/189: symlink d0/d17/l39 0 2024-03-13T14:49:36.274 INFO:tasks.workunit.client.0.smithi176.stdout:5/141: fdatasync d5/d22/f1e 0 2024-03-13T14:49:36.274 INFO:tasks.workunit.client.1.smithi196.stdout:1/317: chown d1/d9/d33/l4d 2 1 2024-03-13T14:49:36.278 INFO:tasks.workunit.client.1.smithi196.stdout:1/318: read - d1/d9/d4e/d55/f6e zero size 2024-03-13T14:49:36.279 INFO:tasks.workunit.client.0.smithi176.stdout:6/157: mkdir d0/d10/d19/d27/d2b/d2e/d3e 0 2024-03-13T14:49:36.279 INFO:tasks.workunit.client.1.smithi196.stdout:5/435: dwrite d4/db/d3c/f84 [0,4194304] 0 2024-03-13T14:49:36.285 INFO:tasks.workunit.client.1.smithi196.stdout:3/314: dread db/f32 [0,4194304] 0 2024-03-13T14:49:36.288 INFO:tasks.workunit.client.1.smithi196.stdout:6/437: rename d6/d7/dd/l12 to d6/d2a/d23/l8b 0 2024-03-13T14:49:36.288 INFO:tasks.workunit.client.0.smithi176.stdout:8/159: truncate d0/d5/d11/f26 462065 0 2024-03-13T14:49:36.291 INFO:tasks.workunit.client.1.smithi196.stdout:6/438: write f4 [2638279,43545] 0 2024-03-13T14:49:36.294 INFO:tasks.workunit.client.1.smithi196.stdout:6/439: chown d6/d9/d11/f70 110 1 2024-03-13T14:49:36.296 INFO:tasks.workunit.client.1.smithi196.stdout:6/440: fdatasync d6/d9/d17/d19/f20 0 2024-03-13T14:49:36.297 INFO:tasks.workunit.client.0.smithi176.stdout:1/154: rmdir d2 39 2024-03-13T14:49:36.299 INFO:tasks.workunit.client.1.smithi196.stdout:6/441: write f4 [1258391,78292] 0 2024-03-13T14:49:36.300 INFO:tasks.workunit.client.0.smithi176.stdout:9/190: unlink d0/d13/l26 0 2024-03-13T14:49:36.303 INFO:tasks.workunit.client.1.smithi196.stdout:5/436: dwrite d4/db/d14/d19/d26/d68/f77 [0,4194304] 0 2024-03-13T14:49:36.303 INFO:tasks.workunit.client.0.smithi176.stdout:7/165: sync 2024-03-13T14:49:36.303 INFO:tasks.workunit.client.0.smithi176.stdout:0/135: sync 2024-03-13T14:49:36.304 INFO:tasks.workunit.client.1.smithi196.stdout:0/391: mknod d8/c85 0 2024-03-13T14:49:36.307 INFO:tasks.workunit.client.1.smithi196.stdout:6/442: dread d6/d2a/f62 [0,4194304] 0 2024-03-13T14:49:36.308 INFO:tasks.workunit.client.0.smithi176.stdout:2/173: creat d1/dc/d40/f44 x:0 0 0 2024-03-13T14:49:36.308 INFO:tasks.workunit.client.1.smithi196.stdout:6/443: chown d6/d57/d82 14054573 1 2024-03-13T14:49:36.310 INFO:tasks.workunit.client.0.smithi176.stdout:2/174: truncate d1/f31 77073 0 2024-03-13T14:49:36.311 INFO:tasks.workunit.client.1.smithi196.stdout:1/319: creat d1/d9/d4e/d55/f6f x:0 0 0 2024-03-13T14:49:36.312 INFO:tasks.workunit.client.0.smithi176.stdout:2/175: stat d1/f15 0 2024-03-13T14:49:36.316 INFO:tasks.workunit.client.0.smithi176.stdout:9/191: fdatasync d0/d17/f19 0 2024-03-13T14:49:36.317 INFO:tasks.workunit.client.1.smithi196.stdout:3/315: rename la to db/d15/d44/l66 0 2024-03-13T14:49:36.321 INFO:tasks.workunit.client.0.smithi176.stdout:7/166: rename d1/d7/f1a to d1/d28/f2d 0 2024-03-13T14:49:36.323 INFO:tasks.workunit.client.0.smithi176.stdout:0/136: dwrite d2/f11 [0,4194304] 0 2024-03-13T14:49:36.323 INFO:tasks.workunit.client.0.smithi176.stdout:8/160: symlink d0/db/dc/l2d 0 2024-03-13T14:49:36.323 INFO:tasks.workunit.client.0.smithi176.stdout:7/167: dread - d1/d28/f2a zero size 2024-03-13T14:49:36.325 INFO:tasks.workunit.client.0.smithi176.stdout:0/137: stat d2/d5/dc 0 2024-03-13T14:49:36.328 INFO:tasks.workunit.client.1.smithi196.stdout:4/283: creat d1/d3/f4e x:0 0 0 2024-03-13T14:49:36.329 INFO:tasks.workunit.client.0.smithi176.stdout:4/162: link d7/d2d/f2e d7/d8/d15/f3e 0 2024-03-13T14:49:36.335 INFO:tasks.workunit.client.0.smithi176.stdout:7/168: dread d1/d13/f14 [0,4194304] 0 2024-03-13T14:49:36.336 INFO:tasks.workunit.client.1.smithi196.stdout:4/284: write d1/d3/d7/d35/f40 [386696,64604] 0 2024-03-13T14:49:36.336 INFO:tasks.workunit.client.0.smithi176.stdout:2/176: rmdir d1/dc/d2d 39 2024-03-13T14:49:36.340 INFO:tasks.workunit.client.1.smithi196.stdout:1/320: dwrite d1/f37 [0,4194304] 0 2024-03-13T14:49:36.350 INFO:tasks.workunit.client.0.smithi176.stdout:9/192: chown d0/d17/l1a 237158989 1 2024-03-13T14:49:36.351 INFO:tasks.workunit.client.0.smithi176.stdout:9/193: readlink d0/le 0 2024-03-13T14:49:36.352 INFO:tasks.workunit.client.0.smithi176.stdout:9/194: chown d0/db 855637301 1 2024-03-13T14:49:36.354 INFO:tasks.workunit.client.1.smithi196.stdout:6/444: truncate d6/d9/d17/d19/f20 5016348 0 2024-03-13T14:49:36.355 INFO:tasks.workunit.client.1.smithi196.stdout:9/288: sync 2024-03-13T14:49:36.363 INFO:tasks.workunit.client.0.smithi176.stdout:8/161: rename d0/d5/l1e to d0/d5/d11/l2e 0 2024-03-13T14:49:36.364 INFO:tasks.workunit.client.1.smithi196.stdout:1/321: dwrite d1/f37 [0,4194304] 0 2024-03-13T14:49:36.368 INFO:tasks.workunit.client.1.smithi196.stdout:1/322: readlink d1/d3f/d5b/l61 0 2024-03-13T14:49:36.379 INFO:tasks.workunit.client.1.smithi196.stdout:4/285: mknod d1/d1e/d39/c4f 0 2024-03-13T14:49:36.382 INFO:tasks.workunit.client.0.smithi176.stdout:0/138: unlink d2/l4 0 2024-03-13T14:49:36.382 INFO:tasks.workunit.client.1.smithi196.stdout:8/273: dwrite d0/d1c/d3b/d3f/f45 [0,4194304] 0 2024-03-13T14:49:36.385 INFO:tasks.workunit.client.1.smithi196.stdout:8/274: write d0/f56 [1176056,21896] 0 2024-03-13T14:49:36.390 INFO:tasks.workunit.client.1.smithi196.stdout:9/289: dwrite d7/de/d4b/f52 [0,4194304] 0 2024-03-13T14:49:36.391 INFO:tasks.workunit.client.0.smithi176.stdout:1/155: mknod d2/de/d10/d2e/c31 0 2024-03-13T14:49:36.392 INFO:tasks.workunit.client.0.smithi176.stdout:2/177: write d1/fa [1100453,42615] 0 2024-03-13T14:49:36.394 INFO:tasks.workunit.client.1.smithi196.stdout:9/290: write d7/d24/d28/f3e [1135315,63472] 0 2024-03-13T14:49:36.395 INFO:tasks.workunit.client.1.smithi196.stdout:1/323: creat d1/d9/d4e/f70 x:0 0 0 2024-03-13T14:49:36.397 INFO:tasks.workunit.client.0.smithi176.stdout:2/178: write d1/f3c [879553,103407] 0 2024-03-13T14:49:36.400 INFO:tasks.workunit.client.1.smithi196.stdout:1/324: chown d1/d9/d4e/c4f 1510828 1 2024-03-13T14:49:36.402 INFO:tasks.workunit.client.0.smithi176.stdout:8/162: symlink d0/db/dc/l2f 0 2024-03-13T14:49:36.403 INFO:tasks.workunit.client.0.smithi176.stdout:4/163: dread d7/fe [0,4194304] 0 2024-03-13T14:49:36.405 INFO:tasks.workunit.client.0.smithi176.stdout:4/164: chown d7/d10/f27 662 1 2024-03-13T14:49:36.408 INFO:tasks.workunit.client.0.smithi176.stdout:5/142: link d5/c11 d5/c32 0 2024-03-13T14:49:36.408 INFO:tasks.workunit.client.1.smithi196.stdout:8/275: mkdir d0/d63 0 2024-03-13T14:49:36.410 INFO:tasks.workunit.client.1.smithi196.stdout:4/286: mknod d1/c50 0 2024-03-13T14:49:36.411 INFO:tasks.workunit.client.0.smithi176.stdout:1/156: creat d2/dd/d19/f32 x:0 0 0 2024-03-13T14:49:36.413 INFO:tasks.workunit.client.1.smithi196.stdout:9/291: mknod d7/d24/c57 0 2024-03-13T14:49:36.414 INFO:tasks.workunit.client.1.smithi196.stdout:6/445: dwrite d6/d9/f7b [0,4194304] 0 2024-03-13T14:49:36.418 INFO:tasks.workunit.client.1.smithi196.stdout:1/325: mknod d1/d28/c71 0 2024-03-13T14:49:36.419 INFO:tasks.workunit.client.0.smithi176.stdout:2/179: write d1/dc/d2d/f2f [533957,78652] 0 2024-03-13T14:49:36.420 INFO:tasks.workunit.client.1.smithi196.stdout:3/316: link l6 db/d37/d64/l67 0 2024-03-13T14:49:36.423 INFO:tasks.workunit.client.0.smithi176.stdout:2/180: read d1/dc/d2d/d38/f3f [1155159,118887] 0 2024-03-13T14:49:36.424 INFO:tasks.workunit.client.0.smithi176.stdout:1/157: rmdir d2/de/d10/d2e 39 2024-03-13T14:49:36.426 INFO:tasks.workunit.client.0.smithi176.stdout:2/181: chown d1/dc/f2c 0 1 2024-03-13T14:49:36.426 INFO:tasks.workunit.client.1.smithi196.stdout:8/276: rmdir d0/d16 39 2024-03-13T14:49:36.426 INFO:tasks.workunit.client.0.smithi176.stdout:1/158: fsync d2/f5 0 2024-03-13T14:49:36.427 INFO:tasks.workunit.client.0.smithi176.stdout:9/195: write d0/d1e/d1f/f2c [663906,113209] 0 2024-03-13T14:49:36.428 INFO:tasks.workunit.client.0.smithi176.stdout:5/143: dwrite d5/d2b/f30 [0,4194304] 0 2024-03-13T14:49:36.431 INFO:tasks.workunit.client.0.smithi176.stdout:5/144: stat d5/d22/f13 0 2024-03-13T14:49:36.431 INFO:tasks.workunit.client.0.smithi176.stdout:1/159: stat d2/de/d10/d14/l2d 0 2024-03-13T14:49:36.443 INFO:tasks.workunit.client.1.smithi196.stdout:1/326: creat d1/d28/d34/d67/f72 x:0 0 0 2024-03-13T14:49:36.445 INFO:tasks.workunit.client.1.smithi196.stdout:1/327: dread - d1/d38/f6a zero size 2024-03-13T14:49:36.449 INFO:tasks.workunit.client.0.smithi176.stdout:8/163: creat d0/f30 x:0 0 0 2024-03-13T14:49:36.451 INFO:tasks.workunit.client.1.smithi196.stdout:4/287: dread d1/d3/d7/f1a [0,4194304] 0 2024-03-13T14:49:36.451 INFO:tasks.workunit.client.1.smithi196.stdout:9/292: creat d7/de/d4b/d1d/d22/d4d/f58 x:0 0 0 2024-03-13T14:49:36.452 INFO:tasks.workunit.client.0.smithi176.stdout:9/196: write d0/d1e/f2e [1732508,40096] 0 2024-03-13T14:49:36.454 INFO:tasks.workunit.client.0.smithi176.stdout:9/197: chown d0/d1e/c34 126756 1 2024-03-13T14:49:36.455 INFO:tasks.workunit.client.0.smithi176.stdout:3/155: sync 2024-03-13T14:49:36.455 INFO:tasks.workunit.client.0.smithi176.stdout:7/169: dwrite d1/d28/f2d [0,4194304] 0 2024-03-13T14:49:36.455 INFO:tasks.workunit.client.0.smithi176.stdout:2/182: creat d1/dc/d40/f45 x:0 0 0 2024-03-13T14:49:36.458 INFO:tasks.workunit.client.0.smithi176.stdout:3/156: chown d4/f5 0 1 2024-03-13T14:49:36.459 INFO:tasks.workunit.client.0.smithi176.stdout:1/160: rename d2/de/d10/l23 to d2/d30/l33 0 2024-03-13T14:49:36.461 INFO:tasks.workunit.client.0.smithi176.stdout:9/198: fdatasync d0/d1e/f2e 0 2024-03-13T14:49:36.462 INFO:tasks.workunit.client.0.smithi176.stdout:1/161: write d2/f6 [2314414,93249] 0 2024-03-13T14:49:36.467 INFO:tasks.workunit.client.0.smithi176.stdout:4/165: getdents d7/d8 0 2024-03-13T14:49:36.470 INFO:tasks.workunit.client.0.smithi176.stdout:2/183: unlink d1/f15 0 2024-03-13T14:49:36.471 INFO:tasks.workunit.client.1.smithi196.stdout:9/293: rename d7/de/d4b/d1d/f3b to d7/de/d4b/f59 0 2024-03-13T14:49:36.472 INFO:tasks.workunit.client.0.smithi176.stdout:8/164: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:36.473 INFO:tasks.workunit.client.1.smithi196.stdout:3/317: write db/d15/f17 [1923531,66768] 0 2024-03-13T14:49:36.473 INFO:tasks.workunit.client.1.smithi196.stdout:4/288: dread d1/d1e/f34 [0,4194304] 0 2024-03-13T14:49:36.476 INFO:tasks.workunit.client.0.smithi176.stdout:9/199: creat d0/d17/f3a x:0 0 0 2024-03-13T14:49:36.476 INFO:tasks.workunit.client.0.smithi176.stdout:1/162: mknod d2/de/d10/d14/d22/c34 0 2024-03-13T14:49:36.477 INFO:tasks.workunit.client.1.smithi196.stdout:1/328: dwrite d1/d38/f6a [0,4194304] 0 2024-03-13T14:49:36.477 INFO:tasks.workunit.client.0.smithi176.stdout:2/184: write d1/d2a/d33/f3a [1187130,3831] 0 2024-03-13T14:49:36.478 INFO:tasks.workunit.client.1.smithi196.stdout:8/277: getdents d0/d1c/d3b 0 2024-03-13T14:49:36.480 INFO:tasks.workunit.client.0.smithi176.stdout:4/166: creat d7/d10/d2c/f3f x:0 0 0 2024-03-13T14:49:36.480 INFO:tasks.workunit.client.0.smithi176.stdout:3/157: dwrite d4/fa [0,4194304] 0 2024-03-13T14:49:36.481 INFO:tasks.workunit.client.0.smithi176.stdout:2/185: read d1/dc/f16 [4183391,49782] 0 2024-03-13T14:49:36.484 INFO:tasks.workunit.client.1.smithi196.stdout:3/318: mkdir db/d19/d22/d3a/d68 0 2024-03-13T14:49:36.487 INFO:tasks.workunit.client.0.smithi176.stdout:4/167: write d7/d8/f11 [6872746,57061] 0 2024-03-13T14:49:36.487 INFO:tasks.workunit.client.0.smithi176.stdout:2/186: dread d1/f13 [0,4194304] 0 2024-03-13T14:49:36.489 INFO:tasks.workunit.client.0.smithi176.stdout:8/165: creat d0/d24/f31 x:0 0 0 2024-03-13T14:49:36.490 INFO:tasks.workunit.client.1.smithi196.stdout:6/446: truncate d6/d7/df/f3a 1147948 0 2024-03-13T14:49:36.492 INFO:tasks.workunit.client.0.smithi176.stdout:2/187: write d1/dc/f12 [2306639,71976] 0 2024-03-13T14:49:36.492 INFO:tasks.workunit.client.1.smithi196.stdout:8/278: creat d0/d1c/d3b/d3f/f64 x:0 0 0 2024-03-13T14:49:36.494 INFO:tasks.workunit.client.1.smithi196.stdout:3/319: creat db/d19/d22/d30/f69 x:0 0 0 2024-03-13T14:49:36.494 INFO:tasks.workunit.client.0.smithi176.stdout:6/158: sync 2024-03-13T14:49:36.495 INFO:tasks.workunit.client.1.smithi196.stdout:8/279: fdatasync d0/f51 0 2024-03-13T14:49:36.497 INFO:tasks.workunit.client.1.smithi196.stdout:6/447: unlink d6/d2a/d23/d28/f2f 0 2024-03-13T14:49:36.502 INFO:tasks.workunit.client.0.smithi176.stdout:5/145: getdents d5/d2b 0 2024-03-13T14:49:36.504 INFO:tasks.workunit.client.1.smithi196.stdout:1/329: dwrite d1/d28/d34/f41 [0,4194304] 0 2024-03-13T14:49:36.507 INFO:tasks.workunit.client.1.smithi196.stdout:1/330: chown d1/l2 33866313 1 2024-03-13T14:49:36.508 INFO:tasks.workunit.client.1.smithi196.stdout:8/280: rmdir d0/d10/d12 39 2024-03-13T14:49:36.508 INFO:tasks.workunit.client.1.smithi196.stdout:1/331: chown d1/d9/d33/f42 120786302 1 2024-03-13T14:49:36.511 INFO:tasks.workunit.client.1.smithi196.stdout:3/320: dread db/f2c [0,4194304] 0 2024-03-13T14:49:36.511 INFO:tasks.workunit.client.1.smithi196.stdout:6/448: mknod d6/d6f/c8c 0 2024-03-13T14:49:36.512 INFO:tasks.workunit.client.1.smithi196.stdout:3/321: stat db/ff 0 2024-03-13T14:49:36.513 INFO:tasks.workunit.client.1.smithi196.stdout:1/332: dread - d1/d9/d4e/f70 zero size 2024-03-13T14:49:36.514 INFO:tasks.workunit.client.0.smithi176.stdout:1/163: dwrite d2/dd/d19/f32 [0,4194304] 0 2024-03-13T14:49:36.514 INFO:tasks.workunit.client.0.smithi176.stdout:4/168: mknod d7/d8/d15/d23/c40 0 2024-03-13T14:49:36.515 INFO:tasks.workunit.client.1.smithi196.stdout:8/281: chown d0/d10/d12/d17 27239 1 2024-03-13T14:49:36.519 INFO:tasks.workunit.client.0.smithi176.stdout:3/158: dread d4/f1d [0,4194304] 0 2024-03-13T14:49:36.521 INFO:tasks.workunit.client.1.smithi196.stdout:3/322: mkdir db/d19/d22/d3a/d6a 0 2024-03-13T14:49:36.523 INFO:tasks.workunit.client.0.smithi176.stdout:3/159: chown f2 4 1 2024-03-13T14:49:36.527 INFO:tasks.workunit.client.0.smithi176.stdout:8/166: dwrite d0/db/fd [0,4194304] 0 2024-03-13T14:49:36.528 INFO:tasks.workunit.client.0.smithi176.stdout:3/160: chown d4/d13/d16/c24 9319269 1 2024-03-13T14:49:36.528 INFO:tasks.workunit.client.0.smithi176.stdout:6/159: dwrite d0/f13 [4194304,4194304] 0 2024-03-13T14:49:36.529 INFO:tasks.workunit.client.0.smithi176.stdout:2/188: creat d1/dc/d2d/d38/f46 x:0 0 0 2024-03-13T14:49:36.533 INFO:tasks.workunit.client.0.smithi176.stdout:6/160: chown d0/d9/l20 20 1 2024-03-13T14:49:36.536 INFO:tasks.workunit.client.0.smithi176.stdout:7/170: link d1/d1c/c21 d1/d13/d1b/d18/c2e 0 2024-03-13T14:49:36.538 INFO:tasks.workunit.client.1.smithi196.stdout:4/289: dwrite d1/d3/d7/d35/f37 [0,4194304] 0 2024-03-13T14:49:36.539 INFO:tasks.workunit.client.1.smithi196.stdout:1/333: dwrite d1/d28/f5a [0,4194304] 0 2024-03-13T14:49:36.539 INFO:tasks.workunit.client.0.smithi176.stdout:7/171: write d1/f19 [1576086,76573] 0 2024-03-13T14:49:36.541 INFO:tasks.workunit.client.0.smithi176.stdout:7/172: fdatasync d1/d2/f25 0 2024-03-13T14:49:36.541 INFO:tasks.workunit.client.1.smithi196.stdout:4/290: dread - d1/d1e/d39/f45 zero size 2024-03-13T14:49:36.544 INFO:tasks.workunit.client.0.smithi176.stdout:8/167: dread d0/db/fd [0,4194304] 0 2024-03-13T14:49:36.544 INFO:tasks.workunit.client.0.smithi176.stdout:7/173: write d1/d13/d1b/d18/f1e [2926928,59352] 0 2024-03-13T14:49:36.547 INFO:tasks.workunit.client.0.smithi176.stdout:8/168: write d0/db/f1a [101550,35710] 0 2024-03-13T14:49:36.547 INFO:tasks.workunit.client.0.smithi176.stdout:9/200: write d0/db/f1d [4024989,78719] 0 2024-03-13T14:49:36.551 INFO:tasks.workunit.client.1.smithi196.stdout:6/449: creat d6/d7/f8d x:0 0 0 2024-03-13T14:49:36.558 INFO:tasks.workunit.client.0.smithi176.stdout:1/164: mkdir d2/de/d10/d14/d35 0 2024-03-13T14:49:36.559 INFO:tasks.workunit.client.0.smithi176.stdout:0/139: sync 2024-03-13T14:49:36.563 INFO:tasks.workunit.client.0.smithi176.stdout:0/140: fsync d2/db/f1c 0 2024-03-13T14:49:36.565 INFO:tasks.workunit.client.0.smithi176.stdout:0/141: dread - d2/d5/dc/f19 zero size 2024-03-13T14:49:36.567 INFO:tasks.workunit.client.1.smithi196.stdout:1/334: dread d1/d9/d33/f62 [0,4194304] 0 2024-03-13T14:49:36.568 INFO:tasks.workunit.client.1.smithi196.stdout:3/323: truncate db/d19/d22/f31 903428 0 2024-03-13T14:49:36.569 INFO:tasks.workunit.client.0.smithi176.stdout:7/174: dwrite d1/d7/f10 [4194304,4194304] 0 2024-03-13T14:49:36.569 INFO:tasks.workunit.client.0.smithi176.stdout:3/161: write d4/f1d [189919,91333] 0 2024-03-13T14:49:36.570 INFO:tasks.workunit.client.0.smithi176.stdout:2/189: symlink d1/d2a/d33/l47 0 2024-03-13T14:49:36.571 INFO:tasks.workunit.client.0.smithi176.stdout:8/169: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:36.572 INFO:tasks.workunit.client.0.smithi176.stdout:4/169: dread d7/d8/f16 [0,4194304] 0 2024-03-13T14:49:36.575 INFO:tasks.workunit.client.0.smithi176.stdout:6/161: readlink d0/d10/l1d 0 2024-03-13T14:49:36.578 INFO:tasks.workunit.client.1.smithi196.stdout:8/282: dread d0/d10/f19 [0,4194304] 0 2024-03-13T14:49:36.579 INFO:tasks.workunit.client.1.smithi196.stdout:6/450: write d6/d7/dd/f7e [1532672,113353] 0 2024-03-13T14:49:36.582 INFO:tasks.workunit.client.0.smithi176.stdout:1/165: dread d2/fc [4194304,4194304] 0 2024-03-13T14:49:36.583 INFO:tasks.workunit.client.1.smithi196.stdout:1/335: write d1/d9/f64 [498924,126573] 0 2024-03-13T14:49:36.584 INFO:tasks.workunit.client.1.smithi196.stdout:8/283: truncate d0/d1c/d3b/d48/f5a 362136 0 2024-03-13T14:49:36.588 INFO:tasks.workunit.client.1.smithi196.stdout:3/324: mknod db/d19/d22/d3a/c6b 0 2024-03-13T14:49:36.590 INFO:tasks.workunit.client.1.smithi196.stdout:3/325: write db/f49 [938352,109579] 0 2024-03-13T14:49:36.594 INFO:tasks.workunit.client.1.smithi196.stdout:3/326: write db/f35 [15123,119820] 0 2024-03-13T14:49:36.598 INFO:tasks.workunit.client.1.smithi196.stdout:6/451: creat d6/d7/df/f8e x:0 0 0 2024-03-13T14:49:36.602 INFO:tasks.workunit.client.0.smithi176.stdout:3/162: mkdir d4/d28/d2e 0 2024-03-13T14:49:36.604 INFO:tasks.workunit.client.1.smithi196.stdout:4/291: dwrite d1/f6 [0,4194304] 0 2024-03-13T14:49:36.605 INFO:tasks.workunit.client.0.smithi176.stdout:2/190: creat d1/dc/d2d/d38/f48 x:0 0 0 2024-03-13T14:49:36.607 INFO:tasks.workunit.client.1.smithi196.stdout:4/292: fdatasync d1/d1e/d39/f38 0 2024-03-13T14:49:36.607 INFO:tasks.workunit.client.0.smithi176.stdout:2/191: write d1/dc/d40/f45 [1010730,110215] 0 2024-03-13T14:49:36.608 INFO:tasks.workunit.client.1.smithi196.stdout:8/284: rename d0/f27 to d0/d1c/d29/f65 0 2024-03-13T14:49:36.614 INFO:tasks.workunit.client.0.smithi176.stdout:4/170: symlink d7/d8/d33/l41 0 2024-03-13T14:49:36.619 INFO:tasks.workunit.client.1.smithi196.stdout:1/336: creat d1/d9/d4e/d55/d63/f73 x:0 0 0 2024-03-13T14:49:36.619 INFO:tasks.workunit.client.0.smithi176.stdout:4/171: write d7/d10/d2c/f3f [795793,102561] 0 2024-03-13T14:49:36.621 INFO:tasks.workunit.client.0.smithi176.stdout:9/201: rename d0/db/c9 to d0/d1e/d1f/c3b 0 2024-03-13T14:49:36.623 INFO:tasks.workunit.client.0.smithi176.stdout:3/163: symlink d4/d13/l2f 0 2024-03-13T14:49:36.624 INFO:tasks.workunit.client.0.smithi176.stdout:2/192: dwrite d1/dc/d1c/f36 [0,4194304] 0 2024-03-13T14:49:36.625 INFO:tasks.workunit.client.1.smithi196.stdout:3/327: mknod db/d19/c6c 0 2024-03-13T14:49:36.627 INFO:tasks.workunit.client.1.smithi196.stdout:3/328: fsync db/d19/f1d 0 2024-03-13T14:49:36.627 INFO:tasks.workunit.client.1.smithi196.stdout:4/293: mkdir d1/d3/d7/d35/d51 0 2024-03-13T14:49:36.631 INFO:tasks.workunit.client.1.smithi196.stdout:1/337: rename d1/d9/d4e/d58 to d1/d3f/d5b/d74 0 2024-03-13T14:49:36.632 INFO:tasks.workunit.client.0.smithi176.stdout:5/146: getdents d5/d22 0 2024-03-13T14:49:36.633 INFO:tasks.workunit.client.0.smithi176.stdout:1/166: mknod d2/de/d10/d2e/c36 0 2024-03-13T14:49:36.633 INFO:tasks.workunit.client.1.smithi196.stdout:1/338: write f0 [5127023,112522] 0 2024-03-13T14:49:36.636 INFO:tasks.workunit.client.1.smithi196.stdout:4/294: creat d1/d3/d7/d35/f52 x:0 0 0 2024-03-13T14:49:36.638 INFO:tasks.workunit.client.0.smithi176.stdout:9/202: unlink d0/d17/c36 0 2024-03-13T14:49:36.639 INFO:tasks.workunit.client.1.smithi196.stdout:3/329: mknod db/d19/d22/d30/c6d 0 2024-03-13T14:49:36.639 INFO:tasks.workunit.client.1.smithi196.stdout:4/295: write d1/d3/f3a [293131,48877] 0 2024-03-13T14:49:36.640 INFO:tasks.workunit.client.1.smithi196.stdout:6/452: dread d6/f8 [0,4194304] 0 2024-03-13T14:49:36.641 INFO:tasks.workunit.client.1.smithi196.stdout:3/330: dread - db/d19/f3f zero size 2024-03-13T14:49:36.643 INFO:tasks.workunit.client.1.smithi196.stdout:6/453: chown d6/d2a/f83 5404776 1 2024-03-13T14:49:36.648 INFO:tasks.workunit.client.0.smithi176.stdout:0/142: rename d2/l22 to d2/d5/l23 0 2024-03-13T14:49:36.650 INFO:tasks.workunit.client.1.smithi196.stdout:4/296: mkdir d1/d1e/d53 0 2024-03-13T14:49:36.653 INFO:tasks.workunit.client.0.smithi176.stdout:9/203: creat d0/d13/d32/f3c x:0 0 0 2024-03-13T14:49:36.656 INFO:tasks.workunit.client.0.smithi176.stdout:3/164: link d4/f25 d4/d13/d2d/f30 0 2024-03-13T14:49:36.656 INFO:tasks.workunit.client.1.smithi196.stdout:8/285: link d0/d1c/d29/l5c d0/d1c/d29/d47/l66 0 2024-03-13T14:49:36.663 INFO:tasks.workunit.client.0.smithi176.stdout:7/175: getdents d1/d13/d1b 0 2024-03-13T14:49:36.663 INFO:tasks.workunit.client.1.smithi196.stdout:6/454: unlink d6/d7/df/d27/d39/d44/c52 0 2024-03-13T14:49:36.667 INFO:tasks.workunit.client.0.smithi176.stdout:5/147: mknod d5/d2c/c33 0 2024-03-13T14:49:36.669 INFO:tasks.workunit.client.1.smithi196.stdout:3/331: dread db/f29 [0,4194304] 0 2024-03-13T14:49:36.670 INFO:tasks.workunit.client.1.smithi196.stdout:8/286: rmdir d0/d10/d12/d1e 39 2024-03-13T14:49:36.671 INFO:tasks.workunit.client.0.smithi176.stdout:9/204: truncate d0/d17/f19 1193995 0 2024-03-13T14:49:36.674 INFO:tasks.workunit.client.1.smithi196.stdout:1/339: rename d1/d3f/d5b/f54 to d1/d9/d33/d4b/f75 0 2024-03-13T14:49:36.675 INFO:tasks.workunit.client.0.smithi176.stdout:3/165: mkdir d4/d28/d31 0 2024-03-13T14:49:36.676 INFO:tasks.workunit.client.1.smithi196.stdout:3/332: write db/d19/d22/d3a/d53/f5c [184528,99980] 0 2024-03-13T14:49:36.678 INFO:tasks.workunit.client.0.smithi176.stdout:7/176: write d1/d13/f14 [4088761,100921] 0 2024-03-13T14:49:36.681 INFO:tasks.workunit.client.0.smithi176.stdout:7/177: write d1/d2/f25 [206387,48554] 0 2024-03-13T14:49:36.681 INFO:tasks.workunit.client.0.smithi176.stdout:8/170: rename d0/f14 to d0/d24/f32 0 2024-03-13T14:49:36.683 INFO:tasks.workunit.client.0.smithi176.stdout:2/193: truncate d1/d2a/d33/f3a 935143 0 2024-03-13T14:49:36.685 INFO:tasks.workunit.client.0.smithi176.stdout:3/166: creat d4/d1a/f32 x:0 0 0 2024-03-13T14:49:36.685 INFO:tasks.workunit.client.0.smithi176.stdout:7/178: truncate d1/d7/f29 469557 0 2024-03-13T14:49:36.685 INFO:tasks.workunit.client.0.smithi176.stdout:2/194: dread - d1/dc/d1c/f37 zero size 2024-03-13T14:49:36.686 INFO:tasks.workunit.client.0.smithi176.stdout:5/148: creat d5/d2e/f34 x:0 0 0 2024-03-13T14:49:36.687 INFO:tasks.workunit.client.0.smithi176.stdout:5/149: stat d5/d22/f29 0 2024-03-13T14:49:36.688 INFO:tasks.workunit.client.0.smithi176.stdout:4/172: rename d7/fc to d7/d8/f42 0 2024-03-13T14:49:36.689 INFO:tasks.workunit.client.0.smithi176.stdout:5/150: write d5/d2b/f30 [3587506,41412] 0 2024-03-13T14:49:36.691 INFO:tasks.workunit.client.0.smithi176.stdout:3/167: fsync f1 0 2024-03-13T14:49:36.693 INFO:tasks.workunit.client.0.smithi176.stdout:2/195: mkdir d1/dc/d2d/d38/d49 0 2024-03-13T14:49:36.694 INFO:tasks.workunit.client.1.smithi196.stdout:4/297: dwrite d1/d3/d7/f2c [0,4194304] 0 2024-03-13T14:49:36.695 INFO:tasks.workunit.client.0.smithi176.stdout:2/196: write d1/dc/d1c/f36 [326882,44585] 0 2024-03-13T14:49:36.696 INFO:tasks.workunit.client.0.smithi176.stdout:2/197: chown d1 473006286 1 2024-03-13T14:49:36.703 INFO:tasks.workunit.client.0.smithi176.stdout:5/151: symlink d5/d2b/l35 0 2024-03-13T14:49:36.703 INFO:tasks.workunit.client.0.smithi176.stdout:8/171: dread d0/f2 [4194304,4194304] 0 2024-03-13T14:49:36.706 INFO:tasks.workunit.client.1.smithi196.stdout:1/340: creat d1/d38/f76 x:0 0 0 2024-03-13T14:49:36.707 INFO:tasks.workunit.client.0.smithi176.stdout:7/179: mknod d1/d1c/d26/c2f 0 2024-03-13T14:49:36.708 INFO:tasks.workunit.client.1.smithi196.stdout:6/455: link d6/d57/f63 d6/d2a/d23/f8f 0 2024-03-13T14:49:36.710 INFO:tasks.workunit.client.0.smithi176.stdout:3/168: mknod d4/c33 0 2024-03-13T14:49:36.710 INFO:tasks.workunit.client.1.smithi196.stdout:6/456: chown d6/d7/df/d27/d39/d44/d74 119222520 1 2024-03-13T14:49:36.712 INFO:tasks.workunit.client.1.smithi196.stdout:0/392: sync 2024-03-13T14:49:36.712 INFO:tasks.workunit.client.1.smithi196.stdout:7/409: sync 2024-03-13T14:49:36.712 INFO:tasks.workunit.client.1.smithi196.stdout:5/437: sync 2024-03-13T14:49:36.713 INFO:tasks.workunit.client.1.smithi196.stdout:2/239: sync 2024-03-13T14:49:36.713 INFO:tasks.workunit.client.1.smithi196.stdout:9/294: sync 2024-03-13T14:49:36.713 INFO:tasks.workunit.client.1.smithi196.stdout:6/457: chown d6/d9/d11/f5e 107474 1 2024-03-13T14:49:36.716 INFO:tasks.workunit.client.1.smithi196.stdout:4/298: unlink d1/d3/f36 0 2024-03-13T14:49:36.717 INFO:tasks.workunit.client.1.smithi196.stdout:2/240: write d0/d6/d1d/f30 [1012356,67977] 0 2024-03-13T14:49:36.717 INFO:tasks.workunit.client.1.smithi196.stdout:9/295: read d7/f45 [1866947,807] 0 2024-03-13T14:49:36.718 INFO:tasks.workunit.client.1.smithi196.stdout:8/287: mknod d0/d10/d12/d1e/c67 0 2024-03-13T14:49:36.719 INFO:tasks.workunit.client.1.smithi196.stdout:0/393: chown d8/da/d25/d26/d35/d3b/d50/d55 455 1 2024-03-13T14:49:36.720 INFO:tasks.workunit.client.0.smithi176.stdout:0/143: rename d2/d5/l23 to d2/d5/d1e/l24 0 2024-03-13T14:49:36.720 INFO:tasks.workunit.client.1.smithi196.stdout:6/458: dread - d6/d7/f8d zero size 2024-03-13T14:49:36.723 INFO:tasks.workunit.client.0.smithi176.stdout:7/180: dwrite d1/f1f [0,4194304] 0 2024-03-13T14:49:36.725 INFO:tasks.workunit.client.0.smithi176.stdout:7/181: chown d1/d7/lb 399891 1 2024-03-13T14:49:36.726 INFO:tasks.workunit.client.0.smithi176.stdout:7/182: chown d1/d1c 547109198 1 2024-03-13T14:49:36.728 INFO:tasks.workunit.client.0.smithi176.stdout:6/162: sync 2024-03-13T14:49:36.737 INFO:tasks.workunit.client.1.smithi196.stdout:4/299: dread d1/d3/fa [0,4194304] 0 2024-03-13T14:49:36.739 INFO:tasks.workunit.client.1.smithi196.stdout:4/300: write d1/d3/d7/f49 [526023,102820] 0 2024-03-13T14:49:36.741 INFO:tasks.workunit.client.0.smithi176.stdout:6/163: dread d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:36.742 INFO:tasks.workunit.client.0.smithi176.stdout:7/183: dwrite d1/d28/f2a [0,4194304] 0 2024-03-13T14:49:36.755 INFO:tasks.workunit.client.0.smithi176.stdout:9/205: sync 2024-03-13T14:49:36.755 INFO:tasks.workunit.client.0.smithi176.stdout:1/167: sync 2024-03-13T14:49:36.760 INFO:tasks.workunit.client.0.smithi176.stdout:7/184: dwrite d1/f1f [0,4194304] 0 2024-03-13T14:49:36.762 INFO:tasks.workunit.client.0.smithi176.stdout:7/185: fsync d1/d13/f14 0 2024-03-13T14:49:36.762 INFO:tasks.workunit.client.0.smithi176.stdout:9/206: dread d0/d1e/d1f/f2c [0,4194304] 0 2024-03-13T14:49:36.777 INFO:tasks.workunit.client.1.smithi196.stdout:7/410: rmdir d0/d9/d1e/d58/d72 39 2024-03-13T14:49:36.778 INFO:tasks.workunit.client.0.smithi176.stdout:5/152: chown d5/c11 3 1 2024-03-13T14:49:36.783 INFO:tasks.workunit.client.1.smithi196.stdout:5/438: creat d4/db/d14/d19/d26/d33/f8c x:0 0 0 2024-03-13T14:49:36.785 INFO:tasks.workunit.client.0.smithi176.stdout:3/169: mkdir d4/d28/d34 0 2024-03-13T14:49:36.797 INFO:tasks.workunit.client.0.smithi176.stdout:0/144: write d2/db/f21 [861410,28481] 0 2024-03-13T14:49:36.802 INFO:tasks.workunit.client.1.smithi196.stdout:9/296: creat d7/de/d4b/d1d/d22/f5a x:0 0 0 2024-03-13T14:49:36.813 INFO:tasks.workunit.client.1.smithi196.stdout:7/411: mkdir d0/d78/d81 0 2024-03-13T14:49:36.821 INFO:tasks.workunit.client.0.smithi176.stdout:9/207: symlink d0/d13/l3d 0 2024-03-13T14:49:36.821 INFO:tasks.workunit.client.1.smithi196.stdout:9/297: dwrite d7/de/d4b/f52 [0,4194304] 0 2024-03-13T14:49:36.826 INFO:tasks.workunit.client.1.smithi196.stdout:3/333: getdents db/d19 0 2024-03-13T14:49:36.829 INFO:tasks.workunit.client.0.smithi176.stdout:7/186: symlink d1/d2/l30 0 2024-03-13T14:49:36.831 INFO:tasks.workunit.client.1.smithi196.stdout:8/288: mkdir d0/d10/d12/d17/d40/d68 0 2024-03-13T14:49:36.842 INFO:tasks.workunit.client.0.smithi176.stdout:4/173: rmdir d7/d8/d33/d3d 0 2024-03-13T14:49:36.845 INFO:tasks.workunit.client.0.smithi176.stdout:4/174: read d7/d8/d15/f3e [245057,10883] 0 2024-03-13T14:49:36.845 INFO:tasks.workunit.client.1.smithi196.stdout:6/459: rename d6/d9/d17/d19/d4e/d66/c72 to d6/d2a/dc/d47/c90 0 2024-03-13T14:49:36.847 INFO:tasks.workunit.client.0.smithi176.stdout:4/175: dread - d7/f3a zero size 2024-03-13T14:49:36.849 INFO:tasks.workunit.client.1.smithi196.stdout:0/394: creat d8/da/d25/d26/d35/d3b/d5f/d33/d44/f86 x:0 0 0 2024-03-13T14:49:36.852 INFO:tasks.workunit.client.0.smithi176.stdout:0/145: creat d2/db/f25 x:0 0 0 2024-03-13T14:49:36.853 INFO:tasks.workunit.client.1.smithi196.stdout:7/412: creat d0/d9/d1e/d3f/f82 x:0 0 0 2024-03-13T14:49:36.856 INFO:tasks.workunit.client.1.smithi196.stdout:5/439: mknod d4/db/d14/d37/d41/d7d/c8d 0 2024-03-13T14:49:36.859 INFO:tasks.workunit.client.1.smithi196.stdout:5/440: chown d4/db/d14/d19/d1c/d2d/c6e 81109639 1 2024-03-13T14:49:36.868 INFO:tasks.workunit.client.1.smithi196.stdout:6/460: mkdir d6/d57/d91 0 2024-03-13T14:49:36.870 INFO:tasks.workunit.client.1.smithi196.stdout:6/461: fsync d6/d9/f7b 0 2024-03-13T14:49:36.872 INFO:tasks.workunit.client.1.smithi196.stdout:3/334: dread db/d19/d22/f31 [0,4194304] 0 2024-03-13T14:49:36.873 INFO:tasks.workunit.client.1.smithi196.stdout:4/301: creat d1/f54 x:0 0 0 2024-03-13T14:49:36.877 INFO:tasks.workunit.client.0.smithi176.stdout:5/153: mknod d5/c36 0 2024-03-13T14:49:36.881 INFO:tasks.workunit.client.0.smithi176.stdout:4/176: mknod d7/d10/d2c/c43 0 2024-03-13T14:49:36.884 INFO:tasks.workunit.client.0.smithi176.stdout:3/170: mknod d4/d28/d34/c35 0 2024-03-13T14:49:36.887 INFO:tasks.workunit.client.0.smithi176.stdout:3/171: stat d4/d28 0 2024-03-13T14:49:36.889 INFO:tasks.workunit.client.1.smithi196.stdout:8/289: dwrite d0/d1c/f30 [0,4194304] 0 2024-03-13T14:49:36.889 INFO:tasks.workunit.client.0.smithi176.stdout:1/168: creat d2/f37 x:0 0 0 2024-03-13T14:49:36.893 INFO:tasks.workunit.client.0.smithi176.stdout:1/169: fsync d2/de/d10/d14/d22/f2b 0 2024-03-13T14:49:36.895 INFO:tasks.workunit.client.1.smithi196.stdout:6/462: symlink d6/d2a/dc/d71/l92 0 2024-03-13T14:49:36.896 INFO:tasks.workunit.client.1.smithi196.stdout:1/341: dwrite d1/d28/d34/f3e [0,4194304] 0 2024-03-13T14:49:36.899 INFO:tasks.workunit.client.0.smithi176.stdout:4/177: creat d7/d10/f44 x:0 0 0 2024-03-13T14:49:36.900 INFO:tasks.workunit.client.1.smithi196.stdout:5/441: dread d4/f8 [0,4194304] 0 2024-03-13T14:49:36.900 INFO:tasks.workunit.client.1.smithi196.stdout:6/463: fsync d6/d9/d17/d19/d4e/f7c 0 2024-03-13T14:49:36.902 INFO:tasks.workunit.client.1.smithi196.stdout:3/335: creat db/d15/d44/d51/f6e x:0 0 0 2024-03-13T14:49:36.902 INFO:tasks.workunit.client.0.smithi176.stdout:4/178: stat d7/d8/l13 0 2024-03-13T14:49:36.905 INFO:tasks.workunit.client.1.smithi196.stdout:5/442: chown d4/db/d14/d19/d26/d68 6602348 1 2024-03-13T14:49:36.910 INFO:tasks.workunit.client.0.smithi176.stdout:3/172: creat d4/f36 x:0 0 0 2024-03-13T14:49:36.911 INFO:tasks.workunit.client.1.smithi196.stdout:6/464: truncate d6/d7/dd/f86 603248 0 2024-03-13T14:49:36.911 INFO:tasks.workunit.client.1.smithi196.stdout:5/443: dread - d4/db/d14/d19/d1c/d25/f49 zero size 2024-03-13T14:49:36.912 INFO:tasks.workunit.client.0.smithi176.stdout:1/170: symlink d2/de/d10/d16/l38 0 2024-03-13T14:49:36.914 INFO:tasks.workunit.client.1.smithi196.stdout:5/444: truncate d4/db/d14/d19/f58 344028 0 2024-03-13T14:49:36.920 INFO:tasks.workunit.client.0.smithi176.stdout:4/179: mkdir d7/d8/d45 0 2024-03-13T14:49:36.922 INFO:tasks.workunit.client.1.smithi196.stdout:7/413: link d0/d6/l3e d0/d9/d1e/d58/d72/l83 0 2024-03-13T14:49:36.927 INFO:tasks.workunit.client.0.smithi176.stdout:3/173: mkdir d4/d37 0 2024-03-13T14:49:36.928 INFO:tasks.workunit.client.0.smithi176.stdout:3/174: write d4/d13/f19 [2695208,35477] 0 2024-03-13T14:49:36.930 INFO:tasks.workunit.client.0.smithi176.stdout:1/171: creat d2/d29/f39 x:0 0 0 2024-03-13T14:49:36.930 INFO:tasks.workunit.client.1.smithi196.stdout:2/241: mknod d0/d7/c4f 0 2024-03-13T14:49:36.931 INFO:tasks.workunit.client.0.smithi176.stdout:3/175: write d4/d1a/f32 [178553,4633] 0 2024-03-13T14:49:36.933 INFO:tasks.workunit.client.0.smithi176.stdout:4/180: mkdir d7/d10/d46 0 2024-03-13T14:49:36.935 INFO:tasks.workunit.client.0.smithi176.stdout:4/181: chown d7/f1c 146 1 2024-03-13T14:49:36.935 INFO:tasks.workunit.client.1.smithi196.stdout:3/336: dwrite db/d15/f41 [0,4194304] 0 2024-03-13T14:49:36.935 INFO:tasks.workunit.client.1.smithi196.stdout:0/395: dread d8/da/d25/d26/d35/d3b/f47 [0,4194304] 0 2024-03-13T14:49:36.937 INFO:tasks.workunit.client.0.smithi176.stdout:4/182: write d7/d8/d15/d23/f35 [584771,88439] 0 2024-03-13T14:49:36.939 INFO:tasks.workunit.client.1.smithi196.stdout:3/337: read - db/d15/d44/d51/f6e zero size 2024-03-13T14:49:36.940 INFO:tasks.workunit.client.1.smithi196.stdout:5/445: stat d4/db/d14/c22 0 2024-03-13T14:49:36.943 INFO:tasks.workunit.client.1.smithi196.stdout:5/446: chown d4/db/fd 3737 1 2024-03-13T14:49:36.943 INFO:tasks.workunit.client.1.smithi196.stdout:0/396: chown d8/da/d25/d26/d35/d3b/d5f/d17/c46 7 1 2024-03-13T14:49:36.947 INFO:tasks.workunit.client.1.smithi196.stdout:5/447: read d4/db/d14/d19/d26/f3a [3612487,14413] 0 2024-03-13T14:49:36.947 INFO:tasks.workunit.client.0.smithi176.stdout:1/172: creat d2/d30/f3a x:0 0 0 2024-03-13T14:49:36.948 INFO:tasks.workunit.client.0.smithi176.stdout:2/198: truncate d1/dc/d1c/f36 1224157 0 2024-03-13T14:49:36.949 INFO:tasks.workunit.client.1.smithi196.stdout:8/290: fdatasync d0/d1c/f30 0 2024-03-13T14:49:36.950 INFO:tasks.workunit.client.0.smithi176.stdout:1/173: fsync d2/de/f26 0 2024-03-13T14:49:36.950 INFO:tasks.workunit.client.1.smithi196.stdout:5/448: dread - d4/db/d14/d37/f78 zero size 2024-03-13T14:49:36.951 INFO:tasks.workunit.client.0.smithi176.stdout:3/176: mkdir d4/d28/d34/d38 0 2024-03-13T14:49:36.952 INFO:tasks.workunit.client.0.smithi176.stdout:3/177: chown d4 15 1 2024-03-13T14:49:36.952 INFO:tasks.workunit.client.0.smithi176.stdout:1/174: chown d2/f6 224639012 1 2024-03-13T14:49:36.954 INFO:tasks.workunit.client.0.smithi176.stdout:3/178: fsync d4/d13/f2a 0 2024-03-13T14:49:36.954 INFO:tasks.workunit.client.0.smithi176.stdout:1/175: fsync d2/f9 0 2024-03-13T14:49:36.955 INFO:tasks.workunit.client.0.smithi176.stdout:9/208: sync 2024-03-13T14:49:36.957 INFO:tasks.workunit.client.0.smithi176.stdout:3/179: write d4/d1a/f32 [1077569,129407] 0 2024-03-13T14:49:36.959 INFO:tasks.workunit.client.0.smithi176.stdout:3/180: fdatasync d4/fa 0 2024-03-13T14:49:36.961 INFO:tasks.workunit.client.1.smithi196.stdout:4/302: rmdir d1/d3/d7/d35 39 2024-03-13T14:49:36.962 INFO:tasks.workunit.client.0.smithi176.stdout:4/183: dwrite d7/d10/f2a [0,4194304] 0 2024-03-13T14:49:36.962 INFO:tasks.workunit.client.1.smithi196.stdout:4/303: stat d1/f22 0 2024-03-13T14:49:36.963 INFO:tasks.workunit.client.0.smithi176.stdout:9/209: fsync d0/d1e/d1f/f2c 0 2024-03-13T14:49:36.965 INFO:tasks.workunit.client.0.smithi176.stdout:4/184: write d7/d10/d2c/f3f [976341,4422] 0 2024-03-13T14:49:36.972 INFO:tasks.workunit.client.1.smithi196.stdout:3/338: mknod db/d19/d50/c6f 0 2024-03-13T14:49:36.972 INFO:tasks.workunit.client.0.smithi176.stdout:1/176: dwrite f1 [0,4194304] 0 2024-03-13T14:49:36.975 INFO:tasks.workunit.client.1.smithi196.stdout:3/339: write db/d19/f2b [61903,84215] 0 2024-03-13T14:49:36.984 INFO:tasks.workunit.client.1.smithi196.stdout:1/342: dread f0 [0,4194304] 0 2024-03-13T14:49:36.984 INFO:tasks.workunit.client.1.smithi196.stdout:4/304: dread - d1/d3/d7/d35/f52 zero size 2024-03-13T14:49:36.986 INFO:tasks.workunit.client.1.smithi196.stdout:4/305: stat d1/cf 0 2024-03-13T14:49:36.988 INFO:tasks.workunit.client.1.smithi196.stdout:0/397: creat d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 x:0 0 0 2024-03-13T14:49:36.988 INFO:tasks.workunit.client.0.smithi176.stdout:3/181: dwrite f2 [4194304,4194304] 0 2024-03-13T14:49:36.990 INFO:tasks.workunit.client.0.smithi176.stdout:1/177: dwrite d2/f37 [0,4194304] 0 2024-03-13T14:49:36.992 INFO:tasks.workunit.client.1.smithi196.stdout:8/291: symlink d0/d10/d12/d17/l69 0 2024-03-13T14:49:36.995 INFO:tasks.workunit.client.0.smithi176.stdout:4/185: dread d7/d8/d15/f2b [0,4194304] 0 2024-03-13T14:49:36.996 INFO:tasks.workunit.client.1.smithi196.stdout:0/398: write d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 [679338,25607] 0 2024-03-13T14:49:36.998 INFO:tasks.workunit.client.0.smithi176.stdout:8/172: dwrite d0/d5/d11/f26 [0,4194304] 0 2024-03-13T14:49:36.999 INFO:tasks.workunit.client.1.smithi196.stdout:3/340: readlink db/d37/d64/l67 0 2024-03-13T14:49:37.000 INFO:tasks.workunit.client.0.smithi176.stdout:4/186: read d7/d8/d2f/f37 [97758,91819] 0 2024-03-13T14:49:37.002 INFO:tasks.workunit.client.1.smithi196.stdout:3/341: write db/d19/f2b [4274481,115487] 0 2024-03-13T14:49:37.004 INFO:tasks.workunit.client.0.smithi176.stdout:6/164: truncate d0/d9/f14 2433 0 2024-03-13T14:49:37.016 INFO:tasks.workunit.client.1.smithi196.stdout:8/292: write d0/d10/d12/d17/f28 [1842691,20892] 0 2024-03-13T14:49:37.017 INFO:tasks.workunit.client.0.smithi176.stdout:1/178: dwrite d2/fc [0,4194304] 0 2024-03-13T14:49:37.018 INFO:tasks.workunit.client.0.smithi176.stdout:1/179: chown d2/f37 4223339 1 2024-03-13T14:49:37.019 INFO:tasks.workunit.client.1.smithi196.stdout:5/449: creat d4/db/d14/d19/d1c/d25/f8e x:0 0 0 2024-03-13T14:49:37.021 INFO:tasks.workunit.client.0.smithi176.stdout:8/173: creat d0/d5/d11/f33 x:0 0 0 2024-03-13T14:49:37.028 INFO:tasks.workunit.client.1.smithi196.stdout:1/343: link d1/f5 d1/d9/d6c/f77 0 2024-03-13T14:49:37.029 INFO:tasks.workunit.client.1.smithi196.stdout:5/450: truncate d4/db/d14/d19/d26/d33/d61/f6f 827619 0 2024-03-13T14:49:37.030 INFO:tasks.workunit.client.0.smithi176.stdout:6/165: creat d0/d10/f3f x:0 0 0 2024-03-13T14:49:37.033 INFO:tasks.workunit.client.0.smithi176.stdout:6/166: dread - d0/d10/d19/d27/d2b/f39 zero size 2024-03-13T14:49:37.035 INFO:tasks.workunit.client.1.smithi196.stdout:4/306: mkdir d1/d3/d7/d35/d51/d55 0 2024-03-13T14:49:37.035 INFO:tasks.workunit.client.0.smithi176.stdout:3/182: getdents d4/d28/d31 0 2024-03-13T14:49:37.036 INFO:tasks.workunit.client.0.smithi176.stdout:5/154: sync 2024-03-13T14:49:37.042 INFO:tasks.workunit.client.1.smithi196.stdout:9/298: rename d7/de/d16/d3c/c46 to d7/de/d4b/c5b 0 2024-03-13T14:49:37.043 INFO:tasks.workunit.client.0.smithi176.stdout:7/187: truncate d1/f19 303351 0 2024-03-13T14:49:37.045 INFO:tasks.workunit.client.1.smithi196.stdout:8/293: dread d0/d10/d12/d1e/f5e [0,4194304] 0 2024-03-13T14:49:37.046 INFO:tasks.workunit.client.1.smithi196.stdout:4/307: dread d1/d3/d7/f49 [0,4194304] 0 2024-03-13T14:49:37.049 INFO:tasks.workunit.client.1.smithi196.stdout:8/294: chown d0/d63 743 1 2024-03-13T14:49:37.051 INFO:tasks.workunit.client.0.smithi176.stdout:8/174: dread d0/f3 [4194304,4194304] 0 2024-03-13T14:49:37.055 INFO:tasks.workunit.client.0.smithi176.stdout:8/175: chown d0/db/dc/f2a 164 1 2024-03-13T14:49:37.056 INFO:tasks.workunit.client.0.smithi176.stdout:4/187: rmdir d7/d2d 39 2024-03-13T14:49:37.058 INFO:tasks.workunit.client.1.smithi196.stdout:3/342: dread db/d19/f2f [0,4194304] 0 2024-03-13T14:49:37.058 INFO:tasks.workunit.client.0.smithi176.stdout:3/183: dwrite d4/d13/f19 [0,4194304] 0 2024-03-13T14:49:37.060 INFO:tasks.workunit.client.0.smithi176.stdout:4/188: chown c4 182313 1 2024-03-13T14:49:37.064 INFO:tasks.workunit.client.0.smithi176.stdout:1/180: mknod d2/de/d10/d14/d35/c3b 0 2024-03-13T14:49:37.066 INFO:tasks.workunit.client.1.smithi196.stdout:9/299: creat d7/de/d4b/d1d/d22/d4d/f5c x:0 0 0 2024-03-13T14:49:37.067 INFO:tasks.workunit.client.1.smithi196.stdout:7/414: getdents d0/d9/d1e/d58/d72 0 2024-03-13T14:49:37.069 INFO:tasks.workunit.client.1.smithi196.stdout:8/295: creat d0/d10/d12/d1e/f6a x:0 0 0 2024-03-13T14:49:37.071 INFO:tasks.workunit.client.1.smithi196.stdout:9/300: read d7/de/d4b/f52 [527342,33482] 0 2024-03-13T14:49:37.075 INFO:tasks.workunit.client.1.smithi196.stdout:1/344: rename d1/d9/c53 to d1/d9/d33/c78 0 2024-03-13T14:49:37.077 INFO:tasks.workunit.client.1.smithi196.stdout:8/296: write d0/d1c/d3b/d3f/f45 [1748621,106921] 0 2024-03-13T14:49:37.078 INFO:tasks.workunit.client.0.smithi176.stdout:3/184: creat d4/f39 x:0 0 0 2024-03-13T14:49:37.083 INFO:tasks.workunit.client.1.smithi196.stdout:3/343: dwrite db/d19/d22/f3c [0,4194304] 0 2024-03-13T14:49:37.083 INFO:tasks.workunit.client.0.smithi176.stdout:4/189: dwrite d7/d10/f2a [0,4194304] 0 2024-03-13T14:49:37.084 INFO:tasks.workunit.client.0.smithi176.stdout:1/181: dread d2/f6 [0,4194304] 0 2024-03-13T14:49:37.087 INFO:tasks.workunit.client.0.smithi176.stdout:1/182: chown d2/de/l13 102 1 2024-03-13T14:49:37.091 INFO:tasks.workunit.client.1.smithi196.stdout:7/415: creat d0/d9/d1e/d58/d72/f84 x:0 0 0 2024-03-13T14:49:37.092 INFO:tasks.workunit.client.0.smithi176.stdout:5/155: creat d5/f37 x:0 0 0 2024-03-13T14:49:37.095 INFO:tasks.workunit.client.0.smithi176.stdout:1/183: fsync d2/d29/f39 0 2024-03-13T14:49:37.099 INFO:tasks.workunit.client.1.smithi196.stdout:9/301: symlink d7/de/d4b/d1d/d22/l5d 0 2024-03-13T14:49:37.101 INFO:tasks.workunit.client.1.smithi196.stdout:9/302: fsync d7/de/d4b/d1d/f54 0 2024-03-13T14:49:37.104 INFO:tasks.workunit.client.0.smithi176.stdout:4/190: creat d7/d8/d15/d23/f47 x:0 0 0 2024-03-13T14:49:37.109 INFO:tasks.workunit.client.1.smithi196.stdout:4/308: rename d1/d1e/d53 to d1/d3/d7/d35/d3e/d56 0 2024-03-13T14:49:37.109 INFO:tasks.workunit.client.1.smithi196.stdout:6/465: write d6/d9/d17/d19/f20 [1472855,86982] 0 2024-03-13T14:49:37.111 INFO:tasks.workunit.client.1.smithi196.stdout:8/297: symlink d0/d1c/d29/l6b 0 2024-03-13T14:49:37.113 INFO:tasks.workunit.client.1.smithi196.stdout:6/466: chown d6/d57/d91 61 1 2024-03-13T14:49:37.113 INFO:tasks.workunit.client.1.smithi196.stdout:8/298: stat d0/d10/d12/d1e/c67 0 2024-03-13T14:49:37.115 INFO:tasks.workunit.client.0.smithi176.stdout:3/185: getdents d4/d28/d2e 0 2024-03-13T14:49:37.115 INFO:tasks.workunit.client.1.smithi196.stdout:6/467: dread - d6/d9/d17/f6c zero size 2024-03-13T14:49:37.117 INFO:tasks.workunit.client.1.smithi196.stdout:6/468: fdatasync d6/d7/df/f35 0 2024-03-13T14:49:37.118 INFO:tasks.workunit.client.0.smithi176.stdout:0/146: rename d2/l7 to d2/d5/dc/d20/l26 0 2024-03-13T14:49:37.120 INFO:tasks.workunit.client.1.smithi196.stdout:7/416: truncate d0/d6/d33/f3c 2014075 0 2024-03-13T14:49:37.121 INFO:tasks.workunit.client.0.smithi176.stdout:4/191: read d7/d2d/f2e [251557,59399] 0 2024-03-13T14:49:37.121 INFO:tasks.workunit.client.0.smithi176.stdout:8/176: dread d0/f3 [0,4194304] 0 2024-03-13T14:49:37.123 INFO:tasks.workunit.client.0.smithi176.stdout:3/186: rename d4/d1a to d4/d28/d3a 0 2024-03-13T14:49:37.125 INFO:tasks.workunit.client.1.smithi196.stdout:1/345: mkdir d1/d9/d6c/d79 0 2024-03-13T14:49:37.130 INFO:tasks.workunit.client.0.smithi176.stdout:9/210: write d0/d1e/f25 [2660716,12979] 0 2024-03-13T14:49:37.131 INFO:tasks.workunit.client.0.smithi176.stdout:9/211: read d0/d1e/d1f/f2c [723064,111809] 0 2024-03-13T14:49:37.132 INFO:tasks.workunit.client.0.smithi176.stdout:2/199: sync 2024-03-13T14:49:37.133 INFO:tasks.workunit.client.0.smithi176.stdout:2/200: chown d1/f35 73306 1 2024-03-13T14:49:37.137 INFO:tasks.workunit.client.0.smithi176.stdout:3/187: rename d4/f5 to d4/d13/d2d/f3b 0 2024-03-13T14:49:37.144 INFO:tasks.workunit.client.1.smithi196.stdout:1/346: symlink d1/d3f/d5b/l7a 0 2024-03-13T14:49:37.149 INFO:tasks.workunit.client.1.smithi196.stdout:4/309: creat d1/d3/d4c/f57 x:0 0 0 2024-03-13T14:49:37.154 INFO:tasks.workunit.client.0.smithi176.stdout:2/201: symlink d1/dc/d40/l4a 0 2024-03-13T14:49:37.155 INFO:tasks.workunit.client.1.smithi196.stdout:9/303: link d7/de/d16/c18 d7/de/d16/d3c/c5e 0 2024-03-13T14:49:37.165 INFO:tasks.workunit.client.1.smithi196.stdout:1/347: rmdir d1/d9/d4e/d55/d63 39 2024-03-13T14:49:37.165 INFO:tasks.workunit.client.1.smithi196.stdout:5/451: truncate d4/db/f81 1692161 0 2024-03-13T14:49:37.169 INFO:tasks.workunit.client.0.smithi176.stdout:2/202: mkdir d1/dc/d2d/d4b 0 2024-03-13T14:49:37.169 INFO:tasks.workunit.client.1.smithi196.stdout:4/310: rmdir d1/d3/d7/d35/d3e 39 2024-03-13T14:49:37.171 INFO:tasks.workunit.client.0.smithi176.stdout:9/212: link d0/l2b d0/d1e/d1f/l3e 0 2024-03-13T14:49:37.171 INFO:tasks.workunit.client.0.smithi176.stdout:6/167: dwrite d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:37.171 INFO:tasks.workunit.client.0.smithi176.stdout:2/203: write d1/ff [1442063,45672] 0 2024-03-13T14:49:37.172 INFO:tasks.workunit.client.1.smithi196.stdout:8/299: symlink d0/d16/d58/l6c 0 2024-03-13T14:49:37.173 INFO:tasks.workunit.client.1.smithi196.stdout:6/469: link d6/d2a/f83 d6/d2a/f93 0 2024-03-13T14:49:37.174 INFO:tasks.workunit.client.0.smithi176.stdout:6/168: dread - d0/d10/d19/d27/d2b/f39 zero size 2024-03-13T14:49:37.177 INFO:tasks.workunit.client.0.smithi176.stdout:6/169: write d0/d10/d19/d27/d2b/d2e/f2d [5199897,89768] 0 2024-03-13T14:49:37.180 INFO:tasks.workunit.client.0.smithi176.stdout:6/170: write d0/de/f3c [142715,36753] 0 2024-03-13T14:49:37.182 INFO:tasks.workunit.client.1.smithi196.stdout:5/452: unlink d4/db/d14/d19/d26/f89 0 2024-03-13T14:49:37.184 INFO:tasks.workunit.client.1.smithi196.stdout:9/304: dwrite d7/d24/f2b [0,4194304] 0 2024-03-13T14:49:37.188 INFO:tasks.workunit.client.0.smithi176.stdout:9/213: dwrite d0/f10 [0,4194304] 0 2024-03-13T14:49:37.189 INFO:tasks.workunit.client.0.smithi176.stdout:9/214: fsync d0/d13/f16 0 2024-03-13T14:49:37.190 INFO:tasks.workunit.client.1.smithi196.stdout:5/453: chown d4/db/d3c/f30 0 1 2024-03-13T14:49:37.191 INFO:tasks.workunit.client.0.smithi176.stdout:6/171: unlink d0/cb 0 2024-03-13T14:49:37.192 INFO:tasks.workunit.client.1.smithi196.stdout:5/454: fsync d4/db/d3c/f30 0 2024-03-13T14:49:37.193 INFO:tasks.workunit.client.0.smithi176.stdout:9/215: write d0/d1e/d1f/f38 [441334,70069] 0 2024-03-13T14:49:37.193 INFO:tasks.workunit.client.0.smithi176.stdout:6/172: chown d0/d9/l1c 485 1 2024-03-13T14:49:37.194 INFO:tasks.workunit.client.1.smithi196.stdout:5/455: chown d4/db/f24 7 1 2024-03-13T14:49:37.194 INFO:tasks.workunit.client.0.smithi176.stdout:9/216: chown d0/d1e/c34 125785757 1 2024-03-13T14:49:37.197 INFO:tasks.workunit.client.1.smithi196.stdout:4/311: mkdir d1/d1e/d39/d58 0 2024-03-13T14:49:37.197 INFO:tasks.workunit.client.1.smithi196.stdout:1/348: dwrite d1/f2a [0,4194304] 0 2024-03-13T14:49:37.198 INFO:tasks.workunit.client.1.smithi196.stdout:7/417: getdents d0/d9/d1e/d3f 0 2024-03-13T14:49:37.203 INFO:tasks.workunit.client.1.smithi196.stdout:1/349: read f0 [2647092,78350] 0 2024-03-13T14:49:37.203 INFO:tasks.workunit.client.1.smithi196.stdout:8/300: dwrite d0/d1c/d29/f3c [0,4194304] 0 2024-03-13T14:49:37.204 INFO:tasks.workunit.client.0.smithi176.stdout:9/217: read d0/d13/f16 [2297501,118719] 0 2024-03-13T14:49:37.205 INFO:tasks.workunit.client.1.smithi196.stdout:9/305: mknod d7/de/d4b/c5f 0 2024-03-13T14:49:37.205 INFO:tasks.workunit.client.1.smithi196.stdout:3/344: write db/f2c [1198240,8407] 0 2024-03-13T14:49:37.207 INFO:tasks.workunit.client.0.smithi176.stdout:0/147: chown d2/d5/dc/d20/l26 137 1 2024-03-13T14:49:37.209 INFO:tasks.workunit.client.0.smithi176.stdout:0/148: readlink d2/d5/dc/d20/l26 0 2024-03-13T14:49:37.211 INFO:tasks.workunit.client.0.smithi176.stdout:2/204: dread d1/dc/f12 [0,4194304] 0 2024-03-13T14:49:37.212 INFO:tasks.workunit.client.1.smithi196.stdout:3/345: stat db/d19/c6c 0 2024-03-13T14:49:37.222 INFO:tasks.workunit.client.0.smithi176.stdout:6/173: truncate d0/d10/d19/d27/d2b/d2e/f31 655396 0 2024-03-13T14:49:37.223 INFO:tasks.workunit.client.0.smithi176.stdout:6/174: chown d0/de/c3b 1668513267 1 2024-03-13T14:49:37.224 INFO:tasks.workunit.client.0.smithi176.stdout:5/156: truncate d5/f7 3136126 0 2024-03-13T14:49:37.229 INFO:tasks.workunit.client.1.smithi196.stdout:7/418: dwrite d0/d6/d33/d57/f7d [0,4194304] 0 2024-03-13T14:49:37.232 INFO:tasks.workunit.client.0.smithi176.stdout:7/188: sync 2024-03-13T14:49:37.233 INFO:tasks.workunit.client.1.smithi196.stdout:7/419: write d0/d9/d1e/d58/d72/d6e/f70 [687352,54516] 0 2024-03-13T14:49:37.234 INFO:tasks.workunit.client.0.smithi176.stdout:0/149: stat d2/d5/d1e/c1f 0 2024-03-13T14:49:37.236 INFO:tasks.workunit.client.0.smithi176.stdout:0/150: truncate d2/d5/dc/f18 4941774 0 2024-03-13T14:49:37.240 INFO:tasks.workunit.client.1.smithi196.stdout:4/312: creat d1/d3/d4c/f59 x:0 0 0 2024-03-13T14:49:37.242 INFO:tasks.workunit.client.0.smithi176.stdout:4/192: truncate d7/d10/f2a 914158 0 2024-03-13T14:49:37.245 INFO:tasks.workunit.client.0.smithi176.stdout:2/205: creat d1/dc/d1c/f4c x:0 0 0 2024-03-13T14:49:37.247 INFO:tasks.workunit.client.1.smithi196.stdout:6/470: dread d6/d9/d11/f13 [0,4194304] 0 2024-03-13T14:49:37.248 INFO:tasks.workunit.client.1.smithi196.stdout:8/301: mknod d0/d1c/d29/c6d 0 2024-03-13T14:49:37.248 INFO:tasks.workunit.client.0.smithi176.stdout:2/206: read d1/dc/f16 [3198731,114769] 0 2024-03-13T14:49:37.249 INFO:tasks.workunit.client.0.smithi176.stdout:8/177: truncate d0/d5/f17 725465 0 2024-03-13T14:49:37.250 INFO:tasks.workunit.client.1.smithi196.stdout:8/302: fsync d0/d1c/d29/d47/f52 0 2024-03-13T14:49:37.251 INFO:tasks.workunit.client.0.smithi176.stdout:5/157: dread d5/d22/f13 [0,4194304] 0 2024-03-13T14:49:37.251 INFO:tasks.workunit.client.0.smithi176.stdout:7/189: dwrite d1/d7/f29 [0,4194304] 0 2024-03-13T14:49:37.251 INFO:tasks.workunit.client.0.smithi176.stdout:2/207: chown d1/ff 1338925 1 2024-03-13T14:49:37.252 INFO:tasks.workunit.client.1.smithi196.stdout:4/313: fsync d1/d3/d4c/f57 0 2024-03-13T14:49:37.252 INFO:tasks.workunit.client.0.smithi176.stdout:5/158: readlink d5/d22/l1b 0 2024-03-13T14:49:37.253 INFO:tasks.workunit.client.0.smithi176.stdout:2/208: chown d1/dc/d1c 16 1 2024-03-13T14:49:37.255 INFO:tasks.workunit.client.0.smithi176.stdout:0/151: dwrite d2/d5/dc/f18 [0,4194304] 0 2024-03-13T14:49:37.255 INFO:tasks.workunit.client.0.smithi176.stdout:7/190: write d1/d13/f14 [1616993,115096] 0 2024-03-13T14:49:37.256 INFO:tasks.workunit.client.1.smithi196.stdout:1/350: mkdir d1/d9/d33/d4b/d66/d7b 0 2024-03-13T14:49:37.258 INFO:tasks.workunit.client.1.smithi196.stdout:4/314: dread - d1/d3/d4c/f59 zero size 2024-03-13T14:49:37.258 INFO:tasks.workunit.client.0.smithi176.stdout:6/175: creat d0/d10/d19/d27/d2b/d2e/f40 x:0 0 0 2024-03-13T14:49:37.259 INFO:tasks.workunit.client.1.smithi196.stdout:1/351: chown d1/d38/f30 31202 1 2024-03-13T14:49:37.261 INFO:tasks.workunit.client.1.smithi196.stdout:1/352: fsync d1/d9/d33/f42 0 2024-03-13T14:49:37.262 INFO:tasks.workunit.client.1.smithi196.stdout:3/346: creat db/f70 x:0 0 0 2024-03-13T14:49:37.266 INFO:tasks.workunit.client.0.smithi176.stdout:0/152: dread d2/db/f1c [0,4194304] 0 2024-03-13T14:49:37.269 INFO:tasks.workunit.client.0.smithi176.stdout:0/153: fsync d2/d5/dc/f19 0 2024-03-13T14:49:37.272 INFO:tasks.workunit.client.0.smithi176.stdout:5/159: dwrite d5/d22/f1e [0,4194304] 0 2024-03-13T14:49:37.274 INFO:tasks.workunit.client.1.smithi196.stdout:7/420: creat d0/d6/d33/d57/f85 x:0 0 0 2024-03-13T14:49:37.283 INFO:tasks.workunit.client.0.smithi176.stdout:5/160: dwrite d5/d2b/f30 [0,4194304] 0 2024-03-13T14:49:37.290 INFO:tasks.workunit.client.1.smithi196.stdout:5/456: write d4/db/fd [3592578,57039] 0 2024-03-13T14:49:37.291 INFO:tasks.workunit.client.1.smithi196.stdout:4/315: fdatasync d1/d1e/f34 0 2024-03-13T14:49:37.295 INFO:tasks.workunit.client.1.smithi196.stdout:3/347: readlink db/d15/d44/d51/l5f 0 2024-03-13T14:49:37.300 INFO:tasks.workunit.client.1.smithi196.stdout:1/353: rename d1/d38/f40 to d1/d9/d33/d4b/d66/f7c 0 2024-03-13T14:49:37.302 INFO:tasks.workunit.client.1.smithi196.stdout:5/457: write d4/db/d14/d19/d1c/d25/d38/f56 [3554258,55096] 0 2024-03-13T14:49:37.306 INFO:tasks.workunit.client.1.smithi196.stdout:6/471: dwrite d6/d9/f30 [0,4194304] 0 2024-03-13T14:49:37.308 INFO:tasks.workunit.client.1.smithi196.stdout:5/458: truncate d4/db/d4a/f8a 87339 0 2024-03-13T14:49:37.315 INFO:tasks.workunit.client.0.smithi176.stdout:9/218: creat d0/db/f3f x:0 0 0 2024-03-13T14:49:37.317 INFO:tasks.workunit.client.1.smithi196.stdout:8/303: symlink d0/d10/d12/d17/l6e 0 2024-03-13T14:49:37.317 INFO:tasks.workunit.client.1.smithi196.stdout:3/348: mkdir db/d71 0 2024-03-13T14:49:37.319 INFO:tasks.workunit.client.1.smithi196.stdout:3/349: chown db/d19/d22/f39 14183 1 2024-03-13T14:49:37.323 INFO:tasks.workunit.client.0.smithi176.stdout:0/154: mknod d2/db/c27 0 2024-03-13T14:49:37.323 INFO:tasks.workunit.client.1.smithi196.stdout:3/350: stat db/f29 0 2024-03-13T14:49:37.324 INFO:tasks.workunit.client.0.smithi176.stdout:0/155: fsync d2/db/f1c 0 2024-03-13T14:49:37.326 INFO:tasks.workunit.client.1.smithi196.stdout:4/316: rmdir d1/d1e/d43 39 2024-03-13T14:49:37.328 INFO:tasks.workunit.client.1.smithi196.stdout:4/317: dread - d1/d3/d4c/f57 zero size 2024-03-13T14:49:37.333 INFO:tasks.workunit.client.0.smithi176.stdout:5/161: creat d5/d2c/f38 x:0 0 0 2024-03-13T14:49:37.338 INFO:tasks.workunit.client.1.smithi196.stdout:3/351: symlink db/d37/l72 0 2024-03-13T14:49:37.341 INFO:tasks.workunit.client.1.smithi196.stdout:3/352: stat db/d19/d22/d3a/d68 0 2024-03-13T14:49:37.341 INFO:tasks.workunit.client.0.smithi176.stdout:0/156: dread d2/d5/dc/f18 [0,4194304] 0 2024-03-13T14:49:37.344 INFO:tasks.workunit.client.0.smithi176.stdout:6/176: symlink d0/de/d38/l41 0 2024-03-13T14:49:37.344 INFO:tasks.workunit.client.0.smithi176.stdout:0/157: fdatasync d2/db/f25 0 2024-03-13T14:49:37.349 INFO:tasks.workunit.client.0.smithi176.stdout:0/158: truncate d2/db/f1c 469882 0 2024-03-13T14:49:37.349 INFO:tasks.workunit.client.0.smithi176.stdout:6/177: write d0/d10/d19/d27/d2b/d2e/f2d [4943430,120650] 0 2024-03-13T14:49:37.354 INFO:tasks.workunit.client.1.smithi196.stdout:6/472: mkdir d6/d9/d17/d19/d5f/d94 0 2024-03-13T14:49:37.358 INFO:tasks.workunit.client.1.smithi196.stdout:3/353: rename db/d19/c6c to db/d4c/c73 0 2024-03-13T14:49:37.368 INFO:tasks.workunit.client.1.smithi196.stdout:3/354: fdatasync db/fe 0 2024-03-13T14:49:37.370 INFO:tasks.workunit.client.0.smithi176.stdout:9/219: mkdir d0/d40 0 2024-03-13T14:49:37.371 INFO:tasks.workunit.client.1.smithi196.stdout:4/318: link d1/c50 d1/d1e/d39/c5a 0 2024-03-13T14:49:37.374 INFO:tasks.workunit.client.1.smithi196.stdout:4/319: chown d1/l2d 3372955 1 2024-03-13T14:49:37.374 INFO:tasks.workunit.client.0.smithi176.stdout:9/220: fdatasync d0/d13/f16 0 2024-03-13T14:49:37.374 INFO:tasks.workunit.client.0.smithi176.stdout:4/193: dwrite d7/d10/f2a [0,4194304] 0 2024-03-13T14:49:37.375 INFO:tasks.workunit.client.0.smithi176.stdout:0/159: mkdir d2/d5/d28 0 2024-03-13T14:49:37.376 INFO:tasks.workunit.client.0.smithi176.stdout:3/188: sync 2024-03-13T14:49:37.376 INFO:tasks.workunit.client.0.smithi176.stdout:1/184: sync 2024-03-13T14:49:37.377 INFO:tasks.workunit.client.1.smithi196.stdout:7/421: dwrite d0/d4d/f63 [0,4194304] 0 2024-03-13T14:49:37.378 INFO:tasks.workunit.client.1.smithi196.stdout:6/473: dwrite d6/d9/d17/d19/d4e/f7c [0,4194304] 0 2024-03-13T14:49:37.378 INFO:tasks.workunit.client.0.smithi176.stdout:6/178: dwrite d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:37.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:37 smithi176 ceph-mon[29064]: pgmap v7: 65 pgs: 65 active+clean; 919 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 109 MiB/s wr, 203 op/s 2024-03-13T14:49:37.380 INFO:tasks.workunit.client.0.smithi176.stdout:1/185: fsync d2/f37 0 2024-03-13T14:49:37.382 INFO:tasks.workunit.client.0.smithi176.stdout:3/189: write d4/f36 [321320,32013] 0 2024-03-13T14:49:37.383 INFO:tasks.workunit.client.0.smithi176.stdout:7/191: truncate d1/f19 1245481 0 2024-03-13T14:49:37.384 INFO:tasks.workunit.client.1.smithi196.stdout:6/474: readlink d6/d2a/dc/l21 0 2024-03-13T14:49:37.386 INFO:tasks.workunit.client.0.smithi176.stdout:0/160: write d2/d5/dc/f19 [833939,84587] 0 2024-03-13T14:49:37.387 INFO:tasks.workunit.client.0.smithi176.stdout:2/209: truncate d1/dc/f12 3515833 0 2024-03-13T14:49:37.389 INFO:tasks.workunit.client.1.smithi196.stdout:7/422: symlink d0/d78/l86 0 2024-03-13T14:49:37.393 INFO:tasks.workunit.client.1.smithi196.stdout:6/475: dread d6/d9/d11/f13 [0,4194304] 0 2024-03-13T14:49:37.395 INFO:tasks.workunit.client.0.smithi176.stdout:2/210: write d1/f35 [937218,108971] 0 2024-03-13T14:49:37.396 INFO:tasks.workunit.client.0.smithi176.stdout:5/162: mknod d5/c39 0 2024-03-13T14:49:37.396 INFO:tasks.workunit.client.0.smithi176.stdout:7/192: chown d1/d13/d1b/d18 708537126 1 2024-03-13T14:49:37.401 INFO:tasks.workunit.client.0.smithi176.stdout:5/163: read d5/d2b/f30 [1201077,127677] 0 2024-03-13T14:49:37.401 INFO:tasks.workunit.client.0.smithi176.stdout:2/211: dread d1/f31 [0,4194304] 0 2024-03-13T14:49:37.404 INFO:tasks.workunit.client.0.smithi176.stdout:5/164: chown d5/f8 869 1 2024-03-13T14:49:37.404 INFO:tasks.workunit.client.0.smithi176.stdout:2/212: write d1/fa [630043,110609] 0 2024-03-13T14:49:37.407 INFO:tasks.workunit.client.1.smithi196.stdout:4/320: rename d1/d3/d7/d35/d51 to d1/d1e/d43/d5b 0 2024-03-13T14:49:37.408 INFO:tasks.workunit.client.0.smithi176.stdout:2/213: dread - d1/dc/d2d/d38/f48 zero size 2024-03-13T14:49:37.410 INFO:tasks.workunit.client.0.smithi176.stdout:4/194: dwrite d7/d8/f39 [0,4194304] 0 2024-03-13T14:49:37.410 INFO:tasks.workunit.client.1.smithi196.stdout:4/321: chown d1/d3/d4c 837938 1 2024-03-13T14:49:37.418 INFO:tasks.workunit.client.1.smithi196.stdout:1/354: dwrite d1/d9/f18 [0,4194304] 0 2024-03-13T14:49:37.418 INFO:tasks.workunit.client.0.smithi176.stdout:9/221: creat d0/d1e/d1f/f41 x:0 0 0 2024-03-13T14:49:37.422 INFO:tasks.workunit.client.0.smithi176.stdout:8/178: sync 2024-03-13T14:49:37.423 INFO:tasks.workunit.client.0.smithi176.stdout:8/179: stat d0/d24/d29 0 2024-03-13T14:49:37.425 INFO:tasks.workunit.client.0.smithi176.stdout:1/186: rename d2/f5 to d2/d29/f3c 0 2024-03-13T14:49:37.426 INFO:tasks.workunit.client.0.smithi176.stdout:1/187: write d2/f9 [3018934,61923] 0 2024-03-13T14:49:37.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:37 smithi196 ceph-mon[37486]: pgmap v7: 65 pgs: 65 active+clean; 919 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 109 MiB/s wr, 203 op/s 2024-03-13T14:49:37.430 INFO:tasks.workunit.client.0.smithi176.stdout:9/222: dwrite d0/f10 [4194304,4194304] 0 2024-03-13T14:49:37.431 INFO:tasks.workunit.client.1.smithi196.stdout:7/423: link d0/f56 d0/d9/d31/d50/f87 0 2024-03-13T14:49:37.433 INFO:tasks.workunit.client.1.smithi196.stdout:3/355: chown db/d19/d22/f23 12268202 1 2024-03-13T14:49:37.436 INFO:tasks.workunit.client.0.smithi176.stdout:1/188: dread f1 [0,4194304] 0 2024-03-13T14:49:37.439 INFO:tasks.workunit.client.1.smithi196.stdout:1/355: mknod d1/d3f/d5b/d74/c7d 0 2024-03-13T14:49:37.441 INFO:tasks.workunit.client.1.smithi196.stdout:4/322: mkdir d1/d1e/d5c 0 2024-03-13T14:49:37.457 INFO:tasks.workunit.client.1.smithi196.stdout:6/476: dread d6/d4b/d50/f5a [0,4194304] 0 2024-03-13T14:49:37.459 INFO:tasks.workunit.client.1.smithi196.stdout:7/424: mknod d0/d9/d1e/d58/d61/c88 0 2024-03-13T14:49:37.461 INFO:tasks.workunit.client.1.smithi196.stdout:7/425: chown d0/d9/d1e/c6d 0 1 2024-03-13T14:49:37.462 INFO:tasks.workunit.client.0.smithi176.stdout:5/165: write d5/d22/d15/f21 [18602,16007] 0 2024-03-13T14:49:37.464 INFO:tasks.workunit.client.1.smithi196.stdout:7/426: truncate d0/f56 1572202 0 2024-03-13T14:49:37.468 INFO:tasks.workunit.client.1.smithi196.stdout:6/477: fsync d6/d2a/d23/d28/f3f 0 2024-03-13T14:49:37.474 INFO:tasks.workunit.client.0.smithi176.stdout:3/190: creat d4/d28/d2e/f3c x:0 0 0 2024-03-13T14:49:37.480 INFO:tasks.workunit.client.1.smithi196.stdout:2/242: sync 2024-03-13T14:49:37.481 INFO:tasks.workunit.client.1.smithi196.stdout:1/356: creat d1/d9/d33/d4b/d66/d7b/f7e x:0 0 0 2024-03-13T14:49:37.483 INFO:tasks.workunit.client.1.smithi196.stdout:2/243: read - d0/f37 zero size 2024-03-13T14:49:37.492 INFO:tasks.workunit.client.1.smithi196.stdout:6/478: fsync d6/d2a/d23/d28/f3f 0 2024-03-13T14:49:37.492 INFO:tasks.workunit.client.0.smithi176.stdout:0/161: creat d2/d5/d28/f29 x:0 0 0 2024-03-13T14:49:37.495 INFO:tasks.workunit.client.0.smithi176.stdout:0/162: chown d2/d5/dc/f16 1582580551 1 2024-03-13T14:49:37.495 INFO:tasks.workunit.client.1.smithi196.stdout:6/479: write d6/d7/dd/d60/f79 [850836,63907] 0 2024-03-13T14:49:37.496 INFO:tasks.workunit.client.0.smithi176.stdout:0/163: chown d2/d5/d1e/l24 475 1 2024-03-13T14:49:37.498 INFO:tasks.workunit.client.1.smithi196.stdout:4/323: creat d1/d3/d7/f5d x:0 0 0 2024-03-13T14:49:37.499 INFO:tasks.workunit.client.1.smithi196.stdout:3/356: dread db/ff [4194304,4194304] 0 2024-03-13T14:49:37.499 INFO:tasks.workunit.client.0.smithi176.stdout:6/179: getdents d0/d10/d19/d27/d2b/d2e/d3e 0 2024-03-13T14:49:37.501 INFO:tasks.workunit.client.1.smithi196.stdout:1/357: creat d1/d9/d33/d4b/d66/d7b/f7f x:0 0 0 2024-03-13T14:49:37.501 INFO:tasks.workunit.client.0.smithi176.stdout:6/180: chown d0/d10/d19/d27/d2b 6 1 2024-03-13T14:49:37.501 INFO:tasks.workunit.client.1.smithi196.stdout:1/358: readlink d1/l2 0 2024-03-13T14:49:37.502 INFO:tasks.workunit.client.1.smithi196.stdout:3/357: read db/f36 [1171652,91503] 0 2024-03-13T14:49:37.503 INFO:tasks.workunit.client.1.smithi196.stdout:3/358: readlink db/d15/l1b 0 2024-03-13T14:49:37.504 INFO:tasks.workunit.client.0.smithi176.stdout:6/181: write d0/d10/f3f [1030067,51269] 0 2024-03-13T14:49:37.504 INFO:tasks.workunit.client.1.smithi196.stdout:1/359: write d1/d28/f5a [4164113,90130] 0 2024-03-13T14:49:37.505 INFO:tasks.workunit.client.0.smithi176.stdout:1/189: truncate d2/de/f17 343198 0 2024-03-13T14:49:37.506 INFO:tasks.workunit.client.1.smithi196.stdout:2/244: truncate d0/d7/f21 319835 0 2024-03-13T14:49:37.506 INFO:tasks.workunit.client.0.smithi176.stdout:7/193: mknod d1/d2/d2b/c31 0 2024-03-13T14:49:37.507 INFO:tasks.workunit.client.1.smithi196.stdout:3/359: write db/d15/d44/d51/f6e [940517,14026] 0 2024-03-13T14:49:37.507 INFO:tasks.workunit.client.0.smithi176.stdout:1/190: chown d2/f9 30460 1 2024-03-13T14:49:37.507 INFO:tasks.workunit.client.0.smithi176.stdout:6/182: read d0/f18 [2212295,85430] 0 2024-03-13T14:49:37.509 INFO:tasks.workunit.client.0.smithi176.stdout:1/191: read - d2/de/d10/f2a zero size 2024-03-13T14:49:37.509 INFO:tasks.workunit.client.1.smithi196.stdout:6/480: rmdir d6/d4b 39 2024-03-13T14:49:37.510 INFO:tasks.workunit.client.0.smithi176.stdout:5/166: symlink d5/d2b/l3a 0 2024-03-13T14:49:37.511 INFO:tasks.workunit.client.0.smithi176.stdout:4/195: mknod d7/d8/d45/c48 0 2024-03-13T14:49:37.511 INFO:tasks.workunit.client.0.smithi176.stdout:1/192: fdatasync d2/de/f26 0 2024-03-13T14:49:37.511 INFO:tasks.workunit.client.1.smithi196.stdout:6/481: chown d6/d7/dd/f7e 1128 1 2024-03-13T14:49:37.513 INFO:tasks.workunit.client.0.smithi176.stdout:3/191: mkdir d4/d13/d16/d3d 0 2024-03-13T14:49:37.513 INFO:tasks.workunit.client.0.smithi176.stdout:4/196: fdatasync d7/d8/d15/d23/f35 0 2024-03-13T14:49:37.516 INFO:tasks.workunit.client.1.smithi196.stdout:1/360: creat d1/d3f/f80 x:0 0 0 2024-03-13T14:49:37.519 INFO:tasks.workunit.client.0.smithi176.stdout:0/164: mkdir d2/d5/dc/d2a 0 2024-03-13T14:49:37.520 INFO:tasks.workunit.client.1.smithi196.stdout:4/324: creat d1/d1e/d43/d5b/d55/f5e x:0 0 0 2024-03-13T14:49:37.520 INFO:tasks.workunit.client.0.smithi176.stdout:7/194: rmdir d1 39 2024-03-13T14:49:37.523 INFO:tasks.workunit.client.1.smithi196.stdout:6/482: unlink d6/d2a/f83 0 2024-03-13T14:49:37.526 INFO:tasks.workunit.client.0.smithi176.stdout:6/183: rmdir d0 39 2024-03-13T14:49:37.528 INFO:tasks.workunit.client.1.smithi196.stdout:7/427: getdents d0/d9/d1e/d58/d72/d6e 0 2024-03-13T14:49:37.531 INFO:tasks.workunit.client.0.smithi176.stdout:5/167: symlink d5/d2c/l3b 0 2024-03-13T14:49:37.532 INFO:tasks.workunit.client.1.smithi196.stdout:7/428: read d0/d9/d1e/d58/d72/d6e/f70 [265267,99268] 0 2024-03-13T14:49:37.533 INFO:tasks.workunit.client.1.smithi196.stdout:4/325: unlink d1/d1e/d39/c4f 0 2024-03-13T14:49:37.534 INFO:tasks.workunit.client.1.smithi196.stdout:2/245: dwrite d0/d7/d40/f36 [0,4194304] 0 2024-03-13T14:49:37.538 INFO:tasks.workunit.client.1.smithi196.stdout:1/361: symlink d1/d9/d6c/d79/l81 0 2024-03-13T14:49:37.541 INFO:tasks.workunit.client.0.smithi176.stdout:0/165: symlink d2/d5/dc/l2b 0 2024-03-13T14:49:37.546 INFO:tasks.workunit.client.0.smithi176.stdout:7/195: write d1/d13/f14 [614904,19010] 0 2024-03-13T14:49:37.550 INFO:tasks.workunit.client.0.smithi176.stdout:2/214: getdents d1 0 2024-03-13T14:49:37.552 INFO:tasks.workunit.client.0.smithi176.stdout:6/184: read - d0/d10/d19/d27/d2b/f39 zero size 2024-03-13T14:49:37.553 INFO:tasks.workunit.client.1.smithi196.stdout:7/429: symlink d0/l89 0 2024-03-13T14:49:37.554 INFO:tasks.workunit.client.0.smithi176.stdout:8/180: dwrite d0/db/f19 [0,4194304] 0 2024-03-13T14:49:37.556 INFO:tasks.workunit.client.1.smithi196.stdout:7/430: truncate d0/f4f 4457476 0 2024-03-13T14:49:37.557 INFO:tasks.workunit.client.1.smithi196.stdout:4/326: creat d1/d1e/d5c/f5f x:0 0 0 2024-03-13T14:49:37.560 INFO:tasks.workunit.client.1.smithi196.stdout:2/246: mkdir d0/d50 0 2024-03-13T14:49:37.562 INFO:tasks.workunit.client.1.smithi196.stdout:6/483: dwrite d6/d2a/d23/d28/f3f [0,4194304] 0 2024-03-13T14:49:37.563 INFO:tasks.workunit.client.0.smithi176.stdout:9/223: link d0/db/c1c d0/d1e/c42 0 2024-03-13T14:49:37.564 INFO:tasks.workunit.client.0.smithi176.stdout:1/193: truncate d2/dd/d19/f32 2947932 0 2024-03-13T14:49:37.565 INFO:tasks.workunit.client.0.smithi176.stdout:7/196: dread d1/d7/f29 [0,4194304] 0 2024-03-13T14:49:37.572 INFO:tasks.workunit.client.1.smithi196.stdout:7/431: readlink d0/d6/l7 0 2024-03-13T14:49:37.574 INFO:tasks.workunit.client.0.smithi176.stdout:6/185: dwrite d0/d10/d19/d27/d2b/d2e/f40 [0,4194304] 0 2024-03-13T14:49:37.577 INFO:tasks.workunit.client.0.smithi176.stdout:2/215: unlink d1/dc/l39 0 2024-03-13T14:49:37.580 INFO:tasks.workunit.client.0.smithi176.stdout:3/192: link d4/d13/c17 d4/d28/c3e 0 2024-03-13T14:49:37.583 INFO:tasks.workunit.client.0.smithi176.stdout:3/193: write d4/f36 [879365,102548] 0 2024-03-13T14:49:37.587 INFO:tasks.workunit.client.0.smithi176.stdout:9/224: rename d0/d13/l22 to d0/d17/l43 0 2024-03-13T14:49:37.588 INFO:tasks.workunit.client.0.smithi176.stdout:1/194: dwrite d2/d29/f39 [0,4194304] 0 2024-03-13T14:49:37.590 INFO:tasks.workunit.client.1.smithi196.stdout:2/247: link d0/d6/d34/d38/f3b d0/d6/d1d/f51 0 2024-03-13T14:49:37.594 INFO:tasks.workunit.client.1.smithi196.stdout:1/362: write d1/d28/f2d [976980,58229] 0 2024-03-13T14:49:37.603 INFO:tasks.workunit.client.0.smithi176.stdout:8/181: link d0/d24/f32 d0/db/d27/f34 0 2024-03-13T14:49:37.603 INFO:tasks.workunit.client.0.smithi176.stdout:6/186: dread d0/d9/f14 [0,4194304] 0 2024-03-13T14:49:37.606 INFO:tasks.workunit.client.0.smithi176.stdout:5/168: truncate d5/d22/f13 519249 0 2024-03-13T14:49:37.606 INFO:tasks.workunit.client.0.smithi176.stdout:6/187: stat d0/d10/d19/d27/d2b/d2e/c34 0 2024-03-13T14:49:37.607 INFO:tasks.workunit.client.0.smithi176.stdout:6/188: fdatasync d0/de/f3c 0 2024-03-13T14:49:37.608 INFO:tasks.workunit.client.1.smithi196.stdout:7/432: write d0/d9/d1e/d3f/f60 [306145,71761] 0 2024-03-13T14:49:37.610 INFO:tasks.workunit.client.0.smithi176.stdout:6/189: write d0/d10/d19/d27/d2b/d2e/f2d [5798273,117572] 0 2024-03-13T14:49:37.614 INFO:tasks.workunit.client.1.smithi196.stdout:1/363: dread d1/d28/f2b [0,4194304] 0 2024-03-13T14:49:37.617 INFO:tasks.workunit.client.1.smithi196.stdout:7/433: mkdir d0/d6/d33/d57/d8a 0 2024-03-13T14:49:37.621 INFO:tasks.workunit.client.0.smithi176.stdout:5/169: dwrite f0 [4194304,4194304] 0 2024-03-13T14:49:37.631 INFO:tasks.workunit.client.0.smithi176.stdout:3/194: write d4/f25 [51464,54048] 0 2024-03-13T14:49:37.634 INFO:tasks.workunit.client.0.smithi176.stdout:2/216: creat d1/dc/d2d/d4b/f4d x:0 0 0 2024-03-13T14:49:37.636 INFO:tasks.workunit.client.0.smithi176.stdout:6/190: rmdir d0/d10/d19/d27/d2b 39 2024-03-13T14:49:37.637 INFO:tasks.workunit.client.0.smithi176.stdout:6/191: fdatasync d0/de/f23 0 2024-03-13T14:49:37.638 INFO:tasks.workunit.client.1.smithi196.stdout:7/434: rename d0/d9/d1e/d58/d72/f3b to d0/d6/f8b 0 2024-03-13T14:49:37.639 INFO:tasks.workunit.client.1.smithi196.stdout:6/484: link d6/d2a/d23/f8f d6/d7/f95 0 2024-03-13T14:49:37.640 INFO:tasks.workunit.client.1.smithi196.stdout:7/435: chown d0/d9/lb 13666 1 2024-03-13T14:49:37.642 INFO:tasks.workunit.client.0.smithi176.stdout:5/170: symlink d5/d2b/l3c 0 2024-03-13T14:49:37.643 INFO:tasks.workunit.client.0.smithi176.stdout:7/197: truncate d1/d7/f29 3940544 0 2024-03-13T14:49:37.644 INFO:tasks.workunit.client.0.smithi176.stdout:1/195: mkdir d2/de/d10/d3d 0 2024-03-13T14:49:37.646 INFO:tasks.workunit.client.1.smithi196.stdout:7/436: creat d0/d9/d1e/d58/d72/f8c x:0 0 0 2024-03-13T14:49:37.647 INFO:tasks.workunit.client.0.smithi176.stdout:8/182: link d0/d5/f17 d0/db/dc/f35 0 2024-03-13T14:49:37.648 INFO:tasks.workunit.client.1.smithi196.stdout:7/437: fsync d0/d6/d33/f80 0 2024-03-13T14:49:37.648 INFO:tasks.workunit.client.0.smithi176.stdout:3/195: creat d4/d28/d34/f3f x:0 0 0 2024-03-13T14:49:37.650 INFO:tasks.workunit.client.1.smithi196.stdout:1/364: write d1/d28/f32 [963158,35080] 0 2024-03-13T14:49:37.652 INFO:tasks.workunit.client.0.smithi176.stdout:6/192: readlink d0/d10/d19/d27/l3d 0 2024-03-13T14:49:37.652 INFO:tasks.workunit.client.0.smithi176.stdout:9/225: link d0/cf d0/d40/c44 0 2024-03-13T14:49:37.653 INFO:tasks.workunit.client.1.smithi196.stdout:7/438: mknod d0/d4d/d74/c8d 0 2024-03-13T14:49:37.655 INFO:tasks.workunit.client.0.smithi176.stdout:5/171: chown d5/lc 0 1 2024-03-13T14:49:37.657 INFO:tasks.workunit.client.0.smithi176.stdout:1/196: mkdir d2/dd/d3e 0 2024-03-13T14:49:37.659 INFO:tasks.workunit.client.0.smithi176.stdout:2/217: unlink d1/l21 0 2024-03-13T14:49:37.660 INFO:tasks.workunit.client.0.smithi176.stdout:8/183: rmdir d0/d5 39 2024-03-13T14:49:37.661 INFO:tasks.workunit.client.0.smithi176.stdout:3/196: symlink d4/d28/d3a/l40 0 2024-03-13T14:49:37.663 INFO:tasks.workunit.client.1.smithi196.stdout:1/365: truncate d1/d28/f35 537226 0 2024-03-13T14:49:37.663 INFO:tasks.workunit.client.1.smithi196.stdout:6/485: link d6/d2a/dc/d1c/d34/l3d d6/l96 0 2024-03-13T14:49:37.667 INFO:tasks.workunit.client.0.smithi176.stdout:4/197: sync 2024-03-13T14:49:37.667 INFO:tasks.workunit.client.0.smithi176.stdout:0/166: sync 2024-03-13T14:49:37.668 INFO:tasks.workunit.client.1.smithi196.stdout:6/486: unlink d6/d2a/dc/f80 0 2024-03-13T14:49:37.671 INFO:tasks.workunit.client.0.smithi176.stdout:6/193: symlink d0/l42 0 2024-03-13T14:49:37.672 INFO:tasks.workunit.client.0.smithi176.stdout:5/172: dwrite f0 [0,4194304] 0 2024-03-13T14:49:37.674 INFO:tasks.workunit.client.0.smithi176.stdout:2/218: rename d1/dc/d2d/d4b/f4d to d1/dc/d2d/d38/f4e 0 2024-03-13T14:49:37.677 INFO:tasks.workunit.client.0.smithi176.stdout:2/219: chown d1/dc/d2d/l3b 47805 1 2024-03-13T14:49:37.679 INFO:tasks.workunit.client.0.smithi176.stdout:9/226: mknod d0/db/c45 0 2024-03-13T14:49:37.683 INFO:tasks.workunit.client.0.smithi176.stdout:8/184: rmdir d0/d5 39 2024-03-13T14:49:37.686 INFO:tasks.workunit.client.0.smithi176.stdout:1/197: dwrite d2/f21 [0,4194304] 0 2024-03-13T14:49:37.688 INFO:tasks.workunit.client.0.smithi176.stdout:0/167: dwrite d2/db/f1c [0,4194304] 0 2024-03-13T14:49:37.692 INFO:tasks.workunit.client.0.smithi176.stdout:7/198: write d1/f19 [796874,91754] 0 2024-03-13T14:49:37.693 INFO:tasks.workunit.client.1.smithi196.stdout:7/439: dwrite d0/d9/d31/d50/f87 [0,4194304] 0 2024-03-13T14:49:37.693 INFO:tasks.workunit.client.0.smithi176.stdout:3/197: creat d4/d13/d16/d3d/f41 x:0 0 0 2024-03-13T14:49:37.696 INFO:tasks.workunit.client.0.smithi176.stdout:4/198: rename d7/d10/l1f to d7/d10/d46/l49 0 2024-03-13T14:49:37.696 INFO:tasks.workunit.client.0.smithi176.stdout:3/198: write d4/d28/f29 [1048614,19631] 0 2024-03-13T14:49:37.697 INFO:tasks.workunit.client.0.smithi176.stdout:2/220: dread d1/dc/d2d/d38/f3f [0,4194304] 0 2024-03-13T14:49:37.698 INFO:tasks.workunit.client.0.smithi176.stdout:4/199: read - d7/d8/d15/d23/f47 zero size 2024-03-13T14:49:37.706 INFO:tasks.workunit.client.0.smithi176.stdout:5/173: creat d5/d2c/f3d x:0 0 0 2024-03-13T14:49:37.707 INFO:tasks.workunit.client.1.smithi196.stdout:1/366: truncate d1/d9/f18 1400041 0 2024-03-13T14:49:37.716 INFO:tasks.workunit.client.0.smithi176.stdout:9/227: mknod d0/d17/c46 0 2024-03-13T14:49:37.723 INFO:tasks.workunit.client.0.smithi176.stdout:1/198: unlink d2/f37 0 2024-03-13T14:49:37.724 INFO:tasks.workunit.client.1.smithi196.stdout:1/367: dwrite d1/d9/d33/f62 [4194304,4194304] 0 2024-03-13T14:49:37.725 INFO:tasks.workunit.client.0.smithi176.stdout:1/199: write d2/fc [6317578,126829] 0 2024-03-13T14:49:37.738 INFO:tasks.workunit.client.0.smithi176.stdout:7/199: rmdir d1/d28 39 2024-03-13T14:49:37.739 INFO:tasks.workunit.client.0.smithi176.stdout:6/194: dread d0/d10/d19/d27/d2b/d2e/f31 [0,4194304] 0 2024-03-13T14:49:37.742 INFO:tasks.workunit.client.0.smithi176.stdout:1/200: dwrite d2/d29/f39 [0,4194304] 0 2024-03-13T14:49:37.744 INFO:tasks.workunit.client.0.smithi176.stdout:6/195: read d0/d10/d19/d27/d2b/d2e/f31 [203075,85901] 0 2024-03-13T14:49:37.747 INFO:tasks.workunit.client.1.smithi196.stdout:1/368: dwrite d1/d9/d33/f42 [0,4194304] 0 2024-03-13T14:49:37.750 INFO:tasks.workunit.client.1.smithi196.stdout:1/369: readlink d1/d9/d6c/d79/l81 0 2024-03-13T14:49:37.767 INFO:tasks.workunit.client.0.smithi176.stdout:8/185: dread d0/f1c [0,4194304] 0 2024-03-13T14:49:37.778 INFO:tasks.workunit.client.0.smithi176.stdout:7/200: symlink d1/d1c/d26/l32 0 2024-03-13T14:49:37.780 INFO:tasks.workunit.client.0.smithi176.stdout:2/221: truncate d1/d2a/d33/f3a 102813 0 2024-03-13T14:49:37.780 INFO:tasks.workunit.client.1.smithi196.stdout:6/487: getdents d6/d2a/dc/d47 0 2024-03-13T14:49:37.782 INFO:tasks.workunit.client.1.smithi196.stdout:6/488: readlink d6/d9/d17/d19/l43 0 2024-03-13T14:49:37.785 INFO:tasks.workunit.client.0.smithi176.stdout:9/228: creat d0/d40/f47 x:0 0 0 2024-03-13T14:49:37.786 INFO:tasks.workunit.client.0.smithi176.stdout:9/229: chown d0/d13/d32 1981 1 2024-03-13T14:49:37.788 INFO:tasks.workunit.client.0.smithi176.stdout:9/230: fsync d0/db/f3f 0 2024-03-13T14:49:37.793 INFO:tasks.workunit.client.0.smithi176.stdout:8/186: creat d0/d24/d29/f36 x:0 0 0 2024-03-13T14:49:37.793 INFO:tasks.workunit.client.0.smithi176.stdout:6/196: mkdir d0/d43 0 2024-03-13T14:49:37.794 INFO:tasks.workunit.client.0.smithi176.stdout:7/201: symlink d1/d2/l33 0 2024-03-13T14:49:37.799 INFO:tasks.workunit.client.0.smithi176.stdout:9/231: chown d0/d17/l43 23 1 2024-03-13T14:49:37.800 INFO:tasks.workunit.client.0.smithi176.stdout:8/187: fdatasync d0/d5/f1f 0 2024-03-13T14:49:37.802 INFO:tasks.workunit.client.0.smithi176.stdout:6/197: unlink d0/d10/d19/d27/l15 0 2024-03-13T14:49:37.803 INFO:tasks.workunit.client.0.smithi176.stdout:1/201: dread d2/d29/f3c [0,4194304] 0 2024-03-13T14:49:37.803 INFO:tasks.workunit.client.1.smithi196.stdout:1/370: write d1/d38/f30 [1243969,39211] 0 2024-03-13T14:49:37.804 INFO:tasks.workunit.client.0.smithi176.stdout:2/222: mkdir d1/dc/d2d/d4f 0 2024-03-13T14:49:37.804 INFO:tasks.workunit.client.0.smithi176.stdout:9/232: fdatasync d0/d1e/d1f/f38 0 2024-03-13T14:49:37.805 INFO:tasks.workunit.client.0.smithi176.stdout:0/168: truncate d2/db/f1c 3147988 0 2024-03-13T14:49:37.806 INFO:tasks.workunit.client.0.smithi176.stdout:8/188: write d0/d24/f31 [337285,76960] 0 2024-03-13T14:49:37.810 INFO:tasks.workunit.client.0.smithi176.stdout:4/200: dwrite d7/d8/d15/f3e [0,4194304] 0 2024-03-13T14:49:37.811 INFO:tasks.workunit.client.1.smithi196.stdout:6/489: dread d6/d2a/f93 [0,4194304] 0 2024-03-13T14:49:37.813 INFO:tasks.workunit.client.0.smithi176.stdout:3/199: truncate d4/d28/d3a/f32 673558 0 2024-03-13T14:49:37.816 INFO:tasks.workunit.client.0.smithi176.stdout:4/201: chown d7/d8/l13 102306 1 2024-03-13T14:49:37.818 INFO:tasks.workunit.client.0.smithi176.stdout:8/189: creat d0/d24/f37 x:0 0 0 2024-03-13T14:49:37.820 INFO:tasks.workunit.client.1.smithi196.stdout:7/440: dwrite d0/d6/d33/f3c [0,4194304] 0 2024-03-13T14:49:37.821 INFO:tasks.workunit.client.0.smithi176.stdout:0/169: dread d2/d5/dc/f19 [0,4194304] 0 2024-03-13T14:49:37.826 INFO:tasks.workunit.client.0.smithi176.stdout:7/202: dwrite d1/d7/f10 [4194304,4194304] 0 2024-03-13T14:49:37.827 INFO:tasks.workunit.client.1.smithi196.stdout:6/490: creat d6/d7/df/f97 x:0 0 0 2024-03-13T14:49:37.827 INFO:tasks.workunit.client.1.smithi196.stdout:1/371: dread d1/d28/f2d [0,4194304] 0 2024-03-13T14:49:37.833 INFO:tasks.workunit.client.0.smithi176.stdout:5/174: sync 2024-03-13T14:49:37.834 INFO:tasks.workunit.client.0.smithi176.stdout:8/190: rename d0/c2b to d0/db/c38 0 2024-03-13T14:49:37.837 INFO:tasks.workunit.client.0.smithi176.stdout:4/202: mkdir d7/d10/d46/d4a 0 2024-03-13T14:49:37.838 INFO:tasks.workunit.client.1.smithi196.stdout:7/441: unlink d0/d6/f23 0 2024-03-13T14:49:37.839 INFO:tasks.workunit.client.0.smithi176.stdout:5/175: chown d5/d22/f1e 0 1 2024-03-13T14:49:37.842 INFO:tasks.workunit.client.0.smithi176.stdout:3/200: symlink d4/d28/d34/d38/l42 0 2024-03-13T14:49:37.842 INFO:tasks.workunit.client.0.smithi176.stdout:9/233: dwrite d0/d40/f47 [0,4194304] 0 2024-03-13T14:49:37.843 INFO:tasks.workunit.client.0.smithi176.stdout:7/203: dread d1/f19 [0,4194304] 0 2024-03-13T14:49:37.845 INFO:tasks.workunit.client.0.smithi176.stdout:2/223: dwrite d1/dc/d2d/d38/f3f [0,4194304] 0 2024-03-13T14:49:37.845 INFO:tasks.workunit.client.0.smithi176.stdout:5/176: dread - d5/d22/f29 zero size 2024-03-13T14:49:37.846 INFO:tasks.workunit.client.0.smithi176.stdout:1/202: dwrite d2/de/d10/d14/d22/f2b [0,4194304] 0 2024-03-13T14:49:37.848 INFO:tasks.workunit.client.1.smithi196.stdout:1/372: dwrite d1/d9/d33/d4b/f48 [0,4194304] 0 2024-03-13T14:49:37.848 INFO:tasks.workunit.client.0.smithi176.stdout:9/234: write d0/d1e/d1f/f41 [177953,19662] 0 2024-03-13T14:49:37.848 INFO:tasks.workunit.client.1.smithi196.stdout:6/491: mknod d6/d4b/c98 0 2024-03-13T14:49:37.851 INFO:tasks.workunit.client.0.smithi176.stdout:9/235: read - d0/db/f31 zero size 2024-03-13T14:49:37.852 INFO:tasks.workunit.client.1.smithi196.stdout:6/492: dread - d6/d7/df/d27/d39/d44/d74/f87 zero size 2024-03-13T14:49:37.854 INFO:tasks.workunit.client.0.smithi176.stdout:2/224: truncate d1/dc/d2d/d38/f46 978324 0 2024-03-13T14:49:37.854 INFO:tasks.workunit.client.0.smithi176.stdout:6/198: getdents d0 0 2024-03-13T14:49:37.854 INFO:tasks.workunit.client.0.smithi176.stdout:0/170: creat d2/db/f2c x:0 0 0 2024-03-13T14:49:37.855 INFO:tasks.workunit.client.1.smithi196.stdout:7/442: rename d0/d9/d1e/l5f to d0/d9/d31/d46/d47/l8e 0 2024-03-13T14:49:37.855 INFO:tasks.workunit.client.0.smithi176.stdout:6/199: stat d0/d10/d19/f1a 0 2024-03-13T14:49:37.856 INFO:tasks.workunit.client.0.smithi176.stdout:6/200: fdatasync d0/de/f3c 0 2024-03-13T14:49:37.859 INFO:tasks.workunit.client.1.smithi196.stdout:1/373: creat d1/d9/d33/f82 x:0 0 0 2024-03-13T14:49:37.864 INFO:tasks.workunit.client.1.smithi196.stdout:1/374: write d1/d9/d33/d4b/f5f [286748,61502] 0 2024-03-13T14:49:37.865 INFO:tasks.workunit.client.1.smithi196.stdout:0/399: sync 2024-03-13T14:49:37.865 INFO:tasks.workunit.client.0.smithi176.stdout:4/203: rmdir d7/d8/d15 39 2024-03-13T14:49:37.870 INFO:tasks.workunit.client.1.smithi196.stdout:1/375: chown d1/d9/f36 1306457607 1 2024-03-13T14:49:37.873 INFO:tasks.workunit.client.0.smithi176.stdout:2/225: unlink d1/dc/d2d/l3b 0 2024-03-13T14:49:37.873 INFO:tasks.workunit.client.0.smithi176.stdout:1/203: write d2/d29/f3c [2792433,130781] 0 2024-03-13T14:49:37.875 INFO:tasks.workunit.client.0.smithi176.stdout:6/201: symlink d0/d10/l44 0 2024-03-13T14:49:37.875 INFO:tasks.workunit.client.0.smithi176.stdout:3/201: mknod d4/d28/d31/c43 0 2024-03-13T14:49:37.876 INFO:tasks.workunit.client.1.smithi196.stdout:1/376: chown d1/d3f/d5b/d74/f65 38 1 2024-03-13T14:49:37.878 INFO:tasks.workunit.client.0.smithi176.stdout:0/171: dwrite d2/f11 [0,4194304] 0 2024-03-13T14:49:37.880 INFO:tasks.workunit.client.0.smithi176.stdout:0/172: rename d2/d5 to d2/d5/dc/d20/d2d 22 2024-03-13T14:49:37.880 INFO:tasks.workunit.client.0.smithi176.stdout:7/204: symlink d1/l34 0 2024-03-13T14:49:37.881 INFO:tasks.workunit.client.0.smithi176.stdout:5/177: fdatasync d5/f7 0 2024-03-13T14:49:37.882 INFO:tasks.workunit.client.1.smithi196.stdout:1/377: chown d1/d9/d6c/d79/l81 1 1 2024-03-13T14:49:37.885 INFO:tasks.workunit.client.1.smithi196.stdout:1/378: chown d1/d9/d33/d4b/f22 19248 1 2024-03-13T14:49:37.888 INFO:tasks.workunit.client.1.smithi196.stdout:7/443: dwrite d0/f76 [0,4194304] 0 2024-03-13T14:49:37.888 INFO:tasks.workunit.client.1.smithi196.stdout:6/493: dread d6/d7/df/f35 [0,4194304] 0 2024-03-13T14:49:37.890 INFO:tasks.workunit.client.0.smithi176.stdout:1/204: dread d2/fc [0,4194304] 0 2024-03-13T14:49:37.891 INFO:tasks.workunit.client.1.smithi196.stdout:0/400: rename d8/da/d25/d26/d35/d3b/d5f/d43 to d8/da/d25/d26/d54/d88 0 2024-03-13T14:49:37.893 INFO:tasks.workunit.client.0.smithi176.stdout:3/202: dwrite d4/f1d [0,4194304] 0 2024-03-13T14:49:37.900 INFO:tasks.workunit.client.1.smithi196.stdout:7/444: fdatasync d0/d6/f45 0 2024-03-13T14:49:37.901 INFO:tasks.workunit.client.0.smithi176.stdout:4/204: creat d7/d10/d46/d4a/f4b x:0 0 0 2024-03-13T14:49:37.902 INFO:tasks.workunit.client.0.smithi176.stdout:4/205: chown d7/d10 381985 1 2024-03-13T14:49:37.907 INFO:tasks.workunit.client.0.smithi176.stdout:6/202: unlink d0/f13 0 2024-03-13T14:49:37.910 INFO:tasks.workunit.client.0.smithi176.stdout:3/203: dread f1 [0,4194304] 0 2024-03-13T14:49:37.912 INFO:tasks.workunit.client.0.smithi176.stdout:3/204: write d4/d28/d2e/f3c [711450,85384] 0 2024-03-13T14:49:37.913 INFO:tasks.workunit.client.1.smithi196.stdout:7/445: mknod d0/d9/d1e/c8f 0 2024-03-13T14:49:37.916 INFO:tasks.workunit.client.0.smithi176.stdout:0/173: creat d2/d5/dc/f2e x:0 0 0 2024-03-13T14:49:37.917 INFO:tasks.workunit.client.1.smithi196.stdout:7/446: dread - d0/d9/d31/d50/f75 zero size 2024-03-13T14:49:37.918 INFO:tasks.workunit.client.1.smithi196.stdout:6/494: mknod d6/d7/c99 0 2024-03-13T14:49:37.922 INFO:tasks.workunit.client.0.smithi176.stdout:5/178: chown d5/f17 2870 1 2024-03-13T14:49:37.922 INFO:tasks.workunit.client.0.smithi176.stdout:5/179: chown d5/d22 363 1 2024-03-13T14:49:37.925 INFO:tasks.workunit.client.1.smithi196.stdout:7/447: mknod d0/d9/d31/d50/c90 0 2024-03-13T14:49:37.926 INFO:tasks.workunit.client.1.smithi196.stdout:6/495: mknod d6/d2a/dc/d1c/d34/c9a 0 2024-03-13T14:49:37.928 INFO:tasks.workunit.client.0.smithi176.stdout:8/191: chown d0/db/c38 342 1 2024-03-13T14:49:37.930 INFO:tasks.workunit.client.0.smithi176.stdout:8/192: write d0/d24/d29/f36 [155305,36935] 0 2024-03-13T14:49:37.931 INFO:tasks.workunit.client.0.smithi176.stdout:8/193: stat d0/d5/d11/f33 0 2024-03-13T14:49:37.932 INFO:tasks.workunit.client.1.smithi196.stdout:7/448: creat d0/d4d/f91 x:0 0 0 2024-03-13T14:49:37.933 INFO:tasks.workunit.client.0.smithi176.stdout:4/206: creat d7/d10/d46/f4c x:0 0 0 2024-03-13T14:49:37.935 INFO:tasks.workunit.client.1.smithi196.stdout:6/496: mknod d6/d2a/d23/d28/c9b 0 2024-03-13T14:49:37.940 INFO:tasks.workunit.client.0.smithi176.stdout:2/226: mknod d1/dc/d2d/d4f/c50 0 2024-03-13T14:49:37.951 INFO:tasks.workunit.client.1.smithi196.stdout:7/449: dwrite d0/d9/d1e/d3f/f82 [0,4194304] 0 2024-03-13T14:49:37.958 INFO:tasks.workunit.client.0.smithi176.stdout:5/180: rename d5/f17 to d5/d31/f3e 0 2024-03-13T14:49:37.961 INFO:tasks.workunit.client.0.smithi176.stdout:9/236: getdents d0/d13/d32 0 2024-03-13T14:49:37.963 INFO:tasks.workunit.client.0.smithi176.stdout:8/194: rename d0/db/d27 to d0/db/d39 0 2024-03-13T14:49:37.966 INFO:tasks.workunit.client.0.smithi176.stdout:9/237: write d0/d1e/d1f/f2c [1800998,111817] 0 2024-03-13T14:49:37.968 INFO:tasks.workunit.client.1.smithi196.stdout:6/497: dread d6/d7/dd/f2c [0,4194304] 0 2024-03-13T14:49:37.970 INFO:tasks.workunit.client.0.smithi176.stdout:4/207: write d7/d8/d15/f3e [2132314,4853] 0 2024-03-13T14:49:37.971 INFO:tasks.workunit.client.1.smithi196.stdout:7/450: creat d0/d9/f92 x:0 0 0 2024-03-13T14:49:37.972 INFO:tasks.workunit.client.0.smithi176.stdout:9/238: read d0/d1e/f25 [1463841,13689] 0 2024-03-13T14:49:37.974 INFO:tasks.workunit.client.0.smithi176.stdout:2/227: write d1/dc/d1c/f4c [899630,56468] 0 2024-03-13T14:49:37.974 INFO:tasks.workunit.client.0.smithi176.stdout:9/239: dread - d0/db/f31 zero size 2024-03-13T14:49:37.976 INFO:tasks.workunit.client.0.smithi176.stdout:9/240: dread - d0/db/f3f zero size 2024-03-13T14:49:37.977 INFO:tasks.workunit.client.1.smithi196.stdout:6/498: creat d6/d7/df/f9c x:0 0 0 2024-03-13T14:49:37.978 INFO:tasks.workunit.client.1.smithi196.stdout:0/401: write d8/da/f10 [5242948,98058] 0 2024-03-13T14:49:37.983 INFO:tasks.workunit.client.0.smithi176.stdout:8/195: dread d0/db/f25 [0,4194304] 0 2024-03-13T14:49:37.983 INFO:tasks.workunit.client.0.smithi176.stdout:0/174: mknod d2/d5/dc/d2a/c2f 0 2024-03-13T14:49:37.984 INFO:tasks.workunit.client.1.smithi196.stdout:0/402: truncate d8/da/d25/d26/d35/d3b/f7d 170090 0 2024-03-13T14:49:37.985 INFO:tasks.workunit.client.0.smithi176.stdout:0/175: stat d2/d5/dc/d20 0 2024-03-13T14:49:37.986 INFO:tasks.workunit.client.1.smithi196.stdout:6/499: creat d6/d7/df/d27/f9d x:0 0 0 2024-03-13T14:49:37.989 INFO:tasks.workunit.client.0.smithi176.stdout:4/208: dwrite d7/d8/d15/d23/f35 [0,4194304] 0 2024-03-13T14:49:37.989 INFO:tasks.workunit.client.1.smithi196.stdout:7/451: mkdir d0/d9/d31/d77/d93 0 2024-03-13T14:49:37.992 INFO:tasks.workunit.client.1.smithi196.stdout:6/500: rmdir d6/d7/dd/d60 39 2024-03-13T14:49:37.995 INFO:tasks.workunit.client.0.smithi176.stdout:5/181: unlink d5/d2c/f38 0 2024-03-13T14:49:37.995 INFO:tasks.workunit.client.1.smithi196.stdout:0/403: dread d8/da/d25/d26/d54/d88/f45 [0,4194304] 0 2024-03-13T14:49:38.003 INFO:tasks.workunit.client.1.smithi196.stdout:0/404: mknod d8/da/d25/d26/d48/c89 0 2024-03-13T14:49:38.006 INFO:tasks.workunit.client.1.smithi196.stdout:7/452: mkdir d0/d9/d31/d77/d93/d94 0 2024-03-13T14:49:38.009 INFO:tasks.workunit.client.1.smithi196.stdout:6/501: mkdir d6/d57/d7a/d9e 0 2024-03-13T14:49:38.011 INFO:tasks.workunit.client.0.smithi176.stdout:2/228: mkdir d1/d51 0 2024-03-13T14:49:38.012 INFO:tasks.workunit.client.0.smithi176.stdout:1/205: dwrite d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:49:38.014 INFO:tasks.workunit.client.0.smithi176.stdout:7/205: truncate d1/d28/f2a 386182 0 2024-03-13T14:49:38.015 INFO:tasks.workunit.client.1.smithi196.stdout:0/405: dread d8/d37/f73 [0,4194304] 0 2024-03-13T14:49:38.015 INFO:tasks.workunit.client.1.smithi196.stdout:7/453: truncate d0/d6/f21 150690 0 2024-03-13T14:49:38.018 INFO:tasks.workunit.client.0.smithi176.stdout:9/241: rename d0/f10 to d0/d17/f48 0 2024-03-13T14:49:38.019 INFO:tasks.workunit.client.1.smithi196.stdout:6/502: chown d6/d2a/f62 2274 1 2024-03-13T14:49:38.020 INFO:tasks.workunit.client.0.smithi176.stdout:9/242: write d0/d1e/d1f/f38 [683913,130128] 0 2024-03-13T14:49:38.021 INFO:tasks.workunit.client.0.smithi176.stdout:8/196: symlink d0/d5/l3a 0 2024-03-13T14:49:38.024 INFO:tasks.workunit.client.0.smithi176.stdout:0/176: symlink d2/l30 0 2024-03-13T14:49:38.026 INFO:tasks.workunit.client.1.smithi196.stdout:7/454: rmdir d0/d9/d31/d77/d93 39 2024-03-13T14:49:38.028 INFO:tasks.workunit.client.0.smithi176.stdout:4/209: readlink d7/d2d/l36 0 2024-03-13T14:49:38.029 INFO:tasks.workunit.client.1.smithi196.stdout:6/503: dread - d6/d9/d17/d19/d4e/f6e zero size 2024-03-13T14:49:38.034 INFO:tasks.workunit.client.0.smithi176.stdout:3/205: getdents d4/d13/d2d 0 2024-03-13T14:49:38.034 INFO:tasks.workunit.client.1.smithi196.stdout:6/504: chown d6/d7 388946 1 2024-03-13T14:49:38.036 INFO:tasks.workunit.client.0.smithi176.stdout:2/229: rmdir d1/dc 39 2024-03-13T14:49:38.038 INFO:tasks.workunit.client.1.smithi196.stdout:7/455: dread d0/d9/d1e/d58/d72/d6e/f70 [0,4194304] 0 2024-03-13T14:49:38.038 INFO:tasks.workunit.client.1.smithi196.stdout:6/505: fsync d6/d9/d17/d19/f2d 0 2024-03-13T14:49:38.039 INFO:tasks.workunit.client.0.smithi176.stdout:1/206: truncate d2/f6 3894087 0 2024-03-13T14:49:38.042 INFO:tasks.workunit.client.0.smithi176.stdout:7/206: rename d1/d2/l12 to d1/d2/l35 0 2024-03-13T14:49:38.047 INFO:tasks.workunit.client.1.smithi196.stdout:6/506: mkdir d6/d57/d7a/d9f 0 2024-03-13T14:49:38.049 INFO:tasks.workunit.client.0.smithi176.stdout:8/197: unlink d0/db/d39/f34 0 2024-03-13T14:49:38.050 INFO:tasks.workunit.client.0.smithi176.stdout:0/177: mkdir d2/db/d31 0 2024-03-13T14:49:38.052 INFO:tasks.workunit.client.0.smithi176.stdout:0/178: truncate d2/db/f2c 6197 0 2024-03-13T14:49:38.053 INFO:tasks.workunit.client.0.smithi176.stdout:0/179: chown d2/f11 37410 1 2024-03-13T14:49:38.055 INFO:tasks.workunit.client.0.smithi176.stdout:4/210: symlink d7/d10/d2c/l4d 0 2024-03-13T14:49:38.056 INFO:tasks.workunit.client.1.smithi196.stdout:0/406: write d8/da/d25/d26/d35/d3b/d5f/f4a [901220,8810] 0 2024-03-13T14:49:38.059 INFO:tasks.workunit.client.0.smithi176.stdout:1/207: unlink d2/de/d10/d14/d35/c3b 0 2024-03-13T14:49:38.062 INFO:tasks.workunit.client.1.smithi196.stdout:0/407: symlink d8/da/d25/l8a 0 2024-03-13T14:49:38.064 INFO:tasks.workunit.client.0.smithi176.stdout:8/198: mkdir d0/d5/d3b 0 2024-03-13T14:49:38.065 INFO:tasks.workunit.client.1.smithi196.stdout:7/456: write d0/d9/f27 [3071980,52798] 0 2024-03-13T14:49:38.066 INFO:tasks.workunit.client.0.smithi176.stdout:8/199: dread - d0/d24/f37 zero size 2024-03-13T14:49:38.068 INFO:tasks.workunit.client.0.smithi176.stdout:3/206: mknod d4/d37/c44 0 2024-03-13T14:49:38.074 INFO:tasks.workunit.client.1.smithi196.stdout:7/457: creat d0/d6/d33/d57/f95 x:0 0 0 2024-03-13T14:49:38.081 INFO:tasks.workunit.client.1.smithi196.stdout:0/408: dwrite d8/da/d25/f7b [0,4194304] 0 2024-03-13T14:49:38.084 INFO:tasks.workunit.client.1.smithi196.stdout:6/507: dwrite d6/d4b/f65 [0,4194304] 0 2024-03-13T14:49:38.086 INFO:tasks.workunit.client.1.smithi196.stdout:7/458: creat d0/d6/d33/f96 x:0 0 0 2024-03-13T14:49:38.087 INFO:tasks.workunit.client.1.smithi196.stdout:6/508: chown d6/d7/df/d27/d39/c7f 97871517 1 2024-03-13T14:49:38.090 INFO:tasks.workunit.client.1.smithi196.stdout:6/509: readlink d6/d2a/dc/l21 0 2024-03-13T14:49:38.092 INFO:tasks.workunit.client.1.smithi196.stdout:0/409: creat d8/da/d25/d26/d48/d82/f8b x:0 0 0 2024-03-13T14:49:38.092 INFO:tasks.workunit.client.1.smithi196.stdout:4/327: sync 2024-03-13T14:49:38.092 INFO:tasks.workunit.client.1.smithi196.stdout:3/360: sync 2024-03-13T14:49:38.093 INFO:tasks.workunit.client.1.smithi196.stdout:9/306: sync 2024-03-13T14:49:38.093 INFO:tasks.workunit.client.1.smithi196.stdout:8/304: sync 2024-03-13T14:49:38.093 INFO:tasks.workunit.client.1.smithi196.stdout:5/459: sync 2024-03-13T14:49:38.093 INFO:tasks.workunit.client.1.smithi196.stdout:2/248: sync 2024-03-13T14:49:38.100 INFO:tasks.workunit.client.1.smithi196.stdout:4/328: truncate d1/f54 945609 0 2024-03-13T14:49:38.100 INFO:tasks.workunit.client.1.smithi196.stdout:0/410: readlink d8/da/d25/d26/d35/d3b/d5f/d17/l78 0 2024-03-13T14:49:38.100 INFO:tasks.workunit.client.1.smithi196.stdout:2/249: symlink d0/d7/d40/l52 0 2024-03-13T14:49:38.103 INFO:tasks.workunit.client.1.smithi196.stdout:3/361: rmdir db/d19/d22/d3a/d53 39 2024-03-13T14:49:38.106 INFO:tasks.workunit.client.1.smithi196.stdout:4/329: read - d1/d3/d4c/f57 zero size 2024-03-13T14:49:38.106 INFO:tasks.workunit.client.1.smithi196.stdout:9/307: creat d7/de/d16/f60 x:0 0 0 2024-03-13T14:49:38.109 INFO:tasks.workunit.client.1.smithi196.stdout:5/460: readlink d4/db/d14/d19/l44 0 2024-03-13T14:49:38.110 INFO:tasks.workunit.client.1.smithi196.stdout:3/362: fsync db/d19/d22/d30/f42 0 2024-03-13T14:49:38.115 INFO:tasks.workunit.client.1.smithi196.stdout:4/330: symlink d1/d1e/d43/d5b/l60 0 2024-03-13T14:49:38.117 INFO:tasks.workunit.client.1.smithi196.stdout:9/308: creat d7/de/f61 x:0 0 0 2024-03-13T14:49:38.118 INFO:tasks.workunit.client.1.smithi196.stdout:5/461: creat d4/db/d14/d19/d26/d33/f8f x:0 0 0 2024-03-13T14:49:38.119 INFO:tasks.workunit.client.1.smithi196.stdout:3/363: fdatasync db/d19/d50/f58 0 2024-03-13T14:49:38.122 INFO:tasks.workunit.client.1.smithi196.stdout:4/331: write d1/d1e/d43/d5b/d55/f5e [170986,102148] 0 2024-03-13T14:49:38.124 INFO:tasks.workunit.client.1.smithi196.stdout:6/510: dwrite d6/d9/f7b [4194304,4194304] 0 2024-03-13T14:49:38.126 INFO:tasks.workunit.client.1.smithi196.stdout:2/250: dread d0/d6/d1d/f51 [0,4194304] 0 2024-03-13T14:49:38.127 INFO:tasks.workunit.client.1.smithi196.stdout:4/332: fsync d1/d1e/d5c/f5f 0 2024-03-13T14:49:38.135 INFO:tasks.workunit.client.1.smithi196.stdout:5/462: truncate d4/db/d14/d19/d26/f43 1339447 0 2024-03-13T14:49:38.135 INFO:tasks.workunit.client.1.smithi196.stdout:2/251: chown d0/d6/l12 179626308 1 2024-03-13T14:49:38.138 INFO:tasks.workunit.client.1.smithi196.stdout:3/364: link db/f32 db/d37/d64/f74 0 2024-03-13T14:49:38.140 INFO:tasks.workunit.client.1.smithi196.stdout:4/333: fsync d1/d3/d7/f1d 0 2024-03-13T14:49:38.146 INFO:tasks.workunit.client.1.smithi196.stdout:3/365: read db/f2d [2821583,1966] 0 2024-03-13T14:49:38.148 INFO:tasks.workunit.client.1.smithi196.stdout:1/379: sync 2024-03-13T14:49:38.149 INFO:tasks.workunit.client.1.smithi196.stdout:7/459: dwrite d0/f6a [0,4194304] 0 2024-03-13T14:49:38.153 INFO:tasks.workunit.client.1.smithi196.stdout:1/380: write d1/d38/f30 [1136275,9645] 0 2024-03-13T14:49:38.155 INFO:tasks.workunit.client.1.smithi196.stdout:5/463: mkdir d4/db/d14/d19/d26/d33/d90 0 2024-03-13T14:49:38.155 INFO:tasks.workunit.client.1.smithi196.stdout:2/252: mknod d0/d6/c53 0 2024-03-13T14:49:38.157 INFO:tasks.workunit.client.1.smithi196.stdout:4/334: dread d1/d3/d7/f1d [0,4194304] 0 2024-03-13T14:49:38.158 INFO:tasks.workunit.client.1.smithi196.stdout:6/511: link d6/d2a/dc/d47/c59 d6/d2a/dc/d1c/ca0 0 2024-03-13T14:49:38.159 INFO:tasks.workunit.client.1.smithi196.stdout:3/366: truncate db/f3e 858543 0 2024-03-13T14:49:38.160 INFO:tasks.workunit.client.1.smithi196.stdout:2/253: dread - d0/d7/f3f zero size 2024-03-13T14:49:38.162 INFO:tasks.workunit.client.1.smithi196.stdout:4/335: stat d1/d1e/d43/d5b/d55 0 2024-03-13T14:49:38.164 INFO:tasks.workunit.client.1.smithi196.stdout:7/460: mkdir d0/d6/d33/d97 0 2024-03-13T14:49:38.166 INFO:tasks.workunit.client.1.smithi196.stdout:8/305: dread d0/d10/d12/f3e [0,4194304] 0 2024-03-13T14:49:38.170 INFO:tasks.workunit.client.1.smithi196.stdout:5/464: mkdir d4/db/d3c/d6a/d91 0 2024-03-13T14:49:38.173 INFO:tasks.workunit.client.1.smithi196.stdout:1/381: rmdir d1/d9/d33/d4b/d66 39 2024-03-13T14:49:38.176 INFO:tasks.workunit.client.1.smithi196.stdout:8/306: rmdir d0/d1c/d29/d47 39 2024-03-13T14:49:38.182 INFO:tasks.workunit.client.1.smithi196.stdout:3/367: mknod db/d19/d22/d3a/c75 0 2024-03-13T14:49:38.182 INFO:tasks.workunit.client.0.smithi176.stdout:0/180: mknod d2/d5/dc/d20/c32 0 2024-03-13T14:49:38.183 INFO:tasks.workunit.client.1.smithi196.stdout:2/254: dread d0/d7/f21 [0,4194304] 0 2024-03-13T14:49:38.183 INFO:tasks.workunit.client.0.smithi176.stdout:8/200: read d0/d5/d11/f1b [344901,94803] 0 2024-03-13T14:49:38.185 INFO:tasks.workunit.client.0.smithi176.stdout:1/208: rename d2/dd to d2/de/d3f 0 2024-03-13T14:49:38.185 INFO:tasks.workunit.client.0.smithi176.stdout:9/243: creat d0/db/f49 x:0 0 0 2024-03-13T14:49:38.187 INFO:tasks.workunit.client.1.smithi196.stdout:2/255: fdatasync d0/f1a 0 2024-03-13T14:49:38.188 INFO:tasks.workunit.client.0.smithi176.stdout:5/182: getdents d5/d2c 0 2024-03-13T14:49:38.190 INFO:tasks.workunit.client.0.smithi176.stdout:6/203: getdents d0/de/d38 0 2024-03-13T14:49:38.190 INFO:tasks.workunit.client.0.smithi176.stdout:4/211: link d7/d10/d46/d4a/f4b d7/d10/d46/f4e 0 2024-03-13T14:49:38.190 INFO:tasks.workunit.client.1.smithi196.stdout:4/336: dwrite d1/d3/d7/f1a [8388608,4194304] 0 2024-03-13T14:49:38.191 INFO:tasks.workunit.client.1.smithi196.stdout:9/309: link d7/de/d4b/d1d/f54 d7/f62 0 2024-03-13T14:49:38.191 INFO:tasks.workunit.client.0.smithi176.stdout:1/209: read d2/f9 [3950601,97262] 0 2024-03-13T14:49:38.193 INFO:tasks.workunit.client.0.smithi176.stdout:4/212: fdatasync d7/d10/f44 0 2024-03-13T14:49:38.194 INFO:tasks.workunit.client.0.smithi176.stdout:6/204: chown d0/de/f12 2731936 1 2024-03-13T14:49:38.195 INFO:tasks.workunit.client.1.smithi196.stdout:3/368: creat db/d37/f76 x:0 0 0 2024-03-13T14:49:38.196 INFO:tasks.workunit.client.1.smithi196.stdout:7/461: link d0/d9/d1e/d58/d72/c32 d0/d9/d31/d46/d47/c98 0 2024-03-13T14:49:38.197 INFO:tasks.workunit.client.0.smithi176.stdout:9/244: chown d0/db/l7 468655 1 2024-03-13T14:49:38.197 INFO:tasks.workunit.client.0.smithi176.stdout:1/210: dread - d2/d30/f3a zero size 2024-03-13T14:49:38.198 INFO:tasks.workunit.client.0.smithi176.stdout:3/207: rename d4/d37 to d4/d28/d31/d45 0 2024-03-13T14:49:38.200 INFO:tasks.workunit.client.1.smithi196.stdout:0/411: dread d8/da/d25/f27 [0,4194304] 0 2024-03-13T14:49:38.202 INFO:tasks.workunit.client.1.smithi196.stdout:8/307: mkdir d0/d10/d12/d6f 0 2024-03-13T14:49:38.204 INFO:tasks.workunit.client.0.smithi176.stdout:5/183: creat d5/d2e/f3f x:0 0 0 2024-03-13T14:49:38.204 INFO:tasks.workunit.client.0.smithi176.stdout:8/201: creat d0/d5/d3b/f3c x:0 0 0 2024-03-13T14:49:38.205 INFO:tasks.workunit.client.1.smithi196.stdout:6/512: dwrite d6/d7/dd/f86 [0,4194304] 0 2024-03-13T14:49:38.205 INFO:tasks.workunit.client.1.smithi196.stdout:1/382: dwrite d1/d3f/d5b/f69 [0,4194304] 0 2024-03-13T14:49:38.207 INFO:tasks.workunit.client.1.smithi196.stdout:4/337: unlink d1/d1e/f34 0 2024-03-13T14:49:38.207 INFO:tasks.workunit.client.0.smithi176.stdout:6/205: dread - d0/fd zero size 2024-03-13T14:49:38.207 INFO:tasks.workunit.client.0.smithi176.stdout:8/202: stat d0/db/dc/l1d 0 2024-03-13T14:49:38.208 INFO:tasks.workunit.client.0.smithi176.stdout:5/184: write d5/f8 [464342,115725] 0 2024-03-13T14:49:38.210 INFO:tasks.workunit.client.1.smithi196.stdout:1/383: readlink d1/d28/l49 0 2024-03-13T14:49:38.212 INFO:tasks.workunit.client.0.smithi176.stdout:9/245: creat d0/d1e/f4a x:0 0 0 2024-03-13T14:49:38.222 INFO:tasks.workunit.client.1.smithi196.stdout:7/462: mkdir d0/d9/d1e/d58/d72/d99 0 2024-03-13T14:49:38.222 INFO:tasks.workunit.client.1.smithi196.stdout:3/369: readlink db/d19/l34 0 2024-03-13T14:49:38.223 INFO:tasks.workunit.client.1.smithi196.stdout:9/310: mknod d7/de/d16/c63 0 2024-03-13T14:49:38.225 INFO:tasks.workunit.client.1.smithi196.stdout:5/465: dread d4/db/d14/f5e [0,4194304] 0 2024-03-13T14:49:38.226 INFO:tasks.workunit.client.0.smithi176.stdout:6/206: truncate d0/fd 169853 0 2024-03-13T14:49:38.229 INFO:tasks.workunit.client.0.smithi176.stdout:9/246: dread d0/d17/f48 [0,4194304] 0 2024-03-13T14:49:38.230 INFO:tasks.workunit.client.0.smithi176.stdout:9/247: stat d0/db/c45 0 2024-03-13T14:49:38.231 INFO:tasks.workunit.client.1.smithi196.stdout:0/412: symlink d8/da/d25/d26/d35/d3b/d50/d55/l8c 0 2024-03-13T14:49:38.232 INFO:tasks.workunit.client.0.smithi176.stdout:5/185: dread d5/d22/f13 [0,4194304] 0 2024-03-13T14:49:38.232 INFO:tasks.workunit.client.1.smithi196.stdout:5/466: stat d4/db/d14/d19/d1c/d25/d38/d47/l7f 0 2024-03-13T14:49:38.234 INFO:tasks.workunit.client.1.smithi196.stdout:0/413: stat d8/da/d25/d26/d48 0 2024-03-13T14:49:38.234 INFO:tasks.workunit.client.0.smithi176.stdout:0/181: getdents d2/d5/dc/d2a 0 2024-03-13T14:49:38.235 INFO:tasks.workunit.client.1.smithi196.stdout:5/467: readlink d4/db/d3c/d6a/l76 0 2024-03-13T14:49:38.238 INFO:tasks.workunit.client.1.smithi196.stdout:6/513: rmdir d6/d2a/dc 39 2024-03-13T14:49:38.240 INFO:tasks.workunit.client.0.smithi176.stdout:3/208: dread d4/d28/f29 [0,4194304] 0 2024-03-13T14:49:38.240 INFO:tasks.workunit.client.1.smithi196.stdout:8/308: getdents d0/d10/d12/d6f 0 2024-03-13T14:49:38.241 INFO:tasks.workunit.client.1.smithi196.stdout:1/384: dwrite d1/d9/d4e/d55/f6e [0,4194304] 0 2024-03-13T14:49:38.241 INFO:tasks.workunit.client.1.smithi196.stdout:5/468: fsync d4/db/d14/d19/d26/f51 0 2024-03-13T14:49:38.242 INFO:tasks.workunit.client.0.smithi176.stdout:4/213: creat d7/d8/f4f x:0 0 0 2024-03-13T14:49:38.244 INFO:tasks.workunit.client.0.smithi176.stdout:4/214: read - d7/d8/f3b zero size 2024-03-13T14:49:38.246 INFO:tasks.workunit.client.1.smithi196.stdout:5/469: write d4/db/fd [811355,40245] 0 2024-03-13T14:49:38.253 INFO:tasks.workunit.client.0.smithi176.stdout:6/207: chown d0/f18 14422228 1 2024-03-13T14:49:38.254 INFO:tasks.workunit.client.1.smithi196.stdout:7/463: rename d0/d9/d1e/d58/d72/f8c to d0/d6/d33/f9a 0 2024-03-13T14:49:38.254 INFO:tasks.workunit.client.1.smithi196.stdout:0/414: dread d8/da/d25/d26/d35/d3b/d50/d5d/f57 [0,4194304] 0 2024-03-13T14:49:38.263 INFO:tasks.workunit.client.0.smithi176.stdout:4/215: dwrite d7/d8/f39 [0,4194304] 0 2024-03-13T14:49:38.265 INFO:tasks.workunit.client.0.smithi176.stdout:9/248: dread d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:38.285 INFO:tasks.workunit.client.1.smithi196.stdout:2/256: link d0/d7/d9/c26 d0/d7/d9/d42/c54 0 2024-03-13T14:49:38.288 INFO:tasks.workunit.client.1.smithi196.stdout:4/338: rmdir d1/d4b 0 2024-03-13T14:49:38.292 INFO:tasks.workunit.client.1.smithi196.stdout:6/514: rename d6/d4b to d6/d9/d17/da1 0 2024-03-13T14:49:38.297 INFO:tasks.workunit.client.1.smithi196.stdout:8/309: mkdir d0/d10/d70 0 2024-03-13T14:49:38.298 INFO:tasks.workunit.client.0.smithi176.stdout:1/211: write d2/f9 [2744826,110634] 0 2024-03-13T14:49:38.302 INFO:tasks.workunit.client.1.smithi196.stdout:1/385: mknod d1/d9/c83 0 2024-03-13T14:49:38.302 INFO:tasks.workunit.client.0.smithi176.stdout:8/203: write d0/db/f19 [1279734,12874] 0 2024-03-13T14:49:38.305 INFO:tasks.workunit.client.0.smithi176.stdout:6/208: mknod d0/d9/c45 0 2024-03-13T14:49:38.305 INFO:tasks.workunit.client.0.smithi176.stdout:9/249: dwrite d0/d17/f3a [0,4194304] 0 2024-03-13T14:49:38.312 INFO:tasks.workunit.client.1.smithi196.stdout:3/370: write db/d37/d64/f74 [4409935,117280] 0 2024-03-13T14:49:38.313 INFO:tasks.workunit.client.1.smithi196.stdout:6/515: unlink d6/d7/dd/f2c 0 2024-03-13T14:49:38.316 INFO:tasks.workunit.client.0.smithi176.stdout:1/212: dread d2/d29/f3c [0,4194304] 0 2024-03-13T14:49:38.321 INFO:tasks.workunit.client.0.smithi176.stdout:8/204: unlink d0/db/dc/l21 0 2024-03-13T14:49:38.323 INFO:tasks.workunit.client.0.smithi176.stdout:8/205: stat d0/db/dc/l2f 0 2024-03-13T14:49:38.325 INFO:tasks.workunit.client.0.smithi176.stdout:3/209: dwrite f1 [0,4194304] 0 2024-03-13T14:49:38.325 INFO:tasks.workunit.client.1.smithi196.stdout:5/470: dread d4/db/d14/d37/d41/f42 [0,4194304] 0 2024-03-13T14:49:38.326 INFO:tasks.workunit.client.0.smithi176.stdout:3/210: read f1 [3603840,79459] 0 2024-03-13T14:49:38.328 INFO:tasks.workunit.client.1.smithi196.stdout:4/339: symlink d1/d3/d7/d35/d3e/d56/l61 0 2024-03-13T14:49:38.329 INFO:tasks.workunit.client.0.smithi176.stdout:5/186: truncate d5/f6 3057763 0 2024-03-13T14:49:38.332 INFO:tasks.workunit.client.0.smithi176.stdout:9/250: mknod d0/d40/c4b 0 2024-03-13T14:49:38.335 INFO:tasks.workunit.client.1.smithi196.stdout:2/257: rename d0/d7/l19 to d0/d50/l55 0 2024-03-13T14:49:38.335 INFO:tasks.workunit.client.1.smithi196.stdout:9/311: dwrite d7/f12 [0,4194304] 0 2024-03-13T14:49:38.339 INFO:tasks.workunit.client.0.smithi176.stdout:4/216: creat d7/f50 x:0 0 0 2024-03-13T14:49:38.342 INFO:tasks.workunit.client.1.smithi196.stdout:7/464: dwrite d0/d6/f19 [0,4194304] 0 2024-03-13T14:49:38.342 INFO:tasks.workunit.client.0.smithi176.stdout:1/213: unlink d2/de/d10/d2e/c31 0 2024-03-13T14:49:38.344 INFO:tasks.workunit.client.1.smithi196.stdout:0/415: dwrite d8/da/d25/d26/f2d [0,4194304] 0 2024-03-13T14:49:38.345 INFO:tasks.workunit.client.0.smithi176.stdout:2/230: sync 2024-03-13T14:49:38.349 INFO:tasks.workunit.client.1.smithi196.stdout:1/386: symlink d1/d9/d4e/d55/d63/l84 0 2024-03-13T14:49:38.351 INFO:tasks.workunit.client.1.smithi196.stdout:8/310: dread d0/d1c/d3b/d3f/f45 [0,4194304] 0 2024-03-13T14:49:38.353 INFO:tasks.workunit.client.0.smithi176.stdout:3/211: creat d4/d28/d2e/f46 x:0 0 0 2024-03-13T14:49:38.353 INFO:tasks.workunit.client.0.smithi176.stdout:9/251: dwrite d0/d40/f47 [0,4194304] 0 2024-03-13T14:49:38.353 INFO:tasks.workunit.client.0.smithi176.stdout:5/187: mkdir d5/d2c/d40 0 2024-03-13T14:49:38.356 INFO:tasks.workunit.client.1.smithi196.stdout:3/371: rename db/d15/d44/d51/l5f to db/d19/d22/d3a/d6a/l77 0 2024-03-13T14:49:38.359 INFO:tasks.workunit.client.0.smithi176.stdout:3/212: fsync d4/d28/d34/f3f 0 2024-03-13T14:49:38.359 INFO:tasks.workunit.client.1.smithi196.stdout:3/372: write db/d4c/f61 [981768,18053] 0 2024-03-13T14:49:38.360 INFO:tasks.workunit.client.1.smithi196.stdout:5/471: dwrite d4/db/d4a/f8a [0,4194304] 0 2024-03-13T14:49:38.363 INFO:tasks.workunit.client.1.smithi196.stdout:9/312: symlink d7/d24/d28/l64 0 2024-03-13T14:49:38.364 INFO:tasks.workunit.client.0.smithi176.stdout:3/213: chown d4/d28/d2e/f46 482109 1 2024-03-13T14:49:38.368 INFO:tasks.workunit.client.0.smithi176.stdout:4/217: dread d7/d2d/f2e [0,4194304] 0 2024-03-13T14:49:38.371 INFO:tasks.workunit.client.1.smithi196.stdout:4/340: mknod d1/d1e/d39/d58/c62 0 2024-03-13T14:49:38.371 INFO:tasks.workunit.client.0.smithi176.stdout:4/218: write d7/d8/f18 [3291433,17461] 0 2024-03-13T14:49:38.373 INFO:tasks.workunit.client.1.smithi196.stdout:6/516: write d6/d9/d11/f5e [266413,79187] 0 2024-03-13T14:49:38.376 INFO:tasks.workunit.client.0.smithi176.stdout:4/219: dread - d7/d8/d15/d23/f47 zero size 2024-03-13T14:49:38.379 INFO:tasks.workunit.client.0.smithi176.stdout:6/209: getdents d0 0 2024-03-13T14:49:38.381 INFO:tasks.workunit.client.1.smithi196.stdout:1/387: mkdir d1/d3f/d5b/d74/d85 0 2024-03-13T14:49:38.381 INFO:tasks.workunit.client.0.smithi176.stdout:1/214: dwrite d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:49:38.384 INFO:tasks.workunit.client.1.smithi196.stdout:0/416: creat d8/da/d25/d26/d35/d3b/d59/d6a/f8d x:0 0 0 2024-03-13T14:49:38.386 INFO:tasks.workunit.client.0.smithi176.stdout:6/210: dread - d0/d10/d19/d27/d2b/f39 zero size 2024-03-13T14:49:38.387 INFO:tasks.workunit.client.1.smithi196.stdout:3/373: unlink f4 0 2024-03-13T14:49:38.389 INFO:tasks.workunit.client.0.smithi176.stdout:9/252: fsync d0/d17/f48 0 2024-03-13T14:49:38.392 INFO:tasks.workunit.client.0.smithi176.stdout:9/253: rename d0 to d0/d17/d4c 22 2024-03-13T14:49:38.395 INFO:tasks.workunit.client.1.smithi196.stdout:5/472: creat d4/db/d14/d19/d26/f92 x:0 0 0 2024-03-13T14:49:38.397 INFO:tasks.workunit.client.1.smithi196.stdout:2/258: dread d0/f1a [0,4194304] 0 2024-03-13T14:49:38.402 INFO:tasks.workunit.client.0.smithi176.stdout:7/207: sync 2024-03-13T14:49:38.402 INFO:tasks.workunit.client.0.smithi176.stdout:0/182: sync 2024-03-13T14:49:38.404 INFO:tasks.workunit.client.0.smithi176.stdout:8/206: truncate d0/db/f19 3210367 0 2024-03-13T14:49:38.407 INFO:tasks.workunit.client.0.smithi176.stdout:0/183: chown d2/d5/d28 1 1 2024-03-13T14:49:38.410 INFO:tasks.workunit.client.0.smithi176.stdout:0/184: chown d2/f11 7398190 1 2024-03-13T14:49:38.412 INFO:tasks.workunit.client.0.smithi176.stdout:6/211: dread d0/de/f2c [0,4194304] 0 2024-03-13T14:49:38.412 INFO:tasks.workunit.client.0.smithi176.stdout:1/215: mkdir d2/de/d10/d16/d40 0 2024-03-13T14:49:38.412 INFO:tasks.workunit.client.0.smithi176.stdout:4/220: dwrite d7/fe [0,4194304] 0 2024-03-13T14:49:38.415 INFO:tasks.workunit.client.1.smithi196.stdout:1/388: write d1/d9/d33/f62 [3099013,8703] 0 2024-03-13T14:49:38.419 INFO:tasks.workunit.client.1.smithi196.stdout:0/417: creat d8/da/d74/f8e x:0 0 0 2024-03-13T14:49:38.422 INFO:tasks.workunit.client.1.smithi196.stdout:9/313: write d7/f45 [4545362,125769] 0 2024-03-13T14:49:38.425 INFO:tasks.workunit.client.1.smithi196.stdout:5/473: dwrite d4/f11 [0,4194304] 0 2024-03-13T14:49:38.425 INFO:tasks.workunit.client.1.smithi196.stdout:3/374: symlink db/d19/d22/l78 0 2024-03-13T14:49:38.427 INFO:tasks.workunit.client.0.smithi176.stdout:9/254: mkdir d0/d13/d4d 0 2024-03-13T14:49:38.434 INFO:tasks.workunit.client.1.smithi196.stdout:3/375: write db/f2c [3086154,9026] 0 2024-03-13T14:49:38.437 INFO:tasks.workunit.client.0.smithi176.stdout:8/207: unlink d0/db/dc/l2f 0 2024-03-13T14:49:38.442 INFO:tasks.workunit.client.1.smithi196.stdout:4/341: dread d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:38.442 INFO:tasks.workunit.client.1.smithi196.stdout:5/474: write d4/db/d14/d19/d1c/d25/d38/f56 [4214914,90034] 0 2024-03-13T14:49:38.444 INFO:tasks.workunit.client.0.smithi176.stdout:7/208: symlink d1/d2/l36 0 2024-03-13T14:49:38.446 INFO:tasks.workunit.client.0.smithi176.stdout:0/185: creat d2/d5/d28/f33 x:0 0 0 2024-03-13T14:49:38.447 INFO:tasks.workunit.client.0.smithi176.stdout:0/186: fsync d2/f3 0 2024-03-13T14:49:38.450 INFO:tasks.workunit.client.0.smithi176.stdout:0/187: read d2/db/f2c [785,97734] 0 2024-03-13T14:49:38.452 INFO:tasks.workunit.client.0.smithi176.stdout:1/216: creat d2/de/f41 x:0 0 0 2024-03-13T14:49:38.452 INFO:tasks.workunit.client.1.smithi196.stdout:8/311: dwrite d0/d10/d12/f33 [0,4194304] 0 2024-03-13T14:49:38.454 INFO:tasks.workunit.client.1.smithi196.stdout:7/465: getdents d0/d4d/d74 0 2024-03-13T14:49:38.455 INFO:tasks.workunit.client.0.smithi176.stdout:9/255: creat d0/d1e/f4e x:0 0 0 2024-03-13T14:49:38.458 INFO:tasks.workunit.client.1.smithi196.stdout:6/517: creat d6/fa2 x:0 0 0 2024-03-13T14:49:38.458 INFO:tasks.workunit.client.1.smithi196.stdout:9/314: dwrite d7/de/d4b/f1a [0,4194304] 0 2024-03-13T14:49:38.460 INFO:tasks.workunit.client.0.smithi176.stdout:5/188: getdents d5/d2e 0 2024-03-13T14:49:38.460 INFO:tasks.workunit.client.1.smithi196.stdout:3/376: fsync db/d19/d22/f23 0 2024-03-13T14:49:38.460 INFO:tasks.workunit.client.1.smithi196.stdout:9/315: readlink d7/d9/d17/l4a 0 2024-03-13T14:49:38.464 INFO:tasks.workunit.client.1.smithi196.stdout:1/389: symlink d1/d9/d33/d4b/d66/l86 0 2024-03-13T14:49:38.466 INFO:tasks.workunit.client.1.smithi196.stdout:5/475: mkdir d4/db/d3c/d6a/d93 0 2024-03-13T14:49:38.472 INFO:tasks.workunit.client.1.smithi196.stdout:1/390: write d1/d3f/f5d [368477,124920] 0 2024-03-13T14:49:38.476 INFO:tasks.workunit.client.0.smithi176.stdout:4/221: symlink d7/d8/l51 0 2024-03-13T14:49:38.477 INFO:tasks.workunit.client.1.smithi196.stdout:8/312: read d0/d10/d12/d1e/f34 [869772,41569] 0 2024-03-13T14:49:38.477 INFO:tasks.workunit.client.1.smithi196.stdout:5/476: fsync d4/db/d14/d19/d26/d33/d61/f6f 0 2024-03-13T14:49:38.478 INFO:tasks.workunit.client.0.smithi176.stdout:6/212: write d0/f8 [4592022,129691] 0 2024-03-13T14:49:38.479 INFO:tasks.workunit.client.1.smithi196.stdout:1/391: chown d1/d38/c1c 3120816 1 2024-03-13T14:49:38.480 INFO:tasks.workunit.client.0.smithi176.stdout:8/208: dwrite d0/db/fd [0,4194304] 0 2024-03-13T14:49:38.483 INFO:tasks.workunit.client.1.smithi196.stdout:3/377: mkdir db/d15/d44/d79 0 2024-03-13T14:49:38.483 INFO:tasks.workunit.client.0.smithi176.stdout:8/209: stat d0/d24/f37 0 2024-03-13T14:49:38.488 INFO:tasks.workunit.client.1.smithi196.stdout:9/316: mknod d7/d24/d28/c65 0 2024-03-13T14:49:38.488 INFO:tasks.workunit.client.1.smithi196.stdout:4/342: write d1/f22 [2979151,100341] 0 2024-03-13T14:49:38.490 INFO:tasks.workunit.client.0.smithi176.stdout:3/214: link d4/d28/c3e d4/d13/c47 0 2024-03-13T14:49:38.491 INFO:tasks.workunit.client.0.smithi176.stdout:2/231: rename d1/dc/f12 to d1/dc/f52 0 2024-03-13T14:49:38.492 INFO:tasks.workunit.client.0.smithi176.stdout:9/256: chown d0/d1e/l24 172285 1 2024-03-13T14:49:38.493 INFO:tasks.workunit.client.1.smithi196.stdout:6/518: fsync d6/d7/df/d27/d39/d44/d74/f88 0 2024-03-13T14:49:38.494 INFO:tasks.workunit.client.0.smithi176.stdout:5/189: symlink d5/d2b/l41 0 2024-03-13T14:49:38.495 INFO:tasks.workunit.client.0.smithi176.stdout:9/257: write d0/d1e/f4a [1048526,85396] 0 2024-03-13T14:49:38.496 INFO:tasks.workunit.client.1.smithi196.stdout:9/317: readlink d7/de/d4b/l42 0 2024-03-13T14:49:38.497 INFO:tasks.workunit.client.0.smithi176.stdout:7/209: dread d1/d13/f14 [0,4194304] 0 2024-03-13T14:49:38.498 INFO:tasks.workunit.client.1.smithi196.stdout:2/259: dread d0/d6/d34/f41 [0,4194304] 0 2024-03-13T14:49:38.499 INFO:tasks.workunit.client.0.smithi176.stdout:7/210: stat d1/d1c/d26/l32 0 2024-03-13T14:49:38.504 INFO:tasks.workunit.client.0.smithi176.stdout:4/222: truncate d7/d8/d2f/f37 1110343 0 2024-03-13T14:49:38.510 INFO:tasks.workunit.client.1.smithi196.stdout:8/313: rename d0/d10/d12/d17/d40 to d0/d16/d71 0 2024-03-13T14:49:38.513 INFO:tasks.workunit.client.0.smithi176.stdout:3/215: stat d4/c14 0 2024-03-13T14:49:38.513 INFO:tasks.workunit.client.0.smithi176.stdout:6/213: write d0/d9/f14 [43188,10493] 0 2024-03-13T14:49:38.513 INFO:tasks.workunit.client.1.smithi196.stdout:8/314: read d0/d10/d12/f3e [2952259,21405] 0 2024-03-13T14:49:38.516 INFO:tasks.workunit.client.1.smithi196.stdout:0/418: getdents d8/da/d25/d26/d35/d3b/d5f/d17 0 2024-03-13T14:49:38.516 INFO:tasks.workunit.client.0.smithi176.stdout:2/232: dread - d1/dc/f2c zero size 2024-03-13T14:49:38.517 INFO:tasks.workunit.client.0.smithi176.stdout:5/190: mknod d5/d2e/c42 0 2024-03-13T14:49:38.517 INFO:tasks.workunit.client.0.smithi176.stdout:9/258: stat d0/l2b 0 2024-03-13T14:49:38.518 INFO:tasks.workunit.client.1.smithi196.stdout:7/466: rmdir d0/d6/d33/d97 0 2024-03-13T14:49:38.520 INFO:tasks.workunit.client.0.smithi176.stdout:0/188: dwrite d2/db/f1c [0,4194304] 0 2024-03-13T14:49:38.520 INFO:tasks.workunit.client.0.smithi176.stdout:5/191: chown d5/d31 64731 1 2024-03-13T14:49:38.520 INFO:tasks.workunit.client.1.smithi196.stdout:2/260: readlink d0/d50/l55 0 2024-03-13T14:49:38.521 INFO:tasks.workunit.client.1.smithi196.stdout:7/467: chown d0/d9/d31 96 1 2024-03-13T14:49:38.523 INFO:tasks.workunit.client.1.smithi196.stdout:0/419: fdatasync d8/da/d25/d26/d35/d3b/d50/d55/f72 0 2024-03-13T14:49:38.523 INFO:tasks.workunit.client.0.smithi176.stdout:5/192: dread - d5/d2e/f3f zero size 2024-03-13T14:49:38.523 INFO:tasks.workunit.client.0.smithi176.stdout:7/211: creat d1/d2/d2b/f37 x:0 0 0 2024-03-13T14:49:38.524 INFO:tasks.workunit.client.1.smithi196.stdout:2/261: fsync d0/d6/d1d/f23 0 2024-03-13T14:49:38.525 INFO:tasks.workunit.client.1.smithi196.stdout:9/318: dwrite d7/d9/d17/f51 [0,4194304] 0 2024-03-13T14:49:38.526 INFO:tasks.workunit.client.0.smithi176.stdout:3/216: creat d4/d13/f48 x:0 0 0 2024-03-13T14:49:38.526 INFO:tasks.workunit.client.0.smithi176.stdout:4/223: symlink d7/d10/d46/l52 0 2024-03-13T14:49:38.526 INFO:tasks.workunit.client.0.smithi176.stdout:2/233: truncate d1/dc/d2d/d38/f4e 543164 0 2024-03-13T14:49:38.527 INFO:tasks.workunit.client.0.smithi176.stdout:9/259: mknod d0/d1e/d1f/c4f 0 2024-03-13T14:49:38.528 INFO:tasks.workunit.client.0.smithi176.stdout:5/193: fsync d5/d22/d15/f27 0 2024-03-13T14:49:38.528 INFO:tasks.workunit.client.0.smithi176.stdout:7/212: read d1/d13/f20 [906057,4400] 0 2024-03-13T14:49:38.529 INFO:tasks.workunit.client.0.smithi176.stdout:8/210: dwrite d0/f2 [0,4194304] 0 2024-03-13T14:49:38.531 INFO:tasks.workunit.client.1.smithi196.stdout:0/420: stat d8/da/d25/d26/d35/d3b/d59/d6a 0 2024-03-13T14:49:38.532 INFO:tasks.workunit.client.1.smithi196.stdout:1/392: dread d1/d9/d33/d4b/f1a [0,4194304] 0 2024-03-13T14:49:38.534 INFO:tasks.workunit.client.0.smithi176.stdout:0/189: creat d2/d5/dc/d20/f34 x:0 0 0 2024-03-13T14:49:38.535 INFO:tasks.workunit.client.0.smithi176.stdout:3/217: read d4/d13/d2d/f30 [78951,66021] 0 2024-03-13T14:49:38.535 INFO:tasks.workunit.client.0.smithi176.stdout:5/194: fsync d5/d22/f29 0 2024-03-13T14:49:38.537 INFO:tasks.workunit.client.0.smithi176.stdout:7/213: chown d1/d13/d1b 312470 1 2024-03-13T14:49:38.538 INFO:tasks.workunit.client.1.smithi196.stdout:4/343: creat d1/d3/d7/d35/f63 x:0 0 0 2024-03-13T14:49:38.539 INFO:tasks.workunit.client.1.smithi196.stdout:7/468: mknod d0/d9/d1e/c9b 0 2024-03-13T14:49:38.539 INFO:tasks.workunit.client.0.smithi176.stdout:5/195: chown d5/d22/d15/c23 18573 1 2024-03-13T14:49:38.539 INFO:tasks.workunit.client.0.smithi176.stdout:0/190: dread - d2/d5/dc/f2e zero size 2024-03-13T14:49:38.540 INFO:tasks.workunit.client.1.smithi196.stdout:7/469: chown d0/d78 191433026 1 2024-03-13T14:49:38.541 INFO:tasks.workunit.client.1.smithi196.stdout:0/421: creat d8/da/d74/f8f x:0 0 0 2024-03-13T14:49:38.541 INFO:tasks.workunit.client.1.smithi196.stdout:2/262: write d0/d6/d1d/f27 [327498,98110] 0 2024-03-13T14:49:38.542 INFO:tasks.workunit.client.0.smithi176.stdout:6/214: getdents d0/d10/d19/d27/d2b/d2e/d3e 0 2024-03-13T14:49:38.545 INFO:tasks.workunit.client.0.smithi176.stdout:7/214: fsync d1/d7/f10 0 2024-03-13T14:49:38.548 INFO:tasks.workunit.client.1.smithi196.stdout:0/422: stat d8/da/d25/d26/d35/l79 0 2024-03-13T14:49:38.550 INFO:tasks.workunit.client.0.smithi176.stdout:3/218: creat d4/d28/f49 x:0 0 0 2024-03-13T14:49:38.555 INFO:tasks.workunit.client.1.smithi196.stdout:1/393: fdatasync d1/d28/f35 0 2024-03-13T14:49:38.557 INFO:tasks.workunit.client.1.smithi196.stdout:1/394: chown d1/d28/f35 12490516 1 2024-03-13T14:49:38.558 INFO:tasks.workunit.client.1.smithi196.stdout:7/470: unlink d0/d6/fc 0 2024-03-13T14:49:38.559 INFO:tasks.workunit.client.1.smithi196.stdout:1/395: write d1/d38/f3b [193806,117559] 0 2024-03-13T14:49:38.559 INFO:tasks.workunit.client.1.smithi196.stdout:7/471: readlink d0/l2f 0 2024-03-13T14:49:38.560 INFO:tasks.workunit.client.0.smithi176.stdout:8/211: symlink d0/l3d 0 2024-03-13T14:49:38.561 INFO:tasks.workunit.client.0.smithi176.stdout:9/260: dread d0/d17/f3a [0,4194304] 0 2024-03-13T14:49:38.563 INFO:tasks.workunit.client.0.smithi176.stdout:1/217: sync 2024-03-13T14:49:38.566 INFO:tasks.workunit.client.1.smithi196.stdout:6/519: truncate d6/d9/d17/d19/d4e/f7c 1111548 0 2024-03-13T14:49:38.566 INFO:tasks.workunit.client.1.smithi196.stdout:4/344: creat d1/d3/d7/d35/d3e/f64 x:0 0 0 2024-03-13T14:49:38.567 INFO:tasks.workunit.client.0.smithi176.stdout:5/196: readlink d5/l26 0 2024-03-13T14:49:38.575 INFO:tasks.workunit.client.0.smithi176.stdout:4/224: write d7/d8/f28 [706067,113422] 0 2024-03-13T14:49:38.576 INFO:tasks.workunit.client.1.smithi196.stdout:8/315: dread d0/d10/f11 [0,4194304] 0 2024-03-13T14:49:38.577 INFO:tasks.workunit.client.1.smithi196.stdout:1/396: symlink d1/d9/d4e/d55/d63/l87 0 2024-03-13T14:49:38.577 INFO:tasks.workunit.client.0.smithi176.stdout:4/225: chown d7/d8/f16 44622 1 2024-03-13T14:49:38.578 INFO:tasks.workunit.client.1.smithi196.stdout:8/316: write d0/d10/d12/f53 [836092,439] 0 2024-03-13T14:49:38.579 INFO:tasks.workunit.client.0.smithi176.stdout:2/234: write d1/dc/f16 [3662296,47897] 0 2024-03-13T14:49:38.581 INFO:tasks.workunit.client.0.smithi176.stdout:8/212: symlink d0/db/l3e 0 2024-03-13T14:49:38.583 INFO:tasks.workunit.client.0.smithi176.stdout:9/261: fsync d0/d17/f19 0 2024-03-13T14:49:38.585 INFO:tasks.workunit.client.0.smithi176.stdout:8/213: write d0/d5/f1f [4982687,95715] 0 2024-03-13T14:49:38.588 INFO:tasks.workunit.client.0.smithi176.stdout:8/214: write d0/db/fd [3053361,47294] 0 2024-03-13T14:49:38.589 INFO:tasks.workunit.client.1.smithi196.stdout:2/263: write d0/d6/d1d/f51 [198757,89745] 0 2024-03-13T14:49:38.593 INFO:tasks.workunit.client.0.smithi176.stdout:5/197: mkdir d5/d2e/d43 0 2024-03-13T14:49:38.595 INFO:tasks.workunit.client.1.smithi196.stdout:7/472: creat d0/d9/d1e/d58/d72/d6e/d7f/f9c x:0 0 0 2024-03-13T14:49:38.595 INFO:tasks.workunit.client.1.smithi196.stdout:8/317: dwrite d0/d10/d12/f50 [0,4194304] 0 2024-03-13T14:49:38.596 INFO:tasks.workunit.client.0.smithi176.stdout:7/215: fsync d1/d28/f2a 0 2024-03-13T14:49:38.597 INFO:tasks.workunit.client.1.smithi196.stdout:7/473: readlink d0/d6/l17 0 2024-03-13T14:49:38.600 INFO:tasks.workunit.client.1.smithi196.stdout:8/318: dread - d0/d1c/d3b/f4a zero size 2024-03-13T14:49:38.600 INFO:tasks.workunit.client.1.smithi196.stdout:0/423: write d8/da/d25/d26/d35/d3b/d5f/f2b [849311,87731] 0 2024-03-13T14:49:38.600 INFO:tasks.workunit.client.1.smithi196.stdout:9/319: truncate d7/de/f50 234980 0 2024-03-13T14:49:38.601 INFO:tasks.workunit.client.0.smithi176.stdout:6/215: mknod d0/d10/d19/d27/c46 0 2024-03-13T14:49:38.602 INFO:tasks.workunit.client.1.smithi196.stdout:9/320: fdatasync d7/de/d16/f49 0 2024-03-13T14:49:38.603 INFO:tasks.workunit.client.0.smithi176.stdout:6/216: chown d0/d10/l32 4133 1 2024-03-13T14:49:38.606 INFO:tasks.workunit.client.0.smithi176.stdout:0/191: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:38.606 INFO:tasks.workunit.client.0.smithi176.stdout:0/192: stat d2/d5/d1e 0 2024-03-13T14:49:38.607 INFO:tasks.workunit.client.1.smithi196.stdout:6/520: mkdir d6/d2a/dc/da3 0 2024-03-13T14:49:38.608 INFO:tasks.workunit.client.1.smithi196.stdout:2/264: dread d0/f1a [0,4194304] 0 2024-03-13T14:49:38.612 INFO:tasks.workunit.client.0.smithi176.stdout:6/217: dread d0/de/f3c [0,4194304] 0 2024-03-13T14:49:38.618 INFO:tasks.workunit.client.1.smithi196.stdout:1/397: mkdir d1/d9/d33/d4b/d88 0 2024-03-13T14:49:38.623 INFO:tasks.workunit.client.1.smithi196.stdout:7/474: dwrite d0/d9/f24 [8388608,4194304] 0 2024-03-13T14:49:38.623 INFO:tasks.workunit.client.0.smithi176.stdout:9/262: symlink d0/d13/d32/l50 0 2024-03-13T14:49:38.626 INFO:tasks.workunit.client.1.smithi196.stdout:0/424: dread d8/da/d25/d26/d35/d3b/d50/f53 [0,4194304] 0 2024-03-13T14:49:38.627 INFO:tasks.workunit.client.0.smithi176.stdout:1/218: link d2/de/d10/f2a d2/de/d10/d2e/f42 0 2024-03-13T14:49:38.632 INFO:tasks.workunit.client.0.smithi176.stdout:2/235: dwrite d1/dc/d2d/d38/f4e [0,4194304] 0 2024-03-13T14:49:38.634 INFO:tasks.workunit.client.1.smithi196.stdout:2/265: dwrite d0/d7/f3f [0,4194304] 0 2024-03-13T14:49:38.642 INFO:tasks.workunit.client.1.smithi196.stdout:2/266: write d0/d6/d34/f41 [925551,96598] 0 2024-03-13T14:49:38.647 INFO:tasks.workunit.client.0.smithi176.stdout:7/216: creat d1/d13/d1b/f38 x:0 0 0 2024-03-13T14:49:38.650 INFO:tasks.workunit.client.0.smithi176.stdout:3/219: link d4/d13/d2d/f30 d4/d13/f4a 0 2024-03-13T14:49:38.655 INFO:tasks.workunit.client.1.smithi196.stdout:4/345: link d1/d3/d4c/f59 d1/d3/d7/f65 0 2024-03-13T14:49:38.657 INFO:tasks.workunit.client.0.smithi176.stdout:6/218: symlink d0/de/d38/l47 0 2024-03-13T14:49:38.659 INFO:tasks.workunit.client.0.smithi176.stdout:6/219: write d0/de/f23 [4711084,94755] 0 2024-03-13T14:49:38.659 INFO:tasks.workunit.client.1.smithi196.stdout:1/398: unlink d1/d3f/d5b/l7a 0 2024-03-13T14:49:38.661 INFO:tasks.workunit.client.0.smithi176.stdout:6/220: read d0/d10/f3f [160316,28586] 0 2024-03-13T14:49:38.663 INFO:tasks.workunit.client.0.smithi176.stdout:0/193: rename d2/d5/dc/d2a to d2/db/d35 0 2024-03-13T14:49:38.666 INFO:tasks.workunit.client.0.smithi176.stdout:9/263: mknod d0/d40/c51 0 2024-03-13T14:49:38.667 INFO:tasks.workunit.client.0.smithi176.stdout:9/264: chown d0/d17/l18 250 1 2024-03-13T14:49:38.668 INFO:tasks.workunit.client.1.smithi196.stdout:7/475: unlink d0/d9/f59 0 2024-03-13T14:49:38.672 INFO:tasks.workunit.client.0.smithi176.stdout:1/219: mknod d2/de/d3f/d19/c43 0 2024-03-13T14:49:38.673 INFO:tasks.workunit.client.0.smithi176.stdout:1/220: chown d2/c1a 458692 1 2024-03-13T14:49:38.673 INFO:tasks.workunit.client.0.smithi176.stdout:2/236: symlink d1/dc/d2d/d38/l53 0 2024-03-13T14:49:38.677 INFO:tasks.workunit.client.0.smithi176.stdout:7/217: read d1/d28/f2a [17747,90342] 0 2024-03-13T14:49:38.681 INFO:tasks.workunit.client.1.smithi196.stdout:6/521: truncate d6/d9/d17/d19/d4e/f8a 1002865 0 2024-03-13T14:49:38.683 INFO:tasks.workunit.client.1.smithi196.stdout:6/522: chown d6/d7/df/d27/d39/d44/d74 6 1 2024-03-13T14:49:38.684 INFO:tasks.workunit.client.0.smithi176.stdout:2/237: dread d1/dc/f16 [0,4194304] 0 2024-03-13T14:49:38.688 INFO:tasks.workunit.client.1.smithi196.stdout:7/476: symlink d0/d9/d31/d46/d47/l9d 0 2024-03-13T14:49:38.689 INFO:tasks.workunit.client.0.smithi176.stdout:1/221: symlink d2/de/l44 0 2024-03-13T14:49:38.690 INFO:tasks.workunit.client.1.smithi196.stdout:7/477: write d0/d9/f5a [388960,39557] 0 2024-03-13T14:49:38.692 INFO:tasks.workunit.client.0.smithi176.stdout:6/221: write d0/fd [722234,76111] 0 2024-03-13T14:49:38.694 INFO:tasks.workunit.client.1.smithi196.stdout:9/321: creat d7/de/d4b/d1d/f66 x:0 0 0 2024-03-13T14:49:38.696 INFO:tasks.workunit.client.0.smithi176.stdout:0/194: dread d2/d5/dc/f18 [0,4194304] 0 2024-03-13T14:49:38.701 INFO:tasks.workunit.client.0.smithi176.stdout:1/222: unlink d2/de/l2c 0 2024-03-13T14:49:38.702 INFO:tasks.workunit.client.0.smithi176.stdout:1/223: dread - d2/de/f26 zero size 2024-03-13T14:49:38.703 INFO:tasks.workunit.client.0.smithi176.stdout:1/224: fsync d2/f21 0 2024-03-13T14:49:38.704 INFO:tasks.workunit.client.0.smithi176.stdout:6/222: mknod d0/de/d38/c48 0 2024-03-13T14:49:38.706 INFO:tasks.workunit.client.0.smithi176.stdout:6/223: chown d0/d10/d19/d27/d2b/d2e 52250133 1 2024-03-13T14:49:38.706 INFO:tasks.workunit.client.0.smithi176.stdout:1/225: chown d2/de/d10/d14/d22/f2b 2 1 2024-03-13T14:49:38.706 INFO:tasks.workunit.client.0.smithi176.stdout:1/226: fsync f1 0 2024-03-13T14:49:38.707 INFO:tasks.workunit.client.0.smithi176.stdout:6/224: truncate d0/fd 1213869 0 2024-03-13T14:49:38.708 INFO:tasks.workunit.client.0.smithi176.stdout:6/225: chown d0/f8 0 1 2024-03-13T14:49:38.709 INFO:tasks.workunit.client.0.smithi176.stdout:0/195: mknod d2/d5/d28/c36 0 2024-03-13T14:49:38.712 INFO:tasks.workunit.client.0.smithi176.stdout:4/226: sync 2024-03-13T14:49:38.712 INFO:tasks.workunit.client.0.smithi176.stdout:5/198: sync 2024-03-13T14:49:38.712 INFO:tasks.workunit.client.0.smithi176.stdout:8/215: sync 2024-03-13T14:49:38.712 INFO:tasks.workunit.client.0.smithi176.stdout:2/238: truncate d1/dc/d1c/f36 162683 0 2024-03-13T14:49:38.714 INFO:tasks.workunit.client.0.smithi176.stdout:9/265: link d0/d13/l35 d0/d1e/l52 0 2024-03-13T14:49:38.716 INFO:tasks.workunit.client.0.smithi176.stdout:3/220: getdents d4/d28/d34 0 2024-03-13T14:49:38.717 INFO:tasks.workunit.client.0.smithi176.stdout:3/221: chown d4/d28/d2e/f3c 37236 1 2024-03-13T14:49:38.720 INFO:tasks.workunit.client.0.smithi176.stdout:3/222: fsync d4/d13/d16/d3d/f41 0 2024-03-13T14:49:38.721 INFO:tasks.workunit.client.0.smithi176.stdout:7/218: truncate d1/f1f 4037089 0 2024-03-13T14:49:38.726 INFO:tasks.workunit.client.0.smithi176.stdout:2/239: dwrite d1/f14 [4194304,4194304] 0 2024-03-13T14:49:38.726 INFO:tasks.workunit.client.0.smithi176.stdout:6/226: write d0/d9/f37 [2092844,128400] 0 2024-03-13T14:49:38.730 INFO:tasks.workunit.client.0.smithi176.stdout:0/196: read d2/d5/dc/f15 [43582,87883] 0 2024-03-13T14:49:38.731 INFO:tasks.workunit.client.0.smithi176.stdout:4/227: mknod d7/d8/d2f/c53 0 2024-03-13T14:49:38.736 INFO:tasks.workunit.client.0.smithi176.stdout:5/199: rename d5/l26 to d5/d2b/l44 0 2024-03-13T14:49:38.738 INFO:tasks.workunit.client.0.smithi176.stdout:5/200: fsync d5/d2e/f3f 0 2024-03-13T14:49:38.738 INFO:tasks.workunit.client.1.smithi196.stdout:4/346: symlink d1/d3/d7/d35/l66 0 2024-03-13T14:49:38.741 INFO:tasks.workunit.client.0.smithi176.stdout:2/240: dread d1/f35 [0,4194304] 0 2024-03-13T14:49:38.741 INFO:tasks.workunit.client.1.smithi196.stdout:4/347: write d1/d3/d7/f5d [333605,93149] 0 2024-03-13T14:49:38.743 INFO:tasks.workunit.client.0.smithi176.stdout:9/266: chown d0/d17/c2f 6 1 2024-03-13T14:49:38.743 INFO:tasks.workunit.client.0.smithi176.stdout:3/223: dwrite d4/f1d [0,4194304] 0 2024-03-13T14:49:38.744 INFO:tasks.workunit.client.0.smithi176.stdout:2/241: dread - d1/dc/d1c/f37 zero size 2024-03-13T14:49:38.744 INFO:tasks.workunit.client.1.smithi196.stdout:4/348: write d1/d3/d7/d35/f37 [2906759,91278] 0 2024-03-13T14:49:38.744 INFO:tasks.workunit.client.1.smithi196.stdout:6/523: unlink d6/d7/l84 0 2024-03-13T14:49:38.746 INFO:tasks.workunit.client.1.smithi196.stdout:1/399: fdatasync d1/d9/d33/d4b/f75 0 2024-03-13T14:49:38.747 INFO:tasks.workunit.client.1.smithi196.stdout:4/349: write d1/d3/d7/d35/f63 [581286,71038] 0 2024-03-13T14:49:38.747 INFO:tasks.workunit.client.0.smithi176.stdout:3/224: stat d4/d13/d2d 0 2024-03-13T14:49:38.749 INFO:tasks.workunit.client.1.smithi196.stdout:4/350: readlink d1/l25 0 2024-03-13T14:49:38.752 INFO:tasks.workunit.client.0.smithi176.stdout:0/197: dwrite d2/d5/dc/d20/f34 [0,4194304] 0 2024-03-13T14:49:38.752 INFO:tasks.workunit.client.1.smithi196.stdout:4/351: dread - d1/d3/d7/f65 zero size 2024-03-13T14:49:38.755 INFO:tasks.workunit.client.1.smithi196.stdout:4/352: dread - d1/d1e/d39/f45 zero size 2024-03-13T14:49:38.761 INFO:tasks.workunit.client.0.smithi176.stdout:6/227: unlink d0/de/d38/l47 0 2024-03-13T14:49:38.768 INFO:tasks.workunit.client.0.smithi176.stdout:3/225: dwrite d4/d13/f48 [0,4194304] 0 2024-03-13T14:49:38.768 INFO:tasks.workunit.client.1.smithi196.stdout:9/322: chown d7/de/f40 9 1 2024-03-13T14:49:38.772 INFO:tasks.workunit.client.0.smithi176.stdout:8/216: link d0/d5/f1f d0/db/f3f 0 2024-03-13T14:49:38.774 INFO:tasks.workunit.client.0.smithi176.stdout:8/217: dread - d0/db/dc/f2a zero size 2024-03-13T14:49:38.775 INFO:tasks.workunit.client.0.smithi176.stdout:8/218: chown d0/db/l3e 3 1 2024-03-13T14:49:38.776 INFO:tasks.workunit.client.0.smithi176.stdout:5/201: rename d5/f7 to d5/d31/f45 0 2024-03-13T14:49:38.779 INFO:tasks.workunit.client.0.smithi176.stdout:9/267: unlink d0/d1e/l33 0 2024-03-13T14:49:38.780 INFO:tasks.workunit.client.0.smithi176.stdout:2/242: rmdir d1/d2a/d33 39 2024-03-13T14:49:38.782 INFO:tasks.workunit.client.1.smithi196.stdout:0/425: getdents d8/da/d25/d26/d35/d3b/d59/d6a 0 2024-03-13T14:49:38.783 INFO:tasks.workunit.client.0.smithi176.stdout:1/227: rmdir d2/de/d10/d3d 0 2024-03-13T14:49:38.786 INFO:tasks.workunit.client.0.smithi176.stdout:0/198: creat d2/d5/dc/d20/f37 x:0 0 0 2024-03-13T14:49:38.787 INFO:tasks.workunit.client.1.smithi196.stdout:8/319: link d0/d10/d12/l39 d0/d10/d12/d17/l72 0 2024-03-13T14:49:38.788 INFO:tasks.workunit.client.1.smithi196.stdout:2/267: getdents d0/d7/d9/d42 0 2024-03-13T14:49:38.791 INFO:tasks.workunit.client.1.smithi196.stdout:9/323: creat d7/de/d4b/d1d/d22/f67 x:0 0 0 2024-03-13T14:49:38.794 INFO:tasks.workunit.client.0.smithi176.stdout:4/228: mknod d7/c54 0 2024-03-13T14:49:38.799 INFO:tasks.workunit.client.0.smithi176.stdout:8/219: mkdir d0/db/dc/d40 0 2024-03-13T14:49:38.799 INFO:tasks.workunit.client.0.smithi176.stdout:3/226: dread d4/d13/f4a [0,4194304] 0 2024-03-13T14:49:38.802 INFO:tasks.workunit.client.0.smithi176.stdout:5/202: creat d5/d2b/f46 x:0 0 0 2024-03-13T14:49:38.805 INFO:tasks.workunit.client.1.smithi196.stdout:7/478: creat d0/d9/d31/d77/d93/d94/f9e x:0 0 0 2024-03-13T14:49:38.805 INFO:tasks.workunit.client.0.smithi176.stdout:9/268: mkdir d0/d40/d53 0 2024-03-13T14:49:38.808 INFO:tasks.workunit.client.1.smithi196.stdout:2/268: mkdir d0/d7/d56 0 2024-03-13T14:49:38.809 INFO:tasks.workunit.client.0.smithi176.stdout:7/219: dread d1/d13/d1b/d18/f1e [0,4194304] 0 2024-03-13T14:49:38.810 INFO:tasks.workunit.client.1.smithi196.stdout:2/269: chown d0/d7/d9/d42 271 1 2024-03-13T14:49:38.810 INFO:tasks.workunit.client.1.smithi196.stdout:6/524: creat d6/d2a/dc/d1c/fa4 x:0 0 0 2024-03-13T14:49:38.810 INFO:tasks.workunit.client.0.smithi176.stdout:6/228: write d0/d10/d19/d27/d2b/d2e/f31 [205190,37410] 0 2024-03-13T14:49:38.812 INFO:tasks.workunit.client.1.smithi196.stdout:6/525: chown d6/d2a/d23/f4c 88 1 2024-03-13T14:49:38.815 INFO:tasks.workunit.client.1.smithi196.stdout:8/320: dread d0/d1c/d3b/d48/f5f [0,4194304] 0 2024-03-13T14:49:38.816 INFO:tasks.workunit.client.1.smithi196.stdout:4/353: rename d1/d1e/d43/d5b/l60 to d1/d3/d7/d35/l67 0 2024-03-13T14:49:38.817 INFO:tasks.workunit.client.0.smithi176.stdout:4/229: mkdir d7/d10/d46/d55 0 2024-03-13T14:49:38.819 INFO:tasks.workunit.client.1.smithi196.stdout:4/354: write d1/d3/d7/d35/f37 [1437930,95580] 0 2024-03-13T14:49:38.821 INFO:tasks.workunit.client.0.smithi176.stdout:1/228: dread d2/f6 [0,4194304] 0 2024-03-13T14:49:38.824 INFO:tasks.workunit.client.1.smithi196.stdout:1/400: getdents d1/d9/d6c/d79 0 2024-03-13T14:49:38.825 INFO:tasks.workunit.client.1.smithi196.stdout:1/401: readlink d1/d9/d6c/d79/l81 0 2024-03-13T14:49:38.825 INFO:tasks.workunit.client.0.smithi176.stdout:0/199: write d2/d5/dc/f13 [1234846,68728] 0 2024-03-13T14:49:38.827 INFO:tasks.workunit.client.1.smithi196.stdout:9/324: symlink d7/de/l68 0 2024-03-13T14:49:38.828 INFO:tasks.workunit.client.1.smithi196.stdout:8/321: readlink d0/d10/d12/d17/l37 0 2024-03-13T14:49:38.828 INFO:tasks.workunit.client.0.smithi176.stdout:4/230: dread d7/d8/d15/d23/f35 [0,4194304] 0 2024-03-13T14:49:38.830 INFO:tasks.workunit.client.1.smithi196.stdout:0/426: rename d8/da/d25/d26/d35/d3b/d5f/c41 to d8/da/d25/d26/d35/d3b/d6c/c90 0 2024-03-13T14:49:38.832 INFO:tasks.workunit.client.1.smithi196.stdout:7/479: mkdir d0/d6/d33/d57/d8a/d9f 0 2024-03-13T14:49:38.834 INFO:tasks.workunit.client.0.smithi176.stdout:7/220: rename d1/d13/l15 to d1/d7/l39 0 2024-03-13T14:49:38.834 INFO:tasks.workunit.client.1.smithi196.stdout:1/402: dread d1/d3f/f5d [0,4194304] 0 2024-03-13T14:49:38.836 INFO:tasks.workunit.client.1.smithi196.stdout:4/355: creat d1/d1e/d39/d58/f68 x:0 0 0 2024-03-13T14:49:38.837 INFO:tasks.workunit.client.1.smithi196.stdout:1/403: fsync d1/d9/d4e/d55/f6f 0 2024-03-13T14:49:38.837 INFO:tasks.workunit.client.1.smithi196.stdout:6/526: symlink d6/d57/d7a/d9e/la5 0 2024-03-13T14:49:38.839 INFO:tasks.workunit.client.1.smithi196.stdout:6/527: dread - d6/d7/f8d zero size 2024-03-13T14:49:38.841 INFO:tasks.workunit.client.0.smithi176.stdout:3/227: getdents d4/d13/d2d 0 2024-03-13T14:49:38.842 INFO:tasks.workunit.client.0.smithi176.stdout:3/228: readlink d4/d28/d34/d38/l42 0 2024-03-13T14:49:38.845 INFO:tasks.workunit.client.1.smithi196.stdout:7/480: mkdir d0/d9/d1e/d58/d61/da0 0 2024-03-13T14:49:38.845 INFO:tasks.workunit.client.0.smithi176.stdout:5/203: write d5/f24 [1009036,70737] 0 2024-03-13T14:49:38.845 INFO:tasks.workunit.client.1.smithi196.stdout:8/322: dwrite d0/d1c/d3b/f1f [0,4194304] 0 2024-03-13T14:49:38.846 INFO:tasks.workunit.client.1.smithi196.stdout:2/270: link d0/c2e d0/d7/d9/d42/c57 0 2024-03-13T14:49:38.850 INFO:tasks.workunit.client.0.smithi176.stdout:0/200: rmdir d2/d5/dc 39 2024-03-13T14:49:38.851 INFO:tasks.workunit.client.0.smithi176.stdout:0/201: dread - d2/db/f25 zero size 2024-03-13T14:49:38.852 INFO:tasks.workunit.client.1.smithi196.stdout:8/323: write d0/d1c/d3b/f60 [61637,6804] 0 2024-03-13T14:49:38.852 INFO:tasks.workunit.client.1.smithi196.stdout:8/324: fsync d0/d10/d12/d17/d42/f4c 0 2024-03-13T14:49:38.856 INFO:tasks.workunit.client.1.smithi196.stdout:8/325: truncate d0/d10/f19 4640803 0 2024-03-13T14:49:38.859 INFO:tasks.workunit.client.1.smithi196.stdout:8/326: truncate d0/d1c/d3b/f60 240036 0 2024-03-13T14:49:38.860 INFO:tasks.workunit.client.0.smithi176.stdout:4/231: creat d7/d10/f56 x:0 0 0 2024-03-13T14:49:38.861 INFO:tasks.workunit.client.1.smithi196.stdout:4/356: truncate d1/d3/fa 475487 0 2024-03-13T14:49:38.862 INFO:tasks.workunit.client.0.smithi176.stdout:2/243: rmdir d1/dc/d2d/d38/d49 0 2024-03-13T14:49:38.864 INFO:tasks.workunit.client.0.smithi176.stdout:2/244: chown d1/dc/d2d/d4f 7322647 1 2024-03-13T14:49:38.864 INFO:tasks.workunit.client.1.smithi196.stdout:9/325: truncate d7/de/f29 1501771 0 2024-03-13T14:49:38.868 INFO:tasks.workunit.client.0.smithi176.stdout:0/202: dwrite d2/d5/d28/f29 [0,4194304] 0 2024-03-13T14:49:38.876 INFO:tasks.workunit.client.0.smithi176.stdout:9/269: truncate d0/d1e/f4a 635776 0 2024-03-13T14:49:38.879 INFO:tasks.workunit.client.0.smithi176.stdout:9/270: chown d0/d1e/l27 2824 1 2024-03-13T14:49:38.881 INFO:tasks.workunit.client.0.smithi176.stdout:2/245: dread d1/f14 [4194304,4194304] 0 2024-03-13T14:49:38.881 INFO:tasks.workunit.client.0.smithi176.stdout:3/229: unlink d4/d13/f4a 0 2024-03-13T14:49:38.882 INFO:tasks.workunit.client.1.smithi196.stdout:0/427: dwrite d8/da/f4e [0,4194304] 0 2024-03-13T14:49:38.884 INFO:tasks.workunit.client.0.smithi176.stdout:6/229: truncate d0/d10/d19/d27/d2b/d2e/f40 698738 0 2024-03-13T14:49:38.886 INFO:tasks.workunit.client.1.smithi196.stdout:4/357: chown d1/d3/c28 21 1 2024-03-13T14:49:38.890 INFO:tasks.workunit.client.0.smithi176.stdout:5/204: dread d5/f24 [0,4194304] 0 2024-03-13T14:49:38.893 INFO:tasks.workunit.client.1.smithi196.stdout:7/481: mknod d0/d9/d31/d77/ca1 0 2024-03-13T14:49:38.893 INFO:tasks.workunit.client.0.smithi176.stdout:5/205: dread d5/d22/f13 [0,4194304] 0 2024-03-13T14:49:38.895 INFO:tasks.workunit.client.0.smithi176.stdout:5/206: chown d5/d22/d15 15200 1 2024-03-13T14:49:38.896 INFO:tasks.workunit.client.1.smithi196.stdout:1/404: creat d1/f89 x:0 0 0 2024-03-13T14:49:38.902 INFO:tasks.workunit.client.0.smithi176.stdout:7/221: truncate d1/d7/f10 2400284 0 2024-03-13T14:49:38.905 INFO:tasks.workunit.client.1.smithi196.stdout:0/428: symlink d8/d37/l91 0 2024-03-13T14:49:38.905 INFO:tasks.workunit.client.1.smithi196.stdout:9/326: dwrite d7/de/d4b/d1d/d22/f5a [0,4194304] 0 2024-03-13T14:49:38.906 INFO:tasks.workunit.client.0.smithi176.stdout:8/220: link d0/db/f19 d0/d5/d3b/f41 0 2024-03-13T14:49:38.907 INFO:tasks.workunit.client.1.smithi196.stdout:4/358: rmdir d1/d1e/d39/d58 39 2024-03-13T14:49:38.911 INFO:tasks.workunit.client.0.smithi176.stdout:3/230: creat d4/d28/d31/f4b x:0 0 0 2024-03-13T14:49:38.913 INFO:tasks.workunit.client.1.smithi196.stdout:1/405: mknod d1/d9/d4e/c8a 0 2024-03-13T14:49:38.914 INFO:tasks.workunit.client.0.smithi176.stdout:6/230: creat d0/de/f49 x:0 0 0 2024-03-13T14:49:38.918 INFO:tasks.workunit.client.1.smithi196.stdout:1/406: chown d1/d3f/d5b/f69 723 1 2024-03-13T14:49:38.922 INFO:tasks.workunit.client.1.smithi196.stdout:4/359: unlink d1/cf 0 2024-03-13T14:49:38.923 INFO:tasks.workunit.client.1.smithi196.stdout:0/429: truncate d8/f1e 1179464 0 2024-03-13T14:49:38.924 INFO:tasks.workunit.client.1.smithi196.stdout:9/327: truncate d7/f15 131037 0 2024-03-13T14:49:38.927 INFO:tasks.workunit.client.1.smithi196.stdout:7/482: rename d0/d9/d1e/d58/d72/c32 to d0/d78/ca2 0 2024-03-13T14:49:38.929 INFO:tasks.workunit.client.0.smithi176.stdout:0/203: unlink d2/d5/dc/f13 0 2024-03-13T14:49:38.929 INFO:tasks.workunit.client.1.smithi196.stdout:2/271: link d0/d6/d1d/l2a d0/d6/l58 0 2024-03-13T14:49:38.932 INFO:tasks.workunit.client.1.smithi196.stdout:1/407: symlink d1/d28/l8b 0 2024-03-13T14:49:38.933 INFO:tasks.workunit.client.1.smithi196.stdout:8/327: getdents d0/d16 0 2024-03-13T14:49:38.935 INFO:tasks.workunit.client.0.smithi176.stdout:7/222: rename d1/d13 to d1/d28/d3a 0 2024-03-13T14:49:38.936 INFO:tasks.workunit.client.1.smithi196.stdout:0/430: dread - d8/da/f5c zero size 2024-03-13T14:49:38.939 INFO:tasks.workunit.client.1.smithi196.stdout:7/483: rename d0/d9/d31/d67 to d0/d9/d31/d46/da3 0 2024-03-13T14:49:38.940 INFO:tasks.workunit.client.0.smithi176.stdout:3/231: creat d4/d13/d16/d3d/f4c x:0 0 0 2024-03-13T14:49:38.940 INFO:tasks.workunit.client.1.smithi196.stdout:2/272: symlink d0/d7/l59 0 2024-03-13T14:49:38.941 INFO:tasks.workunit.client.1.smithi196.stdout:7/484: chown d0/d9/d1e/d58/d72/d6e/d7f/f9c 233209289 1 2024-03-13T14:49:38.943 INFO:tasks.workunit.client.1.smithi196.stdout:8/328: readlink d0/d1c/d29/l5c 0 2024-03-13T14:49:38.944 INFO:tasks.workunit.client.1.smithi196.stdout:4/360: mkdir d1/d1e/d39/d58/d69 0 2024-03-13T14:49:38.947 INFO:tasks.workunit.client.1.smithi196.stdout:4/361: read d1/d3/d7/f1d [35294,35965] 0 2024-03-13T14:49:38.949 INFO:tasks.workunit.client.1.smithi196.stdout:1/408: mkdir d1/d9/d8c 0 2024-03-13T14:49:38.951 INFO:tasks.workunit.client.0.smithi176.stdout:1/229: link d2/d30/l33 d2/l45 0 2024-03-13T14:49:38.953 INFO:tasks.workunit.client.1.smithi196.stdout:7/485: symlink d0/d9/d1e/la4 0 2024-03-13T14:49:38.955 INFO:tasks.workunit.client.1.smithi196.stdout:1/409: write d1/d9/d33/d4b/f48 [5196783,7366] 0 2024-03-13T14:49:38.957 INFO:tasks.workunit.client.0.smithi176.stdout:9/271: creat d0/f54 x:0 0 0 2024-03-13T14:49:38.959 INFO:tasks.workunit.client.1.smithi196.stdout:0/431: dwrite d8/da/d25/d26/d35/d3b/d5f/d33/d44/f86 [0,4194304] 0 2024-03-13T14:49:38.961 INFO:tasks.workunit.client.1.smithi196.stdout:2/273: dread d0/d6/d34/f41 [0,4194304] 0 2024-03-13T14:49:38.962 INFO:tasks.workunit.client.1.smithi196.stdout:9/328: creat d7/f69 x:0 0 0 2024-03-13T14:49:38.965 INFO:tasks.workunit.client.0.smithi176.stdout:7/223: creat d1/d1c/d26/f3b x:0 0 0 2024-03-13T14:49:38.966 INFO:tasks.workunit.client.1.smithi196.stdout:0/432: dread - d8/da/d25/d26/d35/d3b/d50/f6d zero size 2024-03-13T14:49:38.969 INFO:tasks.workunit.client.0.smithi176.stdout:7/224: read - d1/d2/d2b/f37 zero size 2024-03-13T14:49:38.973 INFO:tasks.workunit.client.0.smithi176.stdout:2/246: creat d1/f54 x:0 0 0 2024-03-13T14:49:38.978 INFO:tasks.workunit.client.0.smithi176.stdout:3/232: creat d4/d13/d2d/f4d x:0 0 0 2024-03-13T14:49:38.984 INFO:tasks.workunit.client.1.smithi196.stdout:7/486: fdatasync d0/d6/d33/f40 0 2024-03-13T14:49:38.986 INFO:tasks.workunit.client.1.smithi196.stdout:1/410: creat d1/d9/d33/d4b/d66/d7b/f8d x:0 0 0 2024-03-13T14:49:38.987 INFO:tasks.workunit.client.0.smithi176.stdout:4/232: getdents d7/d8/d45 0 2024-03-13T14:49:38.993 INFO:tasks.workunit.client.1.smithi196.stdout:9/329: creat d7/de/d4b/f6a x:0 0 0 2024-03-13T14:49:38.994 INFO:tasks.workunit.client.0.smithi176.stdout:1/230: symlink d2/de/d10/d16/l46 0 2024-03-13T14:49:38.996 INFO:tasks.workunit.client.0.smithi176.stdout:5/207: dwrite d5/f6 [0,4194304] 0 2024-03-13T14:49:38.997 INFO:tasks.workunit.client.0.smithi176.stdout:5/208: chown d5/d31 0 1 2024-03-13T14:49:38.999 INFO:tasks.workunit.client.0.smithi176.stdout:5/209: write d5/f6 [3762097,47444] 0 2024-03-13T14:49:39.000 INFO:tasks.workunit.client.1.smithi196.stdout:8/329: link d0/d1c/d3b/c2b d0/d10/d70/c73 0 2024-03-13T14:49:39.002 INFO:tasks.workunit.client.0.smithi176.stdout:8/221: link d0/db/f1a d0/f42 0 2024-03-13T14:49:39.008 INFO:tasks.workunit.client.0.smithi176.stdout:7/225: write d1/d28/d3a/d1b/d18/f1e [4950533,114916] 0 2024-03-13T14:49:39.013 INFO:tasks.workunit.client.1.smithi196.stdout:2/274: mkdir d0/d7/d9/d5a 0 2024-03-13T14:49:39.014 INFO:tasks.workunit.client.1.smithi196.stdout:9/330: dwrite d7/de/d4b/d1d/d22/d4d/f58 [0,4194304] 0 2024-03-13T14:49:39.017 INFO:tasks.workunit.client.1.smithi196.stdout:2/275: write d0/d7/f17 [512555,77553] 0 2024-03-13T14:49:39.022 INFO:tasks.workunit.client.0.smithi176.stdout:0/204: truncate d2/db/f1c 1802393 0 2024-03-13T14:49:39.022 INFO:tasks.workunit.client.0.smithi176.stdout:8/222: dwrite d0/db/fd [0,4194304] 0 2024-03-13T14:49:39.024 INFO:tasks.workunit.client.0.smithi176.stdout:6/231: creat d0/d10/d19/d27/f4a x:0 0 0 2024-03-13T14:49:39.025 INFO:tasks.workunit.client.0.smithi176.stdout:8/223: chown d0/d5/f1f 20854 1 2024-03-13T14:49:39.026 INFO:tasks.workunit.client.0.smithi176.stdout:4/233: rename d7/d8/l1b to d7/d10/l57 0 2024-03-13T14:49:39.029 INFO:tasks.workunit.client.1.smithi196.stdout:8/330: chown d0/d10/d12/c20 4816 1 2024-03-13T14:49:39.031 INFO:tasks.workunit.client.0.smithi176.stdout:4/234: readlink d7/d8/d15/l22 0 2024-03-13T14:49:39.031 INFO:tasks.workunit.client.0.smithi176.stdout:5/210: chown d5/l25 3 1 2024-03-13T14:49:39.035 INFO:tasks.workunit.client.1.smithi196.stdout:9/331: creat d7/de/d4b/d1d/d22/d4d/f6b x:0 0 0 2024-03-13T14:49:39.036 INFO:tasks.workunit.client.1.smithi196.stdout:2/276: mkdir d0/d6/d34/d5b 0 2024-03-13T14:49:39.045 INFO:tasks.workunit.client.1.smithi196.stdout:7/487: truncate d0/d9/d31/d50/f87 3821827 0 2024-03-13T14:49:39.047 INFO:tasks.workunit.client.0.smithi176.stdout:9/272: dwrite d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:39.049 INFO:tasks.workunit.client.1.smithi196.stdout:7/488: fdatasync d0/d9/d1e/f51 0 2024-03-13T14:49:39.054 INFO:tasks.workunit.client.0.smithi176.stdout:7/226: fdatasync d1/f1f 0 2024-03-13T14:49:39.054 INFO:tasks.workunit.client.0.smithi176.stdout:9/273: write d0/d17/f19 [769709,91091] 0 2024-03-13T14:49:39.056 INFO:tasks.workunit.client.1.smithi196.stdout:8/331: dwrite d0/f56 [0,4194304] 0 2024-03-13T14:49:39.057 INFO:tasks.workunit.client.0.smithi176.stdout:1/231: write d2/f6 [3316515,21699] 0 2024-03-13T14:49:39.060 INFO:tasks.workunit.client.1.smithi196.stdout:8/332: read d0/d10/d12/f3e [3753032,39266] 0 2024-03-13T14:49:39.061 INFO:tasks.workunit.client.1.smithi196.stdout:1/411: truncate d1/f2a 3535145 0 2024-03-13T14:49:39.063 INFO:tasks.workunit.client.0.smithi176.stdout:4/235: dwrite d7/d10/f27 [0,4194304] 0 2024-03-13T14:49:39.063 INFO:tasks.workunit.client.1.smithi196.stdout:2/277: dwrite d0/d7/f3f [0,4194304] 0 2024-03-13T14:49:39.067 INFO:tasks.workunit.client.1.smithi196.stdout:7/489: symlink d0/d4d/d74/la5 0 2024-03-13T14:49:39.069 INFO:tasks.workunit.client.0.smithi176.stdout:0/205: read d2/db/f2c [3997,59203] 0 2024-03-13T14:49:39.078 INFO:tasks.workunit.client.1.smithi196.stdout:8/333: rmdir d0/d1c/d29 39 2024-03-13T14:49:39.080 INFO:tasks.workunit.client.1.smithi196.stdout:8/334: read d0/f56 [3556884,42965] 0 2024-03-13T14:49:39.084 INFO:tasks.workunit.client.1.smithi196.stdout:2/278: rmdir d0/d7/d9/d42 39 2024-03-13T14:49:39.086 INFO:tasks.workunit.client.1.smithi196.stdout:2/279: chown d0/f37 92 1 2024-03-13T14:49:39.087 INFO:tasks.workunit.client.1.smithi196.stdout:7/490: dread d0/d9/d1e/d3f/f82 [0,4194304] 0 2024-03-13T14:49:39.087 INFO:tasks.workunit.client.0.smithi176.stdout:2/247: getdents d1/dc 0 2024-03-13T14:49:39.091 INFO:tasks.workunit.client.1.smithi196.stdout:7/491: write d0/d6/d33/d57/f7d [2757911,53939] 0 2024-03-13T14:49:39.094 INFO:tasks.workunit.client.0.smithi176.stdout:7/227: dwrite d1/d2/d2b/f37 [0,4194304] 0 2024-03-13T14:49:39.096 INFO:tasks.workunit.client.1.smithi196.stdout:8/335: chown d0/d1c/d29/c6d 7845 1 2024-03-13T14:49:39.099 INFO:tasks.workunit.client.0.smithi176.stdout:4/236: creat d7/d8/d2f/f58 x:0 0 0 2024-03-13T14:49:39.100 INFO:tasks.workunit.client.1.smithi196.stdout:9/332: truncate d7/de/d16/f2a 3248003 0 2024-03-13T14:49:39.106 INFO:tasks.workunit.client.0.smithi176.stdout:4/237: chown d7/d8/d15/d23 9 1 2024-03-13T14:49:39.107 INFO:tasks.workunit.client.0.smithi176.stdout:8/224: dwrite d0/f42 [0,4194304] 0 2024-03-13T14:49:39.110 INFO:tasks.workunit.client.0.smithi176.stdout:9/274: symlink d0/l55 0 2024-03-13T14:49:39.113 INFO:tasks.workunit.client.1.smithi196.stdout:2/280: dwrite d0/d6/f14 [0,4194304] 0 2024-03-13T14:49:39.114 INFO:tasks.workunit.client.0.smithi176.stdout:6/232: dwrite d0/d10/f3f [0,4194304] 0 2024-03-13T14:49:39.116 INFO:tasks.workunit.client.1.smithi196.stdout:8/336: dwrite d0/d10/d12/f50 [0,4194304] 0 2024-03-13T14:49:39.119 INFO:tasks.workunit.client.1.smithi196.stdout:8/337: fsync d0/d10/d12/f53 0 2024-03-13T14:49:39.122 INFO:tasks.workunit.client.1.smithi196.stdout:9/333: mkdir d7/d24/d28/d6c 0 2024-03-13T14:49:39.127 INFO:tasks.workunit.client.0.smithi176.stdout:0/206: dwrite d2/d5/dc/f2e [0,4194304] 0 2024-03-13T14:49:39.130 INFO:tasks.workunit.client.1.smithi196.stdout:7/492: rename d0/d6/l3e to d0/d78/d81/la6 0 2024-03-13T14:49:39.130 INFO:tasks.workunit.client.1.smithi196.stdout:2/281: truncate d0/d7/f21 552813 0 2024-03-13T14:49:39.132 INFO:tasks.workunit.client.0.smithi176.stdout:0/207: chown d2/d5/dc/f2e 1576444 1 2024-03-13T14:49:39.133 INFO:tasks.workunit.client.1.smithi196.stdout:8/338: rmdir d0/d10/d12/d17 39 2024-03-13T14:49:39.134 INFO:tasks.workunit.client.0.smithi176.stdout:2/248: creat d1/dc/d1c/f55 x:0 0 0 2024-03-13T14:49:39.137 INFO:tasks.workunit.client.1.smithi196.stdout:7/493: rename d0/d6/f19 to d0/d9/d31/d46/d47/fa7 0 2024-03-13T14:49:39.138 INFO:tasks.workunit.client.0.smithi176.stdout:0/208: readlink d2/d5/d1e/l24 0 2024-03-13T14:49:39.138 INFO:tasks.workunit.client.0.smithi176.stdout:9/275: mkdir d0/d1e/d1f/d56 0 2024-03-13T14:49:39.139 INFO:tasks.workunit.client.0.smithi176.stdout:8/225: rename d0/d5/d11/f33 to d0/db/d39/f43 0 2024-03-13T14:49:39.142 INFO:tasks.workunit.client.0.smithi176.stdout:2/249: symlink d1/dc/d40/l56 0 2024-03-13T14:49:39.145 INFO:tasks.workunit.client.0.smithi176.stdout:8/226: readlink d0/d5/l3a 0 2024-03-13T14:49:39.154 INFO:tasks.workunit.client.0.smithi176.stdout:9/276: chown d0/d1e/l24 25902953 1 2024-03-13T14:49:39.157 INFO:tasks.workunit.client.0.smithi176.stdout:8/227: chown d0/d5/d3b/f3c 230616334 1 2024-03-13T14:49:39.158 INFO:tasks.workunit.client.1.smithi196.stdout:3/378: sync 2024-03-13T14:49:39.158 INFO:tasks.workunit.client.1.smithi196.stdout:5/477: sync 2024-03-13T14:49:39.158 INFO:tasks.workunit.client.1.smithi196.stdout:2/282: dwrite d0/f25 [0,4194304] 0 2024-03-13T14:49:39.160 INFO:tasks.workunit.client.0.smithi176.stdout:9/277: stat d0/db/f31 0 2024-03-13T14:49:39.160 INFO:tasks.workunit.client.0.smithi176.stdout:4/238: dwrite d7/d8/f11 [0,4194304] 0 2024-03-13T14:49:39.166 INFO:tasks.workunit.client.1.smithi196.stdout:3/379: chown db/d19/d22/d3a/d53/f48 1921696268 1 2024-03-13T14:49:39.170 INFO:tasks.workunit.client.0.smithi176.stdout:6/233: dwrite d0/d9/f37 [0,4194304] 0 2024-03-13T14:49:39.174 INFO:tasks.workunit.client.1.smithi196.stdout:5/478: mkdir d4/db/d14/d19/d1c/d94 0 2024-03-13T14:49:39.178 INFO:tasks.workunit.client.0.smithi176.stdout:3/233: sync 2024-03-13T14:49:39.178 INFO:tasks.workunit.client.1.smithi196.stdout:5/479: chown d4/db/d14/d19/f50 50 1 2024-03-13T14:49:39.179 INFO:tasks.workunit.client.1.smithi196.stdout:8/339: dread d0/d1c/d3b/f38 [0,4194304] 0 2024-03-13T14:49:39.185 INFO:tasks.workunit.client.0.smithi176.stdout:4/239: mknod d7/c59 0 2024-03-13T14:49:39.185 INFO:tasks.workunit.client.0.smithi176.stdout:3/234: read - d4/d13/d2d/f4d zero size 2024-03-13T14:49:39.186 INFO:tasks.workunit.client.0.smithi176.stdout:3/235: stat d4/d13/f48 0 2024-03-13T14:49:39.196 INFO:tasks.workunit.client.1.smithi196.stdout:2/283: rename d0/fe to d0/d7/d9/d42/f5c 0 2024-03-13T14:49:39.200 INFO:tasks.workunit.client.1.smithi196.stdout:9/334: fsync d7/de/d16/f2a 0 2024-03-13T14:49:39.206 INFO:tasks.workunit.client.0.smithi176.stdout:4/240: creat d7/d8/d15/d23/f5a x:0 0 0 2024-03-13T14:49:39.206 INFO:tasks.workunit.client.1.smithi196.stdout:5/480: mkdir d4/db/d14/d19/d26/d95 0 2024-03-13T14:49:39.210 INFO:tasks.workunit.client.0.smithi176.stdout:9/278: rename d0/d1e/d1f/l20 to d0/l57 0 2024-03-13T14:49:39.211 INFO:tasks.workunit.client.1.smithi196.stdout:2/284: mknod d0/d50/c5d 0 2024-03-13T14:49:39.212 INFO:tasks.workunit.client.0.smithi176.stdout:9/279: chown d0/d13/l3d 29 1 2024-03-13T14:49:39.216 INFO:tasks.workunit.client.0.smithi176.stdout:6/234: creat d0/d10/d19/d27/f4b x:0 0 0 2024-03-13T14:49:39.216 INFO:tasks.workunit.client.0.smithi176.stdout:7/228: dread d1/d7/f10 [0,4194304] 0 2024-03-13T14:49:39.216 INFO:tasks.workunit.client.0.smithi176.stdout:4/241: rmdir d7/d10 39 2024-03-13T14:49:39.217 INFO:tasks.workunit.client.1.smithi196.stdout:2/285: write d0/d6/d34/d38/f3b [833000,79460] 0 2024-03-13T14:49:39.218 INFO:tasks.workunit.client.0.smithi176.stdout:0/209: dread d2/d5/dc/f16 [0,4194304] 0 2024-03-13T14:49:39.220 INFO:tasks.workunit.client.0.smithi176.stdout:9/280: mkdir d0/d13/d32/d58 0 2024-03-13T14:49:39.221 INFO:tasks.workunit.client.1.smithi196.stdout:8/340: dwrite d0/d10/d12/d17/f28 [0,4194304] 0 2024-03-13T14:49:39.224 INFO:tasks.workunit.client.1.smithi196.stdout:3/380: getdents db/d15/d44 0 2024-03-13T14:49:39.227 INFO:tasks.workunit.client.1.smithi196.stdout:3/381: chown db/d15/d44/l66 1810 1 2024-03-13T14:49:39.227 INFO:tasks.workunit.client.0.smithi176.stdout:4/242: write d7/d10/d2c/f3f [1546963,24288] 0 2024-03-13T14:49:39.228 INFO:tasks.workunit.client.1.smithi196.stdout:1/412: dwrite d1/f2a [0,4194304] 0 2024-03-13T14:49:39.229 INFO:tasks.workunit.client.1.smithi196.stdout:2/286: creat d0/d6/d1d/f5e x:0 0 0 2024-03-13T14:49:39.230 INFO:tasks.workunit.client.1.smithi196.stdout:1/413: fdatasync d1/f37 0 2024-03-13T14:49:39.231 INFO:tasks.workunit.client.0.smithi176.stdout:6/235: mkdir d0/de/d4c 0 2024-03-13T14:49:39.238 INFO:tasks.workunit.client.1.smithi196.stdout:3/382: stat db/d19/f2f 0 2024-03-13T14:49:39.243 INFO:tasks.workunit.client.1.smithi196.stdout:2/287: rename d0/d6/d34/d38 to d0/d6/d4b/d5f 0 2024-03-13T14:49:39.245 INFO:tasks.workunit.client.0.smithi176.stdout:4/243: symlink d7/d10/d46/l5b 0 2024-03-13T14:49:39.245 INFO:tasks.workunit.client.0.smithi176.stdout:1/232: dwrite d2/de/d3f/d19/f32 [0,4194304] 0 2024-03-13T14:49:39.245 INFO:tasks.workunit.client.0.smithi176.stdout:0/210: dwrite d2/f6 [4194304,4194304] 0 2024-03-13T14:49:39.246 INFO:tasks.workunit.client.0.smithi176.stdout:4/244: chown d7/d8/d15 0 1 2024-03-13T14:49:39.247 INFO:tasks.workunit.client.1.smithi196.stdout:5/481: creat d4/db/d14/d19/f96 x:0 0 0 2024-03-13T14:49:39.252 INFO:tasks.workunit.client.1.smithi196.stdout:8/341: mknod d0/c74 0 2024-03-13T14:49:39.253 INFO:tasks.workunit.client.0.smithi176.stdout:2/250: write d1/dc/d1c/f36 [467943,58467] 0 2024-03-13T14:49:39.254 INFO:tasks.workunit.client.1.smithi196.stdout:8/342: fdatasync d0/f61 0 2024-03-13T14:49:39.260 INFO:tasks.workunit.client.0.smithi176.stdout:9/281: rename d0/d13/d32/f3c to d0/f59 0 2024-03-13T14:49:39.261 INFO:tasks.workunit.client.1.smithi196.stdout:7/494: dwrite d0/d6/f21 [0,4194304] 0 2024-03-13T14:49:39.262 INFO:tasks.workunit.client.0.smithi176.stdout:1/233: fsync d2/de/d10/f2a 0 2024-03-13T14:49:39.262 INFO:tasks.workunit.client.0.smithi176.stdout:9/282: readlink d0/l15 0 2024-03-13T14:49:39.262 INFO:tasks.workunit.client.1.smithi196.stdout:3/383: symlink db/d15/d44/d51/l7a 0 2024-03-13T14:49:39.263 INFO:tasks.workunit.client.1.smithi196.stdout:3/384: readlink db/d15/l2a 0 2024-03-13T14:49:39.263 INFO:tasks.workunit.client.1.smithi196.stdout:7/495: stat d0/d9/d1e/d3f/f60 0 2024-03-13T14:49:39.265 INFO:tasks.workunit.client.1.smithi196.stdout:3/385: fsync db/d15/f41 0 2024-03-13T14:49:39.267 INFO:tasks.workunit.client.0.smithi176.stdout:6/236: dwrite d0/d10/f3f [4194304,4194304] 0 2024-03-13T14:49:39.268 INFO:tasks.workunit.client.0.smithi176.stdout:2/251: dread d1/dc/d2d/d38/f46 [0,4194304] 0 2024-03-13T14:49:39.271 INFO:tasks.workunit.client.0.smithi176.stdout:9/283: write d0/db/f49 [4092,129306] 0 2024-03-13T14:49:39.272 INFO:tasks.workunit.client.1.smithi196.stdout:2/288: write d0/d6/d1d/f27 [399584,2324] 0 2024-03-13T14:49:39.275 INFO:tasks.workunit.client.0.smithi176.stdout:6/237: readlink d0/d10/d19/d27/d2b/d2e/l30 0 2024-03-13T14:49:39.277 INFO:tasks.workunit.client.0.smithi176.stdout:6/238: fdatasync d0/d10/d19/d27/d2b/f3a 0 2024-03-13T14:49:39.279 INFO:tasks.workunit.client.1.smithi196.stdout:5/482: readlink d4/db/d14/d19/d26/d33/d61/l7b 0 2024-03-13T14:49:39.279 INFO:tasks.workunit.client.0.smithi176.stdout:5/211: sync 2024-03-13T14:49:39.280 INFO:tasks.workunit.client.0.smithi176.stdout:6/239: chown d0/d10/d19/d27/d2b/d2e 10090 1 2024-03-13T14:49:39.281 INFO:tasks.workunit.client.1.smithi196.stdout:7/496: dread d0/d9/d31/d46/d47/f4e [0,4194304] 0 2024-03-13T14:49:39.283 INFO:tasks.workunit.client.0.smithi176.stdout:1/234: mkdir d2/de/d10/d16/d47 0 2024-03-13T14:49:39.285 INFO:tasks.workunit.client.0.smithi176.stdout:1/235: write d2/d29/f39 [5033732,23902] 0 2024-03-13T14:49:39.288 INFO:tasks.workunit.client.0.smithi176.stdout:4/245: rename d7/f1c to d7/d8/d45/f5c 0 2024-03-13T14:49:39.288 INFO:tasks.workunit.client.0.smithi176.stdout:1/236: truncate d2/d30/f3a 348313 0 2024-03-13T14:49:39.290 INFO:tasks.workunit.client.0.smithi176.stdout:0/211: link d2/f11 d2/d5/d1e/f38 0 2024-03-13T14:49:39.292 INFO:tasks.workunit.client.0.smithi176.stdout:2/252: symlink d1/dc/d2d/d38/l57 0 2024-03-13T14:49:39.293 INFO:tasks.workunit.client.1.smithi196.stdout:3/386: symlink db/d37/d64/l7b 0 2024-03-13T14:49:39.296 INFO:tasks.workunit.client.1.smithi196.stdout:3/387: write db/d37/d64/f74 [219460,18913] 0 2024-03-13T14:49:39.300 INFO:tasks.workunit.client.0.smithi176.stdout:5/212: mkdir d5/d31/d47 0 2024-03-13T14:49:39.301 INFO:tasks.workunit.client.1.smithi196.stdout:7/497: mknod d0/d9/d31/d50/ca8 0 2024-03-13T14:49:39.302 INFO:tasks.workunit.client.0.smithi176.stdout:7/229: dwrite d1/f19 [0,4194304] 0 2024-03-13T14:49:39.302 INFO:tasks.workunit.client.0.smithi176.stdout:5/213: chown d5/d2b/f46 380635 1 2024-03-13T14:49:39.302 INFO:tasks.workunit.client.0.smithi176.stdout:6/240: rmdir d0 39 2024-03-13T14:49:39.308 INFO:tasks.workunit.client.0.smithi176.stdout:4/246: mkdir d7/d10/d46/d4a/d5d 0 2024-03-13T14:49:39.311 INFO:tasks.workunit.client.1.smithi196.stdout:1/414: truncate d1/d9/d33/d4b/f48 983615 0 2024-03-13T14:49:39.311 INFO:tasks.workunit.client.1.smithi196.stdout:5/483: symlink d4/db/d14/d19/d1c/d94/l97 0 2024-03-13T14:49:39.315 INFO:tasks.workunit.client.1.smithi196.stdout:2/289: link d0/f32 d0/d7/d40/f60 0 2024-03-13T14:49:39.316 INFO:tasks.workunit.client.0.smithi176.stdout:9/284: getdents d0/d40/d53 0 2024-03-13T14:49:39.318 INFO:tasks.workunit.client.1.smithi196.stdout:5/484: truncate d4/db/d14/d19/d1c/d2d/d82/f83 440330 0 2024-03-13T14:49:39.319 INFO:tasks.workunit.client.1.smithi196.stdout:7/498: dwrite d0/d9/f5a [0,4194304] 0 2024-03-13T14:49:39.321 INFO:tasks.workunit.client.1.smithi196.stdout:1/415: rename d1/d28/f2d to d1/d28/d34/f8e 0 2024-03-13T14:49:39.325 INFO:tasks.workunit.client.1.smithi196.stdout:3/388: link db/d19/d50/c6f db/d37/c7c 0 2024-03-13T14:49:39.329 INFO:tasks.workunit.client.0.smithi176.stdout:9/285: dread d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:39.332 INFO:tasks.workunit.client.0.smithi176.stdout:0/212: mkdir d2/db/d39 0 2024-03-13T14:49:39.333 INFO:tasks.workunit.client.1.smithi196.stdout:2/290: mkdir d0/d7/d9/d61 0 2024-03-13T14:49:39.333 INFO:tasks.workunit.client.0.smithi176.stdout:2/253: mknod d1/d2a/c58 0 2024-03-13T14:49:39.334 INFO:tasks.workunit.client.1.smithi196.stdout:2/291: chown d0/d7/c4c 3 1 2024-03-13T14:49:39.335 INFO:tasks.workunit.client.0.smithi176.stdout:5/214: mkdir d5/d48 0 2024-03-13T14:49:39.336 INFO:tasks.workunit.client.1.smithi196.stdout:3/389: symlink db/d19/d22/d3a/l7d 0 2024-03-13T14:49:39.340 INFO:tasks.workunit.client.1.smithi196.stdout:5/485: creat d4/db/d14/d19/f98 x:0 0 0 2024-03-13T14:49:39.341 INFO:tasks.workunit.client.0.smithi176.stdout:5/215: chown d5/l25 421754 1 2024-03-13T14:49:39.341 INFO:tasks.workunit.client.0.smithi176.stdout:1/237: dwrite d2/de/d10/d2e/f42 [0,4194304] 0 2024-03-13T14:49:39.342 INFO:tasks.workunit.client.1.smithi196.stdout:3/390: symlink db/d37/l7e 0 2024-03-13T14:49:39.346 INFO:tasks.workunit.client.0.smithi176.stdout:6/241: dwrite d0/f8 [4194304,4194304] 0 2024-03-13T14:49:39.348 INFO:tasks.workunit.client.1.smithi196.stdout:3/391: truncate db/f36 846677 0 2024-03-13T14:49:39.352 INFO:tasks.workunit.client.0.smithi176.stdout:4/247: dwrite d7/d8/d15/f2b [0,4194304] 0 2024-03-13T14:49:39.353 INFO:tasks.workunit.client.0.smithi176.stdout:7/230: creat d1/f3c x:0 0 0 2024-03-13T14:49:39.354 INFO:tasks.workunit.client.0.smithi176.stdout:6/242: chown d0/d10/d19/d27/d2b/d2e/f2d 1 1 2024-03-13T14:49:39.355 INFO:tasks.workunit.client.0.smithi176.stdout:2/254: chown d1/d2a/d33/l47 164 1 2024-03-13T14:49:39.355 INFO:tasks.workunit.client.1.smithi196.stdout:3/392: mkdir db/d37/d64/d7f 0 2024-03-13T14:49:39.357 INFO:tasks.workunit.client.0.smithi176.stdout:4/248: readlink d7/d8/d15/d23/l29 0 2024-03-13T14:49:39.358 INFO:tasks.workunit.client.1.smithi196.stdout:7/499: dwrite d0/f37 [0,4194304] 0 2024-03-13T14:49:39.371 INFO:tasks.workunit.client.1.smithi196.stdout:5/486: dread d4/f8 [4194304,4194304] 0 2024-03-13T14:49:39.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:39 smithi176 ceph-mon[29064]: pgmap v8: 65 pgs: 65 active+clean; 919 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 84 MiB/s wr, 158 op/s 2024-03-13T14:49:39.388 INFO:tasks.workunit.client.0.smithi176.stdout:0/213: symlink d2/db/l3a 0 2024-03-13T14:49:39.396 INFO:tasks.workunit.client.0.smithi176.stdout:9/286: creat d0/f5a x:0 0 0 2024-03-13T14:49:39.397 INFO:tasks.workunit.client.0.smithi176.stdout:9/287: dread - d0/db/f3f zero size 2024-03-13T14:49:39.398 INFO:tasks.workunit.client.1.smithi196.stdout:7/500: fsync d0/d9/d1e/d3f/f82 0 2024-03-13T14:49:39.399 INFO:tasks.workunit.client.1.smithi196.stdout:5/487: truncate d4/db/d14/d19/d26/f3a 2462082 0 2024-03-13T14:49:39.402 INFO:tasks.workunit.client.0.smithi176.stdout:2/255: mkdir d1/dc/d2d/d59 0 2024-03-13T14:49:39.402 INFO:tasks.workunit.client.1.smithi196.stdout:5/488: write d4/db/f24 [482433,39448] 0 2024-03-13T14:49:39.403 INFO:tasks.workunit.client.0.smithi176.stdout:2/256: chown d1/dc/cd 524 1 2024-03-13T14:49:39.406 INFO:tasks.workunit.client.1.smithi196.stdout:3/393: creat db/d15/d44/d79/f80 x:0 0 0 2024-03-13T14:49:39.408 INFO:tasks.workunit.client.0.smithi176.stdout:7/231: fdatasync d1/d7/f29 0 2024-03-13T14:49:39.412 INFO:tasks.workunit.client.0.smithi176.stdout:7/232: write d1/d1c/d26/f3b [891917,127011] 0 2024-03-13T14:49:39.413 INFO:tasks.workunit.client.0.smithi176.stdout:9/288: unlink d0/d17/l39 0 2024-03-13T14:49:39.415 INFO:tasks.workunit.client.1.smithi196.stdout:3/394: stat db/d19/c27 0 2024-03-13T14:49:39.419 INFO:tasks.workunit.client.0.smithi176.stdout:2/257: creat d1/dc/d2d/d38/f5a x:0 0 0 2024-03-13T14:49:39.426 INFO:tasks.workunit.client.0.smithi176.stdout:7/233: creat d1/d7/f3d x:0 0 0 2024-03-13T14:49:39.426 INFO:tasks.workunit.client.1.smithi196.stdout:3/395: read db/d19/f55 [859776,63803] 0 2024-03-13T14:49:39.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:39 smithi196 ceph-mon[37486]: pgmap v8: 65 pgs: 65 active+clean; 919 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 15 MiB/s rd, 84 MiB/s wr, 158 op/s 2024-03-13T14:49:39.433 INFO:tasks.workunit.client.0.smithi176.stdout:0/214: creat d2/db/f3b x:0 0 0 2024-03-13T14:49:39.437 INFO:tasks.workunit.client.1.smithi196.stdout:7/501: dwrite d0/d9/f5a [4194304,4194304] 0 2024-03-13T14:49:39.437 INFO:tasks.workunit.client.0.smithi176.stdout:2/258: rename d1/dc/d1c/f1d to d1/d2a/d33/f5b 0 2024-03-13T14:49:39.440 INFO:tasks.workunit.client.1.smithi196.stdout:7/502: fdatasync d0/d6/d33/d57/f85 0 2024-03-13T14:49:39.440 INFO:tasks.workunit.client.0.smithi176.stdout:9/289: creat d0/d1e/d1f/d56/f5b x:0 0 0 2024-03-13T14:49:39.444 INFO:tasks.workunit.client.0.smithi176.stdout:4/249: link d7/ca d7/d8/c5e 0 2024-03-13T14:49:39.444 INFO:tasks.workunit.client.1.smithi196.stdout:3/396: write db/d19/d22/d3a/d53/f48 [5152424,80661] 0 2024-03-13T14:49:39.445 INFO:tasks.workunit.client.0.smithi176.stdout:9/290: write d0/d17/f48 [1402422,13554] 0 2024-03-13T14:49:39.446 INFO:tasks.workunit.client.1.smithi196.stdout:7/503: dread - d0/f6b zero size 2024-03-13T14:49:39.447 INFO:tasks.workunit.client.1.smithi196.stdout:5/489: dwrite d4/db/d14/d19/d26/f43 [0,4194304] 0 2024-03-13T14:49:39.450 INFO:tasks.workunit.client.0.smithi176.stdout:2/259: mkdir d1/d51/d5c 0 2024-03-13T14:49:39.450 INFO:tasks.workunit.client.1.smithi196.stdout:7/504: write d0/d9/d1e/d58/d72/f66 [793157,74094] 0 2024-03-13T14:49:39.451 INFO:tasks.workunit.client.0.smithi176.stdout:2/260: readlink d1/l3 0 2024-03-13T14:49:39.452 INFO:tasks.workunit.client.0.smithi176.stdout:9/291: readlink d0/d13/l3d 0 2024-03-13T14:49:39.453 INFO:tasks.workunit.client.0.smithi176.stdout:9/292: fsync d0/db/f49 0 2024-03-13T14:49:39.454 INFO:tasks.workunit.client.0.smithi176.stdout:7/234: dwrite d1/d2/d2b/f37 [0,4194304] 0 2024-03-13T14:49:39.455 INFO:tasks.workunit.client.0.smithi176.stdout:2/261: chown d1/dc/d2d/d38/f3f 3145495 1 2024-03-13T14:49:39.456 INFO:tasks.workunit.client.0.smithi176.stdout:4/250: mknod d7/d10/d46/c5f 0 2024-03-13T14:49:39.457 INFO:tasks.workunit.client.0.smithi176.stdout:2/262: readlink d1/dc/l42 0 2024-03-13T14:49:39.458 INFO:tasks.workunit.client.0.smithi176.stdout:9/293: fsync d0/db/f3f 0 2024-03-13T14:49:39.462 INFO:tasks.workunit.client.0.smithi176.stdout:0/215: dread d2/d5/d1e/f38 [0,4194304] 0 2024-03-13T14:49:39.463 INFO:tasks.workunit.client.0.smithi176.stdout:5/216: dread d5/f14 [4194304,4194304] 0 2024-03-13T14:49:39.466 INFO:tasks.workunit.client.0.smithi176.stdout:2/263: read - d1/dc/d1c/f37 zero size 2024-03-13T14:49:39.466 INFO:tasks.workunit.client.0.smithi176.stdout:5/217: fsync d5/f37 0 2024-03-13T14:49:39.466 INFO:tasks.workunit.client.0.smithi176.stdout:9/294: chown d0/d40/c44 164 1 2024-03-13T14:49:39.466 INFO:tasks.workunit.client.0.smithi176.stdout:7/235: creat d1/d2/f3e x:0 0 0 2024-03-13T14:49:39.470 INFO:tasks.workunit.client.0.smithi176.stdout:9/295: dread d0/db/f49 [0,4194304] 0 2024-03-13T14:49:39.472 INFO:tasks.workunit.client.0.smithi176.stdout:9/296: readlink d0/l2b 0 2024-03-13T14:49:39.478 INFO:tasks.workunit.client.1.smithi196.stdout:3/397: dwrite db/f2c [0,4194304] 0 2024-03-13T14:49:39.484 INFO:tasks.workunit.client.1.smithi196.stdout:3/398: truncate db/d37/f76 369779 0 2024-03-13T14:49:39.488 INFO:tasks.workunit.client.0.smithi176.stdout:7/236: unlink d1/l34 0 2024-03-13T14:49:39.491 INFO:tasks.workunit.client.0.smithi176.stdout:2/264: mkdir d1/dc/d2d/d38/d5d 0 2024-03-13T14:49:39.499 INFO:tasks.workunit.client.0.smithi176.stdout:8/228: sync 2024-03-13T14:49:39.500 INFO:tasks.workunit.client.0.smithi176.stdout:0/216: fdatasync d2/db/f21 0 2024-03-13T14:49:39.502 INFO:tasks.workunit.client.0.smithi176.stdout:5/218: mkdir d5/d48/d49 0 2024-03-13T14:49:39.511 INFO:tasks.workunit.client.1.smithi196.stdout:7/505: dwrite d0/d6/d33/f39 [4194304,4194304] 0 2024-03-13T14:49:39.515 INFO:tasks.workunit.client.1.smithi196.stdout:3/399: getdents db/d19/d22 0 2024-03-13T14:49:39.521 INFO:tasks.workunit.client.0.smithi176.stdout:4/251: truncate d7/d8/f42 1202557 0 2024-03-13T14:49:39.524 INFO:tasks.workunit.client.1.smithi196.stdout:7/506: rename d0/d9/d31/d46 to d0/d9/d1e/d58/d72/da9 0 2024-03-13T14:49:39.527 INFO:tasks.workunit.client.0.smithi176.stdout:5/219: truncate d5/d22/f13 5923 0 2024-03-13T14:49:39.527 INFO:tasks.workunit.client.1.smithi196.stdout:3/400: dread db/d19/d22/d30/f42 [0,4194304] 0 2024-03-13T14:49:39.528 INFO:tasks.workunit.client.0.smithi176.stdout:7/237: mkdir d1/d3f 0 2024-03-13T14:49:39.529 INFO:tasks.workunit.client.1.smithi196.stdout:3/401: rename db/d37/d64 to db/d37/d64/d81 22 2024-03-13T14:49:39.530 INFO:tasks.workunit.client.0.smithi176.stdout:9/297: link d0/d1e/l24 d0/d13/d32/l5c 0 2024-03-13T14:49:39.533 INFO:tasks.workunit.client.0.smithi176.stdout:4/252: unlink d7/d8/d2f/c53 0 2024-03-13T14:49:39.533 INFO:tasks.workunit.client.1.smithi196.stdout:3/402: fdatasync db/d19/f60 0 2024-03-13T14:49:39.534 INFO:tasks.workunit.client.0.smithi176.stdout:0/217: truncate d2/db/f1c 2648691 0 2024-03-13T14:49:39.535 INFO:tasks.workunit.client.0.smithi176.stdout:2/265: dwrite d1/f14 [0,4194304] 0 2024-03-13T14:49:39.538 INFO:tasks.workunit.client.1.smithi196.stdout:7/507: creat d0/d6/faa x:0 0 0 2024-03-13T14:49:39.538 INFO:tasks.workunit.client.0.smithi176.stdout:7/238: mkdir d1/d2/d2b/d40 0 2024-03-13T14:49:39.539 INFO:tasks.workunit.client.0.smithi176.stdout:9/298: rmdir d0/db 39 2024-03-13T14:49:39.541 INFO:tasks.workunit.client.0.smithi176.stdout:8/229: dwrite d0/db/f19 [0,4194304] 0 2024-03-13T14:49:39.542 INFO:tasks.workunit.client.0.smithi176.stdout:2/266: write d1/dc/d2d/d38/f3f [1690578,119509] 0 2024-03-13T14:49:39.542 INFO:tasks.workunit.client.0.smithi176.stdout:4/253: dread d7/d8/f28 [0,4194304] 0 2024-03-13T14:49:39.554 INFO:tasks.workunit.client.0.smithi176.stdout:0/218: truncate d2/f3 2925216 0 2024-03-13T14:49:39.555 INFO:tasks.workunit.client.0.smithi176.stdout:5/220: dwrite f0 [8388608,4194304] 0 2024-03-13T14:49:39.557 INFO:tasks.workunit.client.0.smithi176.stdout:5/221: chown d5/d22/d15/f16 114 1 2024-03-13T14:49:39.557 INFO:tasks.workunit.client.0.smithi176.stdout:5/222: chown d5/l25 101466 1 2024-03-13T14:49:39.558 INFO:tasks.workunit.client.1.smithi196.stdout:3/403: dwrite db/d19/d22/d30/f42 [0,4194304] 0 2024-03-13T14:49:39.561 INFO:tasks.workunit.client.1.smithi196.stdout:3/404: fdatasync db/f70 0 2024-03-13T14:49:39.566 INFO:tasks.workunit.client.0.smithi176.stdout:8/230: rmdir d0/db/dc 39 2024-03-13T14:49:39.569 INFO:tasks.workunit.client.0.smithi176.stdout:6/243: sync 2024-03-13T14:49:39.570 INFO:tasks.workunit.client.0.smithi176.stdout:1/238: sync 2024-03-13T14:49:39.570 INFO:tasks.workunit.client.0.smithi176.stdout:3/236: sync 2024-03-13T14:49:39.570 INFO:tasks.workunit.client.0.smithi176.stdout:7/239: rename d1/d2 to d1/d1c/d26/d41 0 2024-03-13T14:49:39.574 INFO:tasks.workunit.client.1.smithi196.stdout:6/528: sync 2024-03-13T14:49:39.574 INFO:tasks.workunit.client.1.smithi196.stdout:0/433: sync 2024-03-13T14:49:39.574 INFO:tasks.workunit.client.1.smithi196.stdout:4/362: sync 2024-03-13T14:49:39.582 INFO:tasks.workunit.client.0.smithi176.stdout:2/267: rename d1/dc/d2d/d38/f4e to d1/dc/d2d/d38/f5e 0 2024-03-13T14:49:39.583 INFO:tasks.workunit.client.0.smithi176.stdout:2/268: stat d1/dc/d1c/f36 0 2024-03-13T14:49:39.584 INFO:tasks.workunit.client.0.smithi176.stdout:9/299: mkdir d0/d13/d4d/d5d 0 2024-03-13T14:49:39.587 INFO:tasks.workunit.client.0.smithi176.stdout:0/219: chown d2/db/d35/c2f 179 1 2024-03-13T14:49:39.587 INFO:tasks.workunit.client.0.smithi176.stdout:1/239: dwrite d2/de/d10/d2e/f42 [0,4194304] 0 2024-03-13T14:49:39.605 INFO:tasks.workunit.client.1.smithi196.stdout:3/405: rename db/d19/f26 to db/d19/d22/f82 0 2024-03-13T14:49:39.607 INFO:tasks.workunit.client.0.smithi176.stdout:3/237: mknod d4/d13/c4e 0 2024-03-13T14:49:39.607 INFO:tasks.workunit.client.1.smithi196.stdout:7/508: dread d0/d9/d31/d50/f87 [0,4194304] 0 2024-03-13T14:49:39.613 INFO:tasks.workunit.client.0.smithi176.stdout:1/240: fsync d2/de/d10/f2a 0 2024-03-13T14:49:39.614 INFO:tasks.workunit.client.1.smithi196.stdout:6/529: truncate d6/f8 286297 0 2024-03-13T14:49:39.620 INFO:tasks.workunit.client.0.smithi176.stdout:3/238: dwrite d4/d28/d2e/f46 [0,4194304] 0 2024-03-13T14:49:39.621 INFO:tasks.workunit.client.1.smithi196.stdout:4/363: creat d1/d1e/f6a x:0 0 0 2024-03-13T14:49:39.624 INFO:tasks.workunit.client.0.smithi176.stdout:3/239: read d4/d28/d2e/f3c [121275,122126] 0 2024-03-13T14:49:39.633 INFO:tasks.workunit.client.1.smithi196.stdout:0/434: dwrite d8/da/f3e [0,4194304] 0 2024-03-13T14:49:39.635 INFO:tasks.workunit.client.1.smithi196.stdout:0/435: chown d8/da/d25/d26/d35/d3b/d5f/l30 166110 1 2024-03-13T14:49:39.639 INFO:tasks.workunit.client.1.smithi196.stdout:6/530: rename d6/d9/d17/d19/f2d to d6/d57/d7a/fa6 0 2024-03-13T14:49:39.643 INFO:tasks.workunit.client.0.smithi176.stdout:3/240: dwrite d4/fa [0,4194304] 0 2024-03-13T14:49:39.649 INFO:tasks.workunit.client.1.smithi196.stdout:4/364: readlink d1/d3/d7/d35/l67 0 2024-03-13T14:49:39.651 INFO:tasks.workunit.client.0.smithi176.stdout:0/220: truncate d2/d5/dc/f19 1128945 0 2024-03-13T14:49:39.653 INFO:tasks.workunit.client.0.smithi176.stdout:4/254: truncate d7/f24 854505 0 2024-03-13T14:49:39.654 INFO:tasks.workunit.client.0.smithi176.stdout:4/255: stat d7/d8/f28 0 2024-03-13T14:49:39.657 INFO:tasks.workunit.client.1.smithi196.stdout:9/335: sync 2024-03-13T14:49:39.657 INFO:tasks.workunit.client.1.smithi196.stdout:3/406: chown db/d19/l56 81 1 2024-03-13T14:49:39.657 INFO:tasks.workunit.client.1.smithi196.stdout:7/509: mknod d0/d9/d1e/d58/cab 0 2024-03-13T14:49:39.659 INFO:tasks.workunit.client.1.smithi196.stdout:0/436: mknod d8/da/d25/d26/d35/d3b/d5f/d17/c92 0 2024-03-13T14:49:39.659 INFO:tasks.workunit.client.1.smithi196.stdout:9/336: dread - d7/de/d4b/d1d/f54 zero size 2024-03-13T14:49:39.660 INFO:tasks.workunit.client.0.smithi176.stdout:7/240: symlink d1/d28/d3a/d1b/l42 0 2024-03-13T14:49:39.662 INFO:tasks.workunit.client.1.smithi196.stdout:0/437: read d8/da/d25/d26/f2d [3970945,80874] 0 2024-03-13T14:49:39.666 INFO:tasks.workunit.client.0.smithi176.stdout:2/269: creat d1/d51/d5c/f5f x:0 0 0 2024-03-13T14:49:39.667 INFO:tasks.workunit.client.1.smithi196.stdout:0/438: read d8/da/d25/d26/d35/d3b/d5f/d33/d44/f86 [3284526,60829] 0 2024-03-13T14:49:39.668 INFO:tasks.workunit.client.1.smithi196.stdout:7/510: mknod d0/d9/d1e/cac 0 2024-03-13T14:49:39.672 INFO:tasks.workunit.client.1.smithi196.stdout:3/407: dread db/f36 [0,4194304] 0 2024-03-13T14:49:39.674 INFO:tasks.workunit.client.0.smithi176.stdout:3/241: mkdir d4/d28/d4f 0 2024-03-13T14:49:39.675 INFO:tasks.workunit.client.0.smithi176.stdout:3/242: read f1 [3807719,116807] 0 2024-03-13T14:49:39.675 INFO:tasks.workunit.client.1.smithi196.stdout:0/439: truncate d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 1192052 0 2024-03-13T14:49:39.676 INFO:tasks.workunit.client.0.smithi176.stdout:3/243: write d4/f36 [1951206,44091] 0 2024-03-13T14:49:39.679 INFO:tasks.workunit.client.1.smithi196.stdout:4/365: write d1/d3/d7/f65 [900306,89365] 0 2024-03-13T14:49:39.685 INFO:tasks.workunit.client.0.smithi176.stdout:1/241: dread d2/fc [4194304,4194304] 0 2024-03-13T14:49:39.689 INFO:tasks.workunit.client.0.smithi176.stdout:6/244: rename d0/de/f23 to d0/d10/d19/d27/f4d 0 2024-03-13T14:49:39.691 INFO:tasks.workunit.client.1.smithi196.stdout:6/531: write d6/d9/f1f [3877952,20123] 0 2024-03-13T14:49:39.694 INFO:tasks.workunit.client.0.smithi176.stdout:2/270: symlink d1/dc/d1c/l60 0 2024-03-13T14:49:39.695 INFO:tasks.workunit.client.0.smithi176.stdout:9/300: creat d0/f5e x:0 0 0 2024-03-13T14:49:39.695 INFO:tasks.workunit.client.0.smithi176.stdout:2/271: chown d1/f13 29962615 1 2024-03-13T14:49:39.696 INFO:tasks.workunit.client.0.smithi176.stdout:9/301: dread - d0/f5e zero size 2024-03-13T14:49:39.699 INFO:tasks.workunit.client.1.smithi196.stdout:7/511: write d0/d4d/f5b [2762779,38626] 0 2024-03-13T14:49:39.700 INFO:tasks.workunit.client.0.smithi176.stdout:0/221: mknod d2/d5/c3c 0 2024-03-13T14:49:39.705 INFO:tasks.workunit.client.0.smithi176.stdout:8/231: link d0/l3d d0/db/l44 0 2024-03-13T14:49:39.705 INFO:tasks.workunit.client.0.smithi176.stdout:1/242: symlink d2/de/d10/d14/d35/l48 0 2024-03-13T14:49:39.710 INFO:tasks.workunit.client.0.smithi176.stdout:6/245: creat d0/f4e x:0 0 0 2024-03-13T14:49:39.712 INFO:tasks.workunit.client.0.smithi176.stdout:2/272: mkdir d1/dc/d61 0 2024-03-13T14:49:39.718 INFO:tasks.workunit.client.0.smithi176.stdout:8/232: creat d0/d5/f45 x:0 0 0 2024-03-13T14:49:39.720 INFO:tasks.workunit.client.0.smithi176.stdout:0/222: rename d2/d5/d28/f33 to d2/d5/d1e/f3d 0 2024-03-13T14:49:39.722 INFO:tasks.workunit.client.0.smithi176.stdout:1/243: rmdir d2/de/d3f 39 2024-03-13T14:49:39.723 INFO:tasks.workunit.client.1.smithi196.stdout:9/337: symlink d7/de/d4b/d1d/d35/l6d 0 2024-03-13T14:49:39.727 INFO:tasks.workunit.client.0.smithi176.stdout:9/302: mkdir d0/d13/d32/d58/d5f 0 2024-03-13T14:49:39.729 INFO:tasks.workunit.client.0.smithi176.stdout:4/256: getdents d7/d8/d15 0 2024-03-13T14:49:39.733 INFO:tasks.workunit.client.1.smithi196.stdout:2/292: sync 2024-03-13T14:49:39.733 INFO:tasks.workunit.client.1.smithi196.stdout:1/416: sync 2024-03-13T14:49:39.733 INFO:tasks.workunit.client.1.smithi196.stdout:8/343: sync 2024-03-13T14:49:39.733 INFO:tasks.workunit.client.1.smithi196.stdout:5/490: sync 2024-03-13T14:49:39.734 INFO:tasks.workunit.client.0.smithi176.stdout:7/241: getdents d1/d1c/d26/d41/d2b 0 2024-03-13T14:49:39.739 INFO:tasks.workunit.client.1.smithi196.stdout:7/512: creat d0/d9/d1e/d58/d72/da9/da3/fad x:0 0 0 2024-03-13T14:49:39.741 INFO:tasks.workunit.client.1.smithi196.stdout:9/338: dwrite d7/de/d4b/d1d/d22/f5a [0,4194304] 0 2024-03-13T14:49:39.741 INFO:tasks.workunit.client.1.smithi196.stdout:7/513: fdatasync d0/d9/d1e/d58/d72/f7c 0 2024-03-13T14:49:39.743 INFO:tasks.workunit.client.0.smithi176.stdout:0/223: read - d2/db/f25 zero size 2024-03-13T14:49:39.747 INFO:tasks.workunit.client.1.smithi196.stdout:3/408: mknod db/d19/d22/d3a/d68/c83 0 2024-03-13T14:49:39.747 INFO:tasks.workunit.client.0.smithi176.stdout:9/303: write d0/db/f3f [425434,236] 0 2024-03-13T14:49:39.748 INFO:tasks.workunit.client.0.smithi176.stdout:4/257: dread d7/d8/d15/f3e [0,4194304] 0 2024-03-13T14:49:39.749 INFO:tasks.workunit.client.1.smithi196.stdout:3/409: write db/d15/f17 [1623657,90625] 0 2024-03-13T14:49:39.750 INFO:tasks.workunit.client.1.smithi196.stdout:4/366: write d1/d1e/d39/f32 [510025,75885] 0 2024-03-13T14:49:39.752 INFO:tasks.workunit.client.0.smithi176.stdout:7/242: mkdir d1/d28/d43 0 2024-03-13T14:49:39.754 INFO:tasks.workunit.client.1.smithi196.stdout:5/491: symlink d4/db/d4a/l99 0 2024-03-13T14:49:39.755 INFO:tasks.workunit.client.1.smithi196.stdout:0/440: link d8/c6b d8/da/d25/d26/d54/d88/c93 0 2024-03-13T14:49:39.759 INFO:tasks.workunit.client.0.smithi176.stdout:8/233: rename d0/d24/f37 to d0/db/dc/d40/f46 0 2024-03-13T14:49:39.760 INFO:tasks.workunit.client.0.smithi176.stdout:0/224: rmdir d2 39 2024-03-13T14:49:39.761 INFO:tasks.workunit.client.0.smithi176.stdout:1/244: dread d2/d29/f3c [0,4194304] 0 2024-03-13T14:49:39.764 INFO:tasks.workunit.client.0.smithi176.stdout:6/246: readlink d0/d10/d19/d27/d2b/d2e/l30 0 2024-03-13T14:49:39.764 INFO:tasks.workunit.client.0.smithi176.stdout:4/258: chown d7/l3c 4091578 1 2024-03-13T14:49:39.767 INFO:tasks.workunit.client.1.smithi196.stdout:5/492: write d4/db/d14/d19/d1c/d2d/d82/f83 [1373579,61807] 0 2024-03-13T14:49:39.767 INFO:tasks.workunit.client.0.smithi176.stdout:6/247: truncate d0/d10/d19/d27/d2b/f3a 508208 0 2024-03-13T14:49:39.768 INFO:tasks.workunit.client.0.smithi176.stdout:6/248: readlink d0/d10/d19/d27/l3d 0 2024-03-13T14:49:39.768 INFO:tasks.workunit.client.0.smithi176.stdout:7/243: unlink d1/d7/f29 0 2024-03-13T14:49:39.773 INFO:tasks.workunit.client.1.smithi196.stdout:5/493: read d4/db/d14/f5e [893101,65437] 0 2024-03-13T14:49:39.773 INFO:tasks.workunit.client.1.smithi196.stdout:2/293: link d0/f25 d0/d6/d34/f62 0 2024-03-13T14:49:39.774 INFO:tasks.workunit.client.0.smithi176.stdout:8/234: rmdir d0/d5/d11 39 2024-03-13T14:49:39.775 INFO:tasks.workunit.client.1.smithi196.stdout:6/532: dwrite d6/d2a/dc/f3e [0,4194304] 0 2024-03-13T14:49:39.780 INFO:tasks.workunit.client.1.smithi196.stdout:3/410: rename db/d15/d44/d51/f6e to db/d19/d22/f84 0 2024-03-13T14:49:39.780 INFO:tasks.workunit.client.0.smithi176.stdout:5/223: sync 2024-03-13T14:49:39.784 INFO:tasks.workunit.client.1.smithi196.stdout:2/294: chown d0/d6/d1d/f30 1149 1 2024-03-13T14:49:39.786 INFO:tasks.workunit.client.0.smithi176.stdout:6/249: dwrite d0/d10/d19/d27/d2b/f3a [0,4194304] 0 2024-03-13T14:49:39.789 INFO:tasks.workunit.client.1.smithi196.stdout:7/514: dwrite d0/d9/f27 [4194304,4194304] 0 2024-03-13T14:49:39.800 INFO:tasks.workunit.client.0.smithi176.stdout:1/245: write d2/fc [2491166,96497] 0 2024-03-13T14:49:39.805 INFO:tasks.workunit.client.1.smithi196.stdout:0/441: chown d8/da/d25/d26/d54/d88/c80 2 1 2024-03-13T14:49:39.805 INFO:tasks.workunit.client.0.smithi176.stdout:0/225: creat d2/d5/d1e/f3e x:0 0 0 2024-03-13T14:49:39.810 INFO:tasks.workunit.client.1.smithi196.stdout:8/344: truncate d0/d1c/d3b/d3f/f45 1099839 0 2024-03-13T14:49:39.811 INFO:tasks.workunit.client.0.smithi176.stdout:5/224: mkdir d5/d2c/d4a 0 2024-03-13T14:49:39.812 INFO:tasks.workunit.client.0.smithi176.stdout:5/225: chown d5/d31/d47 39944688 1 2024-03-13T14:49:39.813 INFO:tasks.workunit.client.0.smithi176.stdout:6/250: stat d0/c1 0 2024-03-13T14:49:39.814 INFO:tasks.workunit.client.0.smithi176.stdout:9/304: truncate d0/db/f1d 387131 0 2024-03-13T14:49:39.815 INFO:tasks.workunit.client.0.smithi176.stdout:7/244: symlink d1/d1c/d26/d41/d2b/d40/l44 0 2024-03-13T14:49:39.816 INFO:tasks.workunit.client.0.smithi176.stdout:9/305: read - d0/d1e/d1f/d56/f5b zero size 2024-03-13T14:49:39.817 INFO:tasks.workunit.client.1.smithi196.stdout:9/339: creat d7/de/d4b/d1d/f6e x:0 0 0 2024-03-13T14:49:39.818 INFO:tasks.workunit.client.0.smithi176.stdout:1/246: unlink d2/de/d10/c12 0 2024-03-13T14:49:39.820 INFO:tasks.workunit.client.0.smithi176.stdout:8/235: mkdir d0/d5/d11/d47 0 2024-03-13T14:49:39.822 INFO:tasks.workunit.client.1.smithi196.stdout:2/295: dwrite d0/d7/f3f [0,4194304] 0 2024-03-13T14:49:39.823 INFO:tasks.workunit.client.1.smithi196.stdout:3/411: symlink db/d71/l85 0 2024-03-13T14:49:39.823 INFO:tasks.workunit.client.1.smithi196.stdout:1/417: getdents d1/d9/d33/d4b/d66/d7b 0 2024-03-13T14:49:39.824 INFO:tasks.workunit.client.0.smithi176.stdout:0/226: stat d2/f3 0 2024-03-13T14:49:39.826 INFO:tasks.workunit.client.1.smithi196.stdout:9/340: chown d7/de/d4b/d1d/d22/f5a 462118 1 2024-03-13T14:49:39.826 INFO:tasks.workunit.client.0.smithi176.stdout:0/227: dread - d2/d5/d1e/f3d zero size 2024-03-13T14:49:39.830 INFO:tasks.workunit.client.0.smithi176.stdout:1/247: dread d2/de/d10/f2a [0,4194304] 0 2024-03-13T14:49:39.834 INFO:tasks.workunit.client.1.smithi196.stdout:9/341: write d7/f45 [3670355,117172] 0 2024-03-13T14:49:39.836 INFO:tasks.workunit.client.0.smithi176.stdout:7/245: chown d1/d28/d3a/d1b/l1d 2 1 2024-03-13T14:49:39.838 INFO:tasks.workunit.client.0.smithi176.stdout:7/246: rename d1 to d1/d1c/d26/d41/d45 22 2024-03-13T14:49:39.840 INFO:tasks.workunit.client.0.smithi176.stdout:7/247: read d1/f19 [1642230,16775] 0 2024-03-13T14:49:39.841 INFO:tasks.workunit.client.1.smithi196.stdout:7/515: dread d0/d4d/f63 [0,4194304] 0 2024-03-13T14:49:39.844 INFO:tasks.workunit.client.1.smithi196.stdout:2/296: fdatasync d0/d6/d34/f62 0 2024-03-13T14:49:39.844 INFO:tasks.workunit.client.1.smithi196.stdout:0/442: dread d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 [0,4194304] 0 2024-03-13T14:49:39.848 INFO:tasks.workunit.client.0.smithi176.stdout:1/248: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:39.856 INFO:tasks.workunit.client.1.smithi196.stdout:8/345: dwrite d0/d1c/d29/d47/f52 [0,4194304] 0 2024-03-13T14:49:39.856 INFO:tasks.workunit.client.0.smithi176.stdout:8/236: creat d0/db/dc/d40/f48 x:0 0 0 2024-03-13T14:49:39.857 INFO:tasks.workunit.client.0.smithi176.stdout:8/237: truncate d0/f30 802601 0 2024-03-13T14:49:39.861 INFO:tasks.workunit.client.1.smithi196.stdout:4/367: dread d1/d3/d7/d35/f37 [0,4194304] 0 2024-03-13T14:49:39.865 INFO:tasks.workunit.client.0.smithi176.stdout:3/244: sync 2024-03-13T14:49:39.866 INFO:tasks.workunit.client.0.smithi176.stdout:3/245: stat d4/f10 0 2024-03-13T14:49:39.869 INFO:tasks.workunit.client.1.smithi196.stdout:5/494: dread d4/db/d3c/f52 [0,4194304] 0 2024-03-13T14:49:39.873 INFO:tasks.workunit.client.1.smithi196.stdout:6/533: rename d6/d57/d7a to d6/d2a/dc/d1c/da7 0 2024-03-13T14:49:39.877 INFO:tasks.workunit.client.1.smithi196.stdout:6/534: fsync d6/d7/df/d27/f9d 0 2024-03-13T14:49:39.883 INFO:tasks.workunit.client.0.smithi176.stdout:2/273: link d1/dc/f17 d1/dc/d2d/d38/d5d/f62 0 2024-03-13T14:49:39.884 INFO:tasks.workunit.client.0.smithi176.stdout:2/274: stat d1/f13 0 2024-03-13T14:49:39.886 INFO:tasks.workunit.client.1.smithi196.stdout:7/516: rmdir d0/d4d/d74 39 2024-03-13T14:49:39.889 INFO:tasks.workunit.client.1.smithi196.stdout:6/535: dread d6/d9/d11/f48 [0,4194304] 0 2024-03-13T14:49:39.890 INFO:tasks.workunit.client.0.smithi176.stdout:5/226: dread d5/d31/f45 [0,4194304] 0 2024-03-13T14:49:39.890 INFO:tasks.workunit.client.0.smithi176.stdout:9/306: symlink d0/d13/d4d/d5d/l60 0 2024-03-13T14:49:39.892 INFO:tasks.workunit.client.0.smithi176.stdout:9/307: write d0/d17/f48 [5491733,105270] 0 2024-03-13T14:49:39.893 INFO:tasks.workunit.client.0.smithi176.stdout:5/227: write d5/d22/d15/f21 [902288,26154] 0 2024-03-13T14:49:39.897 INFO:tasks.workunit.client.1.smithi196.stdout:2/297: rmdir d0/d6/d4b/d5f 39 2024-03-13T14:49:39.906 INFO:tasks.workunit.client.1.smithi196.stdout:9/342: mkdir d7/d9/d6f 0 2024-03-13T14:49:39.907 INFO:tasks.workunit.client.0.smithi176.stdout:4/259: sync 2024-03-13T14:49:39.909 INFO:tasks.workunit.client.1.smithi196.stdout:4/368: creat d1/d3/d7/d35/d3e/f6b x:0 0 0 2024-03-13T14:49:39.910 INFO:tasks.workunit.client.0.smithi176.stdout:1/249: mknod d2/de/d10/d2e/c49 0 2024-03-13T14:49:39.911 INFO:tasks.workunit.client.0.smithi176.stdout:1/250: readlink d2/lb 0 2024-03-13T14:49:39.913 INFO:tasks.workunit.client.0.smithi176.stdout:1/251: stat d2/de/d10/d2e/f42 0 2024-03-13T14:49:39.914 INFO:tasks.workunit.client.1.smithi196.stdout:1/418: dread d1/d9/d33/d4b/d66/f7c [0,4194304] 0 2024-03-13T14:49:39.915 INFO:tasks.workunit.client.1.smithi196.stdout:3/412: rename db/f32 to db/d4c/f86 0 2024-03-13T14:49:39.918 INFO:tasks.workunit.client.1.smithi196.stdout:7/517: mkdir d0/d6/d33/d57/dae 0 2024-03-13T14:49:39.919 INFO:tasks.workunit.client.0.smithi176.stdout:6/251: truncate d0/d10/d19/d27/d2b/d2e/f2d 127318 0 2024-03-13T14:49:39.924 INFO:tasks.workunit.client.0.smithi176.stdout:3/246: rename d4/d28/d2e to d4/d13/d2d/d50 0 2024-03-13T14:49:39.926 INFO:tasks.workunit.client.1.smithi196.stdout:6/536: truncate d6/d9/d17/d19/f2b 306503 0 2024-03-13T14:49:39.926 INFO:tasks.workunit.client.0.smithi176.stdout:5/228: dread f0 [0,4194304] 0 2024-03-13T14:49:39.927 INFO:tasks.workunit.client.0.smithi176.stdout:5/229: chown d5/d2e/f34 0 1 2024-03-13T14:49:39.935 INFO:tasks.workunit.client.1.smithi196.stdout:2/298: symlink d0/d6/d34/l63 0 2024-03-13T14:49:39.937 INFO:tasks.workunit.client.0.smithi176.stdout:0/228: write d2/db/f1c [2041451,37388] 0 2024-03-13T14:49:39.938 INFO:tasks.workunit.client.0.smithi176.stdout:9/308: mknod d0/d1e/d1f/c61 0 2024-03-13T14:49:39.942 INFO:tasks.workunit.client.0.smithi176.stdout:9/309: write d0/db/f3f [37236,64151] 0 2024-03-13T14:49:39.944 INFO:tasks.workunit.client.1.smithi196.stdout:0/443: rename d8/da/f3e to d8/da/d25/d26/d35/d3b/d5f/d33/f94 0 2024-03-13T14:49:39.948 INFO:tasks.workunit.client.1.smithi196.stdout:0/444: write d8/da/d25/d26/d35/d3b/d5f/f4a [1930570,7594] 0 2024-03-13T14:49:39.951 INFO:tasks.workunit.client.1.smithi196.stdout:1/419: creat d1/d3f/d5b/d74/f8f x:0 0 0 2024-03-13T14:49:39.953 INFO:tasks.workunit.client.1.smithi196.stdout:7/518: mknod d0/d78/caf 0 2024-03-13T14:49:39.955 INFO:tasks.workunit.client.0.smithi176.stdout:8/238: mknod d0/c49 0 2024-03-13T14:49:39.956 INFO:tasks.workunit.client.0.smithi176.stdout:8/239: fsync d0/d5/f20 0 2024-03-13T14:49:39.957 INFO:tasks.workunit.client.1.smithi196.stdout:6/537: creat d6/d9/d17/d19/fa8 x:0 0 0 2024-03-13T14:49:39.959 INFO:tasks.workunit.client.0.smithi176.stdout:1/252: mkdir d2/de/d10/d16/d4a 0 2024-03-13T14:49:39.962 INFO:tasks.workunit.client.0.smithi176.stdout:5/230: dread d5/d22/f1e [0,4194304] 0 2024-03-13T14:49:39.964 INFO:tasks.workunit.client.0.smithi176.stdout:4/260: rename d7/d10/c12 to d7/d10/d2c/c60 0 2024-03-13T14:49:39.966 INFO:tasks.workunit.client.0.smithi176.stdout:4/261: write d7/d10/f27 [3125433,22116] 0 2024-03-13T14:49:39.971 INFO:tasks.workunit.client.0.smithi176.stdout:4/262: read d7/d8/d15/f2b [3727264,30542] 0 2024-03-13T14:49:39.971 INFO:tasks.workunit.client.1.smithi196.stdout:5/495: rename d4/db/d14/d19/d1c/d25/d38/d47/l7f to d4/db/d14/d19/d26/l9a 0 2024-03-13T14:49:39.975 INFO:tasks.workunit.client.1.smithi196.stdout:5/496: readlink d4/db/d14/d19/l20 0 2024-03-13T14:49:39.976 INFO:tasks.workunit.client.0.smithi176.stdout:6/252: dread d0/f8 [4194304,4194304] 0 2024-03-13T14:49:39.976 INFO:tasks.workunit.client.1.smithi196.stdout:9/343: truncate d7/f45 15371 0 2024-03-13T14:49:39.977 INFO:tasks.workunit.client.0.smithi176.stdout:9/310: read d0/d17/f19 [4158064,62106] 0 2024-03-13T14:49:39.979 INFO:tasks.workunit.client.1.smithi196.stdout:0/445: creat d8/da/d25/d26/d35/d3b/d50/d5d/f95 x:0 0 0 2024-03-13T14:49:39.982 INFO:tasks.workunit.client.0.smithi176.stdout:2/275: write d1/d2a/d33/f3a [905937,32815] 0 2024-03-13T14:49:39.983 INFO:tasks.workunit.client.0.smithi176.stdout:7/248: link d1/d1c/d26/d41/l30 d1/d1c/d26/d41/d2b/d40/l46 0 2024-03-13T14:49:39.983 INFO:tasks.workunit.client.1.smithi196.stdout:0/446: fdatasync d8/da/d25/d26/d35/d3b/d5f/d17/f70 0 2024-03-13T14:49:39.984 INFO:tasks.workunit.client.0.smithi176.stdout:4/263: dread d7/d8/d15/f2b [0,4194304] 0 2024-03-13T14:49:39.985 INFO:tasks.workunit.client.1.smithi196.stdout:0/447: chown d8/d37/l91 2 1 2024-03-13T14:49:39.985 INFO:tasks.workunit.client.1.smithi196.stdout:6/538: dread - d6/d7/df/d27/f64 zero size 2024-03-13T14:49:39.986 INFO:tasks.workunit.client.1.smithi196.stdout:0/448: stat d8/da/d25/d26 0 2024-03-13T14:49:39.987 INFO:tasks.workunit.client.1.smithi196.stdout:6/539: fdatasync d6/d9/f1f 0 2024-03-13T14:49:39.989 INFO:tasks.workunit.client.1.smithi196.stdout:1/420: write d1/d28/f56 [858717,109523] 0 2024-03-13T14:49:39.993 INFO:tasks.workunit.client.0.smithi176.stdout:9/311: dread d0/d17/f3a [0,4194304] 0 2024-03-13T14:49:39.996 INFO:tasks.workunit.client.1.smithi196.stdout:7/519: write d0/d9/d31/d50/f87 [577464,61370] 0 2024-03-13T14:49:39.997 INFO:tasks.workunit.client.1.smithi196.stdout:6/540: dread d6/d2a/f62 [0,4194304] 0 2024-03-13T14:49:39.998 INFO:tasks.workunit.client.1.smithi196.stdout:5/497: rmdir d4/db/d4a 39 2024-03-13T14:49:40.001 INFO:tasks.workunit.client.1.smithi196.stdout:9/344: rename d7/d9/d6f to d7/de/d16/d70 0 2024-03-13T14:49:40.003 INFO:tasks.workunit.client.1.smithi196.stdout:9/345: write d7/d9/d17/f51 [2947177,111771] 0 2024-03-13T14:49:40.004 INFO:tasks.workunit.client.0.smithi176.stdout:7/249: dwrite d1/d1c/d26/d41/d2b/f37 [0,4194304] 0 2024-03-13T14:49:40.006 INFO:tasks.workunit.client.0.smithi176.stdout:2/276: creat d1/dc/d2d/d4b/f63 x:0 0 0 2024-03-13T14:49:40.007 INFO:tasks.workunit.client.0.smithi176.stdout:2/277: readlink d1/d2a/d33/l47 0 2024-03-13T14:49:40.009 INFO:tasks.workunit.client.0.smithi176.stdout:2/278: read - d1/dc/d2d/d38/f5a zero size 2024-03-13T14:49:40.011 INFO:tasks.workunit.client.1.smithi196.stdout:4/369: getdents d1/d3/d7/d35/d3e/d56 0 2024-03-13T14:49:40.013 INFO:tasks.workunit.client.0.smithi176.stdout:8/240: mknod d0/c4a 0 2024-03-13T14:49:40.014 INFO:tasks.workunit.client.0.smithi176.stdout:8/241: stat d0/db/f25 0 2024-03-13T14:49:40.016 INFO:tasks.workunit.client.0.smithi176.stdout:1/253: symlink d2/de/d10/l4b 0 2024-03-13T14:49:40.017 INFO:tasks.workunit.client.0.smithi176.stdout:1/254: chown d2/ca 10 1 2024-03-13T14:49:40.017 INFO:tasks.workunit.client.0.smithi176.stdout:5/231: symlink d5/d2c/d4a/l4b 0 2024-03-13T14:49:40.018 INFO:tasks.workunit.client.0.smithi176.stdout:1/255: stat d2/de/d10/f2a 0 2024-03-13T14:49:40.019 INFO:tasks.workunit.client.0.smithi176.stdout:9/312: creat d0/d13/d32/f62 x:0 0 0 2024-03-13T14:49:40.020 INFO:tasks.workunit.client.1.smithi196.stdout:6/541: creat d6/d2a/dc/d1c/d34/fa9 x:0 0 0 2024-03-13T14:49:40.022 INFO:tasks.workunit.client.1.smithi196.stdout:6/542: chown d6/d7/dd/f86 127 1 2024-03-13T14:49:40.023 INFO:tasks.workunit.client.0.smithi176.stdout:7/250: write d1/d28/f2a [1392151,114831] 0 2024-03-13T14:49:40.025 INFO:tasks.workunit.client.1.smithi196.stdout:5/498: rename d4/db/d14/d19/d26/d33/d90 to d4/db/d14/d19/d1c/d2d/d82/d8b/d9b 0 2024-03-13T14:49:40.027 INFO:tasks.workunit.client.0.smithi176.stdout:4/264: rename c6 to d7/d10/c61 0 2024-03-13T14:49:40.033 INFO:tasks.workunit.client.0.smithi176.stdout:1/256: dwrite d2/d30/f3a [0,4194304] 0 2024-03-13T14:49:40.033 INFO:tasks.workunit.client.0.smithi176.stdout:8/242: read d0/d5/d11/f1b [313602,77224] 0 2024-03-13T14:49:40.034 INFO:tasks.workunit.client.1.smithi196.stdout:1/421: truncate d1/f37 2629652 0 2024-03-13T14:49:40.041 INFO:tasks.workunit.client.1.smithi196.stdout:8/346: sync 2024-03-13T14:49:40.043 INFO:tasks.workunit.client.0.smithi176.stdout:6/253: link d0/d10/l44 d0/d10/d19/d27/d2b/d2e/d3e/l4f 0 2024-03-13T14:49:40.045 INFO:tasks.workunit.client.1.smithi196.stdout:2/299: getdents d0/d6/d4b 0 2024-03-13T14:49:40.045 INFO:tasks.workunit.client.0.smithi176.stdout:7/251: fsync d1/d28/d3a/f20 0 2024-03-13T14:49:40.047 INFO:tasks.workunit.client.0.smithi176.stdout:7/252: chown d1/d28/d3a 1 1 2024-03-13T14:49:40.048 INFO:tasks.workunit.client.1.smithi196.stdout:7/520: creat d0/d9/d1e/d3f/d6c/fb0 x:0 0 0 2024-03-13T14:49:40.051 INFO:tasks.workunit.client.1.smithi196.stdout:7/521: write d0/d9/d1e/d3f/f60 [304874,115742] 0 2024-03-13T14:49:40.057 INFO:tasks.workunit.client.0.smithi176.stdout:4/265: write d7/d8/d15/f2b [2815541,99352] 0 2024-03-13T14:49:40.058 INFO:tasks.workunit.client.0.smithi176.stdout:2/279: symlink d1/dc/d61/l64 0 2024-03-13T14:49:40.060 INFO:tasks.workunit.client.0.smithi176.stdout:9/313: write d0/d1e/f25 [1325555,1646] 0 2024-03-13T14:49:40.063 INFO:tasks.workunit.client.0.smithi176.stdout:9/314: write d0/d17/f48 [3195944,109774] 0 2024-03-13T14:49:40.064 INFO:tasks.workunit.client.0.smithi176.stdout:1/257: creat d2/d30/f4c x:0 0 0 2024-03-13T14:49:40.066 INFO:tasks.workunit.client.0.smithi176.stdout:1/258: readlink d2/de/l44 0 2024-03-13T14:49:40.071 INFO:tasks.workunit.client.0.smithi176.stdout:8/243: unlink d0/db/l44 0 2024-03-13T14:49:40.071 INFO:tasks.workunit.client.1.smithi196.stdout:5/499: rename d4/db/d3c/f30 to d4/db/d3c/d6a/f9c 0 2024-03-13T14:49:40.074 INFO:tasks.workunit.client.1.smithi196.stdout:1/422: rmdir d1/d38 39 2024-03-13T14:49:40.079 INFO:tasks.workunit.client.0.smithi176.stdout:7/253: write d1/d28/d3a/d1b/d18/f1e [1928771,112795] 0 2024-03-13T14:49:40.083 INFO:tasks.workunit.client.1.smithi196.stdout:4/370: mkdir d1/d1e/d39/d58/d69/d6c 0 2024-03-13T14:49:40.084 INFO:tasks.workunit.client.0.smithi176.stdout:0/229: sync 2024-03-13T14:49:40.084 INFO:tasks.workunit.client.0.smithi176.stdout:3/247: sync 2024-03-13T14:49:40.084 INFO:tasks.workunit.client.0.smithi176.stdout:2/280: chown d1/dc/d40 22726992 1 2024-03-13T14:49:40.085 INFO:tasks.workunit.client.1.smithi196.stdout:6/543: dwrite d6/d2a/dc/d47/f55 [0,4194304] 0 2024-03-13T14:49:40.092 INFO:tasks.workunit.client.0.smithi176.stdout:8/244: symlink d0/db/dc/l4b 0 2024-03-13T14:49:40.093 INFO:tasks.workunit.client.0.smithi176.stdout:6/254: creat d0/d9/f50 x:0 0 0 2024-03-13T14:49:40.095 INFO:tasks.workunit.client.1.smithi196.stdout:7/522: dwrite d0/d9/d1e/d58/d72/da9/da3/fad [0,4194304] 0 2024-03-13T14:49:40.095 INFO:tasks.workunit.client.1.smithi196.stdout:0/449: link d8/cb d8/da/d25/d26/d35/d3b/d5f/d17/c96 0 2024-03-13T14:49:40.101 INFO:tasks.workunit.client.1.smithi196.stdout:3/413: sync 2024-03-13T14:49:40.103 INFO:tasks.workunit.client.0.smithi176.stdout:7/254: unlink d1/d28/d3a/c16 0 2024-03-13T14:49:40.103 INFO:tasks.workunit.client.0.smithi176.stdout:4/266: truncate d7/d8/d2f/f37 2022729 0 2024-03-13T14:49:40.104 INFO:tasks.workunit.client.1.smithi196.stdout:0/450: write d8/da/d25/d26/d35/d3b/d5f/f2b [224272,64586] 0 2024-03-13T14:49:40.105 INFO:tasks.workunit.client.1.smithi196.stdout:9/346: dwrite d7/f15 [0,4194304] 0 2024-03-13T14:49:40.111 INFO:tasks.workunit.client.0.smithi176.stdout:3/248: creat d4/d13/d2d/f51 x:0 0 0 2024-03-13T14:49:40.111 INFO:tasks.workunit.client.0.smithi176.stdout:1/259: rmdir d2/de/d3f/d19 39 2024-03-13T14:49:40.114 INFO:tasks.workunit.client.0.smithi176.stdout:8/245: creat d0/db/d39/f4c x:0 0 0 2024-03-13T14:49:40.118 INFO:tasks.workunit.client.0.smithi176.stdout:1/260: chown d2/de/d10/d14/d22/c34 23722927 1 2024-03-13T14:49:40.121 INFO:tasks.workunit.client.1.smithi196.stdout:3/414: fsync db/d19/d22/f39 0 2024-03-13T14:49:40.121 INFO:tasks.workunit.client.0.smithi176.stdout:3/249: write d4/d13/d2d/d50/f46 [4919861,122056] 0 2024-03-13T14:49:40.127 INFO:tasks.workunit.client.1.smithi196.stdout:0/451: symlink d8/da/d25/d26/d35/d3b/d59/l97 0 2024-03-13T14:49:40.130 INFO:tasks.workunit.client.0.smithi176.stdout:7/255: symlink d1/d3f/l47 0 2024-03-13T14:49:40.131 INFO:tasks.workunit.client.1.smithi196.stdout:4/371: mknod d1/d3/c6d 0 2024-03-13T14:49:40.131 INFO:tasks.workunit.client.0.smithi176.stdout:7/256: dread - d1/d7/f3d zero size 2024-03-13T14:49:40.134 INFO:tasks.workunit.client.0.smithi176.stdout:4/267: dread d7/d8/f42 [0,4194304] 0 2024-03-13T14:49:40.136 INFO:tasks.workunit.client.0.smithi176.stdout:4/268: chown d7/d8/d15 17483 1 2024-03-13T14:49:40.139 INFO:tasks.workunit.client.0.smithi176.stdout:3/250: dwrite d4/d13/f2a [0,4194304] 0 2024-03-13T14:49:40.143 INFO:tasks.workunit.client.0.smithi176.stdout:3/251: write d4/d13/d16/d3d/f41 [108592,111977] 0 2024-03-13T14:49:40.144 INFO:tasks.workunit.client.0.smithi176.stdout:0/230: truncate d2/d5/dc/f2e 3496864 0 2024-03-13T14:49:40.145 INFO:tasks.workunit.client.0.smithi176.stdout:3/252: read - d4/d28/d31/f4b zero size 2024-03-13T14:49:40.145 INFO:tasks.workunit.client.1.smithi196.stdout:5/500: creat d4/f9d x:0 0 0 2024-03-13T14:49:40.146 INFO:tasks.workunit.client.0.smithi176.stdout:0/231: dread - d2/db/f3b zero size 2024-03-13T14:49:40.148 INFO:tasks.workunit.client.0.smithi176.stdout:0/232: dread - d2/d5/d1e/f3d zero size 2024-03-13T14:49:40.149 INFO:tasks.workunit.client.1.smithi196.stdout:5/501: stat d4/db/d14/d19/d26/d33/c70 0 2024-03-13T14:49:40.149 INFO:tasks.workunit.client.0.smithi176.stdout:0/233: read d2/f11 [3286067,62061] 0 2024-03-13T14:49:40.150 INFO:tasks.workunit.client.0.smithi176.stdout:7/257: dwrite d1/d1c/d26/d41/f25 [0,4194304] 0 2024-03-13T14:49:40.152 INFO:tasks.workunit.client.1.smithi196.stdout:5/502: fdatasync d4/f11 0 2024-03-13T14:49:40.153 INFO:tasks.workunit.client.0.smithi176.stdout:7/258: write d1/d28/d3a/d1b/d18/f1e [4619179,86579] 0 2024-03-13T14:49:40.154 INFO:tasks.workunit.client.1.smithi196.stdout:1/423: rename d1/d3f/d5b/d74/d85 to d1/d9/d4e/d55/d90 0 2024-03-13T14:49:40.158 INFO:tasks.workunit.client.1.smithi196.stdout:8/347: dwrite d0/d1c/d3b/d3f/f45 [0,4194304] 0 2024-03-13T14:49:40.160 INFO:tasks.workunit.client.0.smithi176.stdout:4/269: fsync d7/d8/f42 0 2024-03-13T14:49:40.160 INFO:tasks.workunit.client.0.smithi176.stdout:4/270: chown d7/d10/d46/c5f 1890 1 2024-03-13T14:49:40.161 INFO:tasks.workunit.client.1.smithi196.stdout:8/348: readlink d0/lf 0 2024-03-13T14:49:40.161 INFO:tasks.workunit.client.0.smithi176.stdout:4/271: chown d7/d2d/l36 168627808 1 2024-03-13T14:49:40.163 INFO:tasks.workunit.client.1.smithi196.stdout:0/452: unlink d8/da/d25/d26/d35/d3b/d50/d5d/l4c 0 2024-03-13T14:49:40.164 INFO:tasks.workunit.client.0.smithi176.stdout:2/281: getdents d1/dc/d2d 0 2024-03-13T14:49:40.165 INFO:tasks.workunit.client.1.smithi196.stdout:4/372: unlink d1/d1e/d39/d58/f68 0 2024-03-13T14:49:40.168 INFO:tasks.workunit.client.1.smithi196.stdout:9/347: mkdir d7/d71 0 2024-03-13T14:49:40.169 INFO:tasks.workunit.client.0.smithi176.stdout:6/255: rmdir d0/de/d4c 0 2024-03-13T14:49:40.173 INFO:tasks.workunit.client.1.smithi196.stdout:2/300: getdents d0/d50 0 2024-03-13T14:49:40.175 INFO:tasks.workunit.client.0.smithi176.stdout:3/253: unlink d4/c14 0 2024-03-13T14:49:40.175 INFO:tasks.workunit.client.1.smithi196.stdout:2/301: chown d0/d7/d40/f46 1 1 2024-03-13T14:49:40.176 INFO:tasks.workunit.client.0.smithi176.stdout:3/254: chown d4/d13/d2d/d50/f46 1148933184 1 2024-03-13T14:49:40.178 INFO:tasks.workunit.client.0.smithi176.stdout:3/255: fdatasync d4/d28/d3a/f27 0 2024-03-13T14:49:40.178 INFO:tasks.workunit.client.0.smithi176.stdout:6/256: dread d0/d9/f14 [0,4194304] 0 2024-03-13T14:49:40.178 INFO:tasks.workunit.client.0.smithi176.stdout:0/234: creat d2/db/d35/f3f x:0 0 0 2024-03-13T14:49:40.178 INFO:tasks.workunit.client.1.smithi196.stdout:6/544: dwrite d6/d9/d17/d19/f42 [0,4194304] 0 2024-03-13T14:49:40.179 INFO:tasks.workunit.client.0.smithi176.stdout:3/256: chown d4/d28/f29 18603577 1 2024-03-13T14:49:40.181 INFO:tasks.workunit.client.0.smithi176.stdout:3/257: chown d4/d28/d31/f4b 105748 1 2024-03-13T14:49:40.184 INFO:tasks.workunit.client.0.smithi176.stdout:3/258: dread - d4/d28/d34/f3f zero size 2024-03-13T14:49:40.186 INFO:tasks.workunit.client.0.smithi176.stdout:3/259: stat d4/d28 0 2024-03-13T14:49:40.190 INFO:tasks.workunit.client.0.smithi176.stdout:3/260: chown d4/d13/d2d/d50/f46 219300 1 2024-03-13T14:49:40.191 INFO:tasks.workunit.client.0.smithi176.stdout:3/261: chown d4/d13/d2d 817 1 2024-03-13T14:49:40.192 INFO:tasks.workunit.client.0.smithi176.stdout:1/261: creat d2/de/f4d x:0 0 0 2024-03-13T14:49:40.194 INFO:tasks.workunit.client.1.smithi196.stdout:3/415: write db/d19/f2f [4802532,53087] 0 2024-03-13T14:49:40.195 INFO:tasks.workunit.client.0.smithi176.stdout:4/272: readlink d7/d10/l57 0 2024-03-13T14:49:40.201 INFO:tasks.workunit.client.1.smithi196.stdout:7/523: sync 2024-03-13T14:49:40.202 INFO:tasks.workunit.client.1.smithi196.stdout:1/424: mknod d1/d9/d33/d4b/d66/d7b/c91 0 2024-03-13T14:49:40.202 INFO:tasks.workunit.client.0.smithi176.stdout:3/262: dread d4/d13/f48 [0,4194304] 0 2024-03-13T14:49:40.205 INFO:tasks.workunit.client.0.smithi176.stdout:3/263: write d4/d13/d2d/d50/f46 [1301288,25840] 0 2024-03-13T14:49:40.206 INFO:tasks.workunit.client.1.smithi196.stdout:7/524: chown d0/d9/d1e/d58/d72/c71 2711 1 2024-03-13T14:49:40.207 INFO:tasks.workunit.client.0.smithi176.stdout:6/257: symlink d0/de/l51 0 2024-03-13T14:49:40.208 INFO:tasks.workunit.client.1.smithi196.stdout:1/425: truncate d1/d9/d33/d4b/d66/d7b/f8d 863764 0 2024-03-13T14:49:40.210 INFO:tasks.workunit.client.0.smithi176.stdout:0/235: creat d2/db/d35/f40 x:0 0 0 2024-03-13T14:49:40.211 INFO:tasks.workunit.client.1.smithi196.stdout:4/373: unlink d1/d3/d7/f21 0 2024-03-13T14:49:40.211 INFO:tasks.workunit.client.0.smithi176.stdout:0/236: fdatasync d2/db/d35/f3f 0 2024-03-13T14:49:40.212 INFO:tasks.workunit.client.1.smithi196.stdout:1/426: chown d1/d3f/d5b/f69 326 1 2024-03-13T14:49:40.214 INFO:tasks.workunit.client.1.smithi196.stdout:1/427: chown d1/d28/f5a 2292 1 2024-03-13T14:49:40.215 INFO:tasks.workunit.client.0.smithi176.stdout:9/315: sync 2024-03-13T14:49:40.215 INFO:tasks.workunit.client.0.smithi176.stdout:5/232: sync 2024-03-13T14:49:40.215 INFO:tasks.workunit.client.0.smithi176.stdout:8/246: sync 2024-03-13T14:49:40.215 INFO:tasks.workunit.client.1.smithi196.stdout:9/348: rename d7/d24/c4e to d7/d71/c72 0 2024-03-13T14:49:40.217 INFO:tasks.workunit.client.0.smithi176.stdout:8/247: fdatasync d0/db/d39/f4c 0 2024-03-13T14:49:40.223 INFO:tasks.workunit.client.1.smithi196.stdout:2/302: creat d0/d6/d4b/f64 x:0 0 0 2024-03-13T14:49:40.225 INFO:tasks.workunit.client.1.smithi196.stdout:6/545: dread d6/d2a/d23/d28/f6d [0,4194304] 0 2024-03-13T14:49:40.226 INFO:tasks.workunit.client.0.smithi176.stdout:1/262: symlink d2/de/d10/d16/l4e 0 2024-03-13T14:49:40.229 INFO:tasks.workunit.client.0.smithi176.stdout:0/237: dwrite d2/d5/d28/f29 [0,4194304] 0 2024-03-13T14:49:40.229 INFO:tasks.workunit.client.0.smithi176.stdout:1/263: write d2/de/f4d [980805,53522] 0 2024-03-13T14:49:40.230 INFO:tasks.workunit.client.1.smithi196.stdout:5/503: dwrite d4/db/d14/d19/d1c/f23 [0,4194304] 0 2024-03-13T14:49:40.232 INFO:tasks.workunit.client.0.smithi176.stdout:0/238: write d2/db/f3b [892525,54512] 0 2024-03-13T14:49:40.238 INFO:tasks.workunit.client.1.smithi196.stdout:1/428: dwrite d1/d28/f5a [4194304,4194304] 0 2024-03-13T14:49:40.238 INFO:tasks.workunit.client.0.smithi176.stdout:4/273: creat d7/d8/d45/f62 x:0 0 0 2024-03-13T14:49:40.240 INFO:tasks.workunit.client.0.smithi176.stdout:0/239: write d2/d5/dc/d20/f34 [3304191,14546] 0 2024-03-13T14:49:40.244 INFO:tasks.workunit.client.1.smithi196.stdout:8/349: symlink d0/d10/d12/l75 0 2024-03-13T14:49:40.250 INFO:tasks.workunit.client.1.smithi196.stdout:7/525: fdatasync d0/fa 0 2024-03-13T14:49:40.250 INFO:tasks.workunit.client.0.smithi176.stdout:3/264: stat d4/d13/d2d/f30 0 2024-03-13T14:49:40.251 INFO:tasks.workunit.client.0.smithi176.stdout:2/282: dread d1/dc/d2d/d38/d5d/f62 [0,4194304] 0 2024-03-13T14:49:40.254 INFO:tasks.workunit.client.0.smithi176.stdout:6/258: creat d0/d10/d19/d27/d2b/d2e/f52 x:0 0 0 2024-03-13T14:49:40.257 INFO:tasks.workunit.client.1.smithi196.stdout:7/526: write d0/d6/d33/d57/f7d [2584607,66632] 0 2024-03-13T14:49:40.260 INFO:tasks.workunit.client.1.smithi196.stdout:7/527: dread - d0/d4d/f91 zero size 2024-03-13T14:49:40.262 INFO:tasks.workunit.client.0.smithi176.stdout:5/233: mkdir d5/d48/d4c 0 2024-03-13T14:49:40.262 INFO:tasks.workunit.client.0.smithi176.stdout:9/316: unlink d0/d13/d32/l5c 0 2024-03-13T14:49:40.264 INFO:tasks.workunit.client.1.smithi196.stdout:9/349: dread - d7/de/f40 zero size 2024-03-13T14:49:40.264 INFO:tasks.workunit.client.0.smithi176.stdout:5/234: fdatasync d5/d22/d15/f16 0 2024-03-13T14:49:40.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/303: creat d0/d50/f65 x:0 0 0 2024-03-13T14:49:40.273 INFO:tasks.workunit.client.0.smithi176.stdout:4/274: symlink d7/d8/d33/l63 0 2024-03-13T14:49:40.274 INFO:tasks.workunit.client.1.smithi196.stdout:6/546: rmdir d6/d9/d11 39 2024-03-13T14:49:40.276 INFO:tasks.workunit.client.0.smithi176.stdout:3/265: mkdir d4/d28/d31/d52 0 2024-03-13T14:49:40.278 INFO:tasks.workunit.client.1.smithi196.stdout:6/547: write d6/d7/df/f35 [970169,33981] 0 2024-03-13T14:49:40.278 INFO:tasks.workunit.client.0.smithi176.stdout:3/266: dread - d4/d28/d31/f4b zero size 2024-03-13T14:49:40.279 INFO:tasks.workunit.client.0.smithi176.stdout:2/283: rmdir d1/d2a 39 2024-03-13T14:49:40.281 INFO:tasks.workunit.client.1.smithi196.stdout:1/429: mkdir d1/d9/d4e/d55/d63/d92 0 2024-03-13T14:49:40.283 INFO:tasks.workunit.client.0.smithi176.stdout:2/284: dread d1/f31 [0,4194304] 0 2024-03-13T14:49:40.283 INFO:tasks.workunit.client.1.smithi196.stdout:0/453: truncate d8/da/d25/d26/d35/d3b/d5f/f2b 114566 0 2024-03-13T14:49:40.284 INFO:tasks.workunit.client.0.smithi176.stdout:2/285: dread - d1/f54 zero size 2024-03-13T14:49:40.284 INFO:tasks.workunit.client.0.smithi176.stdout:7/259: getdents d1/d28/d3a/d1b 0 2024-03-13T14:49:40.285 INFO:tasks.workunit.client.0.smithi176.stdout:7/260: chown d1/d7/cd 0 1 2024-03-13T14:49:40.286 INFO:tasks.workunit.client.0.smithi176.stdout:9/317: mknod d0/d1e/c63 0 2024-03-13T14:49:40.287 INFO:tasks.workunit.client.1.smithi196.stdout:0/454: dread - d8/da/d25/d26/d35/d3b/d50/d5d/f95 zero size 2024-03-13T14:49:40.289 INFO:tasks.workunit.client.1.smithi196.stdout:4/374: symlink d1/d3/d7/d35/l6e 0 2024-03-13T14:49:40.291 INFO:tasks.workunit.client.1.smithi196.stdout:7/528: mknod d0/d9/d31/d77/d93/d94/cb1 0 2024-03-13T14:49:40.291 INFO:tasks.workunit.client.1.smithi196.stdout:8/350: dread d0/d1c/d3b/f4b [0,4194304] 0 2024-03-13T14:49:40.293 INFO:tasks.workunit.client.0.smithi176.stdout:5/235: unlink d5/c32 0 2024-03-13T14:49:40.294 INFO:tasks.workunit.client.1.smithi196.stdout:7/529: stat d0/d9/d31/d77/d93/d94/f9e 0 2024-03-13T14:49:40.299 INFO:tasks.workunit.client.1.smithi196.stdout:9/350: rename d7/de/d4b/l42 to d7/d24/d28/l73 0 2024-03-13T14:49:40.299 INFO:tasks.workunit.client.1.smithi196.stdout:2/304: chown d0/f32 0 1 2024-03-13T14:49:40.301 INFO:tasks.workunit.client.1.smithi196.stdout:5/504: creat d4/db/d3c/d6a/d93/f9e x:0 0 0 2024-03-13T14:49:40.302 INFO:tasks.workunit.client.1.smithi196.stdout:2/305: write d0/d50/f65 [273281,74068] 0 2024-03-13T14:49:40.303 INFO:tasks.workunit.client.1.smithi196.stdout:6/548: unlink d6/d2a/dc/f3e 0 2024-03-13T14:49:40.304 INFO:tasks.workunit.client.0.smithi176.stdout:6/259: mkdir d0/d10/d19/d27/d53 0 2024-03-13T14:49:40.305 INFO:tasks.workunit.client.1.smithi196.stdout:3/416: mknod db/d19/d4a/c87 0 2024-03-13T14:49:40.306 INFO:tasks.workunit.client.1.smithi196.stdout:3/417: fsync db/d19/f2f 0 2024-03-13T14:49:40.308 INFO:tasks.workunit.client.1.smithi196.stdout:6/549: write d6/d9/d17/d19/f20 [711310,96637] 0 2024-03-13T14:49:40.315 INFO:tasks.workunit.client.0.smithi176.stdout:4/275: write d7/d8/f16 [2500228,124745] 0 2024-03-13T14:49:40.316 INFO:tasks.workunit.client.0.smithi176.stdout:8/248: creat d0/d5/f4d x:0 0 0 2024-03-13T14:49:40.317 INFO:tasks.workunit.client.0.smithi176.stdout:4/276: chown d7/d8/f16 84673870 1 2024-03-13T14:49:40.319 INFO:tasks.workunit.client.1.smithi196.stdout:8/351: dwrite d0/d1c/d3b/f4a [0,4194304] 0 2024-03-13T14:49:40.319 INFO:tasks.workunit.client.0.smithi176.stdout:4/277: write d7/d8/d15/d23/f47 [943687,119756] 0 2024-03-13T14:49:40.320 INFO:tasks.workunit.client.0.smithi176.stdout:1/264: creat d2/de/d10/f4f x:0 0 0 2024-03-13T14:49:40.320 INFO:tasks.workunit.client.0.smithi176.stdout:4/278: readlink d7/d10/l14 0 2024-03-13T14:49:40.321 INFO:tasks.workunit.client.0.smithi176.stdout:4/279: readlink d7/d8/d33/l63 0 2024-03-13T14:49:40.322 INFO:tasks.workunit.client.0.smithi176.stdout:3/267: fsync d4/d28/d3a/f32 0 2024-03-13T14:49:40.324 INFO:tasks.workunit.client.1.smithi196.stdout:2/306: mkdir d0/d7/d66 0 2024-03-13T14:49:40.333 INFO:tasks.workunit.client.1.smithi196.stdout:1/430: fdatasync d1/d9/d33/d4b/f48 0 2024-03-13T14:49:40.335 INFO:tasks.workunit.client.0.smithi176.stdout:5/236: creat d5/d48/d49/f4d x:0 0 0 2024-03-13T14:49:40.336 INFO:tasks.workunit.client.0.smithi176.stdout:8/249: creat d0/d24/d29/f4e x:0 0 0 2024-03-13T14:49:40.337 INFO:tasks.workunit.client.1.smithi196.stdout:5/505: creat d4/db/d3c/d6a/f9f x:0 0 0 2024-03-13T14:49:40.344 INFO:tasks.workunit.client.0.smithi176.stdout:4/280: symlink d7/d8/d45/l64 0 2024-03-13T14:49:40.345 INFO:tasks.workunit.client.1.smithi196.stdout:7/530: mknod d0/d9/d1e/d58/d61/da0/cb2 0 2024-03-13T14:49:40.347 INFO:tasks.workunit.client.0.smithi176.stdout:5/237: dread f0 [0,4194304] 0 2024-03-13T14:49:40.348 INFO:tasks.workunit.client.0.smithi176.stdout:5/238: chown d5/d2e/c42 34 1 2024-03-13T14:49:40.348 INFO:tasks.workunit.client.1.smithi196.stdout:0/455: truncate d8/f1e 85618 0 2024-03-13T14:49:40.348 INFO:tasks.workunit.client.1.smithi196.stdout:3/418: dwrite db/d19/f1d [0,4194304] 0 2024-03-13T14:49:40.349 INFO:tasks.workunit.client.0.smithi176.stdout:1/265: dread d2/de/d3f/d19/f32 [0,4194304] 0 2024-03-13T14:49:40.351 INFO:tasks.workunit.client.0.smithi176.stdout:6/260: unlink d0/l42 0 2024-03-13T14:49:40.353 INFO:tasks.workunit.client.0.smithi176.stdout:2/286: symlink d1/d2a/l65 0 2024-03-13T14:49:40.354 INFO:tasks.workunit.client.1.smithi196.stdout:8/352: fsync d0/d10/f21 0 2024-03-13T14:49:40.355 INFO:tasks.workunit.client.0.smithi176.stdout:7/261: link d1/d1c/d26/d41/l30 d1/d1c/d26/d41/l48 0 2024-03-13T14:49:40.356 INFO:tasks.workunit.client.0.smithi176.stdout:7/262: fdatasync d1/d1c/d26/d41/f3e 0 2024-03-13T14:49:40.357 INFO:tasks.workunit.client.1.smithi196.stdout:8/353: write d0/d10/d12/f53 [753222,73273] 0 2024-03-13T14:49:40.358 INFO:tasks.workunit.client.0.smithi176.stdout:9/318: link d0/d40/c4b d0/d13/d4d/c64 0 2024-03-13T14:49:40.358 INFO:tasks.workunit.client.0.smithi176.stdout:7/263: write d1/d1c/d26/d41/f3e [620448,25201] 0 2024-03-13T14:49:40.359 INFO:tasks.workunit.client.1.smithi196.stdout:9/351: mkdir d7/de/d74 0 2024-03-13T14:49:40.362 INFO:tasks.workunit.client.1.smithi196.stdout:8/354: write d0/d1c/d29/d47/f52 [3616567,101717] 0 2024-03-13T14:49:40.363 INFO:tasks.workunit.client.1.smithi196.stdout:7/531: dread d0/d9/d1e/d3f/f82 [0,4194304] 0 2024-03-13T14:49:40.365 INFO:tasks.workunit.client.1.smithi196.stdout:9/352: write d7/d24/f3a [1336667,129717] 0 2024-03-13T14:49:40.369 INFO:tasks.workunit.client.1.smithi196.stdout:7/532: write d0/d6/d33/d57/f95 [826909,76748] 0 2024-03-13T14:49:40.372 INFO:tasks.workunit.client.1.smithi196.stdout:3/419: chown db/d19/f2b 11682 1 2024-03-13T14:49:40.373 INFO:tasks.workunit.client.0.smithi176.stdout:3/268: creat d4/d28/d4f/f53 x:0 0 0 2024-03-13T14:49:40.379 INFO:tasks.workunit.client.0.smithi176.stdout:7/264: dwrite d1/f3c [0,4194304] 0 2024-03-13T14:49:40.381 INFO:tasks.workunit.client.0.smithi176.stdout:5/239: creat d5/d48/f4e x:0 0 0 2024-03-13T14:49:40.383 INFO:tasks.workunit.client.0.smithi176.stdout:5/240: chown d5/d22/d15/f27 956775 1 2024-03-13T14:49:40.384 INFO:tasks.workunit.client.1.smithi196.stdout:6/550: dread d6/d9/d17/d19/f20 [0,4194304] 0 2024-03-13T14:49:40.392 INFO:tasks.workunit.client.1.smithi196.stdout:2/307: mkdir d0/d7/d66/d67 0 2024-03-13T14:49:40.392 INFO:tasks.workunit.client.1.smithi196.stdout:5/506: write d4/db/d14/d19/d1c/f6b [2559838,14487] 0 2024-03-13T14:49:40.398 INFO:tasks.workunit.client.1.smithi196.stdout:5/507: stat d4/db/d14/d19/d1c/d2d/d82 0 2024-03-13T14:49:40.403 INFO:tasks.workunit.client.1.smithi196.stdout:8/355: creat d0/d1c/d3b/d3f/f76 x:0 0 0 2024-03-13T14:49:40.405 INFO:tasks.workunit.client.1.smithi196.stdout:5/508: fdatasync d4/db/d14/d37/d41/f5c 0 2024-03-13T14:49:40.412 INFO:tasks.workunit.client.1.smithi196.stdout:7/533: symlink d0/d9/d1e/lb3 0 2024-03-13T14:49:40.416 INFO:tasks.workunit.client.1.smithi196.stdout:6/551: dwrite d6/d2a/dc/d1c/fa4 [0,4194304] 0 2024-03-13T14:49:40.416 INFO:tasks.workunit.client.1.smithi196.stdout:0/456: dwrite d8/da/d25/d26/d35/d3b/f47 [4194304,4194304] 0 2024-03-13T14:49:40.420 INFO:tasks.workunit.client.1.smithi196.stdout:2/308: dread d0/f25 [0,4194304] 0 2024-03-13T14:49:40.426 INFO:tasks.workunit.client.1.smithi196.stdout:3/420: dread db/d19/d22/d3a/d53/f48 [0,4194304] 0 2024-03-13T14:49:40.426 INFO:tasks.workunit.client.1.smithi196.stdout:0/457: stat d8/da/d25/d26/d35/d3b/d5f/d17/c2c 0 2024-03-13T14:49:40.427 INFO:tasks.workunit.client.1.smithi196.stdout:2/309: readlink d0/d6/d34/l63 0 2024-03-13T14:49:40.427 INFO:tasks.workunit.client.1.smithi196.stdout:3/421: readlink l5 0 2024-03-13T14:49:40.427 INFO:tasks.workunit.client.1.smithi196.stdout:2/310: truncate d0/f37 578649 0 2024-03-13T14:49:40.427 INFO:tasks.workunit.client.1.smithi196.stdout:4/375: getdents d1/d1e/d43/d5b/d55 0 2024-03-13T14:49:40.428 INFO:tasks.workunit.client.0.smithi176.stdout:0/240: sync 2024-03-13T14:49:40.428 INFO:tasks.workunit.client.1.smithi196.stdout:2/311: write d0/d7/d40/f36 [5206803,37020] 0 2024-03-13T14:49:40.430 INFO:tasks.workunit.client.1.smithi196.stdout:2/312: chown d0/d6/d1d/l4e 15649 1 2024-03-13T14:49:40.433 INFO:tasks.workunit.client.0.smithi176.stdout:8/250: write d0/db/f3f [5422883,62584] 0 2024-03-13T14:49:40.434 INFO:tasks.workunit.client.1.smithi196.stdout:5/509: read - d4/db/d14/d19/f67 zero size 2024-03-13T14:49:40.436 INFO:tasks.workunit.client.0.smithi176.stdout:4/281: symlink d7/d8/l65 0 2024-03-13T14:49:40.437 INFO:tasks.workunit.client.0.smithi176.stdout:3/269: creat d4/d13/d16/d3d/f54 x:0 0 0 2024-03-13T14:49:40.437 INFO:tasks.workunit.client.0.smithi176.stdout:2/287: write d1/ff [338961,104217] 0 2024-03-13T14:49:40.438 INFO:tasks.workunit.client.1.smithi196.stdout:3/422: creat db/d15/f88 x:0 0 0 2024-03-13T14:49:40.438 INFO:tasks.workunit.client.1.smithi196.stdout:0/458: symlink d8/da/d25/d26/d35/d3b/d50/d5d/d7e/l98 0 2024-03-13T14:49:40.438 INFO:tasks.workunit.client.0.smithi176.stdout:5/241: unlink d5/d2c/c33 0 2024-03-13T14:49:40.441 INFO:tasks.workunit.client.1.smithi196.stdout:4/376: rmdir d1/d1e 39 2024-03-13T14:49:40.441 INFO:tasks.workunit.client.0.smithi176.stdout:6/261: write d0/d10/d19/d27/d2b/f3a [2360570,122577] 0 2024-03-13T14:49:40.442 INFO:tasks.workunit.client.1.smithi196.stdout:9/353: creat d7/de/d4b/d1d/f75 x:0 0 0 2024-03-13T14:49:40.444 INFO:tasks.workunit.client.0.smithi176.stdout:0/241: unlink d2/db/f21 0 2024-03-13T14:49:40.446 INFO:tasks.workunit.client.1.smithi196.stdout:7/534: mkdir d0/d6/db4 0 2024-03-13T14:49:40.450 INFO:tasks.workunit.client.0.smithi176.stdout:1/266: rename d2/de/d10/d2e to d2/de/d3f/d50 0 2024-03-13T14:49:40.451 INFO:tasks.workunit.client.1.smithi196.stdout:6/552: mkdir d6/d9/daa 0 2024-03-13T14:49:40.456 INFO:tasks.workunit.client.0.smithi176.stdout:7/265: dwrite d1/d7/f10 [0,4194304] 0 2024-03-13T14:49:40.458 INFO:tasks.workunit.client.1.smithi196.stdout:5/510: rename d4/db/d14/d37/d41 to d4/db/d14/d19/d1c/d25/d38/d47/da0 0 2024-03-13T14:49:40.461 INFO:tasks.workunit.client.0.smithi176.stdout:4/282: creat d7/d10/d2c/f66 x:0 0 0 2024-03-13T14:49:40.462 INFO:tasks.workunit.client.1.smithi196.stdout:3/423: creat db/d19/d22/d3a/d6a/f89 x:0 0 0 2024-03-13T14:49:40.464 INFO:tasks.workunit.client.0.smithi176.stdout:5/242: symlink d5/d31/l4f 0 2024-03-13T14:49:40.466 INFO:tasks.workunit.client.0.smithi176.stdout:3/270: mknod d4/d13/d16/d3d/c55 0 2024-03-13T14:49:40.469 INFO:tasks.workunit.client.0.smithi176.stdout:2/288: dwrite d1/dc/d1c/f36 [0,4194304] 0 2024-03-13T14:49:40.470 INFO:tasks.workunit.client.0.smithi176.stdout:3/271: truncate d4/d28/d31/f4b 810102 0 2024-03-13T14:49:40.471 INFO:tasks.workunit.client.0.smithi176.stdout:3/272: stat f1 0 2024-03-13T14:49:40.473 INFO:tasks.workunit.client.1.smithi196.stdout:2/313: creat d0/d7/d56/f68 x:0 0 0 2024-03-13T14:49:40.480 INFO:tasks.workunit.client.1.smithi196.stdout:8/356: creat d0/d10/f77 x:0 0 0 2024-03-13T14:49:40.482 INFO:tasks.workunit.client.1.smithi196.stdout:3/424: dwrite db/d19/f1d [0,4194304] 0 2024-03-13T14:49:40.485 INFO:tasks.workunit.client.0.smithi176.stdout:7/266: symlink d1/d1c/d26/d41/d2b/d40/l49 0 2024-03-13T14:49:40.487 INFO:tasks.workunit.client.1.smithi196.stdout:7/535: rename d0/d9/d1e/d58/d72/f3d to d0/d9/d31/fb5 0 2024-03-13T14:49:40.493 INFO:tasks.workunit.client.1.smithi196.stdout:5/511: stat d4/db/d4a/l85 0 2024-03-13T14:49:40.493 INFO:tasks.workunit.client.1.smithi196.stdout:1/431: sync 2024-03-13T14:49:40.495 INFO:tasks.workunit.client.1.smithi196.stdout:1/432: readlink d1/d9/d33/l39 0 2024-03-13T14:49:40.502 INFO:tasks.workunit.client.1.smithi196.stdout:4/377: mknod d1/d1e/d43/c6f 0 2024-03-13T14:49:40.502 INFO:tasks.workunit.client.0.smithi176.stdout:7/267: dwrite d1/d28/d3a/d1b/d18/f1e [4194304,4194304] 0 2024-03-13T14:49:40.509 INFO:tasks.workunit.client.1.smithi196.stdout:6/553: dwrite d6/d2a/dc/d1c/da7/fa6 [0,4194304] 0 2024-03-13T14:49:40.511 INFO:tasks.workunit.client.1.smithi196.stdout:8/357: rmdir d0/d16/d3d 39 2024-03-13T14:49:40.516 INFO:tasks.workunit.client.0.smithi176.stdout:4/283: creat d7/d10/d46/d55/f67 x:0 0 0 2024-03-13T14:49:40.517 INFO:tasks.workunit.client.1.smithi196.stdout:3/425: creat db/d19/d22/d30/f8a x:0 0 0 2024-03-13T14:49:40.518 INFO:tasks.workunit.client.0.smithi176.stdout:6/262: write d0/de/f2c [976770,99780] 0 2024-03-13T14:49:40.518 INFO:tasks.workunit.client.1.smithi196.stdout:7/536: read - d0/d6/d33/f62 zero size 2024-03-13T14:49:40.523 INFO:tasks.workunit.client.1.smithi196.stdout:7/537: write d0/d9/f92 [703675,75858] 0 2024-03-13T14:49:40.526 INFO:tasks.workunit.client.0.smithi176.stdout:8/251: getdents d0/db/d39 0 2024-03-13T14:49:40.526 INFO:tasks.workunit.client.1.smithi196.stdout:0/459: creat d8/da/d25/d26/d35/d3b/d50/f99 x:0 0 0 2024-03-13T14:49:40.528 INFO:tasks.workunit.client.1.smithi196.stdout:8/358: read d0/d10/d12/d17/d42/f4c [2259118,34486] 0 2024-03-13T14:49:40.529 INFO:tasks.workunit.client.1.smithi196.stdout:7/538: truncate d0/f37 5189201 0 2024-03-13T14:49:40.529 INFO:tasks.workunit.client.1.smithi196.stdout:1/433: write d1/d38/f6a [4284786,85916] 0 2024-03-13T14:49:40.530 INFO:tasks.workunit.client.0.smithi176.stdout:5/243: dread d5/d2b/f30 [0,4194304] 0 2024-03-13T14:49:40.532 INFO:tasks.workunit.client.1.smithi196.stdout:5/512: mknod d4/db/d14/d19/d1c/d25/d38/d47/da0/ca1 0 2024-03-13T14:49:40.532 INFO:tasks.workunit.client.0.smithi176.stdout:3/273: truncate d4/d28/d3a/f27 930481 0 2024-03-13T14:49:40.533 INFO:tasks.workunit.client.0.smithi176.stdout:3/274: fsync d4/d28/d4f/f53 0 2024-03-13T14:49:40.533 INFO:tasks.workunit.client.0.smithi176.stdout:3/275: readlink d4/l15 0 2024-03-13T14:49:40.537 INFO:tasks.workunit.client.0.smithi176.stdout:7/268: mkdir d1/d1c/d4a 0 2024-03-13T14:49:40.538 INFO:tasks.workunit.client.1.smithi196.stdout:4/378: creat d1/d1e/d39/d58/f70 x:0 0 0 2024-03-13T14:49:40.539 INFO:tasks.workunit.client.0.smithi176.stdout:0/242: creat d2/db/d31/f41 x:0 0 0 2024-03-13T14:49:40.540 INFO:tasks.workunit.client.1.smithi196.stdout:2/314: truncate d0/d7/f21 508152 0 2024-03-13T14:49:40.542 INFO:tasks.workunit.client.1.smithi196.stdout:2/315: dread - d0/d6/d1d/f5e zero size 2024-03-13T14:49:40.543 INFO:tasks.workunit.client.1.smithi196.stdout:2/316: chown d0/d6/d1d/f23 57336 1 2024-03-13T14:49:40.546 INFO:tasks.workunit.client.0.smithi176.stdout:4/284: rename d7/d8/f18 to d7/d10/d46/d55/f68 0 2024-03-13T14:49:40.548 INFO:tasks.workunit.client.0.smithi176.stdout:8/252: rmdir d0/db/d39 39 2024-03-13T14:49:40.550 INFO:tasks.workunit.client.1.smithi196.stdout:0/460: fsync d8/da/f5c 0 2024-03-13T14:49:40.551 INFO:tasks.workunit.client.0.smithi176.stdout:8/253: chown d0/d5/f20 123036397 1 2024-03-13T14:49:40.551 INFO:tasks.workunit.client.0.smithi176.stdout:7/269: readlink d1/d7/l39 0 2024-03-13T14:49:40.552 INFO:tasks.workunit.client.0.smithi176.stdout:5/244: write d5/f24 [3414183,92014] 0 2024-03-13T14:49:40.553 INFO:tasks.workunit.client.1.smithi196.stdout:7/539: readlink d0/d4d/d74/la5 0 2024-03-13T14:49:40.553 INFO:tasks.workunit.client.0.smithi176.stdout:7/270: chown d1/d1c 2469 1 2024-03-13T14:49:40.555 INFO:tasks.workunit.client.1.smithi196.stdout:7/540: chown d0/d6/c12 2 1 2024-03-13T14:49:40.555 INFO:tasks.workunit.client.1.smithi196.stdout:9/354: getdents d7/d9 0 2024-03-13T14:49:40.555 INFO:tasks.workunit.client.0.smithi176.stdout:5/245: dread - d5/d22/d15/f2d zero size 2024-03-13T14:49:40.558 INFO:tasks.workunit.client.0.smithi176.stdout:5/246: truncate d5/f37 883439 0 2024-03-13T14:49:40.559 INFO:tasks.workunit.client.0.smithi176.stdout:6/263: dwrite d0/fd [0,4194304] 0 2024-03-13T14:49:40.563 INFO:tasks.workunit.client.0.smithi176.stdout:7/271: dread d1/d1c/d26/d41/f3e [0,4194304] 0 2024-03-13T14:49:40.564 INFO:tasks.workunit.client.1.smithi196.stdout:6/554: fsync d6/f8 0 2024-03-13T14:49:40.568 INFO:tasks.workunit.client.0.smithi176.stdout:8/254: creat d0/db/dc/d40/f4f x:0 0 0 2024-03-13T14:49:40.569 INFO:tasks.workunit.client.1.smithi196.stdout:3/426: mknod db/d19/c8b 0 2024-03-13T14:49:40.572 INFO:tasks.workunit.client.0.smithi176.stdout:6/264: mkdir d0/de/d54 0 2024-03-13T14:49:40.574 INFO:tasks.workunit.client.0.smithi176.stdout:6/265: dread - d0/d10/d19/d27/f4b zero size 2024-03-13T14:49:40.575 INFO:tasks.workunit.client.1.smithi196.stdout:8/359: mknod d0/d10/d12/d17/c78 0 2024-03-13T14:49:40.576 INFO:tasks.workunit.client.1.smithi196.stdout:0/461: write d8/d37/f73 [1576278,128630] 0 2024-03-13T14:49:40.581 INFO:tasks.workunit.client.1.smithi196.stdout:0/462: write d8/da/d25/d26/d35/d3b/f7d [276696,24521] 0 2024-03-13T14:49:40.588 INFO:tasks.workunit.client.1.smithi196.stdout:0/463: dread - d8/da/d25/d26/d35/d3b/d59/d6a/f8d zero size 2024-03-13T14:49:40.591 INFO:tasks.workunit.client.1.smithi196.stdout:0/464: chown d8/da/c52 888047 1 2024-03-13T14:49:40.593 INFO:tasks.workunit.client.1.smithi196.stdout:3/427: dwrite db/d37/f76 [0,4194304] 0 2024-03-13T14:49:40.597 INFO:tasks.workunit.client.1.smithi196.stdout:1/434: rmdir d1/d9/d8c 0 2024-03-13T14:49:40.598 INFO:tasks.workunit.client.0.smithi176.stdout:7/272: rmdir d1/d28/d3a 39 2024-03-13T14:49:40.599 INFO:tasks.workunit.client.0.smithi176.stdout:4/285: dwrite d7/d2d/f2e [0,4194304] 0 2024-03-13T14:49:40.600 INFO:tasks.workunit.client.0.smithi176.stdout:0/243: dread d2/d5/dc/f19 [0,4194304] 0 2024-03-13T14:49:40.602 INFO:tasks.workunit.client.1.smithi196.stdout:6/555: write d6/d7/df/f2e [7225227,80300] 0 2024-03-13T14:49:40.604 INFO:tasks.workunit.client.1.smithi196.stdout:4/379: dwrite d1/d3/d7/f1a [0,4194304] 0 2024-03-13T14:49:40.604 INFO:tasks.workunit.client.1.smithi196.stdout:4/380: readlink d1/d3/d7/d35/l6e 0 2024-03-13T14:49:40.606 INFO:tasks.workunit.client.0.smithi176.stdout:0/244: stat d2/db/f2c 0 2024-03-13T14:49:40.609 INFO:tasks.workunit.client.0.smithi176.stdout:0/245: dread - d2/d5/d1e/f3d zero size 2024-03-13T14:49:40.612 INFO:tasks.workunit.client.0.smithi176.stdout:0/246: write d2/db/d35/f3f [390195,128156] 0 2024-03-13T14:49:40.616 INFO:tasks.workunit.client.0.smithi176.stdout:7/273: mkdir d1/d3f/d4b 0 2024-03-13T14:49:40.618 INFO:tasks.workunit.client.1.smithi196.stdout:2/317: rename d0/d7/c4c to d0/d6/c69 0 2024-03-13T14:49:40.620 INFO:tasks.workunit.client.1.smithi196.stdout:2/318: write d0/d6/f14 [4334948,126528] 0 2024-03-13T14:49:40.621 INFO:tasks.workunit.client.0.smithi176.stdout:6/266: creat d0/d9/f55 x:0 0 0 2024-03-13T14:49:40.621 INFO:tasks.workunit.client.1.smithi196.stdout:0/465: creat d8/da/d25/d26/d54/f9a x:0 0 0 2024-03-13T14:49:40.624 INFO:tasks.workunit.client.0.smithi176.stdout:0/247: creat d2/d5/d1e/f42 x:0 0 0 2024-03-13T14:49:40.626 INFO:tasks.workunit.client.0.smithi176.stdout:0/248: dread - d2/db/d35/f40 zero size 2024-03-13T14:49:40.628 INFO:tasks.workunit.client.1.smithi196.stdout:1/435: symlink d1/d28/d34/l93 0 2024-03-13T14:49:40.631 INFO:tasks.workunit.client.1.smithi196.stdout:5/513: getdents d4/db/d14/d19/d1c/d25/d38/d47/da0 0 2024-03-13T14:49:40.633 INFO:tasks.workunit.client.0.smithi176.stdout:4/286: creat d7/d10/d46/d4a/d5d/f69 x:0 0 0 2024-03-13T14:49:40.633 INFO:tasks.workunit.client.1.smithi196.stdout:2/319: write d0/d7/f17 [1143966,124633] 0 2024-03-13T14:49:40.638 INFO:tasks.workunit.client.1.smithi196.stdout:6/556: dwrite d6/fa2 [0,4194304] 0 2024-03-13T14:49:40.640 INFO:tasks.workunit.client.1.smithi196.stdout:8/360: mkdir d0/d10/d79 0 2024-03-13T14:49:40.645 INFO:tasks.workunit.client.1.smithi196.stdout:4/381: mknod d1/d1e/d43/d5b/d55/c71 0 2024-03-13T14:49:40.646 INFO:tasks.workunit.client.0.smithi176.stdout:0/249: truncate d2/d5/dc/f16 5052783 0 2024-03-13T14:49:40.650 INFO:tasks.workunit.client.0.smithi176.stdout:7/274: truncate d1/d28/d3a/f20 216699 0 2024-03-13T14:49:40.653 INFO:tasks.workunit.client.1.smithi196.stdout:3/428: dwrite db/d19/f2f [0,4194304] 0 2024-03-13T14:49:40.655 INFO:tasks.workunit.client.1.smithi196.stdout:7/541: rename d0/d4d/d74/c8d to d0/d9/d31/cb6 0 2024-03-13T14:49:40.659 INFO:tasks.workunit.client.0.smithi176.stdout:8/255: truncate d0/db/dc/f35 896103 0 2024-03-13T14:49:40.662 INFO:tasks.workunit.client.0.smithi176.stdout:7/275: dread d1/f3c [0,4194304] 0 2024-03-13T14:49:40.662 INFO:tasks.workunit.client.0.smithi176.stdout:6/267: creat d0/d10/d19/d27/d53/f56 x:0 0 0 2024-03-13T14:49:40.663 INFO:tasks.workunit.client.1.smithi196.stdout:9/355: truncate d7/de/d4b/f59 63398 0 2024-03-13T14:49:40.666 INFO:tasks.workunit.client.0.smithi176.stdout:6/268: fdatasync d0/d10/d19/d27/f4a 0 2024-03-13T14:49:40.668 INFO:tasks.workunit.client.0.smithi176.stdout:0/250: mknod d2/d5/d28/c43 0 2024-03-13T14:49:40.673 INFO:tasks.workunit.client.1.smithi196.stdout:0/466: creat d8/da/d25/d26/d35/d3b/d59/f9b x:0 0 0 2024-03-13T14:49:40.675 INFO:tasks.workunit.client.1.smithi196.stdout:5/514: creat d4/db/d14/d19/d1c/fa2 x:0 0 0 2024-03-13T14:49:40.676 INFO:tasks.workunit.client.0.smithi176.stdout:8/256: symlink d0/d5/d3b/l50 0 2024-03-13T14:49:40.677 INFO:tasks.workunit.client.1.smithi196.stdout:6/557: truncate d6/d9/d17/d19/f22 482680 0 2024-03-13T14:49:40.678 INFO:tasks.workunit.client.0.smithi176.stdout:7/276: creat d1/d1c/d26/f4c x:0 0 0 2024-03-13T14:49:40.680 INFO:tasks.workunit.client.1.smithi196.stdout:8/361: symlink d0/d1c/d3b/l7a 0 2024-03-13T14:49:40.685 INFO:tasks.workunit.client.0.smithi176.stdout:9/319: sync 2024-03-13T14:49:40.689 INFO:tasks.workunit.client.1.smithi196.stdout:7/542: rename d0/f4f to d0/d9/d1e/d58/d61/da0/fb7 0 2024-03-13T14:49:40.691 INFO:tasks.workunit.client.0.smithi176.stdout:0/251: dread d2/d5/dc/f2e [0,4194304] 0 2024-03-13T14:49:40.694 INFO:tasks.workunit.client.1.smithi196.stdout:9/356: symlink d7/d24/d28/l76 0 2024-03-13T14:49:40.699 INFO:tasks.workunit.client.1.smithi196.stdout:1/436: mkdir d1/d94 0 2024-03-13T14:49:40.701 INFO:tasks.workunit.client.1.smithi196.stdout:1/437: write d1/f89 [947837,20578] 0 2024-03-13T14:49:40.701 INFO:tasks.workunit.client.0.smithi176.stdout:7/277: creat d1/d3f/f4d x:0 0 0 2024-03-13T14:49:40.702 INFO:tasks.workunit.client.1.smithi196.stdout:0/467: fsync d8/da/d25/f2f 0 2024-03-13T14:49:40.706 INFO:tasks.workunit.client.0.smithi176.stdout:6/269: link d0/d10/d19/d27/d2b/f39 d0/de/f57 0 2024-03-13T14:49:40.711 INFO:tasks.workunit.client.0.smithi176.stdout:4/287: getdents d7/d8/d15/d23 0 2024-03-13T14:49:40.715 INFO:tasks.workunit.client.0.smithi176.stdout:7/278: rmdir d1/d1c/d26/d41/d2b 39 2024-03-13T14:49:40.716 INFO:tasks.workunit.client.1.smithi196.stdout:4/382: write d1/d3/d7/f1d [1123297,55637] 0 2024-03-13T14:49:40.720 INFO:tasks.workunit.client.0.smithi176.stdout:9/320: creat d0/d40/d53/f65 x:0 0 0 2024-03-13T14:49:40.725 INFO:tasks.workunit.client.0.smithi176.stdout:4/288: unlink d7/d10/d2c/l4d 0 2024-03-13T14:49:40.736 INFO:tasks.workunit.client.1.smithi196.stdout:6/558: dwrite d6/d2a/f93 [4194304,4194304] 0 2024-03-13T14:49:40.736 INFO:tasks.workunit.client.0.smithi176.stdout:0/252: truncate d2/db/d35/f3f 352059 0 2024-03-13T14:49:40.738 INFO:tasks.workunit.client.0.smithi176.stdout:8/257: rename d0/c28 to d0/db/c51 0 2024-03-13T14:49:40.740 INFO:tasks.workunit.client.0.smithi176.stdout:3/276: sync 2024-03-13T14:49:40.740 INFO:tasks.workunit.client.0.smithi176.stdout:2/289: sync 2024-03-13T14:49:40.740 INFO:tasks.workunit.client.0.smithi176.stdout:1/267: sync 2024-03-13T14:49:40.740 INFO:tasks.workunit.client.0.smithi176.stdout:8/258: chown d0/f30 300630 1 2024-03-13T14:49:40.741 INFO:tasks.workunit.client.0.smithi176.stdout:7/279: symlink d1/l4e 0 2024-03-13T14:49:40.741 INFO:tasks.workunit.client.0.smithi176.stdout:4/289: write d7/d8/d15/d23/f35 [2897918,53487] 0 2024-03-13T14:49:40.745 INFO:tasks.workunit.client.0.smithi176.stdout:3/277: write d4/d28/f49 [723217,17776] 0 2024-03-13T14:49:40.746 INFO:tasks.workunit.client.1.smithi196.stdout:3/429: creat db/d19/d4a/f8c x:0 0 0 2024-03-13T14:49:40.746 INFO:tasks.workunit.client.0.smithi176.stdout:6/270: link d0/de/l51 d0/l58 0 2024-03-13T14:49:40.747 INFO:tasks.workunit.client.0.smithi176.stdout:0/253: creat d2/db/d31/f44 x:0 0 0 2024-03-13T14:49:40.748 INFO:tasks.workunit.client.0.smithi176.stdout:9/321: link d0/d17/l18 d0/d13/l66 0 2024-03-13T14:49:40.749 INFO:tasks.workunit.client.0.smithi176.stdout:3/278: dread - d4/d28/d34/f3f zero size 2024-03-13T14:49:40.750 INFO:tasks.workunit.client.1.smithi196.stdout:5/515: rename d4/db/d14/d19/d26/d33/d61/f6f to d4/db/d4a/fa3 0 2024-03-13T14:49:40.752 INFO:tasks.workunit.client.0.smithi176.stdout:9/322: write d0/d13/d32/f62 [322070,57992] 0 2024-03-13T14:49:40.755 INFO:tasks.workunit.client.0.smithi176.stdout:4/290: mknod d7/d10/c6a 0 2024-03-13T14:49:40.755 INFO:tasks.workunit.client.1.smithi196.stdout:5/516: chown d4/db/d14/d19/f50 33211155 1 2024-03-13T14:49:40.758 INFO:tasks.workunit.client.0.smithi176.stdout:2/290: symlink d1/dc/d1c/l66 0 2024-03-13T14:49:40.759 INFO:tasks.workunit.client.1.smithi196.stdout:5/517: write d4/db/d14/d19/d1c/d2d/d82/f83 [1484111,97978] 0 2024-03-13T14:49:40.761 INFO:tasks.workunit.client.0.smithi176.stdout:1/268: symlink d2/de/d10/d16/l51 0 2024-03-13T14:49:40.765 INFO:tasks.workunit.client.0.smithi176.stdout:3/279: dread d4/d13/f2a [0,4194304] 0 2024-03-13T14:49:40.765 INFO:tasks.workunit.client.1.smithi196.stdout:1/438: write d1/d9/d33/f62 [2484829,129270] 0 2024-03-13T14:49:40.767 INFO:tasks.workunit.client.0.smithi176.stdout:3/280: stat d4/d28/d31/d45/c44 0 2024-03-13T14:49:40.769 INFO:tasks.workunit.client.0.smithi176.stdout:6/271: creat d0/d10/d19/f59 x:0 0 0 2024-03-13T14:49:40.769 INFO:tasks.workunit.client.1.smithi196.stdout:3/430: dwrite db/d19/d4a/f8c [0,4194304] 0 2024-03-13T14:49:40.771 INFO:tasks.workunit.client.1.smithi196.stdout:1/439: dread - d1/d9/d4e/d55/d63/f73 zero size 2024-03-13T14:49:40.774 INFO:tasks.workunit.client.1.smithi196.stdout:4/383: mkdir d1/d3/d4c/d72 0 2024-03-13T14:49:40.776 INFO:tasks.workunit.client.0.smithi176.stdout:4/291: dwrite d7/f50 [0,4194304] 0 2024-03-13T14:49:40.777 INFO:tasks.workunit.client.1.smithi196.stdout:2/320: sync 2024-03-13T14:49:40.789 INFO:tasks.workunit.client.0.smithi176.stdout:9/323: rmdir d0 39 2024-03-13T14:49:40.794 INFO:tasks.workunit.client.1.smithi196.stdout:7/543: creat d0/d9/d1e/d58/d72/d99/fb8 x:0 0 0 2024-03-13T14:49:40.794 INFO:tasks.workunit.client.0.smithi176.stdout:7/280: mknod d1/d3f/d4b/c4f 0 2024-03-13T14:49:40.796 INFO:tasks.workunit.client.1.smithi196.stdout:7/544: chown d0/d9/d1e/c8f 7969 1 2024-03-13T14:49:40.797 INFO:tasks.workunit.client.1.smithi196.stdout:7/545: chown d0/f56 3 1 2024-03-13T14:49:40.798 INFO:tasks.workunit.client.0.smithi176.stdout:1/269: dread d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:49:40.799 INFO:tasks.workunit.client.1.smithi196.stdout:7/546: read - d0/d4d/f91 zero size 2024-03-13T14:49:40.799 INFO:tasks.workunit.client.0.smithi176.stdout:3/281: dread d4/d28/d3a/f32 [0,4194304] 0 2024-03-13T14:49:40.801 INFO:tasks.workunit.client.0.smithi176.stdout:1/270: write d2/de/f41 [667722,112239] 0 2024-03-13T14:49:40.801 INFO:tasks.workunit.client.0.smithi176.stdout:8/259: rename d0/db/f12 to d0/f52 0 2024-03-13T14:49:40.802 INFO:tasks.workunit.client.0.smithi176.stdout:3/282: write d4/d28/d4f/f53 [213468,15249] 0 2024-03-13T14:49:40.803 INFO:tasks.workunit.client.1.smithi196.stdout:5/518: mknod d4/db/d14/d19/d1c/ca4 0 2024-03-13T14:49:40.804 INFO:tasks.workunit.client.1.smithi196.stdout:9/357: dwrite d7/de/d4b/f52 [0,4194304] 0 2024-03-13T14:49:40.810 INFO:tasks.workunit.client.1.smithi196.stdout:0/468: getdents d8/da/d25/d26/d35/d3b/d59/d6a 0 2024-03-13T14:49:40.810 INFO:tasks.workunit.client.0.smithi176.stdout:5/247: sync 2024-03-13T14:49:40.812 INFO:tasks.workunit.client.0.smithi176.stdout:0/254: dwrite d2/f11 [4194304,4194304] 0 2024-03-13T14:49:40.815 INFO:tasks.workunit.client.1.smithi196.stdout:8/362: creat d0/d1c/f7b x:0 0 0 2024-03-13T14:49:40.820 INFO:tasks.workunit.client.1.smithi196.stdout:8/363: chown d0/lf 11846 1 2024-03-13T14:49:40.821 INFO:tasks.workunit.client.1.smithi196.stdout:1/440: creat d1/d9/d33/d4b/d66/f95 x:0 0 0 2024-03-13T14:49:40.822 INFO:tasks.workunit.client.0.smithi176.stdout:1/271: dwrite d2/d29/f39 [4194304,4194304] 0 2024-03-13T14:49:40.824 INFO:tasks.workunit.client.0.smithi176.stdout:8/260: dwrite d0/d5/f45 [0,4194304] 0 2024-03-13T14:49:40.824 INFO:tasks.workunit.client.1.smithi196.stdout:4/384: rename d1/d3/d7/d35/l6e to d1/d1e/d43/l73 0 2024-03-13T14:49:40.826 INFO:tasks.workunit.client.1.smithi196.stdout:4/385: chown d1/d1e/d5c 625277687 1 2024-03-13T14:49:40.828 INFO:tasks.workunit.client.1.smithi196.stdout:2/321: symlink d0/d7/d56/l6a 0 2024-03-13T14:49:40.829 INFO:tasks.workunit.client.1.smithi196.stdout:6/559: creat d6/d2a/dc/da3/fab x:0 0 0 2024-03-13T14:49:40.830 INFO:tasks.workunit.client.0.smithi176.stdout:7/281: rename d1/d7/l39 to d1/d1c/l50 0 2024-03-13T14:49:40.831 INFO:tasks.workunit.client.1.smithi196.stdout:2/322: chown d0/d7/l11 1075762 1 2024-03-13T14:49:40.835 INFO:tasks.workunit.client.1.smithi196.stdout:7/547: creat d0/d9/d1e/d58/d72/da9/da3/fb9 x:0 0 0 2024-03-13T14:49:40.836 INFO:tasks.workunit.client.1.smithi196.stdout:2/323: dread - d0/d7/d9/f48 zero size 2024-03-13T14:49:40.836 INFO:tasks.workunit.client.0.smithi176.stdout:3/283: rmdir d4/d13/d2d/d50 39 2024-03-13T14:49:40.844 INFO:tasks.workunit.client.1.smithi196.stdout:9/358: creat d7/d9/f77 x:0 0 0 2024-03-13T14:49:40.846 INFO:tasks.workunit.client.1.smithi196.stdout:9/359: chown d7/de/d4b/d1d/d22/d4d 7157 1 2024-03-13T14:49:40.847 INFO:tasks.workunit.client.1.smithi196.stdout:3/431: mkdir db/d19/d22/d3a/d6a/d8d 0 2024-03-13T14:49:40.855 INFO:tasks.workunit.client.1.smithi196.stdout:1/441: unlink d1/d3f/f5d 0 2024-03-13T14:49:40.855 INFO:tasks.workunit.client.1.smithi196.stdout:2/324: dwrite d0/d7/d56/f68 [0,4194304] 0 2024-03-13T14:49:40.871 INFO:tasks.workunit.client.1.smithi196.stdout:4/386: creat d1/d1e/d5c/f74 x:0 0 0 2024-03-13T14:49:40.873 INFO:tasks.workunit.client.1.smithi196.stdout:7/548: dread d0/d9/d1e/d58/d61/da0/fb7 [0,4194304] 0 2024-03-13T14:49:40.873 INFO:tasks.workunit.client.0.smithi176.stdout:0/255: symlink d2/d5/d1e/l45 0 2024-03-13T14:49:40.877 INFO:tasks.workunit.client.1.smithi196.stdout:5/519: mknod d4/db/d3c/ca5 0 2024-03-13T14:49:40.880 INFO:tasks.workunit.client.0.smithi176.stdout:9/324: mkdir d0/d13/d32/d58/d67 0 2024-03-13T14:49:40.880 INFO:tasks.workunit.client.1.smithi196.stdout:9/360: truncate d7/de/d16/f2a 2096526 0 2024-03-13T14:49:40.881 INFO:tasks.workunit.client.1.smithi196.stdout:9/361: readlink d7/lb 0 2024-03-13T14:49:40.882 INFO:tasks.workunit.client.1.smithi196.stdout:5/520: write d4/db/d14/d19/d26/f51 [331983,57635] 0 2024-03-13T14:49:40.884 INFO:tasks.workunit.client.0.smithi176.stdout:6/272: dwrite d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:40.887 INFO:tasks.workunit.client.1.smithi196.stdout:1/442: fsync d1/d9/d6c/f77 0 2024-03-13T14:49:40.889 INFO:tasks.workunit.client.0.smithi176.stdout:4/292: link d7/d10/d46/d4a/f4b d7/d2d/f6b 0 2024-03-13T14:49:40.889 INFO:tasks.workunit.client.1.smithi196.stdout:8/364: truncate d0/d1c/d3b/f4b 4129228 0 2024-03-13T14:49:40.890 INFO:tasks.workunit.client.0.smithi176.stdout:2/291: getdents d1/d2a 0 2024-03-13T14:49:40.890 INFO:tasks.workunit.client.0.smithi176.stdout:3/284: write d4/d28/f29 [1400674,75681] 0 2024-03-13T14:49:40.891 INFO:tasks.workunit.client.1.smithi196.stdout:6/560: rmdir d6/d7/dd/d60 39 2024-03-13T14:49:40.892 INFO:tasks.workunit.client.0.smithi176.stdout:3/285: chown d4/c1e 0 1 2024-03-13T14:49:40.892 INFO:tasks.workunit.client.0.smithi176.stdout:5/248: link d5/d22/d15/f2d d5/d2e/d43/f50 0 2024-03-13T14:49:40.893 INFO:tasks.workunit.client.1.smithi196.stdout:8/365: write d0/d1c/d3b/d3f/f64 [471604,79861] 0 2024-03-13T14:49:40.900 INFO:tasks.workunit.client.1.smithi196.stdout:8/366: write d0/d1c/d3b/f60 [1201521,1251] 0 2024-03-13T14:49:40.901 INFO:tasks.workunit.client.0.smithi176.stdout:9/325: mkdir d0/d1e/d1f/d68 0 2024-03-13T14:49:40.905 INFO:tasks.workunit.client.0.smithi176.stdout:8/261: mkdir d0/db/d53 0 2024-03-13T14:49:40.907 INFO:tasks.workunit.client.0.smithi176.stdout:6/273: symlink d0/de/d38/l5a 0 2024-03-13T14:49:40.909 INFO:tasks.workunit.client.0.smithi176.stdout:7/282: symlink d1/l51 0 2024-03-13T14:49:40.914 INFO:tasks.workunit.client.0.smithi176.stdout:4/293: mknod d7/d10/d46/d4a/d5d/c6c 0 2024-03-13T14:49:40.914 INFO:tasks.workunit.client.0.smithi176.stdout:9/326: dread d0/d17/f3a [0,4194304] 0 2024-03-13T14:49:40.916 INFO:tasks.workunit.client.0.smithi176.stdout:2/292: readlink d1/d2a/l43 0 2024-03-13T14:49:40.918 INFO:tasks.workunit.client.0.smithi176.stdout:8/262: dread d0/db/f1a [0,4194304] 0 2024-03-13T14:49:40.920 INFO:tasks.workunit.client.1.smithi196.stdout:6/561: creat d6/d7/df/d27/d39/d44/d74/fac x:0 0 0 2024-03-13T14:49:40.920 INFO:tasks.workunit.client.1.smithi196.stdout:7/549: write d0/d9/d1e/d58/d72/d6e/f70 [1782246,107619] 0 2024-03-13T14:49:40.921 INFO:tasks.workunit.client.0.smithi176.stdout:3/286: unlink d4/d13/l1c 0 2024-03-13T14:49:40.922 INFO:tasks.workunit.client.1.smithi196.stdout:6/562: fdatasync d6/d7/dd/f7e 0 2024-03-13T14:49:40.925 INFO:tasks.workunit.client.1.smithi196.stdout:5/521: creat d4/db/d3c/d6a/d91/fa6 x:0 0 0 2024-03-13T14:49:40.925 INFO:tasks.workunit.client.1.smithi196.stdout:7/550: read d0/d6/f45 [3362842,57455] 0 2024-03-13T14:49:40.926 INFO:tasks.workunit.client.1.smithi196.stdout:3/432: truncate db/d37/d64/f74 1174593 0 2024-03-13T14:49:40.928 INFO:tasks.workunit.client.0.smithi176.stdout:1/272: creat d2/de/d3f/f52 x:0 0 0 2024-03-13T14:49:40.928 INFO:tasks.workunit.client.1.smithi196.stdout:2/325: mknod d0/d6/d4b/c6b 0 2024-03-13T14:49:40.930 INFO:tasks.workunit.client.1.smithi196.stdout:7/551: write d0/d9/d31/d50/f87 [3289428,61300] 0 2024-03-13T14:49:40.931 INFO:tasks.workunit.client.1.smithi196.stdout:4/387: link d1/d1e/l2b d1/d3/d4c/l75 0 2024-03-13T14:49:40.933 INFO:tasks.workunit.client.0.smithi176.stdout:7/283: dread d1/d1c/d26/d41/f3e [0,4194304] 0 2024-03-13T14:49:40.934 INFO:tasks.workunit.client.0.smithi176.stdout:7/284: readlink d1/d7/lb 0 2024-03-13T14:49:40.934 INFO:tasks.workunit.client.0.smithi176.stdout:4/294: creat d7/d8/d33/f6d x:0 0 0 2024-03-13T14:49:40.942 INFO:tasks.workunit.client.0.smithi176.stdout:2/293: readlink d1/dc/l42 0 2024-03-13T14:49:40.944 INFO:tasks.workunit.client.0.smithi176.stdout:1/273: dwrite d2/de/d3f/f52 [0,4194304] 0 2024-03-13T14:49:40.944 INFO:tasks.workunit.client.1.smithi196.stdout:6/563: dwrite d6/d9/d17/d19/fa8 [0,4194304] 0 2024-03-13T14:49:40.945 INFO:tasks.workunit.client.0.smithi176.stdout:8/263: rmdir d0/db 39 2024-03-13T14:49:40.947 INFO:tasks.workunit.client.0.smithi176.stdout:5/249: dread d5/d22/d15/f16 [0,4194304] 0 2024-03-13T14:49:40.953 INFO:tasks.workunit.client.0.smithi176.stdout:5/250: truncate d5/d2e/f3f 310527 0 2024-03-13T14:49:40.954 INFO:tasks.workunit.client.0.smithi176.stdout:0/256: rename d2/d5/dc/f16 to d2/d5/f46 0 2024-03-13T14:49:40.959 INFO:tasks.workunit.client.0.smithi176.stdout:6/274: symlink d0/l5b 0 2024-03-13T14:49:40.964 INFO:tasks.workunit.client.1.smithi196.stdout:5/522: truncate d4/db/d3c/f57 615033 0 2024-03-13T14:49:40.965 INFO:tasks.workunit.client.0.smithi176.stdout:9/327: mknod d0/db/c69 0 2024-03-13T14:49:40.966 INFO:tasks.workunit.client.0.smithi176.stdout:4/295: mknod d7/d10/d2c/c6e 0 2024-03-13T14:49:40.967 INFO:tasks.workunit.client.0.smithi176.stdout:7/285: read d1/d28/d3a/f20 [181164,68531] 0 2024-03-13T14:49:40.967 INFO:tasks.workunit.client.0.smithi176.stdout:9/328: write d0/d1e/d1f/f41 [830566,113293] 0 2024-03-13T14:49:40.969 INFO:tasks.workunit.client.1.smithi196.stdout:5/523: write d4/db/d14/d19/f96 [366925,100266] 0 2024-03-13T14:49:40.969 INFO:tasks.workunit.client.0.smithi176.stdout:4/296: dread - d7/d10/d46/d55/f67 zero size 2024-03-13T14:49:40.972 INFO:tasks.workunit.client.0.smithi176.stdout:2/294: unlink d1/d2a/d33/f3a 0 2024-03-13T14:49:40.975 INFO:tasks.workunit.client.1.smithi196.stdout:5/524: chown d4/db/d14/d19/d26/d33/d61/l7b 1 1 2024-03-13T14:49:40.977 INFO:tasks.workunit.client.0.smithi176.stdout:3/287: rename d4/cb to d4/d28/c56 0 2024-03-13T14:49:40.981 INFO:tasks.workunit.client.1.smithi196.stdout:8/367: link d0/d10/d12/f3e d0/d1c/f7c 0 2024-03-13T14:49:40.982 INFO:tasks.workunit.client.0.smithi176.stdout:0/257: dread d2/db/d35/f3f [0,4194304] 0 2024-03-13T14:49:40.984 INFO:tasks.workunit.client.0.smithi176.stdout:6/275: creat d0/d10/d19/d27/d2b/f5c x:0 0 0 2024-03-13T14:49:40.986 INFO:tasks.workunit.client.0.smithi176.stdout:6/276: dread - d0/d10/d19/d27/d2b/d2e/f52 zero size 2024-03-13T14:49:40.986 INFO:tasks.workunit.client.1.smithi196.stdout:2/326: dwrite d0/d6/d4b/d5f/f3b [0,4194304] 0 2024-03-13T14:49:40.987 INFO:tasks.workunit.client.0.smithi176.stdout:6/277: chown d0/d9/l1c 2014 1 2024-03-13T14:49:40.991 INFO:tasks.workunit.client.1.smithi196.stdout:7/552: truncate d0/d9/d1e/d3f/f82 4079453 0 2024-03-13T14:49:40.994 INFO:tasks.workunit.client.0.smithi176.stdout:7/286: creat d1/d28/d3a/d1b/d18/f52 x:0 0 0 2024-03-13T14:49:40.998 INFO:tasks.workunit.client.1.smithi196.stdout:5/525: rmdir d4/db/d14/d19/d26 39 2024-03-13T14:49:41.000 INFO:tasks.workunit.client.0.smithi176.stdout:4/297: creat d7/d8/d15/d23/f6f x:0 0 0 2024-03-13T14:49:41.001 INFO:tasks.workunit.client.1.smithi196.stdout:8/368: rmdir d0/d16 39 2024-03-13T14:49:41.002 INFO:tasks.workunit.client.0.smithi176.stdout:4/298: write d7/d8/d15/d23/f35 [4078698,19754] 0 2024-03-13T14:49:41.003 INFO:tasks.workunit.client.1.smithi196.stdout:9/362: link d7/d9/l56 d7/de/d4b/d1d/l78 0 2024-03-13T14:49:41.005 INFO:tasks.workunit.client.0.smithi176.stdout:8/264: fsync d0/d24/f32 0 2024-03-13T14:49:41.006 INFO:tasks.workunit.client.1.smithi196.stdout:8/369: read d0/d1c/d3b/f1f [533721,87180] 0 2024-03-13T14:49:41.008 INFO:tasks.workunit.client.0.smithi176.stdout:3/288: unlink d4/f36 0 2024-03-13T14:49:41.011 INFO:tasks.workunit.client.1.smithi196.stdout:4/388: rename d1/d1e/d39 to d1/d3/d7/d35/d76 0 2024-03-13T14:49:41.014 INFO:tasks.workunit.client.1.smithi196.stdout:4/389: chown d1/d3/d7/d35/d76/l3b 2024424609 1 2024-03-13T14:49:41.014 INFO:tasks.workunit.client.0.smithi176.stdout:5/251: symlink d5/d2e/d43/l51 0 2024-03-13T14:49:41.014 INFO:tasks.workunit.client.0.smithi176.stdout:5/252: chown d5/d2e/d43 925876350 1 2024-03-13T14:49:41.017 INFO:tasks.workunit.client.1.smithi196.stdout:3/433: getdents db 0 2024-03-13T14:49:41.017 INFO:tasks.workunit.client.1.smithi196.stdout:7/553: creat d0/d9/d1e/d58/d72/d6e/fba x:0 0 0 2024-03-13T14:49:41.019 INFO:tasks.workunit.client.0.smithi176.stdout:4/299: dwrite d7/d10/d2c/f3f [0,4194304] 0 2024-03-13T14:49:41.019 INFO:tasks.workunit.client.1.smithi196.stdout:6/564: link d6/d7/dd/l61 d6/d9/daa/lad 0 2024-03-13T14:49:41.019 INFO:tasks.workunit.client.1.smithi196.stdout:3/434: chown db/d37/l72 237 1 2024-03-13T14:49:41.020 INFO:tasks.workunit.client.0.smithi176.stdout:9/329: mkdir d0/d1e/d1f/d68/d6a 0 2024-03-13T14:49:41.022 INFO:tasks.workunit.client.1.smithi196.stdout:5/526: unlink d4/db/d4a/l4b 0 2024-03-13T14:49:41.022 INFO:tasks.workunit.client.1.smithi196.stdout:3/435: chown db/d19/d4a/f8c 50837 1 2024-03-13T14:49:41.023 INFO:tasks.workunit.client.1.smithi196.stdout:0/469: sync 2024-03-13T14:49:41.024 INFO:tasks.workunit.client.0.smithi176.stdout:6/278: mknod d0/d10/d19/d27/d2b/d2e/c5d 0 2024-03-13T14:49:41.025 INFO:tasks.workunit.client.0.smithi176.stdout:7/287: unlink d1/f19 0 2024-03-13T14:49:41.029 INFO:tasks.workunit.client.0.smithi176.stdout:1/274: creat d2/de/f53 x:0 0 0 2024-03-13T14:49:41.031 INFO:tasks.workunit.client.0.smithi176.stdout:1/275: stat d2/de/d3f/d50/c36 0 2024-03-13T14:49:41.032 INFO:tasks.workunit.client.1.smithi196.stdout:8/370: dread d0/d10/d12/d1e/f34 [0,4194304] 0 2024-03-13T14:49:41.038 INFO:tasks.workunit.client.0.smithi176.stdout:3/289: creat d4/d13/f57 x:0 0 0 2024-03-13T14:49:41.039 INFO:tasks.workunit.client.0.smithi176.stdout:9/330: dwrite d0/d1e/f25 [0,4194304] 0 2024-03-13T14:49:41.041 INFO:tasks.workunit.client.0.smithi176.stdout:9/331: chown d0/d1e/d1f/f38 742982938 1 2024-03-13T14:49:41.042 INFO:tasks.workunit.client.0.smithi176.stdout:0/258: creat d2/db/d39/f47 x:0 0 0 2024-03-13T14:49:41.044 INFO:tasks.workunit.client.1.smithi196.stdout:3/436: mkdir db/d19/d22/d3a/d68/d8e 0 2024-03-13T14:49:41.044 INFO:tasks.workunit.client.0.smithi176.stdout:0/259: dread - d2/db/d39/f47 zero size 2024-03-13T14:49:41.047 INFO:tasks.workunit.client.0.smithi176.stdout:0/260: truncate d2/d5/d1e/f42 375596 0 2024-03-13T14:49:41.049 INFO:tasks.workunit.client.0.smithi176.stdout:0/261: truncate d2/db/f1c 2822335 0 2024-03-13T14:49:41.050 INFO:tasks.workunit.client.0.smithi176.stdout:0/262: readlink d2/d5/d1e/l24 0 2024-03-13T14:49:41.051 INFO:tasks.workunit.client.1.smithi196.stdout:0/470: dread d8/da/d25/d26/d35/d3b/d5f/f2b [0,4194304] 0 2024-03-13T14:49:41.052 INFO:tasks.workunit.client.0.smithi176.stdout:4/300: rename d7/d8/f11 to d7/d8/d45/f70 0 2024-03-13T14:49:41.053 INFO:tasks.workunit.client.0.smithi176.stdout:2/295: write d1/dc/f34 [916975,10381] 0 2024-03-13T14:49:41.054 INFO:tasks.workunit.client.1.smithi196.stdout:2/327: rename d0/d6/c69 to d0/d7/d66/d67/c6c 0 2024-03-13T14:49:41.058 INFO:tasks.workunit.client.1.smithi196.stdout:4/390: symlink d1/d1e/l77 0 2024-03-13T14:49:41.059 INFO:tasks.workunit.client.1.smithi196.stdout:2/328: read d0/f37 [208437,121529] 0 2024-03-13T14:49:41.060 INFO:tasks.workunit.client.0.smithi176.stdout:9/332: dwrite d0/f5e [0,4194304] 0 2024-03-13T14:49:41.066 INFO:tasks.workunit.client.1.smithi196.stdout:2/329: fsync d0/d7/f17 0 2024-03-13T14:49:41.068 INFO:tasks.workunit.client.1.smithi196.stdout:9/363: dwrite d7/de/d4b/f14 [0,4194304] 0 2024-03-13T14:49:41.068 INFO:tasks.workunit.client.0.smithi176.stdout:3/290: mknod d4/d28/c58 0 2024-03-13T14:49:41.069 INFO:tasks.workunit.client.1.smithi196.stdout:9/364: readlink d7/lb 0 2024-03-13T14:49:41.072 INFO:tasks.workunit.client.1.smithi196.stdout:5/527: mkdir d4/db/d14/d19/da7 0 2024-03-13T14:49:41.075 INFO:tasks.workunit.client.1.smithi196.stdout:3/437: read db/d19/f55 [657025,64271] 0 2024-03-13T14:49:41.075 INFO:tasks.workunit.client.0.smithi176.stdout:3/291: write f2 [5035288,37957] 0 2024-03-13T14:49:41.075 INFO:tasks.workunit.client.1.smithi196.stdout:1/443: sync 2024-03-13T14:49:41.078 INFO:tasks.workunit.client.0.smithi176.stdout:2/296: dwrite d1/f3c [0,4194304] 0 2024-03-13T14:49:41.079 INFO:tasks.workunit.client.0.smithi176.stdout:1/276: rmdir d2 39 2024-03-13T14:49:41.084 INFO:tasks.workunit.client.1.smithi196.stdout:4/391: dwrite d1/d3/f4e [0,4194304] 0 2024-03-13T14:49:41.088 INFO:tasks.workunit.client.0.smithi176.stdout:7/288: symlink d1/l53 0 2024-03-13T14:49:41.090 INFO:tasks.workunit.client.0.smithi176.stdout:7/289: fsync d1/d28/f2a 0 2024-03-13T14:49:41.098 INFO:tasks.workunit.client.1.smithi196.stdout:1/444: dread d1/d9/d33/d4b/d66/d7b/f8d [0,4194304] 0 2024-03-13T14:49:41.105 INFO:tasks.workunit.client.1.smithi196.stdout:4/392: dread d1/d3/f4e [0,4194304] 0 2024-03-13T14:49:41.106 INFO:tasks.workunit.client.0.smithi176.stdout:6/279: link d0/d10/d19/d27/f4a d0/de/d54/f5e 0 2024-03-13T14:49:41.107 INFO:tasks.workunit.client.1.smithi196.stdout:6/565: truncate d6/d9/f30 2023821 0 2024-03-13T14:49:41.109 INFO:tasks.workunit.client.0.smithi176.stdout:4/301: symlink d7/d8/d33/l71 0 2024-03-13T14:49:41.112 INFO:tasks.workunit.client.1.smithi196.stdout:0/471: mknod d8/da/d25/d26/d35/d3b/d50/d55/c9c 0 2024-03-13T14:49:41.114 INFO:tasks.workunit.client.0.smithi176.stdout:9/333: creat d0/d40/d53/f6b x:0 0 0 2024-03-13T14:49:41.123 INFO:tasks.workunit.client.1.smithi196.stdout:9/365: stat d7/de/d16/c18 0 2024-03-13T14:49:41.127 INFO:tasks.workunit.client.0.smithi176.stdout:0/263: creat d2/d5/dc/d20/f48 x:0 0 0 2024-03-13T14:49:41.128 INFO:tasks.workunit.client.0.smithi176.stdout:4/302: dwrite d7/d8/d2f/f58 [0,4194304] 0 2024-03-13T14:49:41.130 INFO:tasks.workunit.client.0.smithi176.stdout:4/303: write d7/d8/d33/f6d [662585,66132] 0 2024-03-13T14:49:41.131 INFO:tasks.workunit.client.1.smithi196.stdout:5/528: chown d4/db/d14/d19/d26/d68/f72 32 1 2024-03-13T14:49:41.131 INFO:tasks.workunit.client.0.smithi176.stdout:1/277: chown d2/d30/f4c 35351768 1 2024-03-13T14:49:41.132 INFO:tasks.workunit.client.1.smithi196.stdout:4/393: dwrite d1/d1e/d5c/f5f [0,4194304] 0 2024-03-13T14:49:41.136 INFO:tasks.workunit.client.0.smithi176.stdout:2/297: mknod d1/dc/d2d/d4f/c67 0 2024-03-13T14:49:41.139 INFO:tasks.workunit.client.0.smithi176.stdout:8/265: write d0/db/dc/f35 [470470,53461] 0 2024-03-13T14:49:41.141 INFO:tasks.workunit.client.1.smithi196.stdout:3/438: truncate db/d19/d22/f31 124149 0 2024-03-13T14:49:41.146 INFO:tasks.workunit.client.1.smithi196.stdout:9/366: dwrite d7/f69 [0,4194304] 0 2024-03-13T14:49:41.147 INFO:tasks.workunit.client.0.smithi176.stdout:7/290: rename d1/l51 to d1/d28/d3a/d1b/d18/l54 0 2024-03-13T14:49:41.148 INFO:tasks.workunit.client.0.smithi176.stdout:1/278: dread d2/de/d3f/d50/f42 [0,4194304] 0 2024-03-13T14:49:41.150 INFO:tasks.workunit.client.0.smithi176.stdout:7/291: write d1/d28/f2d [2520195,52660] 0 2024-03-13T14:49:41.151 INFO:tasks.workunit.client.1.smithi196.stdout:4/394: dread d1/d1e/d43/d5b/d55/f5e [0,4194304] 0 2024-03-13T14:49:41.154 INFO:tasks.workunit.client.0.smithi176.stdout:6/280: write d0/f18 [1276381,2801] 0 2024-03-13T14:49:41.156 INFO:tasks.workunit.client.1.smithi196.stdout:8/371: symlink d0/d16/l7d 0 2024-03-13T14:49:41.158 INFO:tasks.workunit.client.0.smithi176.stdout:6/281: write d0/d10/d19/d27/f4b [983474,7491] 0 2024-03-13T14:49:41.158 INFO:tasks.workunit.client.1.smithi196.stdout:1/445: rename d1/d9/d33/d4b/f5f to d1/d28/f96 0 2024-03-13T14:49:41.159 INFO:tasks.workunit.client.1.smithi196.stdout:4/395: write d1/d1e/d5c/f74 [52172,102084] 0 2024-03-13T14:49:41.160 INFO:tasks.workunit.client.0.smithi176.stdout:9/334: mknod d0/d1e/d1f/d56/c6c 0 2024-03-13T14:49:41.162 INFO:tasks.workunit.client.1.smithi196.stdout:7/554: link d0/d9/d1e/d58/d72/l83 d0/d9/d1e/d58/d72/da9/lbb 0 2024-03-13T14:49:41.163 INFO:tasks.workunit.client.0.smithi176.stdout:9/335: fsync d0/d40/d53/f65 0 2024-03-13T14:49:41.164 INFO:tasks.workunit.client.1.smithi196.stdout:2/330: symlink d0/d7/d9/l6d 0 2024-03-13T14:49:41.164 INFO:tasks.workunit.client.0.smithi176.stdout:5/253: getdents d5 0 2024-03-13T14:49:41.166 INFO:tasks.workunit.client.0.smithi176.stdout:5/254: fsync d5/d2c/f3d 0 2024-03-13T14:49:41.166 INFO:tasks.workunit.client.0.smithi176.stdout:9/336: write d0/d40/d53/f65 [703482,100115] 0 2024-03-13T14:49:41.169 INFO:tasks.workunit.client.0.smithi176.stdout:5/255: write d5/d22/d15/f27 [1236171,89394] 0 2024-03-13T14:49:41.170 INFO:tasks.workunit.client.0.smithi176.stdout:9/337: truncate d0/d17/f3a 5078683 0 2024-03-13T14:49:41.179 INFO:tasks.workunit.client.0.smithi176.stdout:6/282: dwrite d0/d10/d19/f59 [0,4194304] 0 2024-03-13T14:49:41.180 INFO:tasks.workunit.client.0.smithi176.stdout:2/298: dwrite d1/f31 [0,4194304] 0 2024-03-13T14:49:41.184 INFO:tasks.workunit.client.0.smithi176.stdout:2/299: readlink d1/dc/d2d/d38/l53 0 2024-03-13T14:49:41.191 INFO:tasks.workunit.client.1.smithi196.stdout:5/529: write d4/db/d14/d19/d1c/d2d/f4e [1715990,26606] 0 2024-03-13T14:49:41.191 INFO:tasks.workunit.client.1.smithi196.stdout:5/530: readlink d4/db/d14/l45 0 2024-03-13T14:49:41.191 INFO:tasks.workunit.client.1.smithi196.stdout:7/555: dwrite d0/d9/d1e/d58/d72/d99/fb8 [0,4194304] 0 2024-03-13T14:49:41.192 INFO:tasks.workunit.client.0.smithi176.stdout:5/256: mknod d5/c52 0 2024-03-13T14:49:41.198 INFO:tasks.workunit.client.0.smithi176.stdout:8/266: truncate d0/db/dc/f35 182036 0 2024-03-13T14:49:41.198 INFO:tasks.workunit.client.0.smithi176.stdout:3/292: sync 2024-03-13T14:49:41.198 INFO:tasks.workunit.client.0.smithi176.stdout:1/279: creat d2/de/d10/d16/d47/f54 x:0 0 0 2024-03-13T14:49:41.200 INFO:tasks.workunit.client.0.smithi176.stdout:1/280: stat d2/de/d3f/d50 0 2024-03-13T14:49:41.202 INFO:tasks.workunit.client.0.smithi176.stdout:9/338: mkdir d0/d40/d53/d6d 0 2024-03-13T14:49:41.218 INFO:tasks.workunit.client.0.smithi176.stdout:2/300: mknod d1/dc/d2d/c68 0 2024-03-13T14:49:41.220 INFO:tasks.workunit.client.0.smithi176.stdout:2/301: read - d1/d51/d5c/f5f zero size 2024-03-13T14:49:41.223 INFO:tasks.workunit.client.0.smithi176.stdout:0/264: rename d2/db/d35/f40 to d2/d5/dc/f49 0 2024-03-13T14:49:41.226 INFO:tasks.workunit.client.1.smithi196.stdout:6/566: symlink d6/d2a/lae 0 2024-03-13T14:49:41.236 INFO:tasks.workunit.client.1.smithi196.stdout:8/372: dwrite d0/d10/d12/d1e/f5e [0,4194304] 0 2024-03-13T14:49:41.239 INFO:tasks.workunit.client.1.smithi196.stdout:1/446: symlink d1/d9/d33/d4b/d66/l97 0 2024-03-13T14:49:41.240 INFO:tasks.workunit.client.1.smithi196.stdout:9/367: write d7/de/d4b/f43 [4043021,850] 0 2024-03-13T14:49:41.244 INFO:tasks.workunit.client.0.smithi176.stdout:1/281: symlink d2/de/d10/d14/l55 0 2024-03-13T14:49:41.247 INFO:tasks.workunit.client.0.smithi176.stdout:9/339: mkdir d0/d13/d4d/d6e 0 2024-03-13T14:49:41.254 INFO:tasks.workunit.client.0.smithi176.stdout:6/283: symlink d0/d10/d19/d27/d2b/d2e/d3e/l5f 0 2024-03-13T14:49:41.263 INFO:tasks.workunit.client.0.smithi176.stdout:4/304: rename d7/d2d/l36 to d7/d10/d46/l72 0 2024-03-13T14:49:41.264 INFO:tasks.workunit.client.1.smithi196.stdout:8/373: dwrite d0/d1c/d29/d47/f52 [0,4194304] 0 2024-03-13T14:49:41.267 INFO:tasks.workunit.client.1.smithi196.stdout:7/556: dread d0/d6/d33/f3c [0,4194304] 0 2024-03-13T14:49:41.269 INFO:tasks.workunit.client.1.smithi196.stdout:0/472: dwrite d8/da/f13 [0,4194304] 0 2024-03-13T14:49:41.274 INFO:tasks.workunit.client.0.smithi176.stdout:3/293: fsync d4/d13/d2d/d50/f3c 0 2024-03-13T14:49:41.277 INFO:tasks.workunit.client.0.smithi176.stdout:3/294: dread - d4/d13/d2d/f51 zero size 2024-03-13T14:49:41.278 INFO:tasks.workunit.client.1.smithi196.stdout:4/396: rename d1/ce to d1/d1e/d43/c78 0 2024-03-13T14:49:41.279 INFO:tasks.workunit.client.0.smithi176.stdout:1/282: creat d2/de/d3f/d19/f56 x:0 0 0 2024-03-13T14:49:41.280 INFO:tasks.workunit.client.1.smithi196.stdout:4/397: fsync d1/d3/d7/f1d 0 2024-03-13T14:49:41.284 INFO:tasks.workunit.client.1.smithi196.stdout:8/374: read d0/d1c/d3b/f60 [328609,3677] 0 2024-03-13T14:49:41.291 INFO:tasks.workunit.client.0.smithi176.stdout:7/292: rename d1/d1c/d26/f4c to d1/d3f/d4b/f55 0 2024-03-13T14:49:41.294 INFO:tasks.workunit.client.0.smithi176.stdout:4/305: symlink d7/d8/d45/l73 0 2024-03-13T14:49:41.297 INFO:tasks.workunit.client.0.smithi176.stdout:0/265: symlink d2/db/l4a 0 2024-03-13T14:49:41.300 INFO:tasks.workunit.client.0.smithi176.stdout:5/257: mkdir d5/d2c/d4a/d53 0 2024-03-13T14:49:41.302 INFO:tasks.workunit.client.0.smithi176.stdout:9/340: dwrite d0/f59 [0,4194304] 0 2024-03-13T14:49:41.304 INFO:tasks.workunit.client.0.smithi176.stdout:3/295: mknod d4/d28/d34/c59 0 2024-03-13T14:49:41.304 INFO:tasks.workunit.client.1.smithi196.stdout:7/557: symlink d0/d6/d33/d57/d8a/lbc 0 2024-03-13T14:49:41.309 INFO:tasks.workunit.client.1.smithi196.stdout:5/531: dread d4/db/d3c/f59 [0,4194304] 0 2024-03-13T14:49:41.310 INFO:tasks.workunit.client.1.smithi196.stdout:3/439: creat db/d19/d22/d3a/f8f x:0 0 0 2024-03-13T14:49:41.311 INFO:tasks.workunit.client.0.smithi176.stdout:2/302: dwrite d1/d2a/d33/f5b [0,4194304] 0 2024-03-13T14:49:41.315 INFO:tasks.workunit.client.0.smithi176.stdout:6/284: rename d0/d9/c26 to d0/de/d38/c60 0 2024-03-13T14:49:41.315 INFO:tasks.workunit.client.1.smithi196.stdout:4/398: truncate d1/d1e/f33 1832812 0 2024-03-13T14:49:41.317 INFO:tasks.workunit.client.1.smithi196.stdout:1/447: symlink d1/d9/d4e/d55/l98 0 2024-03-13T14:49:41.319 INFO:tasks.workunit.client.1.smithi196.stdout:9/368: truncate d7/d9/f11 1129061 0 2024-03-13T14:49:41.320 INFO:tasks.workunit.client.1.smithi196.stdout:9/369: readlink d7/de/d4b/d1d/l78 0 2024-03-13T14:49:41.322 INFO:tasks.workunit.client.0.smithi176.stdout:4/306: unlink d7/d8/d15/d23/f6f 0 2024-03-13T14:49:41.323 INFO:tasks.workunit.client.1.smithi196.stdout:7/558: rmdir d0/d9/d1e/d58/d72/da9/d47 39 2024-03-13T14:49:41.324 INFO:tasks.workunit.client.0.smithi176.stdout:8/267: getdents d0/db/dc 0 2024-03-13T14:49:41.325 INFO:tasks.workunit.client.1.smithi196.stdout:3/440: creat db/d37/d64/f90 x:0 0 0 2024-03-13T14:49:41.327 INFO:tasks.workunit.client.0.smithi176.stdout:1/283: unlink d2/l45 0 2024-03-13T14:49:41.328 INFO:tasks.workunit.client.1.smithi196.stdout:7/559: write d0/d9/d31/d50/f75 [827122,63850] 0 2024-03-13T14:49:41.329 INFO:tasks.workunit.client.1.smithi196.stdout:6/567: creat d6/d7/dd/faf x:0 0 0 2024-03-13T14:49:41.330 INFO:tasks.workunit.client.1.smithi196.stdout:5/532: symlink d4/db/d14/d19/d26/d33/d61/la8 0 2024-03-13T14:49:41.334 INFO:tasks.workunit.client.1.smithi196.stdout:4/399: symlink d1/d1e/d43/d5b/l79 0 2024-03-13T14:49:41.335 INFO:tasks.workunit.client.1.smithi196.stdout:5/533: fsync d4/db/d14/d19/f50 0 2024-03-13T14:49:41.340 INFO:tasks.workunit.client.0.smithi176.stdout:3/296: dwrite d4/d28/f49 [0,4194304] 0 2024-03-13T14:49:41.340 INFO:tasks.workunit.client.0.smithi176.stdout:4/307: unlink d7/d8/l65 0 2024-03-13T14:49:41.342 INFO:tasks.workunit.client.0.smithi176.stdout:9/341: dwrite d0/d1e/f2e [0,4194304] 0 2024-03-13T14:49:41.342 INFO:tasks.workunit.client.0.smithi176.stdout:4/308: chown d7/fe 7298 1 2024-03-13T14:49:41.344 INFO:tasks.workunit.client.1.smithi196.stdout:2/331: link d0/d6/c45 d0/d7/d40/c6e 0 2024-03-13T14:49:41.346 INFO:tasks.workunit.client.0.smithi176.stdout:5/258: unlink d5/d2e/d43/l51 0 2024-03-13T14:49:41.347 INFO:tasks.workunit.client.0.smithi176.stdout:3/297: write d4/d13/d16/d3d/f54 [91992,122440] 0 2024-03-13T14:49:41.349 INFO:tasks.workunit.client.1.smithi196.stdout:2/332: chown d0/d6/d34/f62 39241767 1 2024-03-13T14:49:41.353 INFO:tasks.workunit.client.0.smithi176.stdout:7/293: link d1/f1f d1/d1c/f56 0 2024-03-13T14:49:41.356 INFO:tasks.workunit.client.0.smithi176.stdout:0/266: truncate d2/d5/dc/f15 697579 0 2024-03-13T14:49:41.357 INFO:tasks.workunit.client.1.smithi196.stdout:8/375: creat d0/f7e x:0 0 0 2024-03-13T14:49:41.357 INFO:tasks.workunit.client.1.smithi196.stdout:2/333: dread - d0/d7/d9/f48 zero size 2024-03-13T14:49:41.358 INFO:tasks.workunit.client.0.smithi176.stdout:0/267: chown d2/db 17305 1 2024-03-13T14:49:41.358 INFO:tasks.workunit.client.0.smithi176.stdout:8/268: read d0/d24/f32 [196625,76963] 0 2024-03-13T14:49:41.359 INFO:tasks.workunit.client.1.smithi196.stdout:6/568: rename d6/d2a/dc/da3/fab to d6/d57/fb0 0 2024-03-13T14:49:41.360 INFO:tasks.workunit.client.0.smithi176.stdout:4/309: mknod d7/d8/d2f/c74 0 2024-03-13T14:49:41.362 INFO:tasks.workunit.client.0.smithi176.stdout:2/303: dwrite d1/dc/f2e [0,4194304] 0 2024-03-13T14:49:41.364 INFO:tasks.workunit.client.1.smithi196.stdout:9/370: mknod d7/d9/c79 0 2024-03-13T14:49:41.366 INFO:tasks.workunit.client.0.smithi176.stdout:6/285: dwrite d0/de/f12 [4194304,4194304] 0 2024-03-13T14:49:41.373 INFO:tasks.workunit.client.0.smithi176.stdout:5/259: write d5/d22/d15/f27 [124386,5756] 0 2024-03-13T14:49:41.374 INFO:tasks.workunit.client.0.smithi176.stdout:7/294: write d1/d28/d3a/f14 [2042490,12572] 0 2024-03-13T14:49:41.374 INFO:tasks.workunit.client.0.smithi176.stdout:2/304: chown d1/dc/d2d/d59 13352 1 2024-03-13T14:49:41.378 INFO:tasks.workunit.client.0.smithi176.stdout:6/286: write d0/d10/d19/d27/d2b/f3a [4236609,50501] 0 2024-03-13T14:49:41.378 INFO:tasks.workunit.client.0.smithi176.stdout:7/295: write d1/d28/f2a [972878,60660] 0 2024-03-13T14:49:41.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:41 smithi176 ceph-mon[29064]: pgmap v9: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 32 MiB/s rd, 155 MiB/s wr, 287 op/s 2024-03-13T14:49:41.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:41 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:41.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:41 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:41.382 INFO:tasks.workunit.client.0.smithi176.stdout:6/287: write d0/d10/d19/d27/d2b/d2e/f31 [1167808,86441] 0 2024-03-13T14:49:41.383 INFO:tasks.workunit.client.1.smithi196.stdout:0/473: getdents d8/da/d25/d26/d35/d3b/d5f 0 2024-03-13T14:49:41.385 INFO:tasks.workunit.client.0.smithi176.stdout:4/310: dwrite d7/d8/f39 [0,4194304] 0 2024-03-13T14:49:41.387 INFO:tasks.workunit.client.0.smithi176.stdout:9/342: getdents d0/d13/d4d/d6e 0 2024-03-13T14:49:41.393 INFO:tasks.workunit.client.0.smithi176.stdout:6/288: dread d0/de/f2c [0,4194304] 0 2024-03-13T14:49:41.395 INFO:tasks.workunit.client.0.smithi176.stdout:3/298: truncate d4/d13/f2a 3790487 0 2024-03-13T14:49:41.399 INFO:tasks.workunit.client.1.smithi196.stdout:7/560: dread d0/d9/f27 [0,4194304] 0 2024-03-13T14:49:41.405 INFO:tasks.workunit.client.0.smithi176.stdout:9/343: dwrite d0/f54 [0,4194304] 0 2024-03-13T14:49:41.407 INFO:tasks.workunit.client.1.smithi196.stdout:1/448: creat d1/d9/f99 x:0 0 0 2024-03-13T14:49:41.409 INFO:tasks.workunit.client.1.smithi196.stdout:5/534: dread d4/f11 [0,4194304] 0 2024-03-13T14:49:41.410 INFO:tasks.workunit.client.1.smithi196.stdout:4/400: mknod d1/c7a 0 2024-03-13T14:49:41.411 INFO:tasks.workunit.client.1.smithi196.stdout:6/569: mkdir d6/d7/df/db1 0 2024-03-13T14:49:41.413 INFO:tasks.workunit.client.0.smithi176.stdout:5/260: dread - d5/d48/d49/f4d zero size 2024-03-13T14:49:41.414 INFO:tasks.workunit.client.0.smithi176.stdout:5/261: readlink d5/d2c/d4a/l4b 0 2024-03-13T14:49:41.414 INFO:tasks.workunit.client.0.smithi176.stdout:9/344: dread d0/d17/f3a [4194304,4194304] 0 2024-03-13T14:49:41.416 INFO:tasks.workunit.client.0.smithi176.stdout:2/305: mkdir d1/dc/d2d/d4b/d69 0 2024-03-13T14:49:41.416 INFO:tasks.workunit.client.0.smithi176.stdout:7/296: unlink d1/d1c/d26/f3b 0 2024-03-13T14:49:41.417 INFO:tasks.workunit.client.0.smithi176.stdout:5/262: write d5/d22/f29 [463602,35556] 0 2024-03-13T14:49:41.417 INFO:tasks.workunit.client.1.smithi196.stdout:0/474: creat d8/da/d25/d26/d35/d3b/d6c/f9d x:0 0 0 2024-03-13T14:49:41.418 INFO:tasks.workunit.client.0.smithi176.stdout:9/345: write d0/d1e/f2e [8826360,11811] 0 2024-03-13T14:49:41.418 INFO:tasks.workunit.client.1.smithi196.stdout:3/441: rmdir db/d37/d64/d7f 0 2024-03-13T14:49:41.419 INFO:tasks.workunit.client.1.smithi196.stdout:0/475: chown d8/da/d25/d26/d35/d3b/d59/d6a/f8d 4072337 1 2024-03-13T14:49:41.421 INFO:tasks.workunit.client.0.smithi176.stdout:7/297: chown d1/d1c/d26/d41/f3e 4 1 2024-03-13T14:49:41.428 INFO:tasks.workunit.client.1.smithi196.stdout:8/376: mkdir d0/d16/d3d/d7f 0 2024-03-13T14:49:41.429 INFO:tasks.workunit.client.1.smithi196.stdout:0/476: truncate d8/da/d74/f8e 1029257 0 2024-03-13T14:49:41.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:41 smithi196 ceph-mon[37486]: pgmap v9: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 32 MiB/s rd, 155 MiB/s wr, 287 op/s 2024-03-13T14:49:41.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:41 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:41.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:41 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:41.430 INFO:tasks.workunit.client.0.smithi176.stdout:4/311: rmdir d7/d8/d45 39 2024-03-13T14:49:41.435 INFO:tasks.workunit.client.1.smithi196.stdout:1/449: fdatasync d1/d28/d34/f52 0 2024-03-13T14:49:41.436 INFO:tasks.workunit.client.0.smithi176.stdout:3/299: mkdir d4/d28/d5a 0 2024-03-13T14:49:41.436 INFO:tasks.workunit.client.1.smithi196.stdout:2/334: dread d0/d7/d9/f18 [0,4194304] 0 2024-03-13T14:49:41.439 INFO:tasks.workunit.client.1.smithi196.stdout:5/535: rmdir d4/db/d14/d19/d26 39 2024-03-13T14:49:41.444 INFO:tasks.workunit.client.0.smithi176.stdout:1/284: getdents d2/de 0 2024-03-13T14:49:41.446 INFO:tasks.workunit.client.1.smithi196.stdout:4/401: rename d1/d1e/d43/d5b/l79 to d1/d3/d7/d35/d3e/l7b 0 2024-03-13T14:49:41.447 INFO:tasks.workunit.client.1.smithi196.stdout:4/402: readlink d1/d1e/d43/l44 0 2024-03-13T14:49:41.452 INFO:tasks.workunit.client.1.smithi196.stdout:8/377: creat d0/d1c/d29/d47/f80 x:0 0 0 2024-03-13T14:49:41.457 INFO:tasks.workunit.client.1.smithi196.stdout:6/570: dread d6/d9/d11/f13 [0,4194304] 0 2024-03-13T14:49:41.462 INFO:tasks.workunit.client.0.smithi176.stdout:2/306: symlink d1/dc/d2d/d4f/l6a 0 2024-03-13T14:49:41.465 INFO:tasks.workunit.client.0.smithi176.stdout:9/346: mkdir d0/d1e/d6f 0 2024-03-13T14:49:41.468 INFO:tasks.workunit.client.0.smithi176.stdout:9/347: write d0/db/f3f [765609,103729] 0 2024-03-13T14:49:41.468 INFO:tasks.workunit.client.0.smithi176.stdout:7/298: symlink d1/d3f/d4b/l57 0 2024-03-13T14:49:41.476 INFO:tasks.workunit.client.1.smithi196.stdout:3/442: mknod db/d19/d22/d3a/c91 0 2024-03-13T14:49:41.477 INFO:tasks.workunit.client.1.smithi196.stdout:7/561: dwrite d0/d6/f26 [0,4194304] 0 2024-03-13T14:49:41.479 INFO:tasks.workunit.client.1.smithi196.stdout:7/562: chown d0/d78 59210 1 2024-03-13T14:49:41.479 INFO:tasks.workunit.client.1.smithi196.stdout:0/477: dwrite d8/da/d25/d26/d35/d3b/d59/d6a/f75 [8388608,4194304] 0 2024-03-13T14:49:41.483 INFO:tasks.workunit.client.0.smithi176.stdout:9/348: dwrite d0/db/f3f [0,4194304] 0 2024-03-13T14:49:41.486 INFO:tasks.workunit.client.1.smithi196.stdout:4/403: symlink d1/d3/d4c/d72/l7c 0 2024-03-13T14:49:41.486 INFO:tasks.workunit.client.1.smithi196.stdout:7/563: truncate d0/d4d/f91 809617 0 2024-03-13T14:49:41.489 INFO:tasks.workunit.client.1.smithi196.stdout:7/564: chown d0/l2f 391765353 1 2024-03-13T14:49:41.489 INFO:tasks.workunit.client.1.smithi196.stdout:5/536: rename d4/db/d14/d19/c48 to d4/db/d14/d19/d1c/d2d/d82/d8b/d9b/ca9 0 2024-03-13T14:49:41.490 INFO:tasks.workunit.client.1.smithi196.stdout:2/335: creat d0/d7/d9/d5a/f6f x:0 0 0 2024-03-13T14:49:41.493 INFO:tasks.workunit.client.1.smithi196.stdout:8/378: mknod d0/d10/d12/d1e/d2d/c81 0 2024-03-13T14:49:41.493 INFO:tasks.workunit.client.1.smithi196.stdout:7/565: chown d0/d6/d33/f80 143335286 1 2024-03-13T14:49:41.496 INFO:tasks.workunit.client.0.smithi176.stdout:3/300: mkdir d4/d28/d3a/d5b 0 2024-03-13T14:49:41.497 INFO:tasks.workunit.client.1.smithi196.stdout:8/379: write d0/d1c/d3b/d3f/f76 [361031,113951] 0 2024-03-13T14:49:41.497 INFO:tasks.workunit.client.0.smithi176.stdout:1/285: mknod d2/de/d10/d14/d22/c57 0 2024-03-13T14:49:41.500 INFO:tasks.workunit.client.1.smithi196.stdout:8/380: chown d0/lf 60565 1 2024-03-13T14:49:41.500 INFO:tasks.workunit.client.1.smithi196.stdout:6/571: fdatasync d6/d9/d17/d19/d4e/f7c 0 2024-03-13T14:49:41.502 INFO:tasks.workunit.client.0.smithi176.stdout:0/268: link d2/d5/dc/f19 d2/d5/d28/f4b 0 2024-03-13T14:49:41.502 INFO:tasks.workunit.client.1.smithi196.stdout:1/450: rmdir d1/d9/d4e/d55/d63/d92 0 2024-03-13T14:49:41.503 INFO:tasks.workunit.client.0.smithi176.stdout:0/269: stat d2/d5/d28 0 2024-03-13T14:49:41.504 INFO:tasks.workunit.client.0.smithi176.stdout:2/307: write d1/dc/f2c [833180,17099] 0 2024-03-13T14:49:41.505 INFO:tasks.workunit.client.1.smithi196.stdout:4/404: mkdir d1/d3/d4c/d72/d7d 0 2024-03-13T14:49:41.505 INFO:tasks.workunit.client.0.smithi176.stdout:1/286: read d2/d29/f3c [875235,35217] 0 2024-03-13T14:49:41.508 INFO:tasks.workunit.client.0.smithi176.stdout:1/287: write d2/f9 [3195033,127836] 0 2024-03-13T14:49:41.508 INFO:tasks.workunit.client.0.smithi176.stdout:2/308: truncate d1/dc/d2d/d4b/f63 916110 0 2024-03-13T14:49:41.509 INFO:tasks.workunit.client.1.smithi196.stdout:7/566: chown d0/c3a 14117 1 2024-03-13T14:49:41.509 INFO:tasks.workunit.client.0.smithi176.stdout:1/288: fdatasync d2/de/f41 0 2024-03-13T14:49:41.516 INFO:tasks.workunit.client.1.smithi196.stdout:2/336: dwrite d0/d7/d40/f46 [0,4194304] 0 2024-03-13T14:49:41.520 INFO:tasks.workunit.client.0.smithi176.stdout:7/299: rename d1/d1c/d26/d41/f25 to d1/d3f/d4b/f58 0 2024-03-13T14:49:41.523 INFO:tasks.workunit.client.0.smithi176.stdout:7/300: dread d1/d1c/d26/d41/f3e [0,4194304] 0 2024-03-13T14:49:41.526 INFO:tasks.workunit.client.0.smithi176.stdout:4/312: mkdir d7/d75 0 2024-03-13T14:49:41.526 INFO:tasks.workunit.client.1.smithi196.stdout:8/381: mknod d0/d16/d71/c82 0 2024-03-13T14:49:41.530 INFO:tasks.workunit.client.1.smithi196.stdout:1/451: dread - d1/d3f/f43 zero size 2024-03-13T14:49:41.531 INFO:tasks.workunit.client.0.smithi176.stdout:6/289: creat d0/d10/d19/d27/f61 x:0 0 0 2024-03-13T14:49:41.533 INFO:tasks.workunit.client.1.smithi196.stdout:6/572: creat d6/d6f/fb2 x:0 0 0 2024-03-13T14:49:41.535 INFO:tasks.workunit.client.1.smithi196.stdout:3/443: dread db/d19/d50/f58 [0,4194304] 0 2024-03-13T14:49:41.537 INFO:tasks.workunit.client.0.smithi176.stdout:0/270: creat d2/db/d31/f4c x:0 0 0 2024-03-13T14:49:41.541 INFO:tasks.workunit.client.1.smithi196.stdout:8/382: dread d0/d1c/d29/f65 [0,4194304] 0 2024-03-13T14:49:41.542 INFO:tasks.workunit.client.0.smithi176.stdout:8/269: sync 2024-03-13T14:49:41.542 INFO:tasks.workunit.client.0.smithi176.stdout:2/309: fdatasync d1/dc/f16 0 2024-03-13T14:49:41.546 INFO:tasks.workunit.client.1.smithi196.stdout:5/537: rename d4/db/d3c/f52 to d4/db/d14/d19/d26/faa 0 2024-03-13T14:49:41.548 INFO:tasks.workunit.client.1.smithi196.stdout:5/538: write d4/db/d14/d19/d26/f51 [887512,40458] 0 2024-03-13T14:49:41.549 INFO:tasks.workunit.client.1.smithi196.stdout:5/539: fdatasync d4/db/d4a/f8a 0 2024-03-13T14:49:41.550 INFO:tasks.workunit.client.1.smithi196.stdout:0/478: truncate d8/da/d25/d26/d35/d3b/d5f/d17/f70 4071221 0 2024-03-13T14:49:41.562 INFO:tasks.workunit.client.1.smithi196.stdout:3/444: rmdir db/d19/d22/d3a/d6a 39 2024-03-13T14:49:41.567 INFO:tasks.workunit.client.1.smithi196.stdout:8/383: creat d0/d10/d12/d1e/f83 x:0 0 0 2024-03-13T14:49:41.569 INFO:tasks.workunit.client.1.smithi196.stdout:2/337: truncate d0/d7/f21 621414 0 2024-03-13T14:49:41.571 INFO:tasks.workunit.client.0.smithi176.stdout:0/271: mknod d2/d5/d1e/c4d 0 2024-03-13T14:49:41.573 INFO:tasks.workunit.client.0.smithi176.stdout:5/263: getdents d5/d48/d49 0 2024-03-13T14:49:41.580 INFO:tasks.workunit.client.0.smithi176.stdout:2/310: truncate d1/dc/d2d/f2f 823191 0 2024-03-13T14:49:41.581 INFO:tasks.workunit.client.1.smithi196.stdout:0/479: symlink d8/da/d25/d26/d35/l9e 0 2024-03-13T14:49:41.583 INFO:tasks.workunit.client.0.smithi176.stdout:7/301: mknod d1/c59 0 2024-03-13T14:49:41.584 INFO:tasks.workunit.client.1.smithi196.stdout:4/405: truncate d1/d3/f13 1273553 0 2024-03-13T14:49:41.585 INFO:tasks.workunit.client.1.smithi196.stdout:0/480: chown d8/da/d25/d26/d35/d3b/d50/d5d 0 1 2024-03-13T14:49:41.585 INFO:tasks.workunit.client.1.smithi196.stdout:6/573: symlink d6/d2a/lb3 0 2024-03-13T14:49:41.586 INFO:tasks.workunit.client.0.smithi176.stdout:4/313: creat d7/d8/d45/f76 x:0 0 0 2024-03-13T14:49:41.587 INFO:tasks.workunit.client.1.smithi196.stdout:4/406: chown d1/d3/l1f 37 1 2024-03-13T14:49:41.588 INFO:tasks.workunit.client.1.smithi196.stdout:3/445: creat db/d19/d4a/f92 x:0 0 0 2024-03-13T14:49:41.589 INFO:tasks.workunit.client.1.smithi196.stdout:0/481: stat d8/da/d25/d26/d35/d3b/d50/d5d/d7e 0 2024-03-13T14:49:41.591 INFO:tasks.workunit.client.1.smithi196.stdout:3/446: write db/d37/d64/f90 [683925,60484] 0 2024-03-13T14:49:41.592 INFO:tasks.workunit.client.0.smithi176.stdout:3/301: link d4/d28/d3a/f32 d4/d28/f5c 0 2024-03-13T14:49:41.593 INFO:tasks.workunit.client.1.smithi196.stdout:5/540: write d4/db/d14/d19/f67 [317318,100994] 0 2024-03-13T14:49:41.595 INFO:tasks.workunit.client.1.smithi196.stdout:0/482: dread - d8/da/d25/d26/d48/d82/f8b zero size 2024-03-13T14:49:41.596 INFO:tasks.workunit.client.0.smithi176.stdout:3/302: write d4/d13/f19 [5162070,80645] 0 2024-03-13T14:49:41.597 INFO:tasks.workunit.client.1.smithi196.stdout:5/541: chown d4/db/d14/d19/d26/d33/f8c 3593435 1 2024-03-13T14:49:41.598 INFO:tasks.workunit.client.1.smithi196.stdout:2/338: symlink d0/d50/l70 0 2024-03-13T14:49:41.600 INFO:tasks.workunit.client.1.smithi196.stdout:5/542: write d4/db/d4a/f8a [5059363,81712] 0 2024-03-13T14:49:41.601 INFO:tasks.workunit.client.1.smithi196.stdout:2/339: dread - d0/d6/d4b/f64 zero size 2024-03-13T14:49:41.609 INFO:tasks.workunit.client.1.smithi196.stdout:1/452: creat d1/d9/f9a x:0 0 0 2024-03-13T14:49:41.610 INFO:tasks.workunit.client.1.smithi196.stdout:0/483: dread d8/da/d25/d26/d35/d3b/d50/d5d/f57 [0,4194304] 0 2024-03-13T14:49:41.611 INFO:tasks.workunit.client.0.smithi176.stdout:1/289: rename d2/de/d3f/d19/c1c to d2/c58 0 2024-03-13T14:49:41.614 INFO:tasks.workunit.client.1.smithi196.stdout:0/484: chown d8/da/d25/d26/d35/d3b/d50/d5d/d7e/l98 2463 1 2024-03-13T14:49:41.615 INFO:tasks.workunit.client.0.smithi176.stdout:1/290: dread - d2/de/d3f/d19/f56 zero size 2024-03-13T14:49:41.616 INFO:tasks.workunit.client.0.smithi176.stdout:1/291: chown d2/d30 57126 1 2024-03-13T14:49:41.618 INFO:tasks.workunit.client.1.smithi196.stdout:2/340: dread d0/d7/f17 [0,4194304] 0 2024-03-13T14:49:41.620 INFO:tasks.workunit.client.1.smithi196.stdout:2/341: chown d0/d6/d1d/f51 11870 1 2024-03-13T14:49:41.620 INFO:tasks.workunit.client.0.smithi176.stdout:9/349: getdents d0/d13 0 2024-03-13T14:49:41.621 INFO:tasks.workunit.client.0.smithi176.stdout:3/303: dread d4/d28/f49 [0,4194304] 0 2024-03-13T14:49:41.622 INFO:tasks.workunit.client.1.smithi196.stdout:7/567: getdents d0/d9/d1e/d58/d72/da9/da3 0 2024-03-13T14:49:41.622 INFO:tasks.workunit.client.0.smithi176.stdout:9/350: fdatasync d0/d13/f16 0 2024-03-13T14:49:41.623 INFO:tasks.workunit.client.1.smithi196.stdout:6/574: dread d6/d9/d17/d19/f22 [0,4194304] 0 2024-03-13T14:49:41.628 INFO:tasks.workunit.client.1.smithi196.stdout:9/371: sync 2024-03-13T14:49:41.631 INFO:tasks.workunit.client.1.smithi196.stdout:8/384: mknod d0/d16/d71/d68/c84 0 2024-03-13T14:49:41.634 INFO:tasks.workunit.client.1.smithi196.stdout:5/543: mkdir d4/db/d14/d19/d26/dab 0 2024-03-13T14:49:41.635 INFO:tasks.workunit.client.0.smithi176.stdout:5/264: dwrite d5/f6 [0,4194304] 0 2024-03-13T14:49:41.636 INFO:tasks.workunit.client.1.smithi196.stdout:1/453: mknod d1/d38/c9b 0 2024-03-13T14:49:41.639 INFO:tasks.workunit.client.0.smithi176.stdout:6/290: rename d0/d10/d19/d27/f4b to d0/d10/d19/d27/d2b/d2e/f62 0 2024-03-13T14:49:41.644 INFO:tasks.workunit.client.1.smithi196.stdout:2/342: mknod d0/d7/d9/d5a/c71 0 2024-03-13T14:49:41.645 INFO:tasks.workunit.client.0.smithi176.stdout:1/292: mkdir d2/de/d10/d16/d59 0 2024-03-13T14:49:41.645 INFO:tasks.workunit.client.0.smithi176.stdout:8/270: dwrite d0/d5/d3b/f41 [0,4194304] 0 2024-03-13T14:49:41.648 INFO:tasks.workunit.client.1.smithi196.stdout:4/407: creat d1/d3/d7/d35/d76/d58/d69/d6c/f7e x:0 0 0 2024-03-13T14:49:41.651 INFO:tasks.workunit.client.1.smithi196.stdout:7/568: read d0/fa [474298,9478] 0 2024-03-13T14:49:41.651 INFO:tasks.workunit.client.0.smithi176.stdout:1/293: readlink d2/de/d10/d16/l46 0 2024-03-13T14:49:41.658 INFO:tasks.workunit.client.0.smithi176.stdout:2/311: dwrite d1/f35 [0,4194304] 0 2024-03-13T14:49:41.658 INFO:tasks.workunit.client.0.smithi176.stdout:4/314: dwrite d7/d10/d2c/f32 [0,4194304] 0 2024-03-13T14:49:41.659 INFO:tasks.workunit.client.0.smithi176.stdout:5/265: creat d5/d2e/f54 x:0 0 0 2024-03-13T14:49:41.660 INFO:tasks.workunit.client.1.smithi196.stdout:8/385: dwrite d0/d10/d12/f53 [0,4194304] 0 2024-03-13T14:49:41.664 INFO:tasks.workunit.client.1.smithi196.stdout:5/544: truncate d4/db/d14/d19/d1c/d25/d38/f3e 1721737 0 2024-03-13T14:49:41.665 INFO:tasks.workunit.client.0.smithi176.stdout:8/271: unlink d0/f1c 0 2024-03-13T14:49:41.665 INFO:tasks.workunit.client.0.smithi176.stdout:4/315: chown d7/d8/f42 299354 1 2024-03-13T14:49:41.666 INFO:tasks.workunit.client.0.smithi176.stdout:4/316: fsync d7/fe 0 2024-03-13T14:49:41.666 INFO:tasks.workunit.client.0.smithi176.stdout:3/304: link d4/d13/d16/d3d/f54 d4/d13/d16/d3d/f5d 0 2024-03-13T14:49:41.667 INFO:tasks.workunit.client.0.smithi176.stdout:0/272: dwrite d2/d5/dc/f15 [0,4194304] 0 2024-03-13T14:49:41.681 INFO:tasks.workunit.client.0.smithi176.stdout:8/272: dwrite d0/d5/f17 [0,4194304] 0 2024-03-13T14:49:41.682 INFO:tasks.workunit.client.0.smithi176.stdout:1/294: creat d2/de/d10/d16/d47/f5a x:0 0 0 2024-03-13T14:49:41.683 INFO:tasks.workunit.client.1.smithi196.stdout:2/343: dread d0/d7/d40/f46 [0,4194304] 0 2024-03-13T14:49:41.685 INFO:tasks.workunit.client.0.smithi176.stdout:1/295: chown d2/de/d10/d16/l51 965313 1 2024-03-13T14:49:41.687 INFO:tasks.workunit.client.1.smithi196.stdout:3/447: rename db/d4c/f86 to db/d19/f93 0 2024-03-13T14:49:41.696 INFO:tasks.workunit.client.1.smithi196.stdout:7/569: dwrite d0/d9/d1e/d3f/f7e [0,4194304] 0 2024-03-13T14:49:41.698 INFO:tasks.workunit.client.0.smithi176.stdout:5/266: rmdir d5/d22 39 2024-03-13T14:49:41.700 INFO:tasks.workunit.client.1.smithi196.stdout:2/344: truncate d0/d50/f65 648089 0 2024-03-13T14:49:41.703 INFO:tasks.workunit.client.1.smithi196.stdout:5/545: dwrite d4/db/d14/d19/f58 [0,4194304] 0 2024-03-13T14:49:41.703 INFO:tasks.workunit.client.0.smithi176.stdout:1/296: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:41.716 INFO:tasks.workunit.client.1.smithi196.stdout:6/575: link d6/d7/df/c67 d6/d2a/dc/d1c/cb4 0 2024-03-13T14:49:41.720 INFO:tasks.workunit.client.1.smithi196.stdout:6/576: write d6/d9/d17/d19/fa8 [3659987,71463] 0 2024-03-13T14:49:41.722 INFO:tasks.workunit.client.1.smithi196.stdout:4/408: creat d1/d1e/f7f x:0 0 0 2024-03-13T14:49:41.724 INFO:tasks.workunit.client.0.smithi176.stdout:1/297: dread d2/de/d3f/f52 [0,4194304] 0 2024-03-13T14:49:41.819 INFO:tasks.workunit.client.0.smithi176.stdout:9/351: link d0/d1e/d1f/c61 d0/d1e/d6f/c70 0 2024-03-13T14:49:41.824 INFO:tasks.workunit.client.0.smithi176.stdout:4/317: creat d7/d10/d2c/f77 x:0 0 0 2024-03-13T14:49:41.826 INFO:tasks.workunit.client.0.smithi176.stdout:0/273: creat d2/d5/d1e/f4e x:0 0 0 2024-03-13T14:49:41.828 INFO:tasks.workunit.client.0.smithi176.stdout:0/274: truncate d2/db/d39/f47 319109 0 2024-03-13T14:49:41.841 INFO:tasks.workunit.client.0.smithi176.stdout:7/302: rename d1/d3f/d4b/f55 to d1/d1c/d26/d41/f5a 0 2024-03-13T14:49:41.843 INFO:tasks.workunit.client.0.smithi176.stdout:5/267: fsync d5/d22/d15/f27 0 2024-03-13T14:49:41.845 INFO:tasks.workunit.client.0.smithi176.stdout:2/312: mknod d1/dc/d2d/d4b/d69/c6b 0 2024-03-13T14:49:41.845 INFO:tasks.workunit.client.0.smithi176.stdout:5/268: readlink d5/d2c/d4a/l4b 0 2024-03-13T14:49:41.847 INFO:tasks.workunit.client.0.smithi176.stdout:5/269: chown d5/d22/l1b 38484401 1 2024-03-13T14:49:41.852 INFO:tasks.workunit.client.0.smithi176.stdout:1/298: fdatasync d2/d29/f3c 0 2024-03-13T14:49:41.856 INFO:tasks.workunit.client.0.smithi176.stdout:4/318: write d7/d8/d45/f70 [3586624,91882] 0 2024-03-13T14:49:41.858 INFO:tasks.workunit.client.0.smithi176.stdout:8/273: unlink d0/db/d39/f43 0 2024-03-13T14:49:41.858 INFO:tasks.workunit.client.0.smithi176.stdout:4/319: dread - d7/d8/f4f zero size 2024-03-13T14:49:41.859 INFO:tasks.workunit.client.0.smithi176.stdout:4/320: stat d7/d8/d15/l1a 0 2024-03-13T14:49:41.861 INFO:tasks.workunit.client.0.smithi176.stdout:4/321: dread - d7/d10/f44 zero size 2024-03-13T14:49:41.864 INFO:tasks.workunit.client.0.smithi176.stdout:4/322: read - d7/d10/d46/f4c zero size 2024-03-13T14:49:41.865 INFO:tasks.workunit.client.0.smithi176.stdout:6/291: getdents d0 0 2024-03-13T14:49:41.865 INFO:tasks.workunit.client.0.smithi176.stdout:5/270: dwrite d5/f6 [0,4194304] 0 2024-03-13T14:49:41.868 INFO:tasks.workunit.client.0.smithi176.stdout:5/271: readlink d5/d22/l1f 0 2024-03-13T14:49:41.868 INFO:tasks.workunit.client.0.smithi176.stdout:4/323: stat d7/d8/d15/d23/c40 0 2024-03-13T14:49:41.870 INFO:tasks.workunit.client.0.smithi176.stdout:9/352: rename d0/f5a to d0/d13/d4d/d6e/f71 0 2024-03-13T14:49:41.871 INFO:tasks.workunit.client.0.smithi176.stdout:7/303: unlink d1/d1c/d26/d41/l33 0 2024-03-13T14:49:41.872 INFO:tasks.workunit.client.0.smithi176.stdout:4/324: chown d7/d8/f4f 143668 1 2024-03-13T14:49:41.875 INFO:tasks.workunit.client.0.smithi176.stdout:2/313: mknod d1/dc/d2d/d4f/c6c 0 2024-03-13T14:49:41.876 INFO:tasks.workunit.client.0.smithi176.stdout:1/299: dwrite d2/de/d3f/d19/f56 [0,4194304] 0 2024-03-13T14:49:41.877 INFO:tasks.workunit.client.0.smithi176.stdout:7/304: chown d1/c2c 391 1 2024-03-13T14:49:41.878 INFO:tasks.workunit.client.0.smithi176.stdout:9/353: truncate d0/d1e/d1f/f41 988414 0 2024-03-13T14:49:41.879 INFO:tasks.workunit.client.0.smithi176.stdout:1/300: chown d2/de/d3f/d19/l28 0 1 2024-03-13T14:49:41.881 INFO:tasks.workunit.client.0.smithi176.stdout:9/354: write d0/d1e/f4e [275622,82203] 0 2024-03-13T14:49:41.893 INFO:tasks.workunit.client.0.smithi176.stdout:6/292: dread d0/d10/d19/d27/d2b/d2e/f31 [0,4194304] 0 2024-03-13T14:49:41.898 INFO:tasks.workunit.client.0.smithi176.stdout:2/314: dwrite d1/dc/d2d/d38/f48 [0,4194304] 0 2024-03-13T14:49:41.905 INFO:tasks.workunit.client.0.smithi176.stdout:4/325: creat d7/d10/d46/d4a/d5d/f78 x:0 0 0 2024-03-13T14:49:41.910 INFO:tasks.workunit.client.0.smithi176.stdout:9/355: rename d0/db/c45 to d0/d13/d32/c72 0 2024-03-13T14:49:41.916 INFO:tasks.workunit.client.1.smithi196.stdout:8/386: creat d0/d63/f85 x:0 0 0 2024-03-13T14:49:41.922 INFO:tasks.workunit.client.1.smithi196.stdout:0/485: rename d8/da/f5c to d8/da/d25/d26/d35/d3b/d59/f9f 0 2024-03-13T14:49:41.922 INFO:tasks.workunit.client.1.smithi196.stdout:8/387: read d0/d10/d12/f33 [1599555,90423] 0 2024-03-13T14:49:41.923 INFO:tasks.workunit.client.0.smithi176.stdout:2/315: dwrite d1/dc/f16 [0,4194304] 0 2024-03-13T14:49:41.924 INFO:tasks.workunit.client.1.smithi196.stdout:9/372: link d7/f45 d7/de/d16/d70/f7a 0 2024-03-13T14:49:41.925 INFO:tasks.workunit.client.0.smithi176.stdout:2/316: chown d1/dc/d2d/d38/d5d/f62 6176 1 2024-03-13T14:49:41.927 INFO:tasks.workunit.client.1.smithi196.stdout:2/345: symlink d0/d6/d1d/l72 0 2024-03-13T14:49:41.928 INFO:tasks.workunit.client.0.smithi176.stdout:4/326: creat d7/d10/d46/d4a/f79 x:0 0 0 2024-03-13T14:49:41.929 INFO:tasks.workunit.client.0.smithi176.stdout:4/327: stat d7/d8/d15 0 2024-03-13T14:49:41.930 INFO:tasks.workunit.client.1.smithi196.stdout:5/546: dread - d4/db/d4a/f5f zero size 2024-03-13T14:49:41.931 INFO:tasks.workunit.client.0.smithi176.stdout:4/328: chown d7/d10/d46/d55/f68 506496 1 2024-03-13T14:49:41.932 INFO:tasks.workunit.client.0.smithi176.stdout:4/329: write d7/f50 [4643356,63216] 0 2024-03-13T14:49:41.933 INFO:tasks.workunit.client.0.smithi176.stdout:3/305: mkdir d4/d13/d5e 0 2024-03-13T14:49:41.938 INFO:tasks.workunit.client.0.smithi176.stdout:2/317: creat d1/dc/d61/f6d x:0 0 0 2024-03-13T14:49:41.938 INFO:tasks.workunit.client.1.smithi196.stdout:4/409: rmdir d1/d3/d7/d35/d76/d58/d69 39 2024-03-13T14:49:41.943 INFO:tasks.workunit.client.0.smithi176.stdout:9/356: mkdir d0/d73 0 2024-03-13T14:49:41.945 INFO:tasks.workunit.client.0.smithi176.stdout:9/357: write d0/d1e/d1f/f38 [532719,27241] 0 2024-03-13T14:49:41.948 INFO:tasks.workunit.client.0.smithi176.stdout:9/358: truncate d0/d1e/d1f/f41 2023064 0 2024-03-13T14:49:41.950 INFO:tasks.workunit.client.0.smithi176.stdout:9/359: chown d0/d1e/c63 0 1 2024-03-13T14:49:41.951 INFO:tasks.workunit.client.0.smithi176.stdout:1/301: getdents d2/d29 0 2024-03-13T14:49:41.952 INFO:tasks.workunit.client.0.smithi176.stdout:9/360: read d0/d1e/f25 [2816294,60774] 0 2024-03-13T14:49:41.954 INFO:tasks.workunit.client.0.smithi176.stdout:3/306: symlink d4/d28/d3a/l5f 0 2024-03-13T14:49:41.961 INFO:tasks.workunit.client.0.smithi176.stdout:2/318: rename d1/dc/d2d/d38/f48 to d1/d2a/d33/f6e 0 2024-03-13T14:49:41.962 INFO:tasks.workunit.client.0.smithi176.stdout:2/319: chown d1/dc/d1c/c29 721 1 2024-03-13T14:49:41.966 INFO:tasks.workunit.client.1.smithi196.stdout:9/373: truncate d7/f45 875798 0 2024-03-13T14:49:41.969 INFO:tasks.workunit.client.1.smithi196.stdout:2/346: mknod d0/d6/d4b/d5f/c73 0 2024-03-13T14:49:41.970 INFO:tasks.workunit.client.0.smithi176.stdout:9/361: creat d0/d13/d32/d58/f74 x:0 0 0 2024-03-13T14:49:41.971 INFO:tasks.workunit.client.1.smithi196.stdout:2/347: chown d0/d7/d40/f60 50146 1 2024-03-13T14:49:41.971 INFO:tasks.workunit.client.1.smithi196.stdout:5/547: rmdir d4/db/d14/d19/d26/d33/d61 39 2024-03-13T14:49:41.971 INFO:tasks.workunit.client.0.smithi176.stdout:9/362: fsync d0/f54 0 2024-03-13T14:49:41.972 INFO:tasks.workunit.client.1.smithi196.stdout:5/548: chown d4/d60/c74 328 1 2024-03-13T14:49:41.972 INFO:tasks.workunit.client.0.smithi176.stdout:3/307: dwrite d4/d13/f19 [4194304,4194304] 0 2024-03-13T14:49:41.979 INFO:tasks.workunit.client.0.smithi176.stdout:2/320: mkdir d1/d51/d5c/d6f 0 2024-03-13T14:49:41.983 INFO:tasks.workunit.client.0.smithi176.stdout:1/302: creat d2/de/d3f/d3e/f5b x:0 0 0 2024-03-13T14:49:41.985 INFO:tasks.workunit.client.1.smithi196.stdout:2/348: dread d0/f25 [0,4194304] 0 2024-03-13T14:49:41.991 INFO:tasks.workunit.client.1.smithi196.stdout:5/549: dread d4/db/d14/d19/d1c/f23 [0,4194304] 0 2024-03-13T14:49:41.994 INFO:tasks.workunit.client.0.smithi176.stdout:3/308: creat d4/d28/d3a/f60 x:0 0 0 2024-03-13T14:49:41.995 INFO:tasks.workunit.client.1.smithi196.stdout:5/550: stat d4/db/d14/d19/d1c/l54 0 2024-03-13T14:49:41.996 INFO:tasks.workunit.client.1.smithi196.stdout:8/388: creat d0/d16/f86 x:0 0 0 2024-03-13T14:49:41.997 INFO:tasks.workunit.client.0.smithi176.stdout:1/303: dwrite d2/f21 [0,4194304] 0 2024-03-13T14:49:41.999 INFO:tasks.workunit.client.0.smithi176.stdout:3/309: dread d4/d13/d16/d3d/f41 [0,4194304] 0 2024-03-13T14:49:42.000 INFO:tasks.workunit.client.0.smithi176.stdout:9/363: link d0/d1e/f2e d0/d13/d32/d58/d5f/f75 0 2024-03-13T14:49:42.001 INFO:tasks.workunit.client.1.smithi196.stdout:2/349: dread d0/d6/d1d/f51 [0,4194304] 0 2024-03-13T14:49:42.002 INFO:tasks.workunit.client.1.smithi196.stdout:2/350: readlink d0/d50/l55 0 2024-03-13T14:49:42.005 INFO:tasks.workunit.client.0.smithi176.stdout:2/321: creat d1/dc/d2d/d38/d5d/f70 x:0 0 0 2024-03-13T14:49:42.007 INFO:tasks.workunit.client.0.smithi176.stdout:2/322: chown d1/d2a/d33/f5b 31542591 1 2024-03-13T14:49:42.008 INFO:tasks.workunit.client.0.smithi176.stdout:2/323: readlink d1/d2a/d33/l3d 0 2024-03-13T14:49:42.012 INFO:tasks.workunit.client.0.smithi176.stdout:9/364: mkdir d0/d13/d32/d76 0 2024-03-13T14:49:42.014 INFO:tasks.workunit.client.0.smithi176.stdout:9/365: write d0/d1e/d1f/f38 [3656,66177] 0 2024-03-13T14:49:42.019 INFO:tasks.workunit.client.1.smithi196.stdout:8/389: mkdir d0/d10/d12/d6f/d87 0 2024-03-13T14:49:42.020 INFO:tasks.workunit.client.1.smithi196.stdout:2/351: mknod d0/d7/d9/d61/c74 0 2024-03-13T14:49:42.020 INFO:tasks.workunit.client.0.smithi176.stdout:9/366: dread d0/d40/d53/f65 [0,4194304] 0 2024-03-13T14:49:42.022 INFO:tasks.workunit.client.0.smithi176.stdout:9/367: read d0/db/f3f [1009056,53625] 0 2024-03-13T14:49:42.023 INFO:tasks.workunit.client.0.smithi176.stdout:9/368: chown d0/d13/l3d 20468388 1 2024-03-13T14:49:42.026 INFO:tasks.workunit.client.1.smithi196.stdout:8/390: mkdir d0/d16/d3d/d7f/d88 0 2024-03-13T14:49:42.027 INFO:tasks.workunit.client.0.smithi176.stdout:2/324: rmdir d1/d51/d5c/d6f 0 2024-03-13T14:49:42.028 INFO:tasks.workunit.client.1.smithi196.stdout:2/352: mknod d0/d7/d9/c75 0 2024-03-13T14:49:42.030 INFO:tasks.workunit.client.1.smithi196.stdout:8/391: creat d0/d16/d71/f89 x:0 0 0 2024-03-13T14:49:42.031 INFO:tasks.workunit.client.0.smithi176.stdout:9/369: mknod d0/d13/d4d/d5d/c77 0 2024-03-13T14:49:42.035 INFO:tasks.workunit.client.0.smithi176.stdout:3/310: dread d4/f10 [0,4194304] 0 2024-03-13T14:49:42.039 INFO:tasks.workunit.client.0.smithi176.stdout:9/370: rename d0/d13/d32/d58/f74 to d0/d40/f78 0 2024-03-13T14:49:42.041 INFO:tasks.workunit.client.0.smithi176.stdout:2/325: rename d1/fa to d1/f71 0 2024-03-13T14:49:42.041 INFO:tasks.workunit.client.0.smithi176.stdout:0/275: dwrite d2/d5/dc/f18 [4194304,4194304] 0 2024-03-13T14:49:42.043 INFO:tasks.workunit.client.0.smithi176.stdout:0/276: chown d2/db/f25 158281043 1 2024-03-13T14:49:42.045 INFO:tasks.workunit.client.0.smithi176.stdout:3/311: rename d4/d13/d16/d3d/f41 to d4/d13/f61 0 2024-03-13T14:49:42.048 INFO:tasks.workunit.client.0.smithi176.stdout:0/277: write d2/d5/f46 [25256,106733] 0 2024-03-13T14:49:42.049 INFO:tasks.workunit.client.0.smithi176.stdout:9/371: symlink d0/d13/d32/d58/d5f/l79 0 2024-03-13T14:49:42.050 INFO:tasks.workunit.client.0.smithi176.stdout:5/272: write d5/d2e/d43/f50 [625894,92947] 0 2024-03-13T14:49:42.051 INFO:tasks.workunit.client.0.smithi176.stdout:5/273: stat d5/d48 0 2024-03-13T14:49:42.052 INFO:tasks.workunit.client.0.smithi176.stdout:3/312: creat d4/d28/d31/f62 x:0 0 0 2024-03-13T14:49:42.058 INFO:tasks.workunit.client.0.smithi176.stdout:5/274: write d5/f37 [81604,268] 0 2024-03-13T14:49:42.058 INFO:tasks.workunit.client.0.smithi176.stdout:2/326: link d1/dc/d2d/d38/l53 d1/d2a/l72 0 2024-03-13T14:49:42.059 INFO:tasks.workunit.client.1.smithi196.stdout:8/392: dread d0/d10/d12/f3e [0,4194304] 0 2024-03-13T14:49:42.060 INFO:tasks.workunit.client.0.smithi176.stdout:3/313: symlink d4/d28/l63 0 2024-03-13T14:49:42.064 INFO:tasks.workunit.client.1.smithi196.stdout:8/393: truncate d0/d10/d12/d17/d42/f4c 5241202 0 2024-03-13T14:49:42.070 INFO:tasks.workunit.client.1.smithi196.stdout:8/394: symlink d0/d63/l8a 0 2024-03-13T14:49:42.073 INFO:tasks.workunit.client.1.smithi196.stdout:8/395: truncate d0/d1c/d29/d47/f80 640195 0 2024-03-13T14:49:42.074 INFO:tasks.workunit.client.0.smithi176.stdout:2/327: link d1/f3c d1/d2a/f73 0 2024-03-13T14:49:42.075 INFO:tasks.workunit.client.0.smithi176.stdout:5/275: creat d5/f55 x:0 0 0 2024-03-13T14:49:42.076 INFO:tasks.workunit.client.0.smithi176.stdout:6/293: creat d0/d10/d19/f63 x:0 0 0 2024-03-13T14:49:42.077 INFO:tasks.workunit.client.0.smithi176.stdout:5/276: chown d5/d22/d15/c20 1171 1 2024-03-13T14:49:42.078 INFO:tasks.workunit.client.1.smithi196.stdout:7/570: write d0/d9/f30 [3209351,81597] 0 2024-03-13T14:49:42.079 INFO:tasks.workunit.client.0.smithi176.stdout:2/328: symlink d1/dc/d2d/d38/l74 0 2024-03-13T14:49:42.081 INFO:tasks.workunit.client.0.smithi176.stdout:6/294: symlink d0/d10/d19/d27/d2b/d2e/d3e/l64 0 2024-03-13T14:49:42.083 INFO:tasks.workunit.client.1.smithi196.stdout:1/454: rename d1/d28/f2b to d1/d28/d34/f9c 0 2024-03-13T14:49:42.083 INFO:tasks.workunit.client.0.smithi176.stdout:6/295: read - d0/d10/d19/d27/d53/f56 zero size 2024-03-13T14:49:42.084 INFO:tasks.workunit.client.0.smithi176.stdout:2/329: creat d1/d2a/f75 x:0 0 0 2024-03-13T14:49:42.086 INFO:tasks.workunit.client.1.smithi196.stdout:7/571: creat d0/d9/d31/d77/d93/fbd x:0 0 0 2024-03-13T14:49:42.089 INFO:tasks.workunit.client.1.smithi196.stdout:3/448: rename db/d19/d22/f23 to db/d19/d4a/f94 0 2024-03-13T14:49:42.090 INFO:tasks.workunit.client.0.smithi176.stdout:6/296: creat d0/d10/d19/d27/d2b/f65 x:0 0 0 2024-03-13T14:49:42.092 INFO:tasks.workunit.client.0.smithi176.stdout:2/330: unlink d1/dc/d40/l56 0 2024-03-13T14:49:42.093 INFO:tasks.workunit.client.1.smithi196.stdout:1/455: creat d1/d9/d33/f9d x:0 0 0 2024-03-13T14:49:42.095 INFO:tasks.workunit.client.1.smithi196.stdout:7/572: fdatasync d0/fa 0 2024-03-13T14:49:42.098 INFO:tasks.workunit.client.0.smithi176.stdout:2/331: mkdir d1/d51/d76 0 2024-03-13T14:49:42.101 INFO:tasks.workunit.client.1.smithi196.stdout:3/449: dread db/d19/d22/f31 [0,4194304] 0 2024-03-13T14:49:42.103 INFO:tasks.workunit.client.1.smithi196.stdout:3/450: fdatasync db/f70 0 2024-03-13T14:49:42.114 INFO:tasks.workunit.client.1.smithi196.stdout:5/551: rename d4/db/d14/d19/d26/d33/f8c to d4/db/d14/d19/fac 0 2024-03-13T14:49:42.117 INFO:tasks.workunit.client.1.smithi196.stdout:5/552: chown d4/db/d3c/d6a/d93 8 1 2024-03-13T14:49:42.126 INFO:tasks.workunit.client.1.smithi196.stdout:7/573: rename d0/l2f to d0/d9/d1e/d3f/lbe 0 2024-03-13T14:49:42.126 INFO:tasks.workunit.client.1.smithi196.stdout:0/486: dwrite d8/da/d25/d26/f2d [0,4194304] 0 2024-03-13T14:49:42.126 INFO:tasks.workunit.client.1.smithi196.stdout:5/553: readlink d4/db/d3c/d6a/l76 0 2024-03-13T14:49:42.128 INFO:tasks.workunit.client.0.smithi176.stdout:6/297: unlink d0/d10/d19/d27/d2b/d2e/d3e/l5f 0 2024-03-13T14:49:42.129 INFO:tasks.workunit.client.0.smithi176.stdout:0/278: rmdir d2/d5/d28 39 2024-03-13T14:49:42.130 INFO:tasks.workunit.client.1.smithi196.stdout:3/451: getdents db/d15/d44/d79 0 2024-03-13T14:49:42.132 INFO:tasks.workunit.client.1.smithi196.stdout:1/456: dread d1/d9/d33/d4b/d66/f7c [0,4194304] 0 2024-03-13T14:49:42.133 INFO:tasks.workunit.client.1.smithi196.stdout:5/554: symlink d4/db/d14/d19/lad 0 2024-03-13T14:49:42.133 INFO:tasks.workunit.client.1.smithi196.stdout:0/487: chown d8/da/d25/d26/d35/d3b/d50/d5d/c24 16574879 1 2024-03-13T14:49:42.134 INFO:tasks.workunit.client.1.smithi196.stdout:3/452: chown db/d19/f2f 0 1 2024-03-13T14:49:42.135 INFO:tasks.workunit.client.0.smithi176.stdout:6/298: fdatasync d0/d10/d19/d27/f4d 0 2024-03-13T14:49:42.135 INFO:tasks.workunit.client.1.smithi196.stdout:5/555: readlink d4/l9 0 2024-03-13T14:49:42.136 INFO:tasks.workunit.client.1.smithi196.stdout:1/457: chown d1/d9/d6c 147 1 2024-03-13T14:49:42.137 INFO:tasks.workunit.client.1.smithi196.stdout:6/577: dwrite d6/d7/f95 [0,4194304] 0 2024-03-13T14:49:42.142 INFO:tasks.workunit.client.1.smithi196.stdout:2/353: truncate d0/d6/d1d/f51 4997083 0 2024-03-13T14:49:42.145 INFO:tasks.workunit.client.1.smithi196.stdout:1/458: chown d1/d3f/d5b/l61 7 1 2024-03-13T14:49:42.145 INFO:tasks.workunit.client.0.smithi176.stdout:2/332: dread d1/dc/d2d/d38/f46 [0,4194304] 0 2024-03-13T14:49:42.145 INFO:tasks.workunit.client.1.smithi196.stdout:6/578: read d6/d2a/dc/d1c/fa4 [571477,125950] 0 2024-03-13T14:49:42.148 INFO:tasks.workunit.client.0.smithi176.stdout:6/299: unlink d0/d10/d19/d27/l33 0 2024-03-13T14:49:42.150 INFO:tasks.workunit.client.1.smithi196.stdout:0/488: fsync d8/da/d25/d26/d35/d3b/d5f/d33/f94 0 2024-03-13T14:49:42.153 INFO:tasks.workunit.client.0.smithi176.stdout:2/333: mkdir d1/d51/d77 0 2024-03-13T14:49:42.155 INFO:tasks.workunit.client.1.smithi196.stdout:5/556: mknod d4/cae 0 2024-03-13T14:49:42.156 INFO:tasks.workunit.client.0.smithi176.stdout:0/279: dwrite d2/d5/dc/d20/f48 [0,4194304] 0 2024-03-13T14:49:42.168 INFO:tasks.workunit.client.0.smithi176.stdout:0/280: symlink d2/db/d35/l4f 0 2024-03-13T14:49:42.169 INFO:tasks.workunit.client.1.smithi196.stdout:4/410: sync 2024-03-13T14:49:42.172 INFO:tasks.workunit.client.0.smithi176.stdout:6/300: dwrite d0/d9/f50 [0,4194304] 0 2024-03-13T14:49:42.173 INFO:tasks.workunit.client.1.smithi196.stdout:6/579: dwrite d6/d7/df/d27/f9d [0,4194304] 0 2024-03-13T14:49:42.190 INFO:tasks.workunit.client.1.smithi196.stdout:0/489: mkdir d8/da/d25/d26/d48/d82/da0 0 2024-03-13T14:49:42.193 INFO:tasks.workunit.client.1.smithi196.stdout:5/557: creat d4/db/d4a/faf x:0 0 0 2024-03-13T14:49:42.196 INFO:tasks.workunit.client.1.smithi196.stdout:0/490: truncate d8/da/d74/f8f 846232 0 2024-03-13T14:49:42.202 INFO:tasks.workunit.client.1.smithi196.stdout:3/453: link db/d37/l72 db/d19/d22/d30/l95 0 2024-03-13T14:49:42.205 INFO:tasks.workunit.client.0.smithi176.stdout:4/330: sync 2024-03-13T14:49:42.205 INFO:tasks.workunit.client.0.smithi176.stdout:8/274: sync 2024-03-13T14:49:42.205 INFO:tasks.workunit.client.0.smithi176.stdout:5/277: sync 2024-03-13T14:49:42.205 INFO:tasks.workunit.client.0.smithi176.stdout:1/304: sync 2024-03-13T14:49:42.205 INFO:tasks.workunit.client.0.smithi176.stdout:7/305: sync 2024-03-13T14:49:42.206 INFO:tasks.workunit.client.0.smithi176.stdout:0/281: unlink d2/d5/dc/f19 0 2024-03-13T14:49:42.210 INFO:tasks.workunit.client.1.smithi196.stdout:1/459: symlink d1/d9/d4e/d55/l9e 0 2024-03-13T14:49:42.210 INFO:tasks.workunit.client.0.smithi176.stdout:5/278: write d5/f55 [319080,128637] 0 2024-03-13T14:49:42.210 INFO:tasks.workunit.client.0.smithi176.stdout:9/372: dwrite d0/d1e/f4a [0,4194304] 0 2024-03-13T14:49:42.211 INFO:tasks.workunit.client.0.smithi176.stdout:1/305: stat d2/de/d10/d16/d47/f5a 0 2024-03-13T14:49:42.214 INFO:tasks.workunit.client.0.smithi176.stdout:1/306: fsync d2/de/d3f/d50/f42 0 2024-03-13T14:49:42.216 INFO:tasks.workunit.client.1.smithi196.stdout:0/491: dread d8/da/d25/d26/d35/d3b/d5f/d17/f4d [0,4194304] 0 2024-03-13T14:49:42.217 INFO:tasks.workunit.client.0.smithi176.stdout:5/279: stat d5/d2c/d4a 0 2024-03-13T14:49:42.218 INFO:tasks.workunit.client.0.smithi176.stdout:1/307: write d2/f9 [1251843,407] 0 2024-03-13T14:49:42.219 INFO:tasks.workunit.client.1.smithi196.stdout:6/580: creat d6/d2a/dc/d1c/da7/fb5 x:0 0 0 2024-03-13T14:49:42.220 INFO:tasks.workunit.client.0.smithi176.stdout:4/331: symlink d7/d10/d2c/l7a 0 2024-03-13T14:49:42.220 INFO:tasks.workunit.client.0.smithi176.stdout:3/314: dwrite d4/d13/d2d/f30 [0,4194304] 0 2024-03-13T14:49:42.222 INFO:tasks.workunit.client.0.smithi176.stdout:1/308: chown d2/de/d10/d14/d22/f2b 55109 1 2024-03-13T14:49:42.223 INFO:tasks.workunit.client.0.smithi176.stdout:1/309: dread - d2/de/f53 zero size 2024-03-13T14:49:42.227 INFO:tasks.workunit.client.0.smithi176.stdout:1/310: write d2/de/d10/d14/d22/f2b [2103913,105603] 0 2024-03-13T14:49:42.229 INFO:tasks.workunit.client.1.smithi196.stdout:8/396: dwrite d0/f51 [0,4194304] 0 2024-03-13T14:49:42.232 INFO:tasks.workunit.client.1.smithi196.stdout:8/397: chown d0/c18 450627 1 2024-03-13T14:49:42.236 INFO:tasks.workunit.client.1.smithi196.stdout:4/411: dwrite d1/d3/f4e [0,4194304] 0 2024-03-13T14:49:42.236 INFO:tasks.workunit.client.0.smithi176.stdout:0/282: dwrite d2/db/d31/f41 [0,4194304] 0 2024-03-13T14:49:42.237 INFO:tasks.workunit.client.1.smithi196.stdout:3/454: write db/d19/d22/f84 [1907432,16921] 0 2024-03-13T14:49:42.241 INFO:tasks.workunit.client.1.smithi196.stdout:1/460: unlink d1/d9/d33/d4b/d66/d7b/f7e 0 2024-03-13T14:49:42.241 INFO:tasks.workunit.client.0.smithi176.stdout:7/306: unlink d1/d7/lf 0 2024-03-13T14:49:42.246 INFO:tasks.workunit.client.0.smithi176.stdout:5/280: dwrite d5/d2e/d43/f50 [0,4194304] 0 2024-03-13T14:49:42.253 INFO:tasks.workunit.client.1.smithi196.stdout:9/374: write d7/f45 [1657477,3346] 0 2024-03-13T14:49:42.261 INFO:tasks.workunit.client.0.smithi176.stdout:3/315: fdatasync d4/d13/f48 0 2024-03-13T14:49:42.262 INFO:tasks.workunit.client.1.smithi196.stdout:7/574: truncate d0/d6/d33/f3c 534900 0 2024-03-13T14:49:42.269 INFO:tasks.workunit.client.1.smithi196.stdout:2/354: link d0/d7/d9/f48 d0/d7/d66/f76 0 2024-03-13T14:49:42.274 INFO:tasks.workunit.client.0.smithi176.stdout:4/332: dread d7/fe [0,4194304] 0 2024-03-13T14:49:42.282 INFO:tasks.workunit.client.1.smithi196.stdout:8/398: rmdir d0 39 2024-03-13T14:49:42.284 INFO:tasks.workunit.client.0.smithi176.stdout:2/334: write d1/dc/f52 [3991282,16490] 0 2024-03-13T14:49:42.287 INFO:tasks.workunit.client.0.smithi176.stdout:1/311: creat d2/de/d10/d16/d47/f5c x:0 0 0 2024-03-13T14:49:42.288 INFO:tasks.workunit.client.0.smithi176.stdout:1/312: read d2/f21 [2307058,75608] 0 2024-03-13T14:49:42.294 INFO:tasks.workunit.client.0.smithi176.stdout:0/283: creat d2/f50 x:0 0 0 2024-03-13T14:49:42.296 INFO:tasks.workunit.client.1.smithi196.stdout:0/492: symlink d8/da/d25/d26/d35/d3b/la1 0 2024-03-13T14:49:42.299 INFO:tasks.workunit.client.0.smithi176.stdout:6/301: dwrite d0/de/f2c [0,4194304] 0 2024-03-13T14:49:42.302 INFO:tasks.workunit.client.0.smithi176.stdout:9/373: write d0/d1e/f25 [3184486,29547] 0 2024-03-13T14:49:42.302 INFO:tasks.workunit.client.0.smithi176.stdout:6/302: chown d0/de/f3c 20569505 1 2024-03-13T14:49:42.310 INFO:tasks.workunit.client.0.smithi176.stdout:9/374: dread d0/d1e/f4e [0,4194304] 0 2024-03-13T14:49:42.310 INFO:tasks.workunit.client.0.smithi176.stdout:8/275: truncate d0/f2 4625492 0 2024-03-13T14:49:42.313 INFO:tasks.workunit.client.0.smithi176.stdout:4/333: creat d7/d8/d15/f7b x:0 0 0 2024-03-13T14:49:42.317 INFO:tasks.workunit.client.0.smithi176.stdout:0/284: dwrite d2/db/d31/f4c [0,4194304] 0 2024-03-13T14:49:42.321 INFO:tasks.workunit.client.1.smithi196.stdout:6/581: dwrite d6/d9/d11/f13 [0,4194304] 0 2024-03-13T14:49:42.332 INFO:tasks.workunit.client.1.smithi196.stdout:9/375: mkdir d7/d24/d28/d6c/d7b 0 2024-03-13T14:49:42.335 INFO:tasks.workunit.client.1.smithi196.stdout:9/376: dread - d7/de/d16/f60 zero size 2024-03-13T14:49:42.341 INFO:tasks.workunit.client.1.smithi196.stdout:1/461: dread d1/d38/f25 [0,4194304] 0 2024-03-13T14:49:42.355 INFO:tasks.workunit.client.1.smithi196.stdout:8/399: rename d0/d1c/d3b to d0/d16/d3d/d8b 0 2024-03-13T14:49:42.356 INFO:tasks.workunit.client.0.smithi176.stdout:5/281: truncate d5/d31/f45 1134485 0 2024-03-13T14:49:42.357 INFO:tasks.workunit.client.0.smithi176.stdout:5/282: fdatasync d5/d22/f29 0 2024-03-13T14:49:42.365 INFO:tasks.workunit.client.1.smithi196.stdout:3/455: dwrite db/d37/d64/f74 [0,4194304] 0 2024-03-13T14:49:42.374 INFO:tasks.workunit.client.1.smithi196.stdout:9/377: creat d7/de/d4b/f7c x:0 0 0 2024-03-13T14:49:42.377 INFO:tasks.workunit.client.1.smithi196.stdout:9/378: chown d7/de/d4b/d1d/l4f 195049084 1 2024-03-13T14:49:42.378 INFO:tasks.workunit.client.0.smithi176.stdout:3/316: sync 2024-03-13T14:49:42.380 INFO:tasks.workunit.client.1.smithi196.stdout:0/493: dread d8/da/d25/d26/d35/d3b/d5f/d17/f70 [0,4194304] 0 2024-03-13T14:49:42.380 INFO:tasks.workunit.client.1.smithi196.stdout:9/379: truncate d7/d9/f77 350778 0 2024-03-13T14:49:42.384 INFO:tasks.workunit.client.0.smithi176.stdout:4/334: mknod d7/d10/d46/c7c 0 2024-03-13T14:49:42.386 INFO:tasks.workunit.client.1.smithi196.stdout:0/494: fdatasync d8/da/d25/d26/d35/d3b/d50/d5d/f95 0 2024-03-13T14:49:42.389 INFO:tasks.workunit.client.1.smithi196.stdout:2/355: creat d0/f77 x:0 0 0 2024-03-13T14:49:42.390 INFO:tasks.workunit.client.0.smithi176.stdout:0/285: creat d2/db/d31/f51 x:0 0 0 2024-03-13T14:49:42.392 INFO:tasks.workunit.client.0.smithi176.stdout:0/286: chown d2/d5/dc/f2e 2109 1 2024-03-13T14:49:42.400 INFO:tasks.workunit.client.0.smithi176.stdout:6/303: write d0/d10/d19/d27/f61 [465231,31504] 0 2024-03-13T14:49:42.401 INFO:tasks.workunit.client.1.smithi196.stdout:4/412: getdents d1/d3/d7/d35 0 2024-03-13T14:49:42.407 INFO:tasks.workunit.client.0.smithi176.stdout:7/307: creat d1/d1c/d26/d41/f5b x:0 0 0 2024-03-13T14:49:42.415 INFO:tasks.workunit.client.0.smithi176.stdout:9/375: getdents d0/d1e/d1f/d68/d6a 0 2024-03-13T14:49:42.417 INFO:tasks.workunit.client.0.smithi176.stdout:9/376: stat d0/d13/d32/d58/d5f 0 2024-03-13T14:49:42.418 INFO:tasks.workunit.client.1.smithi196.stdout:8/400: dread d0/d10/d12/f33 [0,4194304] 0 2024-03-13T14:49:42.418 INFO:tasks.workunit.client.1.smithi196.stdout:3/456: link db/d19/d22/d3a/f8f db/d19/d22/d3a/d68/d8e/f96 0 2024-03-13T14:49:42.418 INFO:tasks.workunit.client.1.smithi196.stdout:5/558: sync 2024-03-13T14:49:42.419 INFO:tasks.workunit.client.1.smithi196.stdout:9/380: mknod d7/de/d74/c7d 0 2024-03-13T14:49:42.421 INFO:tasks.workunit.client.0.smithi176.stdout:8/276: write d0/f9 [5999106,89104] 0 2024-03-13T14:49:42.422 INFO:tasks.workunit.client.0.smithi176.stdout:4/335: truncate d7/d8/d45/f5c 2228598 0 2024-03-13T14:49:42.423 INFO:tasks.workunit.client.0.smithi176.stdout:4/336: stat d7/d10/f56 0 2024-03-13T14:49:42.424 INFO:tasks.workunit.client.1.smithi196.stdout:9/381: fsync d7/de/d4b/d1d/d22/f67 0 2024-03-13T14:49:42.424 INFO:tasks.workunit.client.1.smithi196.stdout:4/413: rename d1/d1e/d43/d5b to d1/d3/d4c/d72/d7d/d80 0 2024-03-13T14:49:42.425 INFO:tasks.workunit.client.0.smithi176.stdout:4/337: chown d7/d8/d45/f62 64 1 2024-03-13T14:49:42.426 INFO:tasks.workunit.client.1.smithi196.stdout:0/495: mknod d8/da/d25/d26/d35/d3b/ca2 0 2024-03-13T14:49:42.427 INFO:tasks.workunit.client.1.smithi196.stdout:0/496: stat d8 0 2024-03-13T14:49:42.427 INFO:tasks.workunit.client.0.smithi176.stdout:0/287: rename d2/d5/dc/f18 to d2/d5/d1e/f52 0 2024-03-13T14:49:42.428 INFO:tasks.workunit.client.0.smithi176.stdout:4/338: write d7/d8/d45/f62 [352345,60518] 0 2024-03-13T14:49:42.428 INFO:tasks.workunit.client.0.smithi176.stdout:1/313: link d2/de/d3f/d19/l28 d2/de/d10/d16/d59/l5d 0 2024-03-13T14:49:42.429 INFO:tasks.workunit.client.1.smithi196.stdout:6/582: link d6/d9/d17/d19/f2b d6/d2a/d23/fb6 0 2024-03-13T14:49:42.431 INFO:tasks.workunit.client.0.smithi176.stdout:1/314: dread - d2/de/d10/d16/d47/f5c zero size 2024-03-13T14:49:42.431 INFO:tasks.workunit.client.0.smithi176.stdout:6/304: symlink d0/d10/d19/l66 0 2024-03-13T14:49:42.434 INFO:tasks.workunit.client.1.smithi196.stdout:1/462: dwrite d1/f37 [0,4194304] 0 2024-03-13T14:49:42.435 INFO:tasks.workunit.client.1.smithi196.stdout:3/457: readlink db/d37/l72 0 2024-03-13T14:49:42.437 INFO:tasks.workunit.client.0.smithi176.stdout:7/308: creat d1/d28/d3a/f5c x:0 0 0 2024-03-13T14:49:42.438 INFO:tasks.workunit.client.0.smithi176.stdout:8/277: symlink d0/d5/d3b/l54 0 2024-03-13T14:49:42.439 INFO:tasks.workunit.client.1.smithi196.stdout:4/414: creat d1/d3/d4c/f81 x:0 0 0 2024-03-13T14:49:42.440 INFO:tasks.workunit.client.1.smithi196.stdout:2/356: creat d0/f78 x:0 0 0 2024-03-13T14:49:42.443 INFO:tasks.workunit.client.0.smithi176.stdout:2/335: getdents d1/dc/d2d 0 2024-03-13T14:49:42.443 INFO:tasks.workunit.client.1.smithi196.stdout:2/357: truncate d0/d50/f65 678708 0 2024-03-13T14:49:42.446 INFO:tasks.workunit.client.1.smithi196.stdout:0/497: mknod d8/da/d25/d26/d35/d3b/d5f/d17/ca3 0 2024-03-13T14:49:42.447 INFO:tasks.workunit.client.1.smithi196.stdout:6/583: readlink d6/d2a/dc/d1c/d34/l3d 0 2024-03-13T14:49:42.449 INFO:tasks.workunit.client.1.smithi196.stdout:3/458: rmdir db/d15/d44 39 2024-03-13T14:49:42.453 INFO:tasks.workunit.client.1.smithi196.stdout:8/401: mkdir d0/d10/d12/d6f/d87/d8c 0 2024-03-13T14:49:42.453 INFO:tasks.workunit.client.1.smithi196.stdout:3/459: chown db/d71/l85 215095493 1 2024-03-13T14:49:42.454 INFO:tasks.workunit.client.0.smithi176.stdout:5/283: link d5/f24 d5/d22/f56 0 2024-03-13T14:49:42.454 INFO:tasks.workunit.client.1.smithi196.stdout:0/498: chown d8/da/c52 898 1 2024-03-13T14:49:42.455 INFO:tasks.workunit.client.1.smithi196.stdout:4/415: mkdir d1/d3/d4c/d72/d7d/d80/d82 0 2024-03-13T14:49:42.465 INFO:tasks.workunit.client.1.smithi196.stdout:6/584: rename d6/d9/d17/f6c to d6/d57/d82/fb7 0 2024-03-13T14:49:42.467 INFO:tasks.workunit.client.1.smithi196.stdout:9/382: dwrite d7/de/d4b/f7c [0,4194304] 0 2024-03-13T14:49:42.469 INFO:tasks.workunit.client.0.smithi176.stdout:3/317: link d4/c12 d4/d28/d31/d52/c64 0 2024-03-13T14:49:42.471 INFO:tasks.workunit.client.1.smithi196.stdout:7/575: sync 2024-03-13T14:49:42.471 INFO:tasks.workunit.client.1.smithi196.stdout:3/460: dread - db/d19/d22/d30/f69 zero size 2024-03-13T14:49:42.473 INFO:tasks.workunit.client.0.smithi176.stdout:7/309: chown d1/d1c/d26/d41/f5b 5434 1 2024-03-13T14:49:42.475 INFO:tasks.workunit.client.1.smithi196.stdout:5/559: dread d4/db/d3c/f84 [0,4194304] 0 2024-03-13T14:49:42.481 INFO:tasks.workunit.client.0.smithi176.stdout:8/278: rename d0/c4a to d0/d24/d29/c55 0 2024-03-13T14:49:42.481 INFO:tasks.workunit.client.1.smithi196.stdout:5/560: write d4/db/d14/d19/d26/f43 [4320580,33195] 0 2024-03-13T14:49:42.483 INFO:tasks.workunit.client.0.smithi176.stdout:2/336: mkdir d1/d51/d5c/d78 0 2024-03-13T14:49:42.484 INFO:tasks.workunit.client.1.smithi196.stdout:3/461: dread db/f3e [0,4194304] 0 2024-03-13T14:49:42.484 INFO:tasks.workunit.client.0.smithi176.stdout:5/284: dread d5/d22/f56 [0,4194304] 0 2024-03-13T14:49:42.486 INFO:tasks.workunit.client.0.smithi176.stdout:2/337: chown d1/dc/d2d/d4f/c67 4183023 1 2024-03-13T14:49:42.487 INFO:tasks.workunit.client.1.smithi196.stdout:4/416: dwrite d1/d3/d7/f5d [0,4194304] 0 2024-03-13T14:49:42.491 INFO:tasks.workunit.client.0.smithi176.stdout:7/310: dwrite d1/d28/f2a [0,4194304] 0 2024-03-13T14:49:42.494 INFO:tasks.workunit.client.1.smithi196.stdout:2/358: creat d0/d6/d4b/f79 x:0 0 0 2024-03-13T14:49:42.502 INFO:tasks.workunit.client.0.smithi176.stdout:1/315: mknod d2/de/d10/c5e 0 2024-03-13T14:49:42.504 INFO:tasks.workunit.client.0.smithi176.stdout:1/316: readlink d2/de/d10/l24 0 2024-03-13T14:49:42.508 INFO:tasks.workunit.client.1.smithi196.stdout:5/561: dwrite d4/db/d14/d19/d1c/fa2 [0,4194304] 0 2024-03-13T14:49:42.509 INFO:tasks.workunit.client.0.smithi176.stdout:0/288: write d2/db/f2c [796816,69583] 0 2024-03-13T14:49:42.510 INFO:tasks.workunit.client.1.smithi196.stdout:4/417: dwrite d1/d3/d7/f1a [8388608,4194304] 0 2024-03-13T14:49:42.511 INFO:tasks.workunit.client.1.smithi196.stdout:5/562: chown d4/d1b/c3d 67555373 1 2024-03-13T14:49:42.513 INFO:tasks.workunit.client.1.smithi196.stdout:9/383: rename d7/de/d16/c18 to d7/d24/c7e 0 2024-03-13T14:49:42.515 INFO:tasks.workunit.client.1.smithi196.stdout:5/563: chown d4/db/d14/d19/d1c/d25/d38 266470302 1 2024-03-13T14:49:42.515 INFO:tasks.workunit.client.1.smithi196.stdout:1/463: write d1/d9/d33/d4b/f1b [5033418,33605] 0 2024-03-13T14:49:42.517 INFO:tasks.workunit.client.1.smithi196.stdout:5/564: chown d4/db/d14/d19/d26/d95 33281151 1 2024-03-13T14:49:42.530 INFO:tasks.workunit.client.1.smithi196.stdout:0/499: dwrite d8/da/d25/d26/d35/d3b/d50/d5d/f57 [0,4194304] 0 2024-03-13T14:49:42.534 INFO:tasks.workunit.client.0.smithi176.stdout:8/279: rmdir d0/d5 39 2024-03-13T14:49:42.537 INFO:tasks.workunit.client.0.smithi176.stdout:2/338: creat d1/dc/d2d/d38/d5d/f79 x:0 0 0 2024-03-13T14:49:42.540 INFO:tasks.workunit.client.1.smithi196.stdout:1/464: dread d1/d38/f3b [0,4194304] 0 2024-03-13T14:49:42.541 INFO:tasks.workunit.client.1.smithi196.stdout:3/462: creat db/d37/f97 x:0 0 0 2024-03-13T14:49:42.542 INFO:tasks.workunit.client.0.smithi176.stdout:6/305: rmdir d0/d43 0 2024-03-13T14:49:42.546 INFO:tasks.workunit.client.1.smithi196.stdout:2/359: creat d0/d6/d1d/f7a x:0 0 0 2024-03-13T14:49:42.550 INFO:tasks.workunit.client.1.smithi196.stdout:6/585: symlink d6/d2a/lb8 0 2024-03-13T14:49:42.554 INFO:tasks.workunit.client.1.smithi196.stdout:4/418: rename d1/d3/d7/f2c to d1/d3/d4c/f83 0 2024-03-13T14:49:42.555 INFO:tasks.workunit.client.1.smithi196.stdout:9/384: rename d7 to d7/d24/d28/d6c/d7f 22 2024-03-13T14:49:42.556 INFO:tasks.workunit.client.0.smithi176.stdout:8/280: mkdir d0/db/d39/d56 0 2024-03-13T14:49:42.556 INFO:tasks.workunit.client.1.smithi196.stdout:4/419: chown d1/l25 357 1 2024-03-13T14:49:42.558 INFO:tasks.workunit.client.0.smithi176.stdout:6/306: dwrite d0/de/f49 [0,4194304] 0 2024-03-13T14:49:42.568 INFO:tasks.workunit.client.1.smithi196.stdout:5/565: creat d4/db/d14/d19/d1c/d25/d38/d47/fb0 x:0 0 0 2024-03-13T14:49:42.571 INFO:tasks.workunit.client.1.smithi196.stdout:8/402: truncate d0/d1c/d29/d47/f52 912934 0 2024-03-13T14:49:42.572 INFO:tasks.workunit.client.0.smithi176.stdout:1/317: mkdir d2/de/d10/d16/d4a/d5f 0 2024-03-13T14:49:42.578 INFO:tasks.workunit.client.0.smithi176.stdout:2/339: rmdir d1/dc/d2d/d38 39 2024-03-13T14:49:42.578 INFO:tasks.workunit.client.0.smithi176.stdout:4/339: sync 2024-03-13T14:49:42.579 INFO:tasks.workunit.client.0.smithi176.stdout:9/377: sync 2024-03-13T14:49:42.579 INFO:tasks.workunit.client.0.smithi176.stdout:5/285: sync 2024-03-13T14:49:42.583 INFO:tasks.workunit.client.0.smithi176.stdout:4/340: write d7/d8/d15/f38 [3953204,39987] 0 2024-03-13T14:49:42.583 INFO:tasks.workunit.client.1.smithi196.stdout:4/420: dwrite d1/f54 [0,4194304] 0 2024-03-13T14:49:42.586 INFO:tasks.workunit.client.0.smithi176.stdout:3/318: dwrite d4/d13/d16/d3d/f54 [0,4194304] 0 2024-03-13T14:49:42.586 INFO:tasks.workunit.client.1.smithi196.stdout:4/421: chown d1/d3/c41 0 1 2024-03-13T14:49:42.588 INFO:tasks.workunit.client.0.smithi176.stdout:3/319: stat d4/d13/d2d/f30 0 2024-03-13T14:49:42.590 INFO:tasks.workunit.client.1.smithi196.stdout:2/360: symlink d0/d6/d4b/d5f/l7b 0 2024-03-13T14:49:42.594 INFO:tasks.workunit.client.0.smithi176.stdout:6/307: unlink d0/d9/f14 0 2024-03-13T14:49:42.607 INFO:tasks.workunit.client.0.smithi176.stdout:8/281: dread d0/db/f1a [0,4194304] 0 2024-03-13T14:49:42.609 INFO:tasks.workunit.client.1.smithi196.stdout:0/500: write d8/da/d25/d26/d35/d3b/d5f/d17/f4d [3942927,101247] 0 2024-03-13T14:49:42.609 INFO:tasks.workunit.client.0.smithi176.stdout:0/289: truncate d2/f11 3585752 0 2024-03-13T14:49:42.612 INFO:tasks.workunit.client.0.smithi176.stdout:5/286: dwrite d5/d22/d15/f2d [0,4194304] 0 2024-03-13T14:49:42.616 INFO:tasks.workunit.client.0.smithi176.stdout:7/311: dwrite d1/d1c/f56 [4194304,4194304] 0 2024-03-13T14:49:42.617 INFO:tasks.workunit.client.1.smithi196.stdout:7/576: link d0/d6/c12 d0/d4d/d74/cbf 0 2024-03-13T14:49:42.617 INFO:tasks.workunit.client.1.smithi196.stdout:0/501: readlink d8/da/d25/d26/d35/d3b/d6c/l83 0 2024-03-13T14:49:42.618 INFO:tasks.workunit.client.0.smithi176.stdout:1/318: write d2/f21 [5159891,99484] 0 2024-03-13T14:49:42.621 INFO:tasks.workunit.client.0.smithi176.stdout:4/341: creat d7/d8/d15/d23/f7d x:0 0 0 2024-03-13T14:49:42.622 INFO:tasks.workunit.client.0.smithi176.stdout:3/320: readlink d4/l22 0 2024-03-13T14:49:42.624 INFO:tasks.workunit.client.0.smithi176.stdout:7/312: chown d1/d28/d3a/d1b/d18/f52 3 1 2024-03-13T14:49:42.625 INFO:tasks.workunit.client.1.smithi196.stdout:3/463: fdatasync db/d19/f3f 0 2024-03-13T14:49:42.625 INFO:tasks.workunit.client.1.smithi196.stdout:6/586: dwrite d6/d57/fb0 [0,4194304] 0 2024-03-13T14:49:42.625 INFO:tasks.workunit.client.0.smithi176.stdout:3/321: chown d4/f1d 3717604 1 2024-03-13T14:49:42.629 INFO:tasks.workunit.client.0.smithi176.stdout:8/282: creat d0/d24/f57 x:0 0 0 2024-03-13T14:49:42.629 INFO:tasks.workunit.client.1.smithi196.stdout:8/403: write d0/d1c/d29/f65 [1876275,103162] 0 2024-03-13T14:49:42.635 INFO:tasks.workunit.client.1.smithi196.stdout:0/502: fsync d8/da/d25/d26/d35/d3b/d6c/f9d 0 2024-03-13T14:49:42.639 INFO:tasks.workunit.client.1.smithi196.stdout:1/465: dwrite d1/d38/f16 [0,4194304] 0 2024-03-13T14:49:42.642 INFO:tasks.workunit.client.1.smithi196.stdout:0/503: fdatasync d8/da/d25/d26/d35/d3b/d5f/f4a 0 2024-03-13T14:49:42.643 INFO:tasks.workunit.client.0.smithi176.stdout:5/287: dwrite d5/f37 [0,4194304] 0 2024-03-13T14:49:42.645 INFO:tasks.workunit.client.1.smithi196.stdout:2/361: creat d0/d7/d9/d5a/f7c x:0 0 0 2024-03-13T14:49:42.654 INFO:tasks.workunit.client.1.smithi196.stdout:9/385: link d7/de/d4b/d1d/f66 d7/de/d4b/d1d/d22/d4d/f80 0 2024-03-13T14:49:42.655 INFO:tasks.workunit.client.1.smithi196.stdout:3/464: dread db/d19/d4a/f8c [0,4194304] 0 2024-03-13T14:49:42.658 INFO:tasks.workunit.client.0.smithi176.stdout:3/322: dwrite d4/f25 [4194304,4194304] 0 2024-03-13T14:49:42.659 INFO:tasks.workunit.client.1.smithi196.stdout:3/465: stat db/d19/d22/d3a/d68/d8e/f96 0 2024-03-13T14:49:42.663 INFO:tasks.workunit.client.1.smithi196.stdout:4/422: mknod d1/d3/d7/d35/d76/d58/c84 0 2024-03-13T14:49:42.670 INFO:tasks.workunit.client.0.smithi176.stdout:4/342: symlink d7/d10/d2c/l7e 0 2024-03-13T14:49:42.677 INFO:tasks.workunit.client.1.smithi196.stdout:7/577: symlink d0/d9/d31/lc0 0 2024-03-13T14:49:42.682 INFO:tasks.workunit.client.1.smithi196.stdout:4/423: dread d1/d3/d7/f5d [0,4194304] 0 2024-03-13T14:49:42.690 INFO:tasks.workunit.client.1.smithi196.stdout:6/587: mknod d6/d2a/dc/cb9 0 2024-03-13T14:49:42.701 INFO:tasks.workunit.client.1.smithi196.stdout:0/504: fdatasync d8/da/d25/d26/d35/d3b/d5f/d33/d44/f86 0 2024-03-13T14:49:42.706 INFO:tasks.workunit.client.1.smithi196.stdout:1/466: unlink d1/d9/d4e/d55/l98 0 2024-03-13T14:49:42.709 INFO:tasks.workunit.client.1.smithi196.stdout:1/467: stat d1/d9/d33/l39 0 2024-03-13T14:49:42.709 INFO:tasks.workunit.client.0.smithi176.stdout:3/323: rename d4/d13/l2f to d4/d28/d34/l65 0 2024-03-13T14:49:42.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.722+0000 7fbe35200700 1 -- 172.21.15.176:0/2305370819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 msgr2=0x7fbe30100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:42.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.722+0000 7fbe35200700 1 --2- 172.21.15.176:0/2305370819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30100e20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009910 tx=0x7fbe20009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:42.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.722+0000 7fbe35200700 1 -- 172.21.15.176:0/2305370819 shutdown_connections 2024-03-13T14:49:42.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.722+0000 7fbe35200700 1 --2- 172.21.15.176:0/2305370819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30101c00 0x7fbe30102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:42.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.722+0000 7fbe35200700 1 --2- 172.21.15.176:0/2305370819 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30100e20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:42.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.722+0000 7fbe35200700 1 -- 172.21.15.176:0/2305370819 >> 172.21.15.176:0/2305370819 conn(0x7fbe300fbf60 msgr2=0x7fbe300fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:42.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.723+0000 7fbe35200700 1 -- 172.21.15.176:0/2305370819 shutdown_connections 2024-03-13T14:49:42.712 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.723+0000 7fbe35200700 1 -- 172.21.15.176:0/2305370819 wait complete. 2024-03-13T14:49:42.714 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.724+0000 7fbe35200700 1 Processor -- start 2024-03-13T14:49:42.714 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.725+0000 7fbe35200700 1 -- start start 2024-03-13T14:49:42.714 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.725+0000 7fbe35200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30193fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:42.715 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.725+0000 7fbe35200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30101c00 0x7fbe301944f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:42.715 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.725+0000 7fbe35200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30194b10 con 0x7fbe30100a00 2024-03-13T14:49:42.715 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.725+0000 7fbe35200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30194c50 con 0x7fbe30101c00 2024-03-13T14:49:42.715 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.726+0000 7fbe2ea00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30101c00 0x7fbe301944f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:42.715 INFO:tasks.workunit.client.0.smithi176.stdout:3/324: fsync d4/d13/d2d/d50/f46 0 2024-03-13T14:49:42.715 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.726+0000 7fbe2f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30193fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:42.716 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.726+0000 7fbe2f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30193fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43212/0 (socket says 172.21.15.176:43212) 2024-03-13T14:49:42.716 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.726+0000 7fbe2f400700 1 -- 172.21.15.176:0/435309251 learned_addr learned my addr 172.21.15.176:0/435309251 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:42.716 INFO:tasks.workunit.client.1.smithi196.stdout:5/566: rename d4/db/c73 to d4/cb1 0 2024-03-13T14:49:42.716 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.728+0000 7fbe2f400700 1 -- 172.21.15.176:0/435309251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30101c00 msgr2=0x7fbe301944f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:42.716 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.728+0000 7fbe2f400700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30101c00 0x7fbe301944f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:42.717 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.728+0000 7fbe2f400700 1 -- 172.21.15.176:0/435309251 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe200095f0 con 0x7fbe30100a00 2024-03-13T14:49:42.718 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.729+0000 7fbe2f400700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30193fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bda0 tx=0x7fbe2000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:42.718 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.729+0000 7fbe27e00700 1 -- 172.21.15.176:0/435309251 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe2001d070 con 0x7fbe30100a00 2024-03-13T14:49:42.718 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.729+0000 7fbe27e00700 1 -- 172.21.15.176:0/435309251 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe2000f460 con 0x7fbe30100a00 2024-03-13T14:49:42.718 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.729+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe301996a0 con 0x7fbe30100a00 2024-03-13T14:49:42.718 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.729+0000 7fbe27e00700 1 -- 172.21.15.176:0/435309251 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe20021620 con 0x7fbe30100a00 2024-03-13T14:49:42.718 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.729+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30199b90 con 0x7fbe30100a00 2024-03-13T14:49:42.719 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.731+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe30105ce0 con 0x7fbe30100a00 2024-03-13T14:49:42.720 INFO:tasks.workunit.client.1.smithi196.stdout:9/386: rmdir d7/de 39 2024-03-13T14:49:42.720 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.732+0000 7fbe27e00700 1 -- 172.21.15.176:0/435309251 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 29) v1 ==== 46247+0+0 (secure 0 0 0) 0x7fbe2000f5d0 con 0x7fbe30100a00 2024-03-13T14:49:42.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.733+0000 7fbe27e00700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fbe14039230 0x7fbe1403b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:42.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.733+0000 7fbe27e00700 1 -- 172.21.15.176:0/435309251 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbe2004e7c0 con 0x7fbe30100a00 2024-03-13T14:49:42.722 INFO:tasks.workunit.client.1.smithi196.stdout:3/466: creat db/d19/d22/f98 x:0 0 0 2024-03-13T14:49:42.722 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.734+0000 7fbe2ea00700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fbe14039230 0x7fbe1403b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:42.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.736+0000 7fbe2ea00700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fbe14039230 0x7fbe1403b6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c005950 tx=0x7fbe1c0058e0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:42.725 INFO:tasks.workunit.client.0.smithi176.stdout:8/283: unlink d0/d5/d3b/l50 0 2024-03-13T14:49:42.728 INFO:tasks.workunit.client.0.smithi176.stdout:5/288: write d5/f14 [11697794,45442] 0 2024-03-13T14:49:42.728 INFO:tasks.workunit.client.0.smithi176.stdout:8/284: write d0/d24/f32 [942733,97495] 0 2024-03-13T14:49:42.731 INFO:tasks.workunit.client.1.smithi196.stdout:7/578: unlink d0/d9/d31/f42 0 2024-03-13T14:49:42.732 INFO:tasks.workunit.client.0.smithi176.stdout:2/340: dread d1/dc/d2d/d38/d5d/f62 [0,4194304] 0 2024-03-13T14:49:42.732 INFO:tasks.workunit.client.1.smithi196.stdout:4/424: rmdir d1/d3/d7/d35/d76 39 2024-03-13T14:49:42.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:42.745+0000 7fbe27e00700 1 -- 172.21.15.176:0/435309251 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fbe2002b440 con 0x7fbe30100a00 2024-03-13T14:49:42.734 INFO:tasks.workunit.client.0.smithi176.stdout:9/378: getdents d0/d13/d4d 0 2024-03-13T14:49:42.738 INFO:tasks.workunit.client.1.smithi196.stdout:8/404: mkdir d0/d10/d70/d8d 0 2024-03-13T14:49:42.739 INFO:tasks.workunit.client.0.smithi176.stdout:2/341: chown d1/dc/d2d/d38/l57 1 1 2024-03-13T14:49:42.739 INFO:tasks.workunit.client.0.smithi176.stdout:9/379: stat d0/d1e/d1f/f2c 0 2024-03-13T14:49:42.742 INFO:tasks.workunit.client.0.smithi176.stdout:6/308: getdents d0/d10/d19/d27/d2b/d2e 0 2024-03-13T14:49:42.742 INFO:tasks.workunit.client.0.smithi176.stdout:9/380: write d0/d1e/f4a [597789,99044] 0 2024-03-13T14:49:42.743 INFO:tasks.workunit.client.0.smithi176.stdout:4/343: truncate d7/d8/d2f/f37 2098262 0 2024-03-13T14:49:42.748 INFO:tasks.workunit.client.0.smithi176.stdout:1/319: getdents d2/de/d10 0 2024-03-13T14:49:42.749 INFO:tasks.workunit.client.0.smithi176.stdout:5/289: mknod d5/d2e/c57 0 2024-03-13T14:49:42.750 INFO:tasks.workunit.client.0.smithi176.stdout:5/290: stat d5/d22/c2f 0 2024-03-13T14:49:42.751 INFO:tasks.workunit.client.0.smithi176.stdout:1/320: dread - d2/d30/f4c zero size 2024-03-13T14:49:42.754 INFO:tasks.workunit.client.0.smithi176.stdout:7/313: write d1/d28/f2a [4784194,20780] 0 2024-03-13T14:49:42.755 INFO:tasks.workunit.client.0.smithi176.stdout:7/314: dread - d1/d3f/f4d zero size 2024-03-13T14:49:42.756 INFO:tasks.workunit.client.0.smithi176.stdout:2/342: dwrite d1/dc/d61/f6d [0,4194304] 0 2024-03-13T14:49:42.763 INFO:tasks.workunit.client.0.smithi176.stdout:7/315: read d1/d3f/d4b/f58 [114073,7017] 0 2024-03-13T14:49:42.764 INFO:tasks.workunit.client.0.smithi176.stdout:9/381: fdatasync d0/d1e/f4e 0 2024-03-13T14:49:42.767 INFO:tasks.workunit.client.0.smithi176.stdout:4/344: creat d7/d8/d33/f7f x:0 0 0 2024-03-13T14:49:42.767 INFO:tasks.workunit.client.0.smithi176.stdout:9/382: read - d0/d40/d53/f6b zero size 2024-03-13T14:49:42.773 INFO:tasks.workunit.client.1.smithi196.stdout:2/362: write d0/d7/f21 [105023,37086] 0 2024-03-13T14:49:42.774 INFO:tasks.workunit.client.0.smithi176.stdout:8/285: getdents d0/db/d39/d56 0 2024-03-13T14:49:42.775 INFO:tasks.workunit.client.1.smithi196.stdout:2/363: write d0/d7/f21 [1525753,30632] 0 2024-03-13T14:49:42.781 INFO:tasks.workunit.client.0.smithi176.stdout:5/291: creat d5/d2e/f58 x:0 0 0 2024-03-13T14:49:42.781 INFO:tasks.workunit.client.0.smithi176.stdout:8/286: truncate d0/db/dc/d40/f4f 303255 0 2024-03-13T14:49:42.782 INFO:tasks.workunit.client.0.smithi176.stdout:2/343: rmdir d1/dc 39 2024-03-13T14:49:42.785 INFO:tasks.workunit.client.0.smithi176.stdout:0/290: truncate d2/d5/d28/f29 2970322 0 2024-03-13T14:49:42.786 INFO:tasks.workunit.client.0.smithi176.stdout:0/291: write d2/d5/f46 [3347506,122891] 0 2024-03-13T14:49:42.789 INFO:tasks.workunit.client.0.smithi176.stdout:9/383: stat d0/d1e/l24 0 2024-03-13T14:49:42.791 INFO:tasks.workunit.client.0.smithi176.stdout:0/292: write d2/d5/dc/d20/f48 [182009,80216] 0 2024-03-13T14:49:42.793 INFO:tasks.workunit.client.1.smithi196.stdout:0/505: rmdir d8/da/d25/d26/d35/d3b/d50/d5d 39 2024-03-13T14:49:42.794 INFO:tasks.workunit.client.1.smithi196.stdout:0/506: chown d8/da/d25/f5a 437876 1 2024-03-13T14:49:42.795 INFO:tasks.workunit.client.0.smithi176.stdout:0/293: write d2/db/f2c [46809,29468] 0 2024-03-13T14:49:42.798 INFO:tasks.workunit.client.1.smithi196.stdout:5/567: rmdir d4/db/d14 39 2024-03-13T14:49:42.799 INFO:tasks.workunit.client.0.smithi176.stdout:0/294: chown d2/d5/cd 122299 1 2024-03-13T14:49:42.801 INFO:tasks.workunit.client.1.smithi196.stdout:3/467: symlink db/d37/l99 0 2024-03-13T14:49:42.801 INFO:tasks.workunit.client.0.smithi176.stdout:3/325: getdents d4/d28/d31/d45 0 2024-03-13T14:49:42.801 INFO:tasks.workunit.client.0.smithi176.stdout:5/292: dread d5/f6 [0,4194304] 0 2024-03-13T14:49:42.804 INFO:tasks.workunit.client.0.smithi176.stdout:3/326: dread d4/d13/f61 [0,4194304] 0 2024-03-13T14:49:42.806 INFO:tasks.workunit.client.1.smithi196.stdout:4/425: creat d1/d1e/d43/f85 x:0 0 0 2024-03-13T14:49:42.808 INFO:tasks.workunit.client.0.smithi176.stdout:2/344: write d1/dc/d2d/d38/f3f [696354,51146] 0 2024-03-13T14:49:42.810 INFO:tasks.workunit.client.0.smithi176.stdout:7/316: truncate d1/d1c/d26/d41/f3e 1560591 0 2024-03-13T14:49:42.812 INFO:tasks.workunit.client.1.smithi196.stdout:8/405: rename d0/d16/d3d/d8b/f1f to d0/d16/d3d/d8b/f8e 0 2024-03-13T14:49:42.814 INFO:tasks.workunit.client.0.smithi176.stdout:9/384: chown d0/d13/d32/c72 713002 1 2024-03-13T14:49:42.821 INFO:tasks.workunit.client.1.smithi196.stdout:5/568: dwrite d4/db/d4a/faf [0,4194304] 0 2024-03-13T14:49:42.827 INFO:tasks.workunit.client.0.smithi176.stdout:6/309: truncate d0/de/f12 5920203 0 2024-03-13T14:49:42.828 INFO:tasks.workunit.client.1.smithi196.stdout:1/468: write d1/d28/d34/d67/f72 [983112,55166] 0 2024-03-13T14:49:42.831 INFO:tasks.workunit.client.1.smithi196.stdout:3/468: creat db/d37/d64/f9a x:0 0 0 2024-03-13T14:49:42.837 INFO:tasks.workunit.client.0.smithi176.stdout:0/295: truncate d2/d5/dc/f2e 1919001 0 2024-03-13T14:49:42.839 INFO:tasks.workunit.client.1.smithi196.stdout:4/426: dwrite d1/d1e/f7f [0,4194304] 0 2024-03-13T14:49:42.839 INFO:tasks.workunit.client.0.smithi176.stdout:0/296: chown d2/d5/dc/d20/f48 101353332 1 2024-03-13T14:49:42.840 INFO:tasks.workunit.client.0.smithi176.stdout:8/287: mkdir d0/db/d58 0 2024-03-13T14:49:42.842 INFO:tasks.workunit.client.0.smithi176.stdout:8/288: fdatasync d0/d24/f57 0 2024-03-13T14:49:42.842 INFO:tasks.workunit.client.1.smithi196.stdout:0/507: write d8/da/d25/d26/d54/f7f [687401,22063] 0 2024-03-13T14:49:42.847 INFO:tasks.workunit.client.0.smithi176.stdout:1/321: sync 2024-03-13T14:49:42.848 INFO:tasks.workunit.client.1.smithi196.stdout:1/469: symlink d1/d28/l9f 0 2024-03-13T14:49:42.848 INFO:tasks.workunit.client.1.smithi196.stdout:0/508: stat d8/da/l20 0 2024-03-13T14:49:42.848 INFO:tasks.workunit.client.0.smithi176.stdout:1/322: chown l0 152445374 1 2024-03-13T14:49:42.850 INFO:tasks.workunit.client.0.smithi176.stdout:7/317: mknod d1/d1c/d26/c5d 0 2024-03-13T14:49:42.852 INFO:tasks.workunit.client.0.smithi176.stdout:7/318: chown d1/d1c/d26/d41 7 1 2024-03-13T14:49:42.853 INFO:tasks.workunit.client.0.smithi176.stdout:9/385: write d0/d40/f78 [126946,36609] 0 2024-03-13T14:49:42.854 INFO:tasks.workunit.client.1.smithi196.stdout:7/579: link d0/d6/l17 d0/d6/d33/d57/dae/lc1 0 2024-03-13T14:49:42.854 INFO:tasks.workunit.client.1.smithi196.stdout:3/469: dread - db/d19/d22/d3a/d6a/f89 zero size 2024-03-13T14:49:42.854 INFO:tasks.workunit.client.1.smithi196.stdout:7/580: fsync d0/d6/d33/f39 0 2024-03-13T14:49:42.855 INFO:tasks.workunit.client.0.smithi176.stdout:9/386: chown d0/d40/f47 27 1 2024-03-13T14:49:42.857 INFO:tasks.workunit.client.1.smithi196.stdout:7/581: chown d0/d6/d33/d57 0 1 2024-03-13T14:49:42.861 INFO:tasks.workunit.client.1.smithi196.stdout:2/364: creat d0/d6/d4b/f7d x:0 0 0 2024-03-13T14:49:42.862 INFO:tasks.workunit.client.0.smithi176.stdout:6/310: creat d0/de/d38/f67 x:0 0 0 2024-03-13T14:49:42.865 INFO:tasks.workunit.client.0.smithi176.stdout:9/387: dread d0/db/f49 [0,4194304] 0 2024-03-13T14:49:42.865 INFO:tasks.workunit.client.0.smithi176.stdout:5/293: creat d5/d31/d47/f59 x:0 0 0 2024-03-13T14:49:42.868 INFO:tasks.workunit.client.1.smithi196.stdout:2/365: dread - d0/d6/d4b/f79 zero size 2024-03-13T14:49:42.870 INFO:tasks.workunit.client.1.smithi196.stdout:1/470: dread d1/d9/d33/d4b/d66/d7b/f8d [0,4194304] 0 2024-03-13T14:49:42.872 INFO:tasks.workunit.client.1.smithi196.stdout:2/366: readlink d0/d7/d56/l6a 0 2024-03-13T14:49:42.874 INFO:tasks.workunit.client.0.smithi176.stdout:7/319: dwrite d1/d1c/f56 [8388608,4194304] 0 2024-03-13T14:49:42.878 INFO:tasks.workunit.client.0.smithi176.stdout:3/327: link d4/d13/d2d/f51 d4/d13/d2d/d50/f66 0 2024-03-13T14:49:42.889 INFO:tasks.workunit.client.0.smithi176.stdout:1/323: unlink d2/de/d10/f2a 0 2024-03-13T14:49:42.890 INFO:tasks.workunit.client.1.smithi196.stdout:9/387: link d7/de/f29 d7/d24/d28/d6c/f81 0 2024-03-13T14:49:42.893 INFO:tasks.workunit.client.0.smithi176.stdout:4/345: getdents d7/d10/d2c 0 2024-03-13T14:49:42.896 INFO:tasks.workunit.client.1.smithi196.stdout:7/582: dwrite d0/d9/d1e/d58/d72/d6e/d7f/f9c [0,4194304] 0 2024-03-13T14:49:42.896 INFO:tasks.workunit.client.0.smithi176.stdout:4/346: read d7/d8/f28 [582828,102803] 0 2024-03-13T14:49:42.898 INFO:tasks.workunit.client.1.smithi196.stdout:1/471: dread d1/d38/f25 [0,4194304] 0 2024-03-13T14:49:42.898 INFO:tasks.workunit.client.0.smithi176.stdout:4/347: fdatasync d7/d8/d15/f2b 0 2024-03-13T14:49:42.902 INFO:tasks.workunit.client.1.smithi196.stdout:3/470: dread db/d19/d22/f3c [0,4194304] 0 2024-03-13T14:49:42.903 INFO:tasks.workunit.client.0.smithi176.stdout:5/294: creat d5/d48/f5a x:0 0 0 2024-03-13T14:49:42.904 INFO:tasks.workunit.client.1.smithi196.stdout:1/472: write d1/d9/d33/f82 [517437,32742] 0 2024-03-13T14:49:42.904 INFO:tasks.workunit.client.0.smithi176.stdout:5/295: read - d5/d31/d47/f59 zero size 2024-03-13T14:49:42.909 INFO:tasks.workunit.client.1.smithi196.stdout:4/427: link d1/d3/f13 d1/d3/d4c/d72/d7d/d80/f86 0 2024-03-13T14:49:42.910 INFO:tasks.workunit.client.1.smithi196.stdout:4/428: fsync d1/d1e/d43/f85 0 2024-03-13T14:49:42.910 INFO:tasks.workunit.client.0.smithi176.stdout:9/388: fsync d0/d13/d32/d58/d5f/f75 0 2024-03-13T14:49:42.914 INFO:tasks.workunit.client.0.smithi176.stdout:0/297: mknod d2/db/c53 0 2024-03-13T14:49:42.916 INFO:tasks.workunit.client.1.smithi196.stdout:0/509: getdents d8/da/d25/d26/d35/d3b/d50 0 2024-03-13T14:49:42.918 INFO:tasks.workunit.client.0.smithi176.stdout:7/320: creat d1/d1c/f5e x:0 0 0 2024-03-13T14:49:42.919 INFO:tasks.workunit.client.1.smithi196.stdout:2/367: truncate d0/d7/d66/f76 189334 0 2024-03-13T14:49:42.920 INFO:tasks.workunit.client.1.smithi196.stdout:7/583: mkdir d0/d9/d1e/d58/d72/da9/da3/dc2 0 2024-03-13T14:49:42.921 INFO:tasks.workunit.client.1.smithi196.stdout:2/368: chown d0/d7/d9/d5a/f7c 339 1 2024-03-13T14:49:42.922 INFO:tasks.workunit.client.1.smithi196.stdout:3/471: mkdir db/d19/d4a/d9b 0 2024-03-13T14:49:42.923 INFO:tasks.workunit.client.0.smithi176.stdout:8/289: symlink d0/db/d39/d56/l59 0 2024-03-13T14:49:42.924 INFO:tasks.workunit.client.1.smithi196.stdout:7/584: write d0/d9/d1e/d3f/d6c/fb0 [504829,80839] 0 2024-03-13T14:49:42.924 INFO:tasks.workunit.client.1.smithi196.stdout:1/473: mknod d1/d28/ca0 0 2024-03-13T14:49:42.925 INFO:tasks.workunit.client.0.smithi176.stdout:3/328: dread d4/d28/d3a/f27 [0,4194304] 0 2024-03-13T14:49:42.926 INFO:tasks.workunit.client.1.smithi196.stdout:2/369: truncate d0/f78 400319 0 2024-03-13T14:49:42.930 INFO:tasks.workunit.client.0.smithi176.stdout:2/345: rename d1/d2a/d33/f6e to d1/dc/d2d/f7a 0 2024-03-13T14:49:42.931 INFO:tasks.workunit.client.1.smithi196.stdout:3/472: mknod db/d19/d22/d3a/d53/c9c 0 2024-03-13T14:49:42.935 INFO:tasks.workunit.client.1.smithi196.stdout:7/585: rmdir d0/d78 39 2024-03-13T14:49:42.940 INFO:tasks.workunit.client.0.smithi176.stdout:4/348: symlink d7/d10/d46/l80 0 2024-03-13T14:49:42.940 INFO:tasks.workunit.client.1.smithi196.stdout:9/388: dread f5 [0,4194304] 0 2024-03-13T14:49:42.948 INFO:tasks.workunit.client.1.smithi196.stdout:3/473: creat db/d19/d22/d30/f9d x:0 0 0 2024-03-13T14:49:42.950 INFO:tasks.workunit.client.0.smithi176.stdout:9/389: mkdir d0/d13/d7a 0 2024-03-13T14:49:42.951 INFO:tasks.workunit.client.1.smithi196.stdout:0/510: write d8/da/d25/d26/d54/d88/f84 [347487,67293] 0 2024-03-13T14:49:42.953 INFO:tasks.workunit.client.0.smithi176.stdout:9/390: truncate d0/d13/d32/f62 409276 0 2024-03-13T14:49:42.956 INFO:tasks.workunit.client.0.smithi176.stdout:9/391: write d0/d17/f48 [5578785,3692] 0 2024-03-13T14:49:42.957 INFO:tasks.workunit.client.0.smithi176.stdout:7/321: creat d1/d1c/f5f x:0 0 0 2024-03-13T14:49:42.961 INFO:tasks.workunit.client.0.smithi176.stdout:3/329: creat d4/d28/f67 x:0 0 0 2024-03-13T14:49:42.962 INFO:tasks.workunit.client.1.smithi196.stdout:1/474: symlink d1/d9/d4e/d55/la1 0 2024-03-13T14:49:42.963 INFO:tasks.workunit.client.0.smithi176.stdout:2/346: mknod d1/d51/d5c/c7b 0 2024-03-13T14:49:42.965 INFO:tasks.workunit.client.0.smithi176.stdout:2/347: fsync d1/d2a/d33/f5b 0 2024-03-13T14:49:42.965 INFO:tasks.workunit.client.1.smithi196.stdout:0/511: mknod d8/da/d25/d26/d54/d88/ca4 0 2024-03-13T14:49:42.966 INFO:tasks.workunit.client.1.smithi196.stdout:3/474: dread db/d19/d22/d3a/d53/f48 [4194304,4194304] 0 2024-03-13T14:49:42.967 INFO:tasks.workunit.client.1.smithi196.stdout:2/370: dwrite d0/d7/d9/d42/f5c [0,4194304] 0 2024-03-13T14:49:42.969 INFO:tasks.workunit.client.0.smithi176.stdout:4/349: creat d7/d8/d15/f81 x:0 0 0 2024-03-13T14:49:42.970 INFO:tasks.workunit.client.0.smithi176.stdout:5/296: write d5/d2b/f30 [5016580,10718] 0 2024-03-13T14:49:42.970 INFO:tasks.workunit.client.1.smithi196.stdout:0/512: fsync d8/da/d25/d26/f2d 0 2024-03-13T14:49:42.972 INFO:tasks.workunit.client.0.smithi176.stdout:1/324: read d2/de/d10/d14/d22/f2b [2505544,114296] 0 2024-03-13T14:49:42.975 INFO:tasks.workunit.client.1.smithi196.stdout:3/475: chown db/d19/d22/d3a/d53/f48 62863 1 2024-03-13T14:49:42.975 INFO:tasks.workunit.client.1.smithi196.stdout:6/588: sync 2024-03-13T14:49:42.975 INFO:tasks.workunit.client.1.smithi196.stdout:0/513: write d8/da/f4e [998406,46140] 0 2024-03-13T14:49:42.978 INFO:tasks.workunit.client.0.smithi176.stdout:1/325: write d2/f6 [356035,73638] 0 2024-03-13T14:49:42.979 INFO:tasks.workunit.client.1.smithi196.stdout:6/589: dread - d6/d7/df/d27/d39/d44/d74/fac zero size 2024-03-13T14:49:42.981 INFO:tasks.workunit.client.0.smithi176.stdout:6/311: creat d0/d10/d19/d27/f68 x:0 0 0 2024-03-13T14:49:42.982 INFO:tasks.workunit.client.1.smithi196.stdout:7/586: creat d0/d9/d1e/d58/fc3 x:0 0 0 2024-03-13T14:49:42.982 INFO:tasks.workunit.client.1.smithi196.stdout:0/514: write d8/da/d25/d26/d35/d3b/d5f/d33/f34 [4070702,51039] 0 2024-03-13T14:49:42.984 INFO:tasks.workunit.client.0.smithi176.stdout:6/312: stat d0/d9/l1c 0 2024-03-13T14:49:42.986 INFO:tasks.workunit.client.1.smithi196.stdout:7/587: write d0/d6/faa [134775,97969] 0 2024-03-13T14:49:42.987 INFO:tasks.workunit.client.1.smithi196.stdout:4/429: creat d1/d3/d7/d35/d76/d58/f87 x:0 0 0 2024-03-13T14:49:42.989 INFO:tasks.workunit.client.1.smithi196.stdout:9/389: dread d7/de/d4b/f52 [0,4194304] 0 2024-03-13T14:49:42.990 INFO:tasks.workunit.client.1.smithi196.stdout:1/475: mknod d1/d3f/d5b/d74/ca2 0 2024-03-13T14:49:42.992 INFO:tasks.workunit.client.0.smithi176.stdout:6/313: write d0/d10/d19/d27/f61 [1515799,31142] 0 2024-03-13T14:49:42.992 INFO:tasks.workunit.client.1.smithi196.stdout:9/390: readlink d7/de/d4b/d1d/d22/l5d 0 2024-03-13T14:49:42.994 INFO:tasks.workunit.client.1.smithi196.stdout:1/476: write d1/d38/f16 [4093735,127016] 0 2024-03-13T14:49:42.998 INFO:tasks.workunit.client.0.smithi176.stdout:3/330: dread d4/d28/f29 [0,4194304] 0 2024-03-13T14:49:43.000 INFO:tasks.workunit.client.1.smithi196.stdout:3/476: rename db/d19/d22/d30/l38 to db/l9e 0 2024-03-13T14:49:43.001 INFO:tasks.workunit.client.0.smithi176.stdout:7/322: dread d1/d28/d3a/d1b/d18/f1e [0,4194304] 0 2024-03-13T14:49:43.012 INFO:tasks.workunit.client.1.smithi196.stdout:0/515: mkdir d8/da/d25/d26/d35/d3b/d50/d5d/da5 0 2024-03-13T14:49:43.013 INFO:tasks.workunit.client.1.smithi196.stdout:7/588: readlink d0/d9/d1e/d3f/lbe 0 2024-03-13T14:49:43.021 INFO:tasks.workunit.client.0.smithi176.stdout:2/348: rmdir d1/d51/d5c 39 2024-03-13T14:49:43.022 INFO:tasks.workunit.client.0.smithi176.stdout:4/350: mknod d7/d2d/c82 0 2024-03-13T14:49:43.023 INFO:tasks.workunit.client.1.smithi196.stdout:2/371: truncate d0/f1a 2886112 0 2024-03-13T14:49:43.026 INFO:tasks.workunit.client.0.smithi176.stdout:5/297: dread d5/d2b/f30 [0,4194304] 0 2024-03-13T14:49:43.027 INFO:tasks.workunit.client.0.smithi176.stdout:5/298: fdatasync f0 0 2024-03-13T14:49:43.029 INFO:tasks.workunit.client.1.smithi196.stdout:1/477: mkdir d1/d28/da3 0 2024-03-13T14:49:43.029 INFO:tasks.workunit.client.1.smithi196.stdout:9/391: dread d7/f20 [0,4194304] 0 2024-03-13T14:49:43.032 INFO:tasks.workunit.client.0.smithi176.stdout:0/298: creat d2/d5/d28/f54 x:0 0 0 2024-03-13T14:49:43.034 INFO:tasks.workunit.client.1.smithi196.stdout:9/392: dread d7/f20 [0,4194304] 0 2024-03-13T14:49:43.034 INFO:tasks.workunit.client.1.smithi196.stdout:6/590: mkdir d6/d7/dd/dba 0 2024-03-13T14:49:43.036 INFO:tasks.workunit.client.0.smithi176.stdout:9/392: creat d0/d1e/d6f/f7b x:0 0 0 2024-03-13T14:49:43.037 INFO:tasks.workunit.client.0.smithi176.stdout:6/314: unlink d0/de/f3c 0 2024-03-13T14:49:43.037 INFO:tasks.workunit.client.1.smithi196.stdout:0/516: mknod d8/da/d25/d26/d35/ca6 0 2024-03-13T14:49:43.038 INFO:tasks.workunit.client.0.smithi176.stdout:8/290: link d0/db/dc/l4b d0/d24/l5a 0 2024-03-13T14:49:43.039 INFO:tasks.workunit.client.0.smithi176.stdout:3/331: rmdir d4/d28/d31/d45 39 2024-03-13T14:49:43.040 INFO:tasks.workunit.client.0.smithi176.stdout:3/332: fsync f2 0 2024-03-13T14:49:43.041 INFO:tasks.workunit.client.1.smithi196.stdout:6/591: dread d6/d9/d11/f48 [0,4194304] 0 2024-03-13T14:49:43.046 INFO:tasks.workunit.client.0.smithi176.stdout:0/299: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:43.050 INFO:tasks.workunit.client.0.smithi176.stdout:2/349: symlink d1/dc/d40/l7c 0 2024-03-13T14:49:43.050 INFO:tasks.workunit.client.0.smithi176.stdout:2/350: dread d1/f13 [0,4194304] 0 2024-03-13T14:49:43.051 INFO:tasks.workunit.client.0.smithi176.stdout:5/299: creat d5/d2b/f5b x:0 0 0 2024-03-13T14:49:43.054 INFO:tasks.workunit.client.1.smithi196.stdout:9/393: write d7/de/d4b/f52 [4992631,55328] 0 2024-03-13T14:49:43.059 INFO:tasks.workunit.client.0.smithi176.stdout:5/300: truncate d5/d48/d49/f4d 938684 0 2024-03-13T14:49:43.062 INFO:tasks.workunit.client.1.smithi196.stdout:8/406: sync 2024-03-13T14:49:43.062 INFO:tasks.workunit.client.1.smithi196.stdout:3/477: link db/d19/d22/d30/c4d db/d19/d22/d3a/d53/c9f 0 2024-03-13T14:49:43.066 INFO:tasks.workunit.client.1.smithi196.stdout:7/589: creat d0/d4d/fc4 x:0 0 0 2024-03-13T14:49:43.067 INFO:tasks.workunit.client.0.smithi176.stdout:8/291: dwrite d0/db/f3f [4194304,4194304] 0 2024-03-13T14:49:43.068 INFO:tasks.workunit.client.1.smithi196.stdout:7/590: stat d0/d9/d31/d50/f87 0 2024-03-13T14:49:43.073 INFO:tasks.workunit.client.0.smithi176.stdout:3/333: rename d4/l7 to d4/l68 0 2024-03-13T14:49:43.073 INFO:tasks.workunit.client.0.smithi176.stdout:3/334: chown d4/d13/c4e 353975 1 2024-03-13T14:49:43.074 INFO:tasks.workunit.client.0.smithi176.stdout:7/323: link d1/d28/d3a/d1b/f38 d1/d28/d43/f60 0 2024-03-13T14:49:43.077 INFO:tasks.workunit.client.0.smithi176.stdout:0/300: symlink d2/d5/d1e/l55 0 2024-03-13T14:49:43.077 INFO:tasks.workunit.client.1.smithi196.stdout:1/478: dwrite d1/d28/f32 [0,4194304] 0 2024-03-13T14:49:43.078 INFO:tasks.workunit.client.0.smithi176.stdout:3/335: read f1 [1528305,12195] 0 2024-03-13T14:49:43.079 INFO:tasks.workunit.client.1.smithi196.stdout:1/479: chown d1/d9/c83 2515150 1 2024-03-13T14:49:43.079 INFO:tasks.workunit.client.0.smithi176.stdout:0/301: stat d2/db/d31/f51 0 2024-03-13T14:49:43.082 INFO:tasks.workunit.client.1.smithi196.stdout:4/430: rename d1/d1e/c48 to d1/d3/d7/c88 0 2024-03-13T14:49:43.083 INFO:tasks.workunit.client.0.smithi176.stdout:0/302: write d2/db/d31/f41 [1985899,97742] 0 2024-03-13T14:49:43.085 INFO:tasks.workunit.client.0.smithi176.stdout:1/326: mknod d2/de/d10/d16/d4a/d5f/c60 0 2024-03-13T14:49:43.085 INFO:tasks.workunit.client.1.smithi196.stdout:8/407: rmdir d0/d16/d3d/d7f 39 2024-03-13T14:49:43.088 INFO:tasks.workunit.client.0.smithi176.stdout:1/327: readlink d2/de/d10/d14/d35/l48 0 2024-03-13T14:49:43.090 INFO:tasks.workunit.client.1.smithi196.stdout:2/372: getdents d0/d7/d56 0 2024-03-13T14:49:43.090 INFO:tasks.workunit.client.0.smithi176.stdout:2/351: mknod d1/dc/d2d/d4b/d69/c7d 0 2024-03-13T14:49:43.095 INFO:tasks.workunit.client.1.smithi196.stdout:3/478: symlink db/d19/d22/d3a/d68/la0 0 2024-03-13T14:49:43.095 INFO:tasks.workunit.client.0.smithi176.stdout:2/352: write d1/dc/d1c/f36 [488985,91177] 0 2024-03-13T14:49:43.096 INFO:tasks.workunit.client.1.smithi196.stdout:3/479: stat db/d71 0 2024-03-13T14:49:43.103 INFO:tasks.workunit.client.1.smithi196.stdout:7/591: mknod d0/d9/d1e/d3f/cc5 0 2024-03-13T14:49:43.107 INFO:tasks.workunit.client.0.smithi176.stdout:8/292: unlink d0/d5/f1f 0 2024-03-13T14:49:43.109 INFO:tasks.workunit.client.1.smithi196.stdout:6/592: rmdir d6/d9/d17/d19/d5f/d94 0 2024-03-13T14:49:43.111 INFO:tasks.workunit.client.1.smithi196.stdout:6/593: readlink d6/d2a/lae 0 2024-03-13T14:49:43.111 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.123+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe3019b100 con 0x7fbe14039230 2024-03-13T14:49:43.112 INFO:tasks.workunit.client.1.smithi196.stdout:5/569: sync 2024-03-13T14:49:43.114 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.126+0000 7fbe27e00700 1 -- 172.21.15.176:0/435309251 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fbe3019b100 con 0x7fbe14039230 2024-03-13T14:49:43.116 INFO:tasks.workunit.client.1.smithi196.stdout:0/517: rename d8/da/d25/d26/d35/d3b/d50/f53 to d8/da/d25/d26/d54/fa7 0 2024-03-13T14:49:43.117 INFO:tasks.workunit.client.0.smithi176.stdout:3/336: mknod d4/d13/c69 0 2024-03-13T14:49:43.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fbe14039230 msgr2=0x7fbe1403b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:43.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fbe14039230 0x7fbe1403b6f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c005950 tx=0x7fbe1c0058e0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 msgr2=0x7fbe30193fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:43.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30193fb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bda0 tx=0x7fbe2000be80 comp rx=0 tx=0).stop 2024-03-13T14:49:43.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 shutdown_connections 2024-03-13T14:49:43.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fbe14039230 0x7fbe1403b6f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30100a00 0x7fbe30193fb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 --2- 172.21.15.176:0/435309251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30101c00 0x7fbe301944f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 >> 172.21.15.176:0/435309251 conn(0x7fbe300fbf60 msgr2=0x7fbe30102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:43.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 shutdown_connections 2024-03-13T14:49:43.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.131+0000 7fbe35200700 1 -- 172.21.15.176:0/435309251 wait complete. 2024-03-13T14:49:43.123 INFO:tasks.workunit.client.0.smithi176.stdout:8/293: truncate d0/d24/f31 1321820 0 2024-03-13T14:49:43.126 INFO:tasks.workunit.client.1.smithi196.stdout:8/408: symlink d0/d10/d12/d1e/l8f 0 2024-03-13T14:49:43.126 INFO:tasks.workunit.client.0.smithi176.stdout:5/301: write d5/f6 [1345502,57241] 0 2024-03-13T14:49:43.128 INFO:tasks.workunit.client.1.smithi196.stdout:2/373: truncate d0/d6/d1d/f30 801719 0 2024-03-13T14:49:43.131 INFO:tasks.workunit.client.1.smithi196.stdout:1/480: mknod d1/d9/ca4 0 2024-03-13T14:49:43.132 INFO:tasks.workunit.client.0.smithi176.stdout:6/315: dread d0/d10/f3f [0,4194304] 0 2024-03-13T14:49:43.133 INFO:tasks.workunit.client.0.smithi176.stdout:5/302: fdatasync d5/d2b/f5b 0 2024-03-13T14:49:43.135 INFO:tasks.workunit.client.0.smithi176.stdout:1/328: dwrite d2/d29/f39 [4194304,4194304] 0 2024-03-13T14:49:43.137 INFO:tasks.workunit.client.0.smithi176.stdout:4/351: dwrite d7/f24 [0,4194304] 0 2024-03-13T14:49:43.137 INFO:tasks.workunit.client.1.smithi196.stdout:0/518: mknod d8/d37/ca8 0 2024-03-13T14:49:43.138 INFO:tasks.workunit.client.0.smithi176.stdout:5/303: chown d5/d2c/d4a 285 1 2024-03-13T14:49:43.139 INFO:tasks.workunit.client.1.smithi196.stdout:4/431: creat d1/d3/d7/d35/d3e/f89 x:0 0 0 2024-03-13T14:49:43.140 INFO:tasks.workunit.client.0.smithi176.stdout:6/316: chown d0/d10/d19 10192 1 2024-03-13T14:49:43.146 INFO:tasks.workunit.client.0.smithi176.stdout:6/317: chown d0/d10/d19/d27/c46 7 1 2024-03-13T14:49:43.148 INFO:tasks.workunit.client.0.smithi176.stdout:6/318: dread - d0/de/d38/f67 zero size 2024-03-13T14:49:43.152 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:49:43.156 INFO:tasks.workunit.client.0.smithi176.stdout:5/304: dread d5/d22/f56 [0,4194304] 0 2024-03-13T14:49:43.158 INFO:tasks.workunit.client.1.smithi196.stdout:0/519: dwrite d8/da/d25/d26/d48/d82/f8b [0,4194304] 0 2024-03-13T14:49:43.161 INFO:tasks.workunit.client.1.smithi196.stdout:9/394: rename d7/d24/d28/d6c/f81 to d7/d24/d28/f82 0 2024-03-13T14:49:43.161 INFO:tasks.workunit.client.0.smithi176.stdout:9/393: getdents d0 0 2024-03-13T14:49:43.161 INFO:tasks.workunit.client.0.smithi176.stdout:5/305: dread - d5/d2c/f3d zero size 2024-03-13T14:49:43.165 INFO:tasks.workunit.client.0.smithi176.stdout:2/353: mkdir d1/d51/d7e 0 2024-03-13T14:49:43.168 INFO:tasks.workunit.client.1.smithi196.stdout:9/395: chown d7/d24/f31 432395 1 2024-03-13T14:49:43.168 INFO:tasks.workunit.client.1.smithi196.stdout:5/570: mkdir d4/db/d14/d19/d1c/d25/d38/d47/da0/d63/db2 0 2024-03-13T14:49:43.170 INFO:tasks.workunit.client.0.smithi176.stdout:2/354: write d1/dc/d2d/d38/d5d/f70 [155745,105818] 0 2024-03-13T14:49:43.171 INFO:tasks.workunit.client.0.smithi176.stdout:4/352: unlink d7/d8/d15/d23/c40 0 2024-03-13T14:49:43.173 INFO:tasks.workunit.client.0.smithi176.stdout:2/355: chown d1/dc/d2d/d4b/d69/c6b 1402062536 1 2024-03-13T14:49:43.177 INFO:tasks.workunit.client.0.smithi176.stdout:7/324: creat d1/f61 x:0 0 0 2024-03-13T14:49:43.177 INFO:tasks.workunit.client.1.smithi196.stdout:7/592: dread d0/d6/f45 [0,4194304] 0 2024-03-13T14:49:43.178 INFO:tasks.workunit.client.1.smithi196.stdout:4/432: creat d1/d3/d7/d35/d3e/d56/f8a x:0 0 0 2024-03-13T14:49:43.183 INFO:tasks.workunit.client.0.smithi176.stdout:0/303: creat d2/d5/d28/f56 x:0 0 0 2024-03-13T14:49:43.187 INFO:tasks.workunit.client.1.smithi196.stdout:6/594: creat d6/d2a/dc/d1c/fbb x:0 0 0 2024-03-13T14:49:43.188 INFO:tasks.workunit.client.0.smithi176.stdout:6/319: rename d0/fd to d0/f69 0 2024-03-13T14:49:43.189 INFO:tasks.workunit.client.0.smithi176.stdout:5/306: creat d5/d31/d47/f5c x:0 0 0 2024-03-13T14:49:43.189 INFO:tasks.workunit.client.1.smithi196.stdout:0/520: mkdir d8/da/d25/d26/d54/da9 0 2024-03-13T14:49:43.190 INFO:tasks.workunit.client.1.smithi196.stdout:1/481: dwrite d1/d3f/d5b/f60 [0,4194304] 0 2024-03-13T14:49:43.194 INFO:tasks.workunit.client.1.smithi196.stdout:1/482: chown d1/d9/c83 1 1 2024-03-13T14:49:43.198 INFO:tasks.workunit.client.1.smithi196.stdout:3/480: getdents db/d37 0 2024-03-13T14:49:43.202 INFO:tasks.workunit.client.1.smithi196.stdout:3/481: chown db/d19/f93 2393 1 2024-03-13T14:49:43.206 INFO:tasks.workunit.client.1.smithi196.stdout:2/374: fsync d0/d6/d1d/f30 0 2024-03-13T14:49:43.207 INFO:tasks.workunit.client.0.smithi176.stdout:0/304: creat d2/db/d31/f57 x:0 0 0 2024-03-13T14:49:43.209 INFO:tasks.workunit.client.1.smithi196.stdout:6/595: dwrite d6/d2a/dc/d1c/da7/fb5 [0,4194304] 0 2024-03-13T14:49:43.209 INFO:tasks.workunit.client.1.smithi196.stdout:2/375: chown d0/d7/d9/c75 1988634 1 2024-03-13T14:49:43.211 INFO:tasks.workunit.client.1.smithi196.stdout:6/596: fsync d6/d9/f1f 0 2024-03-13T14:49:43.211 INFO:tasks.workunit.client.0.smithi176.stdout:8/294: creat d0/f5b x:0 0 0 2024-03-13T14:49:43.211 INFO:tasks.workunit.client.0.smithi176.stdout:6/320: mkdir d0/d10/d19/d6a 0 2024-03-13T14:49:43.215 INFO:tasks.workunit.client.0.smithi176.stdout:9/394: rename d0/d13/l66 to d0/d1e/d1f/l7c 0 2024-03-13T14:49:43.218 INFO:tasks.workunit.client.0.smithi176.stdout:5/307: mknod d5/d2b/c5d 0 2024-03-13T14:49:43.218 INFO:tasks.workunit.client.0.smithi176.stdout:6/321: write d0/f4e [594375,72588] 0 2024-03-13T14:49:43.220 INFO:tasks.workunit.client.1.smithi196.stdout:5/571: mkdir d4/db/d14/d19/d26/d33/db3 0 2024-03-13T14:49:43.223 INFO:tasks.workunit.client.0.smithi176.stdout:7/325: creat d1/d1c/d4a/f62 x:0 0 0 2024-03-13T14:49:43.224 INFO:tasks.workunit.client.0.smithi176.stdout:6/322: truncate d0/d10/d19/d27/f68 423108 0 2024-03-13T14:49:43.226 INFO:tasks.workunit.client.1.smithi196.stdout:0/521: unlink d8/da/d25/d26/d35/d3b/d5f/d17/l78 0 2024-03-13T14:49:43.226 INFO:tasks.workunit.client.0.smithi176.stdout:3/337: getdents d4/d13/d16/d3d 0 2024-03-13T14:49:43.229 INFO:tasks.workunit.client.0.smithi176.stdout:3/338: fdatasync d4/d28/d3a/f60 0 2024-03-13T14:49:43.231 INFO:tasks.workunit.client.0.smithi176.stdout:3/339: fsync d4/d13/d16/d3d/f4c 0 2024-03-13T14:49:43.233 INFO:tasks.workunit.client.1.smithi196.stdout:7/593: fsync d0/d6/d33/f3c 0 2024-03-13T14:49:43.233 INFO:tasks.workunit.client.1.smithi196.stdout:2/376: dwrite d0/f77 [0,4194304] 0 2024-03-13T14:49:43.238 INFO:tasks.workunit.client.1.smithi196.stdout:6/597: dwrite d6/d7/df/f2e [0,4194304] 0 2024-03-13T14:49:43.238 INFO:tasks.workunit.client.0.smithi176.stdout:4/353: rename d7/d8/d45/f70 to d7/d10/f83 0 2024-03-13T14:49:43.239 INFO:tasks.workunit.client.0.smithi176.stdout:9/395: creat d0/d13/d4d/d5d/f7d x:0 0 0 2024-03-13T14:49:43.241 INFO:tasks.workunit.client.0.smithi176.stdout:4/354: write d7/d10/f56 [783268,73273] 0 2024-03-13T14:49:43.244 INFO:tasks.workunit.client.0.smithi176.stdout:7/326: dread - d1/d1c/d26/d41/f5a zero size 2024-03-13T14:49:43.245 INFO:tasks.workunit.client.1.smithi196.stdout:6/598: truncate d6/d2a/dc/d1c/fbb 838997 0 2024-03-13T14:49:43.246 INFO:tasks.workunit.client.0.smithi176.stdout:6/323: mkdir d0/de/d38/d6b 0 2024-03-13T14:49:43.247 INFO:tasks.workunit.client.1.smithi196.stdout:1/483: dread d1/d28/f5a [0,4194304] 0 2024-03-13T14:49:43.250 INFO:tasks.workunit.client.1.smithi196.stdout:8/409: getdents d0/d16/d3d/d7f 0 2024-03-13T14:49:43.251 INFO:tasks.workunit.client.1.smithi196.stdout:5/572: dread d4/db/d3c/f84 [0,4194304] 0 2024-03-13T14:49:43.253 INFO:tasks.workunit.client.0.smithi176.stdout:8/295: dwrite d0/d24/d29/f4e [0,4194304] 0 2024-03-13T14:49:43.257 INFO:tasks.workunit.client.0.smithi176.stdout:1/329: sync 2024-03-13T14:49:43.260 INFO:tasks.workunit.client.1.smithi196.stdout:3/482: write db/d19/d22/d30/f69 [90098,78978] 0 2024-03-13T14:49:43.260 INFO:tasks.workunit.client.1.smithi196.stdout:5/573: stat d4/db/d14/d19/d1c/d2d 0 2024-03-13T14:49:43.261 INFO:tasks.workunit.client.0.smithi176.stdout:5/308: mknod d5/d2c/d4a/d53/c5e 0 2024-03-13T14:49:43.267 INFO:tasks.workunit.client.1.smithi196.stdout:2/377: creat d0/d6/d34/f7e x:0 0 0 2024-03-13T14:49:43.270 INFO:tasks.workunit.client.0.smithi176.stdout:4/355: mknod d7/d2d/c84 0 2024-03-13T14:49:43.276 INFO:tasks.workunit.client.1.smithi196.stdout:1/484: creat d1/d9/d33/fa5 x:0 0 0 2024-03-13T14:49:43.278 INFO:tasks.workunit.client.0.smithi176.stdout:4/356: chown d7/d10/d2c/c60 20 1 2024-03-13T14:49:43.279 INFO:tasks.workunit.client.0.smithi176.stdout:6/324: creat d0/f6c x:0 0 0 2024-03-13T14:49:43.282 INFO:tasks.workunit.client.0.smithi176.stdout:3/340: creat d4/d28/d31/d52/f6a x:0 0 0 2024-03-13T14:49:43.283 INFO:tasks.workunit.client.0.smithi176.stdout:9/396: link d0/d1e/f25 d0/d13/d32/d58/d67/f7e 0 2024-03-13T14:49:43.285 INFO:tasks.workunit.client.1.smithi196.stdout:8/410: mknod d0/d16/d71/d68/c90 0 2024-03-13T14:49:43.286 INFO:tasks.workunit.client.0.smithi176.stdout:4/357: fdatasync d7/d8/d33/f6d 0 2024-03-13T14:49:43.287 INFO:tasks.workunit.client.0.smithi176.stdout:1/330: creat d2/de/d3f/d3e/f61 x:0 0 0 2024-03-13T14:49:43.288 INFO:tasks.workunit.client.0.smithi176.stdout:5/309: dread d5/d22/f13 [0,4194304] 0 2024-03-13T14:49:43.290 INFO:tasks.workunit.client.0.smithi176.stdout:0/305: getdents d2/d5/d28 0 2024-03-13T14:49:43.290 INFO:tasks.workunit.client.0.smithi176.stdout:9/397: write d0/d40/f47 [3342108,55386] 0 2024-03-13T14:49:43.291 INFO:tasks.workunit.client.1.smithi196.stdout:5/574: creat d4/db/d14/fb4 x:0 0 0 2024-03-13T14:49:43.292 INFO:tasks.workunit.client.1.smithi196.stdout:7/594: dread d0/d6/d33/d57/f7d [0,4194304] 0 2024-03-13T14:49:43.293 INFO:tasks.workunit.client.0.smithi176.stdout:0/306: chown d2/d5/d28/c36 12496 1 2024-03-13T14:49:43.294 INFO:tasks.workunit.client.1.smithi196.stdout:6/599: dwrite d6/d2a/d23/d28/f6d [0,4194304] 0 2024-03-13T14:49:43.298 INFO:tasks.workunit.client.1.smithi196.stdout:5/575: write d4/f11 [5386172,4769] 0 2024-03-13T14:49:43.298 INFO:tasks.workunit.client.1.smithi196.stdout:4/433: rename d1/d3/c1c to d1/d1e/c8b 0 2024-03-13T14:49:43.299 INFO:tasks.workunit.client.0.smithi176.stdout:2/356: rename d1/dc/d40/f45 to d1/f7f 0 2024-03-13T14:49:43.302 INFO:tasks.workunit.client.1.smithi196.stdout:8/411: creat d0/d16/d3d/d8b/f91 x:0 0 0 2024-03-13T14:49:43.302 INFO:tasks.workunit.client.1.smithi196.stdout:1/485: dwrite d1/d38/f16 [0,4194304] 0 2024-03-13T14:49:43.303 INFO:tasks.workunit.client.0.smithi176.stdout:6/325: mknod d0/d10/d19/d27/d53/c6d 0 2024-03-13T14:49:43.304 INFO:tasks.workunit.client.1.smithi196.stdout:3/483: mkdir db/d15/da1 0 2024-03-13T14:49:43.308 INFO:tasks.workunit.client.0.smithi176.stdout:7/327: dwrite d1/d28/d43/f60 [0,4194304] 0 2024-03-13T14:49:43.308 INFO:tasks.workunit.client.0.smithi176.stdout:6/326: readlink d0/de/d38/l5a 0 2024-03-13T14:49:43.309 INFO:tasks.workunit.client.1.smithi196.stdout:5/576: rename d4/f9d to d4/db/d14/d37/fb5 0 2024-03-13T14:49:43.309 INFO:tasks.workunit.client.1.smithi196.stdout:7/595: creat d0/d9/d31/d77/d93/fc6 x:0 0 0 2024-03-13T14:49:43.311 INFO:tasks.workunit.client.1.smithi196.stdout:4/434: unlink d1/d3/d4c/d72/l7c 0 2024-03-13T14:49:43.312 INFO:tasks.workunit.client.0.smithi176.stdout:1/331: dwrite d2/f9 [0,4194304] 0 2024-03-13T14:49:43.315 INFO:tasks.workunit.client.0.smithi176.stdout:6/327: rename d0 to d0/d9/d6e 22 2024-03-13T14:49:43.316 INFO:tasks.workunit.client.0.smithi176.stdout:4/358: creat d7/d10/f85 x:0 0 0 2024-03-13T14:49:43.317 INFO:tasks.workunit.client.1.smithi196.stdout:1/486: read - d1/d3f/d5b/d74/f65 zero size 2024-03-13T14:49:43.318 INFO:tasks.workunit.client.0.smithi176.stdout:6/328: fdatasync d0/f4e 0 2024-03-13T14:49:43.318 INFO:tasks.workunit.client.0.smithi176.stdout:7/328: chown d1/f3c 85055425 1 2024-03-13T14:49:43.319 INFO:tasks.workunit.client.0.smithi176.stdout:5/310: mkdir d5/d2e/d5f 0 2024-03-13T14:49:43.323 INFO:tasks.workunit.client.0.smithi176.stdout:8/296: creat d0/f5c x:0 0 0 2024-03-13T14:49:43.323 INFO:tasks.workunit.client.0.smithi176.stdout:9/398: chown d0/d1e/d1f/c3b 62857 1 2024-03-13T14:49:43.323 INFO:tasks.workunit.client.0.smithi176.stdout:4/359: fdatasync d7/d10/d46/d4a/d5d/f69 0 2024-03-13T14:49:43.328 INFO:tasks.workunit.client.1.smithi196.stdout:2/378: dwrite d0/d7/d40/f46 [0,4194304] 0 2024-03-13T14:49:43.329 INFO:tasks.workunit.client.1.smithi196.stdout:1/487: write d1/d38/f16 [962217,109119] 0 2024-03-13T14:49:43.329 INFO:tasks.workunit.client.0.smithi176.stdout:4/360: chown d7/d10/d2c/l7e 383968 1 2024-03-13T14:49:43.333 INFO:tasks.workunit.client.1.smithi196.stdout:6/600: dwrite d6/d7/f8d [0,4194304] 0 2024-03-13T14:49:43.333 INFO:tasks.workunit.client.1.smithi196.stdout:7/596: symlink d0/d9/d1e/d58/d72/d6e/lc7 0 2024-03-13T14:49:43.333 INFO:tasks.workunit.client.1.smithi196.stdout:2/379: write d0/d7/d40/f46 [3346295,84097] 0 2024-03-13T14:49:43.335 INFO:tasks.workunit.client.0.smithi176.stdout:5/311: mkdir d5/d31/d60 0 2024-03-13T14:49:43.335 INFO:tasks.workunit.client.0.smithi176.stdout:4/361: fdatasync d7/d8/d33/f7f 0 2024-03-13T14:49:43.336 INFO:tasks.workunit.client.1.smithi196.stdout:6/601: read d6/d57/fb0 [1612757,59247] 0 2024-03-13T14:49:43.336 INFO:tasks.workunit.client.1.smithi196.stdout:8/412: dread d0/d10/d12/f33 [0,4194304] 0 2024-03-13T14:49:43.336 INFO:tasks.workunit.client.0.smithi176.stdout:7/329: read d1/d3f/d4b/f58 [2423520,9417] 0 2024-03-13T14:49:43.338 INFO:tasks.workunit.client.0.smithi176.stdout:6/329: truncate d0/d10/d19/d27/f4a 161504 0 2024-03-13T14:49:43.338 INFO:tasks.workunit.client.0.smithi176.stdout:3/341: link d4/d28/d3a/l5f d4/d13/d2d/d50/l6b 0 2024-03-13T14:49:43.340 INFO:tasks.workunit.client.0.smithi176.stdout:9/399: mknod d0/d73/c7f 0 2024-03-13T14:49:43.342 INFO:tasks.workunit.client.0.smithi176.stdout:2/357: dwrite d1/d2a/d33/f5b [0,4194304] 0 2024-03-13T14:49:43.351 INFO:tasks.workunit.client.1.smithi196.stdout:5/577: dwrite d4/db/d4a/fa3 [0,4194304] 0 2024-03-13T14:49:43.356 INFO:tasks.workunit.client.1.smithi196.stdout:1/488: fsync d1/d9/d33/d4b/f22 0 2024-03-13T14:49:43.358 INFO:tasks.workunit.client.1.smithi196.stdout:1/489: fsync d1/d9/f99 0 2024-03-13T14:49:43.360 INFO:tasks.workunit.client.1.smithi196.stdout:2/380: rename d0/f25 to d0/d6/d1d/f7f 0 2024-03-13T14:49:43.362 INFO:tasks.workunit.client.1.smithi196.stdout:1/490: write d1/d3f/d5b/d74/f8f [142076,4526] 0 2024-03-13T14:49:43.365 INFO:tasks.workunit.client.0.smithi176.stdout:2/358: fsync d1/dc/d2d/d38/d5d/f70 0 2024-03-13T14:49:43.365 INFO:tasks.workunit.client.0.smithi176.stdout:6/330: write d0/de/d38/f67 [1017629,53129] 0 2024-03-13T14:49:43.366 INFO:tasks.workunit.client.0.smithi176.stdout:4/362: dread - d7/d10/d46/f4e zero size 2024-03-13T14:49:43.369 INFO:tasks.workunit.client.0.smithi176.stdout:3/342: mknod d4/d13/d2d/d50/c6c 0 2024-03-13T14:49:43.370 INFO:tasks.workunit.client.1.smithi196.stdout:6/602: fsync d6/d7/df/d27/d39/d44/d74/f88 0 2024-03-13T14:49:43.375 INFO:tasks.workunit.client.1.smithi196.stdout:3/484: getdents db/d19/d22/d3a/d68 0 2024-03-13T14:49:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:43 smithi176 ceph-mon[29064]: pgmap v10: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 29 MiB/s rd, 142 MiB/s wr, 262 op/s 2024-03-13T14:49:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:43 smithi176 ceph-mon[29064]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:49:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:43 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:49:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:43 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:49:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:43 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:49:43.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:43 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:49:43.382 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.392+0000 7f33b0000700 1 -- 172.21.15.176:0/1625568418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 msgr2=0x7f33a8106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:43.382 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.392+0000 7f33b0000700 1 --2- 172.21.15.176:0/1625568418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a8106380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f339c009870 tx=0x7f339c009b80 comp rx=0 tx=0).stop 2024-03-13T14:49:43.383 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.393+0000 7f33b0000700 1 -- 172.21.15.176:0/1625568418 shutdown_connections 2024-03-13T14:49:43.383 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.393+0000 7f33b0000700 1 --2- 172.21.15.176:0/1625568418 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a8106380 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.383 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.393+0000 7f33b0000700 1 --2- 172.21.15.176:0/1625568418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a8101660 0x7f33a8103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.383 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.393+0000 7f33b0000700 1 -- 172.21.15.176:0/1625568418 >> 172.21.15.176:0/1625568418 conn(0x7f33a80fb050 msgr2=0x7f33a80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:43.383 INFO:tasks.workunit.client.0.smithi176.stdout:4/363: write d7/d8/d45/f76 [324200,57671] 0 2024-03-13T14:49:43.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.396+0000 7f33b0000700 1 -- 172.21.15.176:0/1625568418 shutdown_connections 2024-03-13T14:49:43.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.396+0000 7f33b0000700 1 -- 172.21.15.176:0/1625568418 wait complete. 2024-03-13T14:49:43.386 INFO:tasks.workunit.client.0.smithi176.stdout:1/332: sync 2024-03-13T14:49:43.386 INFO:tasks.workunit.client.1.smithi196.stdout:8/413: symlink d0/d10/l92 0 2024-03-13T14:49:43.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.399+0000 7f33b0000700 1 Processor -- start 2024-03-13T14:49:43.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.399+0000 7f33b0000700 1 -- start start 2024-03-13T14:49:43.388 INFO:tasks.workunit.client.0.smithi176.stdout:5/312: dwrite d5/d48/f5a [0,4194304] 0 2024-03-13T14:49:43.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.399+0000 7f33b0000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a8101660 0x7f33a8194180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:43.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.399+0000 7f33b0000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a81946c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:43.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.399+0000 7f33b0000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8194ce0 con 0x7f33a8103f90 2024-03-13T14:49:43.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.399+0000 7f33b0000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a8194e20 con 0x7f33a8101660 2024-03-13T14:49:43.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.400+0000 7f33acc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a81946c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:43.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.400+0000 7f33acc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a81946c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43242/0 (socket says 172.21.15.176:43242) 2024-03-13T14:49:43.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.400+0000 7f33acc00700 1 -- 172.21.15.176:0/1930331437 learned_addr learned my addr 172.21.15.176:0/1930331437 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:43.390 INFO:tasks.workunit.client.1.smithi196.stdout:0/522: dwrite d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 [0,4194304] 0 2024-03-13T14:49:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.400+0000 7f33ad600700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a8101660 0x7f33a8194180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.401+0000 7f33acc00700 1 -- 172.21.15.176:0/1930331437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a8101660 msgr2=0x7f33a8194180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.401+0000 7f33acc00700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a8101660 0x7f33a8194180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.401+0000 7f33acc00700 1 -- 172.21.15.176:0/1930331437 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c009520 con 0x7f33a8103f90 2024-03-13T14:49:43.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.401+0000 7f33acc00700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a81946c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f339c00f690 tx=0x7f339c00f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:43.392 INFO:tasks.workunit.client.0.smithi176.stdout:8/297: dread d0/db/f25 [0,4194304] 0 2024-03-13T14:49:43.392 INFO:tasks.workunit.client.0.smithi176.stdout:3/343: rename d4/d13/d2d/f3b to d4/d28/d3a/f6d 0 2024-03-13T14:49:43.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.402+0000 7f33a6000700 1 -- 172.21.15.176:0/1930331437 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339c01c070 con 0x7f33a8103f90 2024-03-13T14:49:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.402+0000 7f33a6000700 1 -- 172.21.15.176:0/1930331437 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f339c00fc70 con 0x7f33a8103f90 2024-03-13T14:49:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.402+0000 7f33a6000700 1 -- 172.21.15.176:0/1930331437 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339c017620 con 0x7f33a8103f90 2024-03-13T14:49:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.403+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a8199870 con 0x7f33a8103f90 2024-03-13T14:49:43.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.403+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a8199de0 con 0x7f33a8103f90 2024-03-13T14:49:43.396 INFO:tasks.workunit.client.1.smithi196.stdout:4/435: dread d1/d3/d7/d35/f37 [0,4194304] 0 2024-03-13T14:49:43.396 INFO:tasks.workunit.client.0.smithi176.stdout:2/359: read d1/f35 [3307135,111904] 0 2024-03-13T14:49:43.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.409+0000 7f33a6000700 1 -- 172.21.15.176:0/1930331437 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f339c017890 con 0x7f33a8103f90 2024-03-13T14:49:43.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.409+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3388005320 con 0x7f33a8103f90 2024-03-13T14:49:43.399 INFO:tasks.workunit.client.0.smithi176.stdout:1/333: read d2/de/f4d [297219,39313] 0 2024-03-13T14:49:43.402 INFO:tasks.workunit.client.0.smithi176.stdout:9/400: dread d0/d1e/d1f/f38 [0,4194304] 0 2024-03-13T14:49:43.404 INFO:tasks.workunit.client.0.smithi176.stdout:4/364: rmdir d7/d8/d2f 39 2024-03-13T14:49:43.404 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.415+0000 7f33a6000700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f339006ddf0 0x7f33900702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:43.404 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.415+0000 7f33a6000700 1 -- 172.21.15.176:0/1930331437 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f339c090230 con 0x7f33a8103f90 2024-03-13T14:49:43.404 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.415+0000 7f33ad600700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f339006ddf0 0x7f33900702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:43.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.417+0000 7f33ad600700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f339006ddf0 0x7f33900702b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e7e0 tx=0x7f33a0011040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:43.408 INFO:tasks.workunit.client.1.smithi196.stdout:8/414: stat d0/d10/d12/d17/l72 0 2024-03-13T14:49:43.409 INFO:tasks.workunit.client.1.smithi196.stdout:2/381: unlink d0/c2e 0 2024-03-13T14:49:43.414 INFO:tasks.workunit.client.1.smithi196.stdout:7/597: dread d0/d6/f2c [0,4194304] 0 2024-03-13T14:49:43.415 INFO:tasks.workunit.client.0.smithi176.stdout:7/330: dwrite d1/d28/d3a/f5c [0,4194304] 0 2024-03-13T14:49:43.416 INFO:tasks.workunit.client.1.smithi196.stdout:7/598: write d0/d6/f26 [1045302,82541] 0 2024-03-13T14:49:43.417 INFO:tasks.workunit.client.1.smithi196.stdout:0/523: mkdir d8/da/d25/d26/d35/d3b/d5f/d17/daa 0 2024-03-13T14:49:43.418 INFO:tasks.workunit.client.0.smithi176.stdout:0/307: dwrite d2/d5/d28/f29 [0,4194304] 0 2024-03-13T14:49:43.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.424+0000 7f33a6000700 1 -- 172.21.15.176:0/1930331437 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f339c05d470 con 0x7f33a8103f90 2024-03-13T14:49:43.420 INFO:tasks.workunit.client.0.smithi176.stdout:6/331: rename d0/f18 to d0/d9/f6f 0 2024-03-13T14:49:43.425 INFO:tasks.workunit.client.1.smithi196.stdout:3/485: truncate db/d19/d22/f3c 2850374 0 2024-03-13T14:49:43.427 INFO:tasks.workunit.client.0.smithi176.stdout:2/360: write d1/dc/f34 [1242259,22625] 0 2024-03-13T14:49:43.427 INFO:tasks.workunit.client.1.smithi196.stdout:6/603: mkdir d6/d7/dd/dbc 0 2024-03-13T14:49:43.428 INFO:tasks.workunit.client.0.smithi176.stdout:3/344: creat d4/d28/d34/f6e x:0 0 0 2024-03-13T14:49:43.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:43 smithi196 ceph-mon[37486]: pgmap v10: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 29 MiB/s rd, 142 MiB/s wr, 262 op/s 2024-03-13T14:49:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:43 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:49:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:43 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:49:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:43 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:49:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:43 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:49:43.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:43 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3736007843' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:49:43.431 INFO:tasks.workunit.client.1.smithi196.stdout:4/436: chown d1/d3/c31 1156 1 2024-03-13T14:49:43.431 INFO:tasks.workunit.client.1.smithi196.stdout:1/491: dwrite d1/d9/d33/f46 [0,4194304] 0 2024-03-13T14:49:43.433 INFO:tasks.workunit.client.1.smithi196.stdout:8/415: creat d0/d16/d3d/d8b/d3f/f93 x:0 0 0 2024-03-13T14:49:43.434 INFO:tasks.workunit.client.0.smithi176.stdout:5/313: creat d5/d31/f61 x:0 0 0 2024-03-13T14:49:43.445 INFO:tasks.workunit.client.0.smithi176.stdout:9/401: mknod d0/d1e/d1f/d68/c80 0 2024-03-13T14:49:43.446 INFO:tasks.workunit.client.1.smithi196.stdout:2/382: link d0/d6/d4b/f79 d0/d7/d9/d42/f80 0 2024-03-13T14:49:43.446 INFO:tasks.workunit.client.0.smithi176.stdout:4/365: dwrite d7/d8/d33/f6d [0,4194304] 0 2024-03-13T14:49:43.448 INFO:tasks.workunit.client.1.smithi196.stdout:2/383: readlink d0/d7/l11 0 2024-03-13T14:49:43.448 INFO:tasks.workunit.client.1.smithi196.stdout:8/416: creat d0/d63/f94 x:0 0 0 2024-03-13T14:49:43.451 INFO:tasks.workunit.client.1.smithi196.stdout:2/384: stat d0/f77 0 2024-03-13T14:49:43.451 INFO:tasks.workunit.client.1.smithi196.stdout:4/437: dread d1/d3/d7/d35/f63 [0,4194304] 0 2024-03-13T14:49:43.451 INFO:tasks.workunit.client.1.smithi196.stdout:1/492: mkdir d1/d9/d4e/d55/d63/da6 0 2024-03-13T14:49:43.452 INFO:tasks.workunit.client.0.smithi176.stdout:5/314: truncate d5/d2b/f5b 1005032 0 2024-03-13T14:49:43.453 INFO:tasks.workunit.client.1.smithi196.stdout:0/524: creat d8/da/d25/d26/d54/da9/fab x:0 0 0 2024-03-13T14:49:43.454 INFO:tasks.workunit.client.0.smithi176.stdout:6/332: mkdir d0/de/d38/d70 0 2024-03-13T14:49:43.455 INFO:tasks.workunit.client.0.smithi176.stdout:0/308: symlink d2/d5/d1e/l58 0 2024-03-13T14:49:43.456 INFO:tasks.workunit.client.0.smithi176.stdout:0/309: readlink d2/l14 0 2024-03-13T14:49:43.456 INFO:tasks.workunit.client.0.smithi176.stdout:5/315: write d5/f6 [1445160,48508] 0 2024-03-13T14:49:43.458 INFO:tasks.workunit.client.0.smithi176.stdout:3/345: readlink d4/d28/d3a/l5f 0 2024-03-13T14:49:43.458 INFO:tasks.workunit.client.0.smithi176.stdout:8/298: symlink d0/d5/l5d 0 2024-03-13T14:49:43.460 INFO:tasks.workunit.client.0.smithi176.stdout:5/316: dread - d5/d2e/f58 zero size 2024-03-13T14:49:43.460 INFO:tasks.workunit.client.0.smithi176.stdout:2/361: rename d1/d51 to d1/dc/d2d/d59/d80 0 2024-03-13T14:49:43.463 INFO:tasks.workunit.client.1.smithi196.stdout:2/385: creat d0/d6/d4b/d5f/f81 x:0 0 0 2024-03-13T14:49:43.469 INFO:tasks.workunit.client.1.smithi196.stdout:2/386: chown d0/d6/l58 170 1 2024-03-13T14:49:43.471 INFO:tasks.workunit.client.1.smithi196.stdout:1/493: dwrite d1/d28/d34/f8e [4194304,4194304] 0 2024-03-13T14:49:43.473 INFO:tasks.workunit.client.1.smithi196.stdout:2/387: write d0/d6/d1d/f5e [505405,90874] 0 2024-03-13T14:49:43.473 INFO:tasks.workunit.client.1.smithi196.stdout:1/494: readlink d1/d28/d34/d67/l6b 0 2024-03-13T14:49:43.473 INFO:tasks.workunit.client.0.smithi176.stdout:9/402: rmdir d0/d17 39 2024-03-13T14:49:43.477 INFO:tasks.workunit.client.1.smithi196.stdout:0/525: getdents d8/da/d25/d26 0 2024-03-13T14:49:43.481 INFO:tasks.workunit.client.0.smithi176.stdout:5/317: fdatasync d5/d31/f3e 0 2024-03-13T14:49:43.483 INFO:tasks.workunit.client.1.smithi196.stdout:2/388: mknod d0/d6/d34/d5b/c82 0 2024-03-13T14:49:43.483 INFO:tasks.workunit.client.1.smithi196.stdout:0/526: write d8/da/d25/d26/d35/d3b/d5f/d33/d44/f86 [2160434,108374] 0 2024-03-13T14:49:43.484 INFO:tasks.workunit.client.0.smithi176.stdout:3/346: stat d4/d28/d31/d45/c44 0 2024-03-13T14:49:43.486 INFO:tasks.workunit.client.0.smithi176.stdout:2/362: symlink d1/d2a/d33/l81 0 2024-03-13T14:49:43.486 INFO:tasks.workunit.client.1.smithi196.stdout:2/389: chown d0/d6/d1d/f7a 9 1 2024-03-13T14:49:43.487 INFO:tasks.workunit.client.0.smithi176.stdout:3/347: stat d4/d28/d3a/d5b 0 2024-03-13T14:49:43.487 INFO:tasks.workunit.client.0.smithi176.stdout:4/366: creat d7/d75/f86 x:0 0 0 2024-03-13T14:49:43.487 INFO:tasks.workunit.client.0.smithi176.stdout:1/334: link d2/de/d10/d16/l46 d2/de/d10/d14/d22/l62 0 2024-03-13T14:49:43.488 INFO:tasks.workunit.client.1.smithi196.stdout:0/527: symlink d8/da/d25/lac 0 2024-03-13T14:49:43.488 INFO:tasks.workunit.client.0.smithi176.stdout:2/363: chown d1/dc/d61 22864 1 2024-03-13T14:49:43.489 INFO:tasks.workunit.client.1.smithi196.stdout:3/486: dread db/d37/f76 [0,4194304] 0 2024-03-13T14:49:43.490 INFO:tasks.workunit.client.0.smithi176.stdout:5/318: creat d5/d48/d49/f62 x:0 0 0 2024-03-13T14:49:43.490 INFO:tasks.workunit.client.1.smithi196.stdout:1/495: dwrite d1/d9/d33/f42 [0,4194304] 0 2024-03-13T14:49:43.490 INFO:tasks.workunit.client.1.smithi196.stdout:2/390: mkdir d0/d7/d9/d61/d83 0 2024-03-13T14:49:43.491 INFO:tasks.workunit.client.1.smithi196.stdout:2/391: chown d0 1222480107 1 2024-03-13T14:49:43.492 INFO:tasks.workunit.client.0.smithi176.stdout:6/333: rmdir d0/d10/d19/d6a 0 2024-03-13T14:49:43.496 INFO:tasks.workunit.client.1.smithi196.stdout:1/496: mknod d1/d9/d4e/d55/d63/ca7 0 2024-03-13T14:49:43.497 INFO:tasks.workunit.client.0.smithi176.stdout:8/299: write d0/db/dc/d40/f46 [832522,98054] 0 2024-03-13T14:49:43.498 INFO:tasks.workunit.client.1.smithi196.stdout:1/497: chown d1/d9/d4e/c8a 3369711 1 2024-03-13T14:49:43.499 INFO:tasks.workunit.client.1.smithi196.stdout:0/528: mknod d8/cad 0 2024-03-13T14:49:43.499 INFO:tasks.workunit.client.0.smithi176.stdout:0/310: getdents d2/db/d31 0 2024-03-13T14:49:43.502 INFO:tasks.workunit.client.0.smithi176.stdout:8/300: readlink d0/d5/l5d 0 2024-03-13T14:49:43.503 INFO:tasks.workunit.client.0.smithi176.stdout:3/348: rename d4/c2c to d4/d28/d3a/d5b/c6f 0 2024-03-13T14:49:43.504 INFO:tasks.workunit.client.1.smithi196.stdout:7/599: dread d0/d9/d1e/d58/d72/da9/d47/fa7 [0,4194304] 0 2024-03-13T14:49:43.505 INFO:tasks.workunit.client.1.smithi196.stdout:2/392: creat d0/d6/f84 x:0 0 0 2024-03-13T14:49:43.508 INFO:tasks.workunit.client.1.smithi196.stdout:3/487: rename db/d19/d22/f84 to db/d19/fa2 0 2024-03-13T14:49:43.508 INFO:tasks.workunit.client.0.smithi176.stdout:5/319: creat d5/f63 x:0 0 0 2024-03-13T14:49:43.508 INFO:tasks.workunit.client.0.smithi176.stdout:0/311: write d2/db/d31/f51 [93413,41195] 0 2024-03-13T14:49:43.509 INFO:tasks.workunit.client.1.smithi196.stdout:7/600: write d0/d9/d1e/d58/d72/da9/da3/fad [5149545,118922] 0 2024-03-13T14:49:43.510 INFO:tasks.workunit.client.1.smithi196.stdout:1/498: link d1/d3f/d5b/f60 d1/d9/d33/d4b/d66/fa8 0 2024-03-13T14:49:43.510 INFO:tasks.workunit.client.1.smithi196.stdout:9/396: sync 2024-03-13T14:49:43.511 INFO:tasks.workunit.client.0.smithi176.stdout:0/312: write d2/d5/f46 [1811973,91242] 0 2024-03-13T14:49:43.515 INFO:tasks.workunit.client.1.smithi196.stdout:3/488: mknod db/d37/ca3 0 2024-03-13T14:49:43.520 INFO:tasks.workunit.client.0.smithi176.stdout:2/364: dread d1/dc/d2d/d4b/f63 [0,4194304] 0 2024-03-13T14:49:43.521 INFO:tasks.workunit.client.1.smithi196.stdout:1/499: creat d1/d9/d33/d4b/d66/d7b/fa9 x:0 0 0 2024-03-13T14:49:43.521 INFO:tasks.workunit.client.1.smithi196.stdout:3/489: truncate db/d19/d22/f98 699680 0 2024-03-13T14:49:43.521 INFO:tasks.workunit.client.1.smithi196.stdout:1/500: stat d1/d28/ca0 0 2024-03-13T14:49:43.521 INFO:tasks.workunit.client.1.smithi196.stdout:7/601: write d0/d9/d1e/d58/d61/da0/fb7 [5333309,49724] 0 2024-03-13T14:49:43.521 INFO:tasks.workunit.client.1.smithi196.stdout:9/397: mkdir d7/d9/d83 0 2024-03-13T14:49:43.522 INFO:tasks.workunit.client.0.smithi176.stdout:0/313: mkdir d2/d5/dc/d59 0 2024-03-13T14:49:43.523 INFO:tasks.workunit.client.1.smithi196.stdout:9/398: chown d7/de/d16 5061 1 2024-03-13T14:49:43.523 INFO:tasks.workunit.client.0.smithi176.stdout:5/320: creat d5/f64 x:0 0 0 2024-03-13T14:49:43.524 INFO:tasks.workunit.client.0.smithi176.stdout:2/365: unlink d1/l11 0 2024-03-13T14:49:43.525 INFO:tasks.workunit.client.1.smithi196.stdout:9/399: write d7/d9/f77 [332225,51798] 0 2024-03-13T14:49:43.526 INFO:tasks.workunit.client.1.smithi196.stdout:7/602: unlink d0/d9/d31/lc0 0 2024-03-13T14:49:43.528 INFO:tasks.workunit.client.0.smithi176.stdout:2/366: creat d1/d2a/f82 x:0 0 0 2024-03-13T14:49:43.529 INFO:tasks.workunit.client.0.smithi176.stdout:0/314: creat d2/d5/dc/d59/f5a x:0 0 0 2024-03-13T14:49:43.529 INFO:tasks.workunit.client.1.smithi196.stdout:9/400: mknod d7/d9/d83/c84 0 2024-03-13T14:49:43.530 INFO:tasks.workunit.client.0.smithi176.stdout:3/349: dwrite d4/d13/f19 [0,4194304] 0 2024-03-13T14:49:43.531 INFO:tasks.workunit.client.1.smithi196.stdout:7/603: creat d0/d4d/fc8 x:0 0 0 2024-03-13T14:49:43.532 INFO:tasks.workunit.client.0.smithi176.stdout:2/367: symlink d1/dc/d2d/d4f/l83 0 2024-03-13T14:49:43.533 INFO:tasks.workunit.client.0.smithi176.stdout:5/321: link d5/d48/f5a d5/d48/d4c/f65 0 2024-03-13T14:49:43.534 INFO:tasks.workunit.client.0.smithi176.stdout:3/350: fdatasync d4/f39 0 2024-03-13T14:49:43.535 INFO:tasks.workunit.client.1.smithi196.stdout:7/604: write d0/d6/d33/f9a [211596,109529] 0 2024-03-13T14:49:43.535 INFO:tasks.workunit.client.0.smithi176.stdout:5/322: write d5/f63 [197494,83297] 0 2024-03-13T14:49:43.536 INFO:tasks.workunit.client.0.smithi176.stdout:2/368: creat d1/dc/d2d/d59/d80/d77/f84 x:0 0 0 2024-03-13T14:49:43.537 INFO:tasks.workunit.client.0.smithi176.stdout:3/351: chown d4/d13/d16/d3d/f5d 62694617 1 2024-03-13T14:49:43.538 INFO:tasks.workunit.client.1.smithi196.stdout:9/401: getdents d7/d24/d28 0 2024-03-13T14:49:43.539 INFO:tasks.workunit.client.0.smithi176.stdout:2/369: truncate d1/dc/f2e 4612897 0 2024-03-13T14:49:43.539 INFO:tasks.workunit.client.0.smithi176.stdout:5/323: creat d5/d2c/f66 x:0 0 0 2024-03-13T14:49:43.541 INFO:tasks.workunit.client.1.smithi196.stdout:9/402: creat d7/de/d74/f85 x:0 0 0 2024-03-13T14:49:43.543 INFO:tasks.workunit.client.0.smithi176.stdout:3/352: symlink d4/d28/d3a/l70 0 2024-03-13T14:49:43.544 INFO:tasks.workunit.client.0.smithi176.stdout:5/324: creat d5/d22/d15/f67 x:0 0 0 2024-03-13T14:49:43.545 INFO:tasks.workunit.client.0.smithi176.stdout:2/370: symlink d1/dc/d40/l85 0 2024-03-13T14:49:43.546 INFO:tasks.workunit.client.0.smithi176.stdout:3/353: stat d4/d28/c56 0 2024-03-13T14:49:43.546 INFO:tasks.workunit.client.0.smithi176.stdout:5/325: stat d5/d22/l12 0 2024-03-13T14:49:43.551 INFO:tasks.workunit.client.0.smithi176.stdout:2/371: creat d1/dc/d2d/d59/f86 x:0 0 0 2024-03-13T14:49:43.551 INFO:tasks.workunit.client.1.smithi196.stdout:7/605: dwrite d0/d9/d1e/d58/d72/d6e/d7f/f9c [4194304,4194304] 0 2024-03-13T14:49:43.558 INFO:tasks.workunit.client.0.smithi176.stdout:2/372: write d1/dc/d2d/d38/f3f [4934944,93359] 0 2024-03-13T14:49:43.558 INFO:tasks.workunit.client.1.smithi196.stdout:7/606: unlink d0/d9/d1e/d58/d61/c88 0 2024-03-13T14:49:43.568 INFO:tasks.workunit.client.1.smithi196.stdout:7/607: dwrite d0/d4d/f5b [4194304,4194304] 0 2024-03-13T14:49:43.570 INFO:tasks.workunit.client.1.smithi196.stdout:7/608: fdatasync d0/d9/d1e/d58/d72/d99/fb8 0 2024-03-13T14:49:43.570 INFO:tasks.workunit.client.0.smithi176.stdout:5/326: dwrite d5/f55 [0,4194304] 0 2024-03-13T14:49:43.575 INFO:tasks.workunit.client.0.smithi176.stdout:3/354: dwrite f2 [8388608,4194304] 0 2024-03-13T14:49:43.580 INFO:tasks.workunit.client.0.smithi176.stdout:5/327: mknod d5/d48/c68 0 2024-03-13T14:49:43.588 INFO:tasks.workunit.client.0.smithi176.stdout:5/328: symlink d5/d2b/l69 0 2024-03-13T14:49:43.590 INFO:tasks.workunit.client.0.smithi176.stdout:2/373: dwrite d1/dc/d2d/d38/f5a [0,4194304] 0 2024-03-13T14:49:43.602 INFO:tasks.workunit.client.0.smithi176.stdout:2/374: mknod d1/dc/d1c/c87 0 2024-03-13T14:49:43.607 INFO:tasks.workunit.client.0.smithi176.stdout:3/355: dwrite d4/d13/f48 [4194304,4194304] 0 2024-03-13T14:49:43.631 INFO:tasks.workunit.client.0.smithi176.stdout:8/301: sync 2024-03-13T14:49:43.633 INFO:tasks.workunit.client.0.smithi176.stdout:8/302: chown d0/d5/f20 1476 1 2024-03-13T14:49:43.638 INFO:tasks.workunit.client.0.smithi176.stdout:8/303: creat d0/db/d39/d56/f5e x:0 0 0 2024-03-13T14:49:43.640 INFO:tasks.workunit.client.0.smithi176.stdout:8/304: chown d0/db/l3e 28599522 1 2024-03-13T14:49:43.644 INFO:tasks.workunit.client.0.smithi176.stdout:8/305: getdents d0/db/d58 0 2024-03-13T14:49:43.683 INFO:tasks.workunit.client.1.smithi196.stdout:8/417: dwrite d0/d16/d3d/d8b/d48/f5a [0,4194304] 0 2024-03-13T14:49:43.688 INFO:tasks.workunit.client.1.smithi196.stdout:8/418: fsync d0/f61 0 2024-03-13T14:49:43.689 INFO:tasks.workunit.client.1.smithi196.stdout:8/419: chown d0/d1c/d29/d47/f80 2 1 2024-03-13T14:49:43.690 INFO:tasks.workunit.client.0.smithi176.stdout:0/315: sync 2024-03-13T14:49:43.692 INFO:tasks.workunit.client.0.smithi176.stdout:8/306: dread d0/db/dc/f35 [0,4194304] 0 2024-03-13T14:49:43.693 INFO:tasks.workunit.client.0.smithi176.stdout:1/335: rename d2/de/d10/d14 to d2/d30/d63 0 2024-03-13T14:49:43.695 INFO:tasks.workunit.client.0.smithi176.stdout:0/316: unlink d2/l14 0 2024-03-13T14:49:43.697 INFO:tasks.workunit.client.0.smithi176.stdout:7/331: write d1/d1c/d26/d41/f3e [602128,71380] 0 2024-03-13T14:49:43.699 INFO:tasks.workunit.client.0.smithi176.stdout:7/332: write d1/d1c/f5e [78311,57217] 0 2024-03-13T14:49:43.701 INFO:tasks.workunit.client.0.smithi176.stdout:7/333: dread - d1/d3f/f4d zero size 2024-03-13T14:49:43.712 INFO:tasks.workunit.client.0.smithi176.stdout:5/329: rename d5/d2c/d4a to d5/d2c/d40/d6a 0 2024-03-13T14:49:43.712 INFO:tasks.workunit.client.0.smithi176.stdout:4/367: getdents d7/d75 0 2024-03-13T14:49:43.717 INFO:tasks.workunit.client.0.smithi176.stdout:8/307: truncate d0/db/fd 5242562 0 2024-03-13T14:49:43.717 INFO:tasks.workunit.client.0.smithi176.stdout:0/317: symlink d2/db/d31/l5b 0 2024-03-13T14:49:43.720 INFO:tasks.workunit.client.0.smithi176.stdout:7/334: mkdir d1/d3f/d4b/d63 0 2024-03-13T14:49:43.725 INFO:tasks.workunit.client.0.smithi176.stdout:2/375: rename d1/l1b to d1/dc/d2d/d59/d80/d5c/l88 0 2024-03-13T14:49:43.727 INFO:tasks.workunit.client.1.smithi196.stdout:0/529: dwrite d8/da/fc [4194304,4194304] 0 2024-03-13T14:49:43.727 INFO:tasks.workunit.client.0.smithi176.stdout:5/330: mkdir d5/d2c/d40/d6b 0 2024-03-13T14:49:43.732 INFO:tasks.workunit.client.0.smithi176.stdout:9/403: dwrite d0/d1e/f25 [0,4194304] 0 2024-03-13T14:49:43.740 INFO:tasks.workunit.client.0.smithi176.stdout:6/334: truncate d0/d9/f6f 2314133 0 2024-03-13T14:49:43.741 INFO:tasks.workunit.client.0.smithi176.stdout:2/376: symlink d1/dc/d2d/d4b/l89 0 2024-03-13T14:49:43.745 INFO:tasks.workunit.client.0.smithi176.stdout:6/335: truncate d0/d10/d19/d27/d2b/f39 168703 0 2024-03-13T14:49:43.747 INFO:tasks.workunit.client.1.smithi196.stdout:2/393: dwrite d0/d6/d4b/d5f/f3b [0,4194304] 0 2024-03-13T14:49:43.748 INFO:tasks.workunit.client.0.smithi176.stdout:6/336: chown d0/d10/d19/d27/d2b/d2e/f62 3697 1 2024-03-13T14:49:43.754 INFO:tasks.workunit.client.1.smithi196.stdout:1/501: write d1/d38/f25 [2086902,64558] 0 2024-03-13T14:49:43.756 INFO:tasks.workunit.client.1.smithi196.stdout:9/403: getdents d7/d9/d83 0 2024-03-13T14:49:43.758 INFO:tasks.workunit.client.1.smithi196.stdout:9/404: dread - d7/de/d16/f60 zero size 2024-03-13T14:49:43.763 INFO:tasks.workunit.client.0.smithi176.stdout:0/318: mknod d2/db/c5c 0 2024-03-13T14:49:43.763 INFO:tasks.workunit.client.0.smithi176.stdout:7/335: rename d1/d28/d43 to d1/d1c/d26/d41/d2b/d40/d64 0 2024-03-13T14:49:43.764 INFO:tasks.workunit.client.1.smithi196.stdout:1/502: dread d1/d28/d34/d67/f72 [0,4194304] 0 2024-03-13T14:49:43.764 INFO:tasks.workunit.client.0.smithi176.stdout:2/377: creat d1/dc/d2d/d59/f8a x:0 0 0 2024-03-13T14:49:43.765 INFO:tasks.workunit.client.0.smithi176.stdout:4/368: link d7/d8/d33/l63 d7/d10/l87 0 2024-03-13T14:49:43.767 INFO:tasks.workunit.client.1.smithi196.stdout:5/578: sync 2024-03-13T14:49:43.768 INFO:tasks.workunit.client.0.smithi176.stdout:9/404: mkdir d0/d13/d4d/d6e/d81 0 2024-03-13T14:49:43.773 INFO:tasks.workunit.client.0.smithi176.stdout:6/337: rename d0/l5b to d0/de/d38/d6b/l71 0 2024-03-13T14:49:43.775 INFO:tasks.workunit.client.1.smithi196.stdout:7/609: getdents d0/d9/d1e/d58/d61 0 2024-03-13T14:49:43.784 INFO:tasks.workunit.client.0.smithi176.stdout:4/369: dwrite d7/d8/d15/d23/f47 [0,4194304] 0 2024-03-13T14:49:43.785 INFO:tasks.workunit.client.0.smithi176.stdout:3/356: write d4/d28/d3a/f2b [820687,101931] 0 2024-03-13T14:49:43.788 INFO:tasks.workunit.client.1.smithi196.stdout:0/530: creat d8/da/d74/fae x:0 0 0 2024-03-13T14:49:43.789 INFO:tasks.workunit.client.0.smithi176.stdout:9/405: mkdir d0/d13/d32/d76/d82 0 2024-03-13T14:49:43.790 INFO:tasks.workunit.client.0.smithi176.stdout:0/319: creat d2/d5/dc/f5d x:0 0 0 2024-03-13T14:49:43.790 INFO:tasks.workunit.client.0.smithi176.stdout:8/308: sync 2024-03-13T14:49:43.791 INFO:tasks.workunit.client.1.smithi196.stdout:2/394: rename d0/d7/d56/f68 to d0/d7/d66/d67/f85 0 2024-03-13T14:49:43.793 INFO:tasks.workunit.client.1.smithi196.stdout:9/405: rmdir d7/d24/d28 39 2024-03-13T14:49:43.795 INFO:tasks.workunit.client.0.smithi176.stdout:4/370: chown d7/d10/d46/d4a/d5d/c6c 45 1 2024-03-13T14:49:43.795 INFO:tasks.workunit.client.1.smithi196.stdout:1/503: read - d1/d9/d4e/f70 zero size 2024-03-13T14:49:43.796 INFO:tasks.workunit.client.0.smithi176.stdout:6/338: dread d0/de/f49 [0,4194304] 0 2024-03-13T14:49:43.802 INFO:tasks.workunit.client.0.smithi176.stdout:0/320: creat d2/d5/d28/f5e x:0 0 0 2024-03-13T14:49:43.802 INFO:tasks.workunit.client.0.smithi176.stdout:4/371: write d7/d10/d46/f4c [176660,80890] 0 2024-03-13T14:49:43.804 INFO:tasks.workunit.client.0.smithi176.stdout:2/378: dwrite d1/dc/d2d/d38/f5e [0,4194304] 0 2024-03-13T14:49:43.806 INFO:tasks.workunit.client.0.smithi176.stdout:8/309: creat d0/d5/d11/f5f x:0 0 0 2024-03-13T14:49:43.808 INFO:tasks.workunit.client.0.smithi176.stdout:9/406: creat d0/d13/d32/d58/f83 x:0 0 0 2024-03-13T14:49:43.809 INFO:tasks.workunit.client.0.smithi176.stdout:0/321: write d2/d5/d1e/f3e [142635,130184] 0 2024-03-13T14:49:43.810 INFO:tasks.workunit.client.1.smithi196.stdout:8/420: dwrite d0/d10/d12/d17/f57 [0,4194304] 0 2024-03-13T14:49:43.812 INFO:tasks.workunit.client.1.smithi196.stdout:2/395: rmdir d0/d7/d66 39 2024-03-13T14:49:43.812 INFO:tasks.workunit.client.0.smithi176.stdout:2/379: chown d1/dc/d2d/d59/d80/d7e 860 1 2024-03-13T14:49:43.815 INFO:tasks.workunit.client.1.smithi196.stdout:2/396: fsync d0/d6/d1d/f27 0 2024-03-13T14:49:43.815 INFO:tasks.workunit.client.0.smithi176.stdout:6/339: read d0/d10/d19/d27/d2b/d2e/f40 [696776,37098] 0 2024-03-13T14:49:43.818 INFO:tasks.workunit.client.0.smithi176.stdout:5/331: rmdir d5 39 2024-03-13T14:49:43.821 INFO:tasks.workunit.client.0.smithi176.stdout:6/340: stat d0/d9/c21 0 2024-03-13T14:49:43.822 INFO:tasks.workunit.client.1.smithi196.stdout:5/579: dwrite d4/db/d14/d37/fb5 [0,4194304] 0 2024-03-13T14:49:43.823 INFO:tasks.workunit.client.0.smithi176.stdout:1/336: write d2/de/d10/d16/d47/f5c [605448,74799] 0 2024-03-13T14:49:43.824 INFO:tasks.workunit.client.1.smithi196.stdout:7/610: rename d0/d9/d1e/d58/d61/da0/fb7 to d0/d78/d81/fc9 0 2024-03-13T14:49:43.826 INFO:tasks.workunit.client.0.smithi176.stdout:8/310: rmdir d0/d24/d29 39 2024-03-13T14:49:43.828 INFO:tasks.workunit.client.0.smithi176.stdout:3/357: dwrite d4/d13/d16/d3d/f5d [0,4194304] 0 2024-03-13T14:49:43.831 INFO:tasks.workunit.client.0.smithi176.stdout:2/380: mkdir d1/dc/d2d/d59/d8b 0 2024-03-13T14:49:43.836 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.844+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3388000bf0 con 0x7f339006ddf0 2024-03-13T14:49:43.836 INFO:tasks.workunit.client.0.smithi176.stdout:6/341: dread - d0/d10/d19/d27/d2b/f5c zero size 2024-03-13T14:49:43.837 INFO:tasks.workunit.client.0.smithi176.stdout:3/358: fsync d4/d13/f48 0 2024-03-13T14:49:43.838 INFO:tasks.workunit.client.0.smithi176.stdout:4/372: rename d7/d10/d46/l49 to d7/d10/d2c/l88 0 2024-03-13T14:49:43.841 INFO:tasks.workunit.client.0.smithi176.stdout:7/336: rmdir d1/d1c/d26/d41/d2b/d40/d64 39 2024-03-13T14:49:43.841 INFO:tasks.workunit.client.1.smithi196.stdout:5/580: creat d4/db/d14/d37/fb6 x:0 0 0 2024-03-13T14:49:43.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.856+0000 7f33a6000700 1 -- 172.21.15.176:0/1930331437 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f3388000bf0 con 0x7f339006ddf0 2024-03-13T14:49:43.845 INFO:tasks.workunit.client.0.smithi176.stdout:1/337: creat d2/de/d10/d16/d4a/d5f/f64 x:0 0 0 2024-03-13T14:49:43.846 INFO:tasks.workunit.client.0.smithi176.stdout:2/381: mkdir d1/dc/d2d/d59/d8c 0 2024-03-13T14:49:43.847 INFO:tasks.workunit.client.1.smithi196.stdout:2/397: symlink d0/l86 0 2024-03-13T14:49:43.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f339006ddf0 msgr2=0x7f33900702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:43.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f339006ddf0 0x7f33900702b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f33a000e7e0 tx=0x7f33a0011040 comp rx=0 tx=0).stop 2024-03-13T14:49:43.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 msgr2=0x7f33a81946c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:43.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a81946c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f339c00f690 tx=0x7f339c00f6c0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 shutdown_connections 2024-03-13T14:49:43.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f339006ddf0 0x7f33900702b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33a8101660 0x7f33a8194180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 --2- 172.21.15.176:0/1930331437 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33a8103f90 0x7f33a81946c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:43.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.861+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 >> 172.21.15.176:0/1930331437 conn(0x7f33a80fb050 msgr2=0x7f33a80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:43.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.862+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 shutdown_connections 2024-03-13T14:49:43.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:43.862+0000 7f33b0000700 1 -- 172.21.15.176:0/1930331437 wait complete. 2024-03-13T14:49:43.851 INFO:tasks.workunit.client.0.smithi176.stdout:1/338: write d2/f6 [1278401,3345] 0 2024-03-13T14:49:43.854 INFO:tasks.workunit.client.1.smithi196.stdout:1/504: getdents d1/d3f 0 2024-03-13T14:49:43.855 INFO:tasks.workunit.client.0.smithi176.stdout:5/332: mknod d5/d2b/c6c 0 2024-03-13T14:49:43.855 INFO:tasks.workunit.client.0.smithi176.stdout:2/382: chown d1/dc/d2d/d59/f86 31788 1 2024-03-13T14:49:43.855 INFO:tasks.workunit.client.0.smithi176.stdout:9/407: rename d0/d1e/d1f/d68/d6a to d0/d13/d32/d58/d5f/d84 0 2024-03-13T14:49:43.856 INFO:tasks.workunit.client.1.smithi196.stdout:2/398: rmdir d0/d7/d56 39 2024-03-13T14:49:43.856 INFO:tasks.workunit.client.0.smithi176.stdout:4/373: creat d7/d75/f89 x:0 0 0 2024-03-13T14:49:43.857 INFO:tasks.workunit.client.0.smithi176.stdout:5/333: fsync d5/d2e/f58 0 2024-03-13T14:49:43.858 INFO:tasks.workunit.client.0.smithi176.stdout:2/383: stat d1/dc/d2d/d4b 0 2024-03-13T14:49:43.859 INFO:tasks.workunit.client.0.smithi176.stdout:0/322: dwrite d2/d5/d1e/f4e [0,4194304] 0 2024-03-13T14:49:43.859 INFO:tasks.workunit.client.1.smithi196.stdout:1/505: fdatasync d1/d38/f76 0 2024-03-13T14:49:43.869 INFO:tasks.workunit.client.0.smithi176.stdout:6/342: rename d0/de/d38/f67 to d0/de/d38/f72 0 2024-03-13T14:49:43.872 INFO:tasks.workunit.client.1.smithi196.stdout:0/531: write d8/da/d25/f27 [4040720,119095] 0 2024-03-13T14:49:43.874 INFO:tasks.workunit.client.1.smithi196.stdout:1/506: truncate d1/d9/d33/d4b/f1a 2965726 0 2024-03-13T14:49:43.877 INFO:tasks.workunit.client.1.smithi196.stdout:7/611: dread d0/f37 [0,4194304] 0 2024-03-13T14:49:43.885 INFO:tasks.workunit.client.0.smithi176.stdout:4/374: read d7/d8/f16 [531029,129196] 0 2024-03-13T14:49:43.885 INFO:tasks.workunit.client.0.smithi176.stdout:5/334: mknod d5/d2c/d40/c6d 0 2024-03-13T14:49:43.886 INFO:tasks.workunit.client.1.smithi196.stdout:8/421: write d0/d10/f26 [388471,77671] 0 2024-03-13T14:49:43.887 INFO:tasks.workunit.client.0.smithi176.stdout:2/384: mkdir d1/dc/d2d/d4b/d8d 0 2024-03-13T14:49:43.890 INFO:tasks.workunit.client.1.smithi196.stdout:1/507: rename d1/d3f/d5b/l5e to d1/d94/laa 0 2024-03-13T14:49:43.890 INFO:tasks.workunit.client.0.smithi176.stdout:9/408: read d0/d1e/d1f/f2c [1333709,95352] 0 2024-03-13T14:49:43.893 INFO:tasks.workunit.client.1.smithi196.stdout:9/406: dwrite d7/de/d4b/f19 [0,4194304] 0 2024-03-13T14:49:43.894 INFO:tasks.workunit.client.0.smithi176.stdout:7/337: dwrite d1/d7/f10 [4194304,4194304] 0 2024-03-13T14:49:43.895 INFO:tasks.workunit.client.1.smithi196.stdout:7/612: truncate d0/d6/d33/f40 1653019 0 2024-03-13T14:49:43.895 INFO:tasks.workunit.client.1.smithi196.stdout:8/422: truncate d0/d16/d71/f89 698343 0 2024-03-13T14:49:43.898 INFO:tasks.workunit.client.1.smithi196.stdout:7/613: fsync d0/d9/f30 0 2024-03-13T14:49:43.899 INFO:tasks.workunit.client.1.smithi196.stdout:8/423: read - d0/d16/d3d/d8b/f91 zero size 2024-03-13T14:49:43.899 INFO:tasks.workunit.client.0.smithi176.stdout:6/343: dwrite d0/d9/f55 [0,4194304] 0 2024-03-13T14:49:43.903 INFO:tasks.workunit.client.1.smithi196.stdout:1/508: creat d1/d9/d6c/fab x:0 0 0 2024-03-13T14:49:43.906 INFO:tasks.workunit.client.1.smithi196.stdout:0/532: dwrite d8/da/d25/d26/d35/d3b/d5f/f4a [0,4194304] 0 2024-03-13T14:49:43.908 INFO:tasks.workunit.client.1.smithi196.stdout:6/604: sync 2024-03-13T14:49:43.908 INFO:tasks.workunit.client.1.smithi196.stdout:3/490: sync 2024-03-13T14:49:43.908 INFO:tasks.workunit.client.1.smithi196.stdout:4/438: sync 2024-03-13T14:49:43.912 INFO:tasks.workunit.client.0.smithi176.stdout:4/375: creat d7/d10/f8a x:0 0 0 2024-03-13T14:49:43.913 INFO:tasks.workunit.client.1.smithi196.stdout:9/407: dread - d7/de/d4b/d1d/d22/f53 zero size 2024-03-13T14:49:43.913 INFO:tasks.workunit.client.0.smithi176.stdout:5/335: creat d5/d2c/d40/d6a/f6e x:0 0 0 2024-03-13T14:49:43.916 INFO:tasks.workunit.client.1.smithi196.stdout:8/424: read d0/d16/d3d/d8b/d48/f5a [4135642,89883] 0 2024-03-13T14:49:43.918 INFO:tasks.workunit.client.1.smithi196.stdout:9/408: read d7/de/d4b/f19 [3386549,124122] 0 2024-03-13T14:49:43.921 INFO:tasks.workunit.client.1.smithi196.stdout:4/439: stat d1/d3/c31 0 2024-03-13T14:49:43.921 INFO:tasks.workunit.client.1.smithi196.stdout:8/425: dread - d0/d10/d12/d1e/f83 zero size 2024-03-13T14:49:43.926 INFO:tasks.workunit.client.1.smithi196.stdout:5/581: dwrite d4/db/d3c/f59 [0,4194304] 0 2024-03-13T14:49:43.927 INFO:tasks.workunit.client.0.smithi176.stdout:1/339: dread d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:49:43.928 INFO:tasks.workunit.client.1.smithi196.stdout:5/582: read - d4/db/d14/d37/fb6 zero size 2024-03-13T14:49:43.930 INFO:tasks.workunit.client.0.smithi176.stdout:9/409: creat d0/d1e/d1f/d68/f85 x:0 0 0 2024-03-13T14:49:43.932 INFO:tasks.workunit.client.1.smithi196.stdout:2/399: write d0/d6/d1d/f7f [4608758,53679] 0 2024-03-13T14:49:43.932 INFO:tasks.workunit.client.0.smithi176.stdout:0/323: symlink d2/d5/dc/l5f 0 2024-03-13T14:49:43.934 INFO:tasks.workunit.client.0.smithi176.stdout:0/324: chown d2/db 34036062 1 2024-03-13T14:49:43.937 INFO:tasks.workunit.client.0.smithi176.stdout:0/325: write d2/db/d31/f4c [4486430,102932] 0 2024-03-13T14:49:43.938 INFO:tasks.workunit.client.0.smithi176.stdout:0/326: readlink d2/db/l3a 0 2024-03-13T14:49:43.939 INFO:tasks.workunit.client.0.smithi176.stdout:8/311: getdents d0/d5/d11 0 2024-03-13T14:49:43.940 INFO:tasks.workunit.client.0.smithi176.stdout:3/359: truncate d4/f1d 814917 0 2024-03-13T14:49:43.940 INFO:tasks.workunit.client.0.smithi176.stdout:0/327: readlink d2/d5/d1e/l45 0 2024-03-13T14:49:43.941 INFO:tasks.workunit.client.0.smithi176.stdout:7/338: mkdir d1/d3f/d65 0 2024-03-13T14:49:43.942 INFO:tasks.workunit.client.1.smithi196.stdout:6/605: creat d6/d2a/dc/d1c/d34/fbd x:0 0 0 2024-03-13T14:49:43.943 INFO:tasks.workunit.client.0.smithi176.stdout:6/344: fsync d0/d10/d19/f1a 0 2024-03-13T14:49:43.944 INFO:tasks.workunit.client.1.smithi196.stdout:9/409: mknod d7/de/d74/c86 0 2024-03-13T14:49:43.945 INFO:tasks.workunit.client.0.smithi176.stdout:4/376: creat d7/d8/d15/d23/f8b x:0 0 0 2024-03-13T14:49:43.947 INFO:tasks.workunit.client.0.smithi176.stdout:5/336: rename d5/f64 to d5/d2e/d5f/f6f 0 2024-03-13T14:49:43.948 INFO:tasks.workunit.client.1.smithi196.stdout:9/410: chown d7/de/d4b/d1d/d22 631 1 2024-03-13T14:49:43.949 INFO:tasks.workunit.client.0.smithi176.stdout:1/340: fsync d2/d29/f3c 0 2024-03-13T14:49:43.951 INFO:tasks.workunit.client.1.smithi196.stdout:9/411: chown d7/de/d4b/d1d/f23 5547 1 2024-03-13T14:49:43.952 INFO:tasks.workunit.client.1.smithi196.stdout:7/614: mkdir d0/d6/db4/dca 0 2024-03-13T14:49:43.960 INFO:tasks.workunit.client.0.smithi176.stdout:3/360: fdatasync d4/d28/f29 0 2024-03-13T14:49:43.960 INFO:tasks.workunit.client.1.smithi196.stdout:1/509: write d1/d9/d33/d4b/f75 [173454,83300] 0 2024-03-13T14:49:43.961 INFO:tasks.workunit.client.0.smithi176.stdout:7/339: dwrite d1/d28/d3a/d1b/d18/f52 [0,4194304] 0 2024-03-13T14:49:43.966 INFO:tasks.workunit.client.0.smithi176.stdout:4/377: fdatasync d7/f3a 0 2024-03-13T14:49:43.966 INFO:tasks.workunit.client.0.smithi176.stdout:4/378: read - d7/d10/d46/d4a/d5d/f69 zero size 2024-03-13T14:49:43.969 INFO:tasks.workunit.client.1.smithi196.stdout:4/440: mknod d1/d1e/c8c 0 2024-03-13T14:49:43.972 INFO:tasks.workunit.client.1.smithi196.stdout:8/426: mknod d0/d10/d70/d8d/c95 0 2024-03-13T14:49:43.973 INFO:tasks.workunit.client.0.smithi176.stdout:5/337: rename d5/d22/f56 to d5/d2b/f70 0 2024-03-13T14:49:43.973 INFO:tasks.workunit.client.0.smithi176.stdout:3/361: dread d4/d28/d31/f4b [0,4194304] 0 2024-03-13T14:49:43.975 INFO:tasks.workunit.client.0.smithi176.stdout:1/341: rmdir d2/de/d10/d16/d4a 39 2024-03-13T14:49:43.976 INFO:tasks.workunit.client.1.smithi196.stdout:7/615: symlink d0/d9/d1e/d58/d72/da9/d47/lcb 0 2024-03-13T14:49:43.979 INFO:tasks.workunit.client.1.smithi196.stdout:5/583: creat d4/db/d14/d19/d26/dab/fb7 x:0 0 0 2024-03-13T14:49:43.979 INFO:tasks.workunit.client.1.smithi196.stdout:8/427: read d0/d10/f21 [1216740,102668] 0 2024-03-13T14:49:43.981 INFO:tasks.workunit.client.0.smithi176.stdout:0/328: unlink d2/d5/dc/f2e 0 2024-03-13T14:49:43.981 INFO:tasks.workunit.client.0.smithi176.stdout:8/312: dwrite d0/d5/d3b/f41 [0,4194304] 0 2024-03-13T14:49:43.983 INFO:tasks.workunit.client.1.smithi196.stdout:0/533: truncate d8/da/d25/f27 13196 0 2024-03-13T14:49:43.984 INFO:tasks.workunit.client.0.smithi176.stdout:2/385: sync 2024-03-13T14:49:43.984 INFO:tasks.workunit.client.1.smithi196.stdout:0/534: chown d8/da/d25/f2f 21 1 2024-03-13T14:49:43.987 INFO:tasks.workunit.client.1.smithi196.stdout:4/441: mknod d1/d3/d4c/d72/d7d/d80/d55/c8d 0 2024-03-13T14:49:43.989 INFO:tasks.workunit.client.0.smithi176.stdout:4/379: creat d7/d75/f8c x:0 0 0 2024-03-13T14:49:43.993 INFO:tasks.workunit.client.0.smithi176.stdout:5/338: symlink d5/d2c/l71 0 2024-03-13T14:49:43.993 INFO:tasks.workunit.client.0.smithi176.stdout:4/380: write d7/d10/d2c/f66 [369147,115295] 0 2024-03-13T14:49:43.994 INFO:tasks.workunit.client.1.smithi196.stdout:2/400: creat d0/d7/f87 x:0 0 0 2024-03-13T14:49:43.994 INFO:tasks.workunit.client.0.smithi176.stdout:4/381: fsync d7/d75/f86 0 2024-03-13T14:49:43.995 INFO:tasks.workunit.client.0.smithi176.stdout:6/345: dwrite d0/de/d38/f72 [0,4194304] 0 2024-03-13T14:49:43.995 INFO:tasks.workunit.client.0.smithi176.stdout:3/362: symlink d4/l71 0 2024-03-13T14:49:43.997 INFO:tasks.workunit.client.1.smithi196.stdout:2/401: write d0/d6/d1d/f5e [1577101,50357] 0 2024-03-13T14:49:43.997 INFO:tasks.workunit.client.1.smithi196.stdout:1/510: symlink d1/d9/d4e/d55/d63/da6/lac 0 2024-03-13T14:49:44.001 INFO:tasks.workunit.client.0.smithi176.stdout:6/346: write d0/d10/d19/d27/d2b/d2e/f2d [2912103,100422] 0 2024-03-13T14:49:44.002 INFO:tasks.workunit.client.0.smithi176.stdout:6/347: chown d0 135963436 1 2024-03-13T14:49:44.004 INFO:tasks.workunit.client.1.smithi196.stdout:9/412: rename d7/de/d4b/f52 to d7/de/f87 0 2024-03-13T14:49:44.005 INFO:tasks.workunit.client.1.smithi196.stdout:6/606: getdents d6/d2a/dc/d1c/da7 0 2024-03-13T14:49:44.008 INFO:tasks.workunit.client.1.smithi196.stdout:6/607: stat d6/d2a/dc/d1c/da7/d9f 0 2024-03-13T14:49:44.013 INFO:tasks.workunit.client.1.smithi196.stdout:1/511: chown d1/d9/c12 3 1 2024-03-13T14:49:44.017 INFO:tasks.workunit.client.0.smithi176.stdout:5/339: mknod d5/d2c/d40/d6a/c72 0 2024-03-13T14:49:44.020 INFO:tasks.workunit.client.0.smithi176.stdout:5/340: truncate d5/d22/f1e 4328997 0 2024-03-13T14:49:44.021 INFO:tasks.workunit.client.1.smithi196.stdout:9/413: creat d7/de/d4b/d1d/d22/f88 x:0 0 0 2024-03-13T14:49:44.022 INFO:tasks.workunit.client.0.smithi176.stdout:4/382: creat d7/d8/d45/f8d x:0 0 0 2024-03-13T14:49:44.023 INFO:tasks.workunit.client.0.smithi176.stdout:5/341: dread - d5/d2c/d40/d6a/f6e zero size 2024-03-13T14:49:44.025 INFO:tasks.workunit.client.1.smithi196.stdout:8/428: dwrite d0/d10/f11 [0,4194304] 0 2024-03-13T14:49:44.025 INFO:tasks.workunit.client.0.smithi176.stdout:5/342: write d5/d31/d47/f59 [454206,2170] 0 2024-03-13T14:49:44.026 INFO:tasks.workunit.client.0.smithi176.stdout:3/363: symlink d4/d28/d3a/d5b/l72 0 2024-03-13T14:49:44.028 INFO:tasks.workunit.client.1.smithi196.stdout:6/608: dwrite d6/d7/f8d [0,4194304] 0 2024-03-13T14:49:44.029 INFO:tasks.workunit.client.0.smithi176.stdout:3/364: dread - d4/d13/d16/d3d/f4c zero size 2024-03-13T14:49:44.029 INFO:tasks.workunit.client.0.smithi176.stdout:0/329: mkdir d2/d5/dc/d60 0 2024-03-13T14:49:44.030 INFO:tasks.workunit.client.1.smithi196.stdout:0/535: link d8/cad d8/da/d25/d26/d35/d3b/d5f/caf 0 2024-03-13T14:49:44.031 INFO:tasks.workunit.client.1.smithi196.stdout:4/442: write d1/f22 [3989732,109613] 0 2024-03-13T14:49:44.035 INFO:tasks.workunit.client.0.smithi176.stdout:6/348: rename d0/d10/d19/d27/d2b/d2e/d3e/l64 to d0/de/d54/l73 0 2024-03-13T14:49:44.036 INFO:tasks.workunit.client.0.smithi176.stdout:6/349: fsync d0/f6c 0 2024-03-13T14:49:44.038 INFO:tasks.workunit.client.0.smithi176.stdout:6/350: dread - d0/d10/d19/f1a zero size 2024-03-13T14:49:44.039 INFO:tasks.workunit.client.0.smithi176.stdout:9/410: getdents d0/d40/d53 0 2024-03-13T14:49:44.040 INFO:tasks.workunit.client.1.smithi196.stdout:9/414: dwrite d7/de/d74/f85 [0,4194304] 0 2024-03-13T14:49:44.041 INFO:tasks.workunit.client.0.smithi176.stdout:7/340: truncate d1/d28/f2d 324834 0 2024-03-13T14:49:44.046 INFO:tasks.workunit.client.1.smithi196.stdout:4/443: symlink d1/d3/d7/d35/d3e/l8e 0 2024-03-13T14:49:44.048 INFO:tasks.workunit.client.0.smithi176.stdout:4/383: creat d7/d10/d46/d4a/d5d/f8e x:0 0 0 2024-03-13T14:49:44.050 INFO:tasks.workunit.client.0.smithi176.stdout:3/365: creat d4/d13/f73 x:0 0 0 2024-03-13T14:49:44.050 INFO:tasks.workunit.client.1.smithi196.stdout:6/609: creat d6/d7/dd/dbc/fbe x:0 0 0 2024-03-13T14:49:44.054 INFO:tasks.workunit.client.0.smithi176.stdout:8/313: dread d0/f2 [0,4194304] 0 2024-03-13T14:49:44.057 INFO:tasks.workunit.client.1.smithi196.stdout:8/429: truncate d0/d16/d3d/d8b/d48/f5f 4546348 0 2024-03-13T14:49:44.058 INFO:tasks.workunit.client.1.smithi196.stdout:8/430: chown d0/d16/d3d/d7f/d88 0 1 2024-03-13T14:49:44.059 INFO:tasks.workunit.client.1.smithi196.stdout:9/415: creat d7/de/d16/d70/f89 x:0 0 0 2024-03-13T14:49:44.066 INFO:tasks.workunit.client.0.smithi176.stdout:0/330: mkdir d2/d5/dc/d60/d61 0 2024-03-13T14:49:44.067 INFO:tasks.workunit.client.1.smithi196.stdout:4/444: mkdir d1/d8f 0 2024-03-13T14:49:44.067 INFO:tasks.workunit.client.1.smithi196.stdout:8/431: mknod d0/d16/c96 0 2024-03-13T14:49:44.068 INFO:tasks.workunit.client.0.smithi176.stdout:3/366: symlink d4/d28/d4f/l74 0 2024-03-13T14:49:44.068 INFO:tasks.workunit.client.0.smithi176.stdout:2/386: link d1/dc/l28 d1/dc/d2d/d59/d80/d7e/l8e 0 2024-03-13T14:49:44.069 INFO:tasks.workunit.client.0.smithi176.stdout:9/411: dwrite d0/d13/d32/d58/d67/f7e [4194304,4194304] 0 2024-03-13T14:49:44.070 INFO:tasks.workunit.client.1.smithi196.stdout:9/416: creat d7/d24/d28/f8a x:0 0 0 2024-03-13T14:49:44.072 INFO:tasks.workunit.client.0.smithi176.stdout:2/387: dread d1/f13 [0,4194304] 0 2024-03-13T14:49:44.076 INFO:tasks.workunit.client.1.smithi196.stdout:1/512: dread d1/d9/d33/d4b/f48 [0,4194304] 0 2024-03-13T14:49:44.077 INFO:tasks.workunit.client.1.smithi196.stdout:4/445: symlink d1/d3/d7/d35/d76/l90 0 2024-03-13T14:49:44.078 INFO:tasks.workunit.client.0.smithi176.stdout:5/343: getdents d5/d2e 0 2024-03-13T14:49:44.082 INFO:tasks.workunit.client.1.smithi196.stdout:2/402: truncate d0/d7/d9/f18 3791216 0 2024-03-13T14:49:44.085 INFO:tasks.workunit.client.1.smithi196.stdout:2/403: chown d0/d6/d1d/l4e 860 1 2024-03-13T14:49:44.086 INFO:tasks.workunit.client.0.smithi176.stdout:9/412: mknod d0/d1e/d1f/d68/c86 0 2024-03-13T14:49:44.086 INFO:tasks.workunit.client.1.smithi196.stdout:0/536: write d8/f9 [1335394,128837] 0 2024-03-13T14:49:44.088 INFO:tasks.workunit.client.1.smithi196.stdout:1/513: symlink d1/d28/d34/d67/lad 0 2024-03-13T14:49:44.090 INFO:tasks.workunit.client.0.smithi176.stdout:4/384: link d7/f3a d7/d10/d46/f8f 0 2024-03-13T14:49:44.093 INFO:tasks.workunit.client.0.smithi176.stdout:7/341: dwrite d1/d1c/d26/d41/d2b/d40/d64/f60 [0,4194304] 0 2024-03-13T14:49:44.093 INFO:tasks.workunit.client.0.smithi176.stdout:8/314: mknod d0/db/d53/c60 0 2024-03-13T14:49:44.099 INFO:tasks.workunit.client.1.smithi196.stdout:1/514: creat d1/d9/d33/fae x:0 0 0 2024-03-13T14:49:44.102 INFO:tasks.workunit.client.0.smithi176.stdout:5/344: dread d5/d2e/d43/f50 [0,4194304] 0 2024-03-13T14:49:44.104 INFO:tasks.workunit.client.1.smithi196.stdout:8/432: dread d0/d16/d3d/d8b/f4a [0,4194304] 0 2024-03-13T14:49:44.104 INFO:tasks.workunit.client.0.smithi176.stdout:9/413: mknod d0/d13/d32/d58/d5f/c87 0 2024-03-13T14:49:44.107 INFO:tasks.workunit.client.1.smithi196.stdout:8/433: fsync d0/d10/f26 0 2024-03-13T14:49:44.112 INFO:tasks.workunit.client.1.smithi196.stdout:9/417: getdents d7/de/d16 0 2024-03-13T14:49:44.115 INFO:tasks.workunit.client.1.smithi196.stdout:8/434: chown d0/d10/d70/d8d/c95 965125 1 2024-03-13T14:49:44.115 INFO:tasks.workunit.client.1.smithi196.stdout:9/418: fsync d7/de/d4b/d1d/d22/f67 0 2024-03-13T14:49:44.115 INFO:tasks.workunit.client.1.smithi196.stdout:0/537: dwrite d8/da/d25/d26/d35/d3b/d59/d6a/f75 [4194304,4194304] 0 2024-03-13T14:49:44.117 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:44 smithi196 ceph-mon[37486]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:44.117 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:44 smithi196 ceph-mon[37486]: mgrmap e30: smithi176.tzfdwd(active, since 15s), standbys: smithi196.vdktbk 2024-03-13T14:49:44.117 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:49:44.117 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.118 INFO:tasks.workunit.client.0.smithi176.stdout:7/342: dwrite d1/d1c/f5f [0,4194304] 0 2024-03-13T14:49:44.118 INFO:tasks.workunit.client.0.smithi176.stdout:3/367: creat d4/d28/d31/f75 x:0 0 0 2024-03-13T14:49:44.118 INFO:tasks.workunit.client.1.smithi196.stdout:6/610: dwrite d6/d2a/d23/f4c [0,4194304] 0 2024-03-13T14:49:44.120 INFO:tasks.workunit.client.1.smithi196.stdout:4/446: rmdir d1/d3/d4c/d72/d7d/d80/d82 0 2024-03-13T14:49:44.120 INFO:tasks.workunit.client.0.smithi176.stdout:7/343: write d1/d28/f2a [1526705,104187] 0 2024-03-13T14:49:44.126 INFO:tasks.workunit.client.1.smithi196.stdout:1/515: rename d1/d9/d33/d4b/f22 to d1/d3f/d5b/faf 0 2024-03-13T14:49:44.132 INFO:tasks.workunit.client.0.smithi176.stdout:4/385: mknod d7/d8/d2f/c90 0 2024-03-13T14:49:44.136 INFO:tasks.workunit.client.0.smithi176.stdout:9/414: rename d0/d1e/d1f/d56/c6c to d0/d40/d53/c88 0 2024-03-13T14:49:44.141 INFO:tasks.workunit.client.0.smithi176.stdout:3/368: mknod d4/d13/c76 0 2024-03-13T14:49:44.141 INFO:tasks.workunit.client.1.smithi196.stdout:1/516: dread d1/d28/f96 [0,4194304] 0 2024-03-13T14:49:44.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.152+0000 7f830f800700 1 -- 172.21.15.176:0/3235928832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8308072b80 msgr2=0x7f8308109da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:44.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.152+0000 7f830f800700 1 --2- 172.21.15.176:0/3235928832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8308072b80 0x7f8308109da0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f830000b0d0 tx=0x7f830000b3e0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.153+0000 7f830f800700 1 -- 172.21.15.176:0/3235928832 shutdown_connections 2024-03-13T14:49:44.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.153+0000 7f830f800700 1 --2- 172.21.15.176:0/3235928832 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f830810a370 0x7f830810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.153+0000 7f830f800700 1 --2- 172.21.15.176:0/3235928832 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8308072b80 0x7f8308109da0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.153+0000 7f830f800700 1 -- 172.21.15.176:0/3235928832 >> 172.21.15.176:0/3235928832 conn(0x7f830806d9f0 msgr2=0x7f830806fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:44.143 INFO:tasks.workunit.client.0.smithi176.stdout:0/331: getdents d2/d5/d1e 0 2024-03-13T14:49:44.144 INFO:tasks.workunit.client.0.smithi176.stdout:9/415: write d0/d40/d53/f6b [476796,112904] 0 2024-03-13T14:49:44.145 INFO:tasks.workunit.client.1.smithi196.stdout:1/517: write d1/d9/d33/d4b/d66/f95 [548940,102241] 0 2024-03-13T14:49:44.146 INFO:tasks.workunit.client.1.smithi196.stdout:2/404: creat d0/d7/d9/d42/f88 x:0 0 0 2024-03-13T14:49:44.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.157+0000 7f830f800700 1 -- 172.21.15.176:0/3235928832 shutdown_connections 2024-03-13T14:49:44.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.157+0000 7f830f800700 1 -- 172.21.15.176:0/3235928832 wait complete. 2024-03-13T14:49:44.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.158+0000 7f830f800700 1 Processor -- start 2024-03-13T14:49:44.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.158+0000 7f830f800700 1 -- start start 2024-03-13T14:49:44.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f830f800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308072b80 0x7f83081a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:44.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f830f800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f830810a370 0x7f83081a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:44.148 INFO:tasks.workunit.client.0.smithi176.stdout:1/342: sync 2024-03-13T14:49:44.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f830f800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a5cc0 con 0x7f830810a370 2024-03-13T14:49:44.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f830f800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83081a5e00 con 0x7f8308072b80 2024-03-13T14:49:44.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f8307e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f830810a370 0x7f83081a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:44.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f8307e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f830810a370 0x7f83081a56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43258/0 (socket says 172.21.15.176:43258) 2024-03-13T14:49:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f8307e00700 1 -- 172.21.15.176:0/889746065 learned_addr learned my addr 172.21.15.176:0/889746065 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f8307e00700 1 -- 172.21.15.176:0/889746065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308072b80 msgr2=0x7f83081a5160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-13T14:49:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f8307e00700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308072b80 0x7f83081a5160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.151 INFO:tasks.workunit.client.0.smithi176.stdout:3/369: chown d4/d13/l26 184716679 1 2024-03-13T14:49:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.159+0000 7f8307e00700 1 -- 172.21.15.176:0/889746065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8300009d00 con 0x7f830810a370 2024-03-13T14:49:44.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.160+0000 7f8307e00700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f830810a370 0x7f83081a56a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f82f800d8d0 tx=0x7f82f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:44.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.160+0000 7f8305600700 1 -- 172.21.15.176:0/889746065 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f8009940 con 0x7f830810a370 2024-03-13T14:49:44.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.160+0000 7f8305600700 1 -- 172.21.15.176:0/889746065 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f82f8010460 con 0x7f830810a370 2024-03-13T14:49:44.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.160+0000 7f8305600700 1 -- 172.21.15.176:0/889746065 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82f8009940 con 0x7f830810a370 2024-03-13T14:49:44.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.160+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83081aa8b0 con 0x7f830810a370 2024-03-13T14:49:44.152 INFO:tasks.workunit.client.1.smithi196.stdout:1/518: read d1/d3f/d5b/faf [3169240,26123] 0 2024-03-13T14:49:44.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.161+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83081aad40 con 0x7f830810a370 2024-03-13T14:49:44.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.162+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830810c850 con 0x7f830810a370 2024-03-13T14:49:44.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.164+0000 7f8305600700 1 -- 172.21.15.176:0/889746065 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f82f80105d0 con 0x7f830810a370 2024-03-13T14:49:44.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.166+0000 7f8305600700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f82ec06ddb0 0x7f82ec070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:44.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.167+0000 7f8305600700 1 -- 172.21.15.176:0/889746065 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f82f8062c40 con 0x7f830810a370 2024-03-13T14:49:44.156 INFO:tasks.workunit.client.0.smithi176.stdout:7/344: dwrite d1/d28/d3a/d1b/d18/f52 [4194304,4194304] 0 2024-03-13T14:49:44.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.168+0000 7f830ce00700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f82ec06ddb0 0x7f82ec070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:44.160 INFO:tasks.workunit.client.0.smithi176.stdout:0/332: dread - d2/db/d31/f44 zero size 2024-03-13T14:49:44.160 INFO:tasks.workunit.client.0.smithi176.stdout:2/388: dwrite d1/dc/d2d/d38/d5d/f62 [0,4194304] 0 2024-03-13T14:49:44.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.171+0000 7f830ce00700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f82ec06ddb0 0x7f82ec070270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8300009fd0 tx=0x7f830000b040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:44.162 INFO:tasks.workunit.client.0.smithi176.stdout:3/370: readlink d4/d28/d34/d38/l42 0 2024-03-13T14:49:44.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.175+0000 7f8305600700 1 -- 172.21.15.176:0/889746065 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f82f805a300 con 0x7f830810a370 2024-03-13T14:49:44.165 INFO:tasks.workunit.client.1.smithi196.stdout:1/519: creat d1/d9/d4e/d55/d63/da6/fb0 x:0 0 0 2024-03-13T14:49:44.165 INFO:tasks.workunit.client.0.smithi176.stdout:7/345: chown d1/d7/cd 11 1 2024-03-13T14:49:44.165 INFO:tasks.workunit.client.1.smithi196.stdout:3/491: sync 2024-03-13T14:49:44.167 INFO:tasks.workunit.client.1.smithi196.stdout:8/435: getdents d0/d16/d3d/d8b 0 2024-03-13T14:49:44.168 INFO:tasks.workunit.client.0.smithi176.stdout:2/389: chown d1/dc/d2d/d38/l57 2207359 1 2024-03-13T14:49:44.170 INFO:tasks.workunit.client.1.smithi196.stdout:2/405: creat d0/d7/f89 x:0 0 0 2024-03-13T14:49:44.174 INFO:tasks.workunit.client.1.smithi196.stdout:6/611: getdents d6/d7/dd/d60 0 2024-03-13T14:49:44.174 INFO:tasks.workunit.client.0.smithi176.stdout:5/345: rmdir d5/d31/d60 0 2024-03-13T14:49:44.174 INFO:tasks.workunit.client.0.smithi176.stdout:9/416: dread d0/db/f49 [0,4194304] 0 2024-03-13T14:49:44.175 INFO:tasks.workunit.client.1.smithi196.stdout:1/520: creat d1/d9/d4e/d55/d90/fb1 x:0 0 0 2024-03-13T14:49:44.178 INFO:tasks.workunit.client.0.smithi176.stdout:7/346: rmdir d1/d3f 39 2024-03-13T14:49:44.178 INFO:tasks.workunit.client.0.smithi176.stdout:2/390: rename d1/dc/f2c to d1/dc/d2d/d59/d8c/f8f 0 2024-03-13T14:49:44.179 INFO:tasks.workunit.client.1.smithi196.stdout:3/492: truncate db/d19/d22/f31 103892 0 2024-03-13T14:49:44.180 INFO:tasks.workunit.client.1.smithi196.stdout:9/419: dread d7/d24/f3a [0,4194304] 0 2024-03-13T14:49:44.181 INFO:tasks.workunit.client.0.smithi176.stdout:9/417: write d0/d1e/d1f/d68/f85 [736440,52246] 0 2024-03-13T14:49:44.181 INFO:tasks.workunit.client.0.smithi176.stdout:6/351: sync 2024-03-13T14:49:44.181 INFO:tasks.workunit.client.0.smithi176.stdout:8/315: getdents d0/d24/d29 0 2024-03-13T14:49:44.183 INFO:tasks.workunit.client.1.smithi196.stdout:0/538: dwrite d8/da/d25/d26/d54/fa7 [0,4194304] 0 2024-03-13T14:49:44.185 INFO:tasks.workunit.client.1.smithi196.stdout:6/612: rmdir d6/d2a/dc/d71 39 2024-03-13T14:49:44.185 INFO:tasks.workunit.client.1.smithi196.stdout:9/420: write d7/de/d4b/d1d/d22/f5a [349080,120685] 0 2024-03-13T14:49:44.187 INFO:tasks.workunit.client.0.smithi176.stdout:6/352: write d0/f4e [151270,49532] 0 2024-03-13T14:49:44.189 INFO:tasks.workunit.client.1.smithi196.stdout:4/447: write d1/d3/fa [441992,40494] 0 2024-03-13T14:49:44.190 INFO:tasks.workunit.client.1.smithi196.stdout:1/521: fdatasync d1/d9/d33/d4b/f44 0 2024-03-13T14:49:44.193 INFO:tasks.workunit.client.1.smithi196.stdout:3/493: creat db/d37/fa4 x:0 0 0 2024-03-13T14:49:44.194 INFO:tasks.workunit.client.1.smithi196.stdout:1/522: write d1/f37 [2285623,113412] 0 2024-03-13T14:49:44.194 INFO:tasks.workunit.client.0.smithi176.stdout:5/346: mkdir d5/d31/d47/d73 0 2024-03-13T14:49:44.197 INFO:tasks.workunit.client.1.smithi196.stdout:6/613: truncate d6/d2a/d23/d28/f3f 1305250 0 2024-03-13T14:49:44.198 INFO:tasks.workunit.client.0.smithi176.stdout:7/347: symlink d1/d28/d3a/d1b/l66 0 2024-03-13T14:49:44.203 INFO:tasks.workunit.client.0.smithi176.stdout:2/391: mkdir d1/dc/d2d/d90 0 2024-03-13T14:49:44.206 INFO:tasks.workunit.client.1.smithi196.stdout:0/539: dwrite d8/da/d74/f8f [0,4194304] 0 2024-03-13T14:49:44.209 INFO:tasks.workunit.client.0.smithi176.stdout:3/371: dwrite d4/d28/d4f/f53 [0,4194304] 0 2024-03-13T14:49:44.209 INFO:tasks.workunit.client.0.smithi176.stdout:5/347: fsync d5/d48/d4c/f65 0 2024-03-13T14:49:44.211 INFO:tasks.workunit.client.1.smithi196.stdout:4/448: mkdir d1/d8f/d91 0 2024-03-13T14:49:44.213 INFO:tasks.workunit.client.1.smithi196.stdout:6/614: rename d6/c73 to d6/d7/dd/d60/cbf 0 2024-03-13T14:49:44.214 INFO:tasks.workunit.client.0.smithi176.stdout:1/343: link d2/de/d3f/d50/f42 d2/d30/f65 0 2024-03-13T14:49:44.220 INFO:tasks.workunit.client.0.smithi176.stdout:9/418: rename d0/d13 to d0/db/d89 0 2024-03-13T14:49:44.224 INFO:tasks.workunit.client.0.smithi176.stdout:4/386: dread d7/d10/d2c/f32 [0,4194304] 0 2024-03-13T14:49:44.229 INFO:tasks.workunit.client.1.smithi196.stdout:4/449: dread d1/f2e [0,4194304] 0 2024-03-13T14:49:44.229 INFO:tasks.workunit.client.0.smithi176.stdout:3/372: truncate d4/d28/f29 2256316 0 2024-03-13T14:49:44.230 INFO:tasks.workunit.client.1.smithi196.stdout:6/615: getdents d6/d2a/dc/d47 0 2024-03-13T14:49:44.231 INFO:tasks.workunit.client.0.smithi176.stdout:3/373: readlink d4/d28/l63 0 2024-03-13T14:49:44.233 INFO:tasks.workunit.client.1.smithi196.stdout:8/436: dread d0/d16/d3d/d8b/d48/f5f [0,4194304] 0 2024-03-13T14:49:44.234 INFO:tasks.workunit.client.1.smithi196.stdout:8/437: chown d0/d63 61591 1 2024-03-13T14:49:44.235 INFO:tasks.workunit.client.1.smithi196.stdout:0/540: creat d8/da/fb0 x:0 0 0 2024-03-13T14:49:44.236 INFO:tasks.workunit.client.0.smithi176.stdout:4/387: rmdir d7/d2d 39 2024-03-13T14:49:44.237 INFO:tasks.workunit.client.1.smithi196.stdout:8/438: dread - d0/d10/d12/d1e/f83 zero size 2024-03-13T14:49:44.238 INFO:tasks.workunit.client.0.smithi176.stdout:1/344: mkdir d2/de/d10/d16/d4a/d5f/d66 0 2024-03-13T14:49:44.238 INFO:tasks.workunit.client.1.smithi196.stdout:8/439: chown d0/d10/d12/d17/d42 21098306 1 2024-03-13T14:49:44.241 INFO:tasks.workunit.client.1.smithi196.stdout:6/616: stat d6/d9/f30 0 2024-03-13T14:49:44.243 INFO:tasks.workunit.client.1.smithi196.stdout:8/440: read - d0/d16/d71/f43 zero size 2024-03-13T14:49:44.244 INFO:tasks.workunit.client.0.smithi176.stdout:2/392: link d1/dc/d61/l64 d1/dc/d2d/d4f/l91 0 2024-03-13T14:49:44.244 INFO:tasks.workunit.client.0.smithi176.stdout:9/419: rmdir d0/d17 39 2024-03-13T14:49:44.245 INFO:tasks.workunit.client.0.smithi176.stdout:3/374: truncate d4/d28/d3a/f32 640657 0 2024-03-13T14:49:44.248 INFO:tasks.workunit.client.1.smithi196.stdout:9/421: write d7/d24/f38 [864388,11822] 0 2024-03-13T14:49:44.250 INFO:tasks.workunit.client.0.smithi176.stdout:9/420: write d0/d1e/d1f/f41 [911270,79572] 0 2024-03-13T14:49:44.250 INFO:tasks.workunit.client.1.smithi196.stdout:4/450: dwrite d1/d3/f4e [0,4194304] 0 2024-03-13T14:49:44.251 INFO:tasks.workunit.client.0.smithi176.stdout:7/348: readlink d1/d28/d3a/d1b/l66 0 2024-03-13T14:49:44.255 INFO:tasks.workunit.client.0.smithi176.stdout:1/345: getdents d2/d29 0 2024-03-13T14:49:44.256 INFO:tasks.workunit.client.1.smithi196.stdout:4/451: creat d1/d1e/d43/f92 x:0 0 0 2024-03-13T14:49:44.259 INFO:tasks.workunit.client.0.smithi176.stdout:7/349: dread - d1/d1c/d26/d41/f5a zero size 2024-03-13T14:49:44.260 INFO:tasks.workunit.client.1.smithi196.stdout:3/494: write db/f4e [280907,42708] 0 2024-03-13T14:49:44.262 INFO:tasks.workunit.client.1.smithi196.stdout:3/495: truncate db/d37/fa4 631909 0 2024-03-13T14:49:44.265 INFO:tasks.workunit.client.1.smithi196.stdout:4/452: fdatasync d1/d3/f13 0 2024-03-13T14:49:44.268 INFO:tasks.workunit.client.0.smithi176.stdout:0/333: sync 2024-03-13T14:49:44.272 INFO:tasks.workunit.client.0.smithi176.stdout:1/346: creat d2/de/d3f/d3e/f67 x:0 0 0 2024-03-13T14:49:44.276 INFO:tasks.workunit.client.0.smithi176.stdout:9/421: getdents d0/d1e 0 2024-03-13T14:49:44.276 INFO:tasks.workunit.client.1.smithi196.stdout:4/453: rename d1/d3/d7/d35/d76/d58/c84 to d1/d3/d7/d35/d3e/d56/c93 0 2024-03-13T14:49:44.278 INFO:tasks.workunit.client.1.smithi196.stdout:1/523: dwrite d1/d9/f18 [0,4194304] 0 2024-03-13T14:49:44.279 INFO:tasks.workunit.client.0.smithi176.stdout:7/350: creat d1/d1c/d26/d41/d2b/f67 x:0 0 0 2024-03-13T14:49:44.287 INFO:tasks.workunit.client.0.smithi176.stdout:5/348: truncate d5/d2b/f30 2196342 0 2024-03-13T14:49:44.288 INFO:tasks.workunit.client.1.smithi196.stdout:3/496: link db/d19/f62 db/d71/fa5 0 2024-03-13T14:49:44.294 INFO:tasks.workunit.client.1.smithi196.stdout:9/422: write d7/de/f40 [121089,25629] 0 2024-03-13T14:49:44.295 INFO:tasks.workunit.client.0.smithi176.stdout:1/347: write d2/f9 [3454949,16747] 0 2024-03-13T14:49:44.295 INFO:tasks.workunit.client.0.smithi176.stdout:6/353: dwrite d0/d10/d19/f1a [0,4194304] 0 2024-03-13T14:49:44.297 INFO:tasks.workunit.client.1.smithi196.stdout:6/617: dwrite d6/fa [0,4194304] 0 2024-03-13T14:49:44.298 INFO:tasks.workunit.client.1.smithi196.stdout:3/497: truncate db/f36 246023 0 2024-03-13T14:49:44.301 INFO:tasks.workunit.client.0.smithi176.stdout:4/388: dwrite d7/d8/d15/f3e [0,4194304] 0 2024-03-13T14:49:44.303 INFO:tasks.workunit.client.0.smithi176.stdout:8/316: dwrite d0/db/f1a [4194304,4194304] 0 2024-03-13T14:49:44.305 INFO:tasks.workunit.client.0.smithi176.stdout:7/351: chown d1/d28/f2d 68 1 2024-03-13T14:49:44.307 INFO:tasks.workunit.client.1.smithi196.stdout:9/423: symlink d7/de/d4b/d1d/l8b 0 2024-03-13T14:49:44.310 INFO:tasks.workunit.client.1.smithi196.stdout:6/618: mknod d6/d2a/dc/d1c/d34/cc0 0 2024-03-13T14:49:44.313 INFO:tasks.workunit.client.0.smithi176.stdout:5/349: rename d5/d31/d47/d73 to d5/d2c/d74 0 2024-03-13T14:49:44.313 INFO:tasks.workunit.client.0.smithi176.stdout:9/422: mkdir d0/db/d89/d4d/d6e/d81/d8a 0 2024-03-13T14:49:44.314 INFO:tasks.workunit.client.1.smithi196.stdout:1/524: dread d1/f37 [0,4194304] 0 2024-03-13T14:49:44.317 INFO:tasks.workunit.client.0.smithi176.stdout:8/317: symlink d0/db/dc/d40/l61 0 2024-03-13T14:49:44.317 INFO:tasks.workunit.client.0.smithi176.stdout:6/354: creat d0/d10/d19/d27/d53/f74 x:0 0 0 2024-03-13T14:49:44.320 INFO:tasks.workunit.client.1.smithi196.stdout:9/424: mknod d7/d24/d28/d6c/d7b/c8c 0 2024-03-13T14:49:44.321 INFO:tasks.workunit.client.1.smithi196.stdout:1/525: creat d1/d38/fb2 x:0 0 0 2024-03-13T14:49:44.323 INFO:tasks.workunit.client.0.smithi176.stdout:5/350: mkdir d5/d75 0 2024-03-13T14:49:44.323 INFO:tasks.workunit.client.1.smithi196.stdout:9/425: fsync d7/d24/f38 0 2024-03-13T14:49:44.329 INFO:tasks.workunit.client.0.smithi176.stdout:9/423: chown d0/db/d89/d4d/d5d/l60 3 1 2024-03-13T14:49:44.329 INFO:tasks.workunit.client.0.smithi176.stdout:7/352: symlink d1/d3f/d4b/l68 0 2024-03-13T14:49:44.331 INFO:tasks.workunit.client.1.smithi196.stdout:4/454: dread d1/d3/d7/f1a [4194304,4194304] 0 2024-03-13T14:49:44.331 INFO:tasks.workunit.client.1.smithi196.stdout:1/526: rename d1/d9/d33/f46 to d1/d3f/fb3 0 2024-03-13T14:49:44.333 INFO:tasks.workunit.client.1.smithi196.stdout:8/441: dread d0/d1c/f30 [0,4194304] 0 2024-03-13T14:49:44.333 INFO:tasks.workunit.client.1.smithi196.stdout:6/619: link d6/d9/d11/f70 d6/fc1 0 2024-03-13T14:49:44.338 INFO:tasks.workunit.client.1.smithi196.stdout:1/527: dread - d1/d9/d6c/fab zero size 2024-03-13T14:49:44.338 INFO:tasks.workunit.client.1.smithi196.stdout:9/426: truncate d7/de/d4b/d1d/d22/f53 790060 0 2024-03-13T14:49:44.342 INFO:tasks.workunit.client.1.smithi196.stdout:1/528: creat d1/d9/d33/d4b/d66/d7b/fb4 x:0 0 0 2024-03-13T14:49:44.343 INFO:tasks.workunit.client.0.smithi176.stdout:2/393: dread d1/dc/d2d/d38/f3f [0,4194304] 0 2024-03-13T14:49:44.346 INFO:tasks.workunit.client.0.smithi176.stdout:9/424: mkdir d0/db/d89/d7a/d8b 0 2024-03-13T14:49:44.349 INFO:tasks.workunit.client.0.smithi176.stdout:4/389: read d7/d10/d2c/f30 [1518404,78843] 0 2024-03-13T14:49:44.350 INFO:tasks.workunit.client.1.smithi196.stdout:9/427: read d7/de/d4b/f1a [3318175,3751] 0 2024-03-13T14:49:44.351 INFO:tasks.workunit.client.0.smithi176.stdout:2/394: fsync d1/dc/d2d/d4b/f63 0 2024-03-13T14:49:44.351 INFO:tasks.workunit.client.0.smithi176.stdout:0/334: dread d2/d5/f46 [0,4194304] 0 2024-03-13T14:49:44.354 INFO:tasks.workunit.client.1.smithi196.stdout:6/620: dwrite d6/d2a/dc/d1c/fbb [0,4194304] 0 2024-03-13T14:49:44.354 INFO:tasks.workunit.client.0.smithi176.stdout:0/335: fdatasync d2/d5/dc/d20/f48 0 2024-03-13T14:49:44.359 INFO:tasks.workunit.client.0.smithi176.stdout:3/375: dread d4/d13/f2a [0,4194304] 0 2024-03-13T14:49:44.367 INFO:tasks.workunit.client.1.smithi196.stdout:7/616: sync 2024-03-13T14:49:44.368 INFO:tasks.workunit.client.1.smithi196.stdout:5/584: sync 2024-03-13T14:49:44.369 INFO:tasks.workunit.client.0.smithi176.stdout:5/351: rmdir d5/d2c/d40/d6b 0 2024-03-13T14:49:44.374 INFO:tasks.workunit.client.1.smithi196.stdout:3/498: fsync db/d19/d22/f31 0 2024-03-13T14:49:44.378 INFO:tasks.workunit.client.0.smithi176.stdout:9/425: truncate d0/db/f31 964131 0 2024-03-13T14:49:44.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:44 smithi176 ceph-mon[29064]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:44 smithi176 ceph-mon[29064]: mgrmap e30: smithi176.tzfdwd(active, since 15s), standbys: smithi196.vdktbk 2024-03-13T14:49:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:49:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.384 INFO:tasks.workunit.client.0.smithi176.stdout:2/395: rename d1/d2a/d33/l81 to d1/dc/d40/l92 0 2024-03-13T14:49:44.385 INFO:tasks.workunit.client.1.smithi196.stdout:6/621: rename d6/d7/dd/f7e to d6/d9/daa/fc2 0 2024-03-13T14:49:44.386 INFO:tasks.workunit.client.0.smithi176.stdout:0/336: creat d2/db/d31/f62 x:0 0 0 2024-03-13T14:49:44.388 INFO:tasks.workunit.client.0.smithi176.stdout:3/376: mknod d4/d28/d31/d45/c77 0 2024-03-13T14:49:44.389 INFO:tasks.workunit.client.0.smithi176.stdout:2/396: truncate d1/dc/d1c/f37 972281 0 2024-03-13T14:49:44.389 INFO:tasks.workunit.client.0.smithi176.stdout:0/337: chown d2/d5/dc/f5d 14318 1 2024-03-13T14:49:44.391 INFO:tasks.workunit.client.0.smithi176.stdout:3/377: chown d4/d28/d34/d38/l42 12358563 1 2024-03-13T14:49:44.391 INFO:tasks.workunit.client.1.smithi196.stdout:8/442: link d0/d16/d71/c82 d0/d10/d12/d17/c97 0 2024-03-13T14:49:44.395 INFO:tasks.workunit.client.0.smithi176.stdout:3/378: chown d4/d13/d2d/f4d 3060 1 2024-03-13T14:49:44.395 INFO:tasks.workunit.client.0.smithi176.stdout:9/426: truncate d0/d1e/f4e 323646 0 2024-03-13T14:49:44.396 INFO:tasks.workunit.client.0.smithi176.stdout:1/348: dread d2/fc [0,4194304] 0 2024-03-13T14:49:44.399 INFO:tasks.workunit.client.0.smithi176.stdout:7/353: write d1/d28/f2d [237921,49796] 0 2024-03-13T14:49:44.400 INFO:tasks.workunit.client.1.smithi196.stdout:6/622: dwrite d6/d9/d17/d19/fa8 [4194304,4194304] 0 2024-03-13T14:49:44.404 INFO:tasks.workunit.client.1.smithi196.stdout:8/443: dread d0/d1c/d29/f65 [0,4194304] 0 2024-03-13T14:49:44.405 INFO:tasks.workunit.client.0.smithi176.stdout:9/427: write d0/d1e/f4a [3296980,44096] 0 2024-03-13T14:49:44.409 INFO:tasks.workunit.client.0.smithi176.stdout:1/349: symlink d2/d30/d63/l68 0 2024-03-13T14:49:44.411 INFO:tasks.workunit.client.1.smithi196.stdout:1/529: write d1/d9/d33/d4b/d66/fa8 [1299930,31645] 0 2024-03-13T14:49:44.412 INFO:tasks.workunit.client.0.smithi176.stdout:7/354: write d1/d28/d3a/f5c [4729003,34892] 0 2024-03-13T14:49:44.413 INFO:tasks.workunit.client.0.smithi176.stdout:8/318: truncate d0/db/dc/f35 4155099 0 2024-03-13T14:49:44.417 INFO:tasks.workunit.client.0.smithi176.stdout:3/379: rename d4/d13/d2d/d50/f66 to d4/f78 0 2024-03-13T14:49:44.421 INFO:tasks.workunit.client.0.smithi176.stdout:8/319: write d0/d5/d11/f5f [664335,12585] 0 2024-03-13T14:49:44.423 INFO:tasks.workunit.client.1.smithi196.stdout:8/444: truncate d0/d1c/f30 3823932 0 2024-03-13T14:49:44.424 INFO:tasks.workunit.client.0.smithi176.stdout:6/355: dwrite d0/de/f57 [0,4194304] 0 2024-03-13T14:49:44.425 INFO:tasks.workunit.client.1.smithi196.stdout:7/617: link d0/d9/d1e/la4 d0/d78/lcc 0 2024-03-13T14:49:44.427 INFO:tasks.workunit.client.1.smithi196.stdout:7/618: write d0/d9/f30 [404004,43378] 0 2024-03-13T14:49:44.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:44.429 INFO:tasks.workunit.client.0.smithi176.stdout:6/356: readlink d0/d10/d19/d27/l3d 0 2024-03-13T14:49:44.429 INFO:tasks.workunit.client.1.smithi196.stdout:1/530: mkdir d1/d3f/d5b/d74/db5 0 2024-03-13T14:49:44.429 INFO:tasks.workunit.client.1.smithi196.stdout:5/585: write d4/db/d14/d37/f78 [10627,77977] 0 2024-03-13T14:49:44.432 INFO:tasks.workunit.client.1.smithi196.stdout:1/531: chown d1/d28/f35 33381545 1 2024-03-13T14:49:44.432 INFO:tasks.workunit.client.0.smithi176.stdout:1/350: creat d2/de/d10/d16/d4a/d5f/f69 x:0 0 0 2024-03-13T14:49:44.432 INFO:tasks.workunit.client.0.smithi176.stdout:5/352: dwrite d5/d2e/d43/f50 [0,4194304] 0 2024-03-13T14:49:44.434 INFO:tasks.workunit.client.0.smithi176.stdout:4/390: truncate d7/d8/d2f/f58 2698377 0 2024-03-13T14:49:44.435 INFO:tasks.workunit.client.0.smithi176.stdout:0/338: dwrite d2/d5/d1e/f3e [0,4194304] 0 2024-03-13T14:49:44.435 INFO:tasks.workunit.client.1.smithi196.stdout:8/445: creat d0/d1c/d29/d47/f98 x:0 0 0 2024-03-13T14:49:44.435 INFO:tasks.workunit.client.0.smithi176.stdout:4/391: chown c4 375 1 2024-03-13T14:49:44.436 INFO:tasks.workunit.client.1.smithi196.stdout:9/428: write d7/d9/f11 [1644292,30444] 0 2024-03-13T14:49:44.438 INFO:tasks.workunit.client.1.smithi196.stdout:7/619: symlink d0/d9/d31/d77/d93/lcd 0 2024-03-13T14:49:44.439 INFO:tasks.workunit.client.0.smithi176.stdout:6/357: truncate d0/d10/d19/d27/f61 2299975 0 2024-03-13T14:49:44.441 INFO:tasks.workunit.client.1.smithi196.stdout:9/429: write d7/d24/f38 [549932,123853] 0 2024-03-13T14:49:44.444 INFO:tasks.workunit.client.1.smithi196.stdout:5/586: creat d4/db/d14/d19/d1c/d25/d38/d47/fb8 x:0 0 0 2024-03-13T14:49:44.449 INFO:tasks.workunit.client.1.smithi196.stdout:1/532: mkdir d1/d9/d33/d4b/d66/db6 0 2024-03-13T14:49:44.454 INFO:tasks.workunit.client.1.smithi196.stdout:8/446: truncate d0/d10/d12/f3e 1006558 0 2024-03-13T14:49:44.454 INFO:tasks.workunit.client.0.smithi176.stdout:5/353: rmdir d5/d48 39 2024-03-13T14:49:44.457 INFO:tasks.workunit.client.1.smithi196.stdout:7/620: truncate d0/f37 892417 0 2024-03-13T14:49:44.461 INFO:tasks.workunit.client.1.smithi196.stdout:3/499: write db/d71/fa5 [682553,105222] 0 2024-03-13T14:49:44.462 INFO:tasks.workunit.client.0.smithi176.stdout:0/339: rmdir d2/d5/dc 39 2024-03-13T14:49:44.464 INFO:tasks.workunit.client.1.smithi196.stdout:9/430: symlink d7/d24/d28/d6c/l8d 0 2024-03-13T14:49:44.464 INFO:tasks.workunit.client.0.smithi176.stdout:4/392: symlink d7/d8/d15/l91 0 2024-03-13T14:49:44.468 INFO:tasks.workunit.client.0.smithi176.stdout:3/380: creat d4/d13/d5e/f79 x:0 0 0 2024-03-13T14:49:44.470 INFO:tasks.workunit.client.1.smithi196.stdout:5/587: dwrite d4/db/d3c/f59 [0,4194304] 0 2024-03-13T14:49:44.471 INFO:tasks.workunit.client.0.smithi176.stdout:3/381: write d4/d13/f73 [620435,94755] 0 2024-03-13T14:49:44.476 INFO:tasks.workunit.client.1.smithi196.stdout:1/533: dread d1/d28/d34/f9c [0,4194304] 0 2024-03-13T14:49:44.479 INFO:tasks.workunit.client.1.smithi196.stdout:9/431: rmdir d7/d24/d28/d6c/d7b 39 2024-03-13T14:49:44.485 INFO:tasks.workunit.client.0.smithi176.stdout:9/428: dread d0/d1e/f4a [0,4194304] 0 2024-03-13T14:49:44.485 INFO:tasks.workunit.client.0.smithi176.stdout:1/351: mkdir d2/de/d3f/d6a 0 2024-03-13T14:49:44.489 INFO:tasks.workunit.client.0.smithi176.stdout:9/429: fsync d0/d1e/f25 0 2024-03-13T14:49:44.489 INFO:tasks.workunit.client.0.smithi176.stdout:0/340: symlink d2/db/d39/l63 0 2024-03-13T14:49:44.492 INFO:tasks.workunit.client.1.smithi196.stdout:1/534: dread d1/d38/f16 [0,4194304] 0 2024-03-13T14:49:44.492 INFO:tasks.workunit.client.0.smithi176.stdout:4/393: creat d7/d8/d45/f92 x:0 0 0 2024-03-13T14:49:44.494 INFO:tasks.workunit.client.1.smithi196.stdout:8/447: getdents d0/d10/d12/d1e/d2d 0 2024-03-13T14:49:44.497 INFO:tasks.workunit.client.0.smithi176.stdout:2/397: sync 2024-03-13T14:49:44.503 INFO:tasks.workunit.client.0.smithi176.stdout:3/382: readlink d4/l68 0 2024-03-13T14:49:44.506 INFO:tasks.workunit.client.1.smithi196.stdout:3/500: link db/d19/d22/d30/l33 db/d15/d44/d51/la6 0 2024-03-13T14:49:44.508 INFO:tasks.workunit.client.0.smithi176.stdout:4/394: unlink d7/d8/d15/f3e 0 2024-03-13T14:49:44.509 INFO:tasks.workunit.client.1.smithi196.stdout:8/448: symlink d0/d1c/d29/l99 0 2024-03-13T14:49:44.512 INFO:tasks.workunit.client.0.smithi176.stdout:3/383: creat d4/d28/d34/f7a x:0 0 0 2024-03-13T14:49:44.512 INFO:tasks.workunit.client.1.smithi196.stdout:8/449: mkdir d0/d16/d71/d9a 0 2024-03-13T14:49:44.513 INFO:tasks.workunit.client.1.smithi196.stdout:8/450: fsync d0/d16/d3d/d8b/d3f/f93 0 2024-03-13T14:49:44.514 INFO:tasks.workunit.client.0.smithi176.stdout:3/384: write f2 [986289,63289] 0 2024-03-13T14:49:44.517 INFO:tasks.workunit.client.0.smithi176.stdout:4/395: dread - d7/d10/d46/d4a/d5d/f78 zero size 2024-03-13T14:49:44.520 INFO:tasks.workunit.client.1.smithi196.stdout:3/501: read db/d37/d64/f74 [524856,79316] 0 2024-03-13T14:49:44.521 INFO:tasks.workunit.client.0.smithi176.stdout:6/358: dread d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:44.525 INFO:tasks.workunit.client.0.smithi176.stdout:1/352: getdents d2/de/d10/d16 0 2024-03-13T14:49:44.525 INFO:tasks.workunit.client.1.smithi196.stdout:3/502: creat db/d15/d44/d79/fa7 x:0 0 0 2024-03-13T14:49:44.526 INFO:tasks.workunit.client.1.smithi196.stdout:8/451: dread d0/d10/d12/d17/f57 [0,4194304] 0 2024-03-13T14:49:44.526 INFO:tasks.workunit.client.1.smithi196.stdout:7/621: write d0/d6/d33/f4c [576447,47203] 0 2024-03-13T14:49:44.530 INFO:tasks.workunit.client.0.smithi176.stdout:6/359: write d0/d10/d19/d27/d2b/f39 [131132,19892] 0 2024-03-13T14:49:44.532 INFO:tasks.workunit.client.1.smithi196.stdout:3/503: dread db/d19/d22/f82 [4194304,4194304] 0 2024-03-13T14:49:44.533 INFO:tasks.workunit.client.1.smithi196.stdout:5/588: truncate d4/db/d14/d19/f58 536466 0 2024-03-13T14:49:44.535 INFO:tasks.workunit.client.1.smithi196.stdout:7/622: truncate d0/d6/f45 1483541 0 2024-03-13T14:49:44.544 INFO:tasks.workunit.client.1.smithi196.stdout:3/504: rename db/d19/l45 to db/d19/d22/d3a/d6a/d8d/la8 0 2024-03-13T14:49:44.544 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.557+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f83080611d0 con 0x7f82ec06ddb0 2024-03-13T14:49:44.547 INFO:tasks.workunit.client.0.smithi176.stdout:9/430: dwrite d0/d17/f3a [0,4194304] 0 2024-03-13T14:49:44.549 INFO:tasks.workunit.client.1.smithi196.stdout:5/589: truncate d4/db/d14/d19/d1c/f23 2056217 0 2024-03-13T14:49:44.550 INFO:tasks.workunit.client.0.smithi176.stdout:8/320: dread d0/d24/f32 [0,4194304] 0 2024-03-13T14:49:44.551 INFO:tasks.workunit.client.1.smithi196.stdout:3/505: mkdir db/d19/d22/d30/da9 0 2024-03-13T14:49:44.551 INFO:tasks.workunit.client.1.smithi196.stdout:9/432: dwrite d7/de/d4b/d1d/d22/d4d/f80 [0,4194304] 0 2024-03-13T14:49:44.552 INFO:tasks.workunit.client.0.smithi176.stdout:6/360: dread d0/de/d38/f72 [0,4194304] 0 2024-03-13T14:49:44.556 INFO:tasks.workunit.client.0.smithi176.stdout:0/341: dwrite d2/d5/dc/f49 [0,4194304] 0 2024-03-13T14:49:44.556 INFO:tasks.workunit.client.1.smithi196.stdout:5/590: creat d4/db/d14/d19/d26/dab/fb9 x:0 0 0 2024-03-13T14:49:44.556 INFO:tasks.workunit.client.0.smithi176.stdout:2/398: write d1/dc/d40/f44 [59911,16608] 0 2024-03-13T14:49:44.557 INFO:tasks.workunit.client.1.smithi196.stdout:7/623: link d0/d78/ca2 d0/d9/d1e/cce 0 2024-03-13T14:49:44.559 INFO:tasks.workunit.client.0.smithi176.stdout:6/361: dread - d0/d10/d19/f63 zero size 2024-03-13T14:49:44.559 INFO:tasks.workunit.client.1.smithi196.stdout:5/591: read d4/db/d3c/f84 [2925700,110622] 0 2024-03-13T14:49:44.560 INFO:tasks.workunit.client.0.smithi176.stdout:0/342: fdatasync d2/db/d31/f44 0 2024-03-13T14:49:44.561 INFO:tasks.workunit.client.0.smithi176.stdout:9/431: symlink d0/db/d89/d32/d58/d5f/l8c 0 2024-03-13T14:49:44.561 INFO:tasks.workunit.client.0.smithi176.stdout:6/362: readlink d0/d10/d19/l66 0 2024-03-13T14:49:44.562 INFO:tasks.workunit.client.1.smithi196.stdout:8/452: dread d0/d1c/d29/f3c [0,4194304] 0 2024-03-13T14:49:44.562 INFO:tasks.workunit.client.1.smithi196.stdout:3/506: truncate db/d15/f1a 1033307 0 2024-03-13T14:49:44.564 INFO:tasks.workunit.client.0.smithi176.stdout:8/321: creat d0/db/dc/d40/f62 x:0 0 0 2024-03-13T14:49:44.565 INFO:tasks.workunit.client.0.smithi176.stdout:9/432: stat d0/db/d89/d32 0 2024-03-13T14:49:44.569 INFO:tasks.workunit.client.0.smithi176.stdout:4/396: write d7/d10/d46/f4e [724991,11643] 0 2024-03-13T14:49:44.569 INFO:tasks.workunit.client.0.smithi176.stdout:1/353: dwrite d2/d30/f3a [4194304,4194304] 0 2024-03-13T14:49:44.570 INFO:tasks.workunit.client.0.smithi176.stdout:9/433: write d0/db/d89/d32/f62 [1297564,41967] 0 2024-03-13T14:49:44.571 INFO:tasks.workunit.client.1.smithi196.stdout:5/592: write d4/db/d3c/d6a/f9c [1096902,129042] 0 2024-03-13T14:49:44.578 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:49:44.578 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (9m) 0s ago 11m 13.9M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (11m) 0s ago 11m 7784k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (10m) 4s ago 10m 7696k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (11m) 0s ago 11m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (9m) 4s ago 9m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (8m) 0s ago 10m 74.9M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (3m) 0s ago 3m 108M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (4m) 0s ago 4m 13.8M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (3m) 4s ago 3m 166M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (3m) 4s ago 3m 13.8M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (33s) 0s ago 13m 473M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (95s) 4s ago 9m 71.3M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5187659249a7 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (13m) 0s ago 13m 51.7M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (9m) 4s ago 9m 41.0M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:49:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (11m) 0s ago 11m 7680k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (9m) 4s ago 9m 8082k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (8m) 0s ago 8m 139M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (8m) 0s ago 8m 156M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (7m) 0s ago 7m 138M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (6m) 4s ago 6m 178M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (6m) 4s ago 6m 166M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (5m) 4s ago 5m 139M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:49:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (43s) 0s ago 10m 45.2M - 2.43.0 a07b618ecd1d 987cea8f7b3b 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.586+0000 7f8305600700 1 -- 172.21.15.176:0/889746065 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f83080611d0 con 0x7f82ec06ddb0 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.591+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f82ec06ddb0 msgr2=0x7f82ec070270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.591+0000 7f830f800700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f82ec06ddb0 0x7f82ec070270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8300009fd0 tx=0x7f830000b040 comp rx=0 tx=0).stop 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.591+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f830810a370 msgr2=0x7f83081a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.591+0000 7f830f800700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f830810a370 0x7f83081a56a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f82f800d8d0 tx=0x7f82f800dc90 comp rx=0 tx=0).stop 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.592+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 shutdown_connections 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.592+0000 7f830f800700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f82ec06ddb0 0x7f82ec070270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.592+0000 7f830f800700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8308072b80 0x7f83081a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.592+0000 7f830f800700 1 --2- 172.21.15.176:0/889746065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f830810a370 0x7f83081a56a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.592+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 >> 172.21.15.176:0/889746065 conn(0x7f830806d9f0 msgr2=0x7f830806fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:44.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.592+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 shutdown_connections 2024-03-13T14:49:44.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.592+0000 7f830f800700 1 -- 172.21.15.176:0/889746065 wait complete. 2024-03-13T14:49:44.582 INFO:tasks.workunit.client.1.smithi196.stdout:7/624: mkdir d0/d9/d31/d77/dcf 0 2024-03-13T14:49:44.583 INFO:tasks.workunit.client.0.smithi176.stdout:7/355: sync 2024-03-13T14:49:44.586 INFO:tasks.workunit.client.1.smithi196.stdout:9/433: getdents d7/d9 0 2024-03-13T14:49:44.586 INFO:tasks.workunit.client.1.smithi196.stdout:2/406: sync 2024-03-13T14:49:44.591 INFO:tasks.workunit.client.0.smithi176.stdout:0/343: creat d2/d5/f64 x:0 0 0 2024-03-13T14:49:44.592 INFO:tasks.workunit.client.1.smithi196.stdout:7/625: chown d0/d9/d31/cb6 449262529 1 2024-03-13T14:49:44.592 INFO:tasks.workunit.client.1.smithi196.stdout:8/453: dwrite d0/d10/d12/f53 [0,4194304] 0 2024-03-13T14:49:44.594 INFO:tasks.workunit.client.0.smithi176.stdout:9/434: mkdir d0/d1e/d8d 0 2024-03-13T14:49:44.601 INFO:tasks.workunit.client.0.smithi176.stdout:7/356: rename d1/d1c/d26/d41/d2b/d40/l44 to d1/d1c/l69 0 2024-03-13T14:49:44.603 INFO:tasks.workunit.client.1.smithi196.stdout:5/593: rename d4/db/d4a/f8a to d4/db/d14/d19/d1c/d25/fba 0 2024-03-13T14:49:44.606 INFO:tasks.workunit.client.0.smithi176.stdout:6/363: getdents d0/de 0 2024-03-13T14:49:44.607 INFO:tasks.workunit.client.1.smithi196.stdout:8/454: truncate d0/d10/d12/d1e/f34 1755254 0 2024-03-13T14:49:44.609 INFO:tasks.workunit.client.1.smithi196.stdout:2/407: stat d0/d7/d9/d42/f5c 0 2024-03-13T14:49:44.610 INFO:tasks.workunit.client.0.smithi176.stdout:3/385: dwrite d4/d28/d3a/f32 [0,4194304] 0 2024-03-13T14:49:44.611 INFO:tasks.workunit.client.1.smithi196.stdout:5/594: symlink d4/d60/lbb 0 2024-03-13T14:49:44.612 INFO:tasks.workunit.client.1.smithi196.stdout:7/626: getdents d0/d6/d33/d57/d8a/d9f 0 2024-03-13T14:49:44.613 INFO:tasks.workunit.client.1.smithi196.stdout:8/455: creat d0/d10/d12/d1e/f9b x:0 0 0 2024-03-13T14:49:44.614 INFO:tasks.workunit.client.0.smithi176.stdout:9/435: rename d0/d40/c44 to d0/d17/c8e 0 2024-03-13T14:49:44.616 INFO:tasks.workunit.client.1.smithi196.stdout:9/434: getdents d7 0 2024-03-13T14:49:44.618 INFO:tasks.workunit.client.0.smithi176.stdout:6/364: creat d0/de/d54/f75 x:0 0 0 2024-03-13T14:49:44.618 INFO:tasks.workunit.client.0.smithi176.stdout:8/322: dwrite d0/f30 [0,4194304] 0 2024-03-13T14:49:44.619 INFO:tasks.workunit.client.1.smithi196.stdout:7/627: rmdir d0/d9/d1e/d3f 39 2024-03-13T14:49:44.623 INFO:tasks.workunit.client.1.smithi196.stdout:9/435: mknod d7/d71/c8e 0 2024-03-13T14:49:44.623 INFO:tasks.workunit.client.1.smithi196.stdout:2/408: chown d0/d7/d66/d67/c6c 1324 1 2024-03-13T14:49:44.623 INFO:tasks.workunit.client.0.smithi176.stdout:4/397: dwrite d7/d10/d46/d4a/d5d/f78 [0,4194304] 0 2024-03-13T14:49:44.624 INFO:tasks.workunit.client.1.smithi196.stdout:5/595: truncate d4/db/d14/d19/d26/faa 2583678 0 2024-03-13T14:49:44.624 INFO:tasks.workunit.client.0.smithi176.stdout:5/354: sync 2024-03-13T14:49:44.627 INFO:tasks.workunit.client.1.smithi196.stdout:7/628: mkdir d0/d4d/dd0 0 2024-03-13T14:49:44.628 INFO:tasks.workunit.client.0.smithi176.stdout:5/355: chown d5/d31/l4f 8289849 1 2024-03-13T14:49:44.630 INFO:tasks.workunit.client.1.smithi196.stdout:8/456: getdents d0/d10/d12/d17 0 2024-03-13T14:49:44.630 INFO:tasks.workunit.client.0.smithi176.stdout:0/344: rename d2/d5/dc/f5d to d2/d5/dc/f65 0 2024-03-13T14:49:44.632 INFO:tasks.workunit.client.1.smithi196.stdout:8/457: readlink d0/d10/d12/d17/l69 0 2024-03-13T14:49:44.636 INFO:tasks.workunit.client.1.smithi196.stdout:5/596: fdatasync d4/db/f66 0 2024-03-13T14:49:44.638 INFO:tasks.workunit.client.1.smithi196.stdout:9/436: truncate d7/f62 283078 0 2024-03-13T14:49:44.641 INFO:tasks.workunit.client.0.smithi176.stdout:7/357: creat d1/d1c/d26/d41/f6a x:0 0 0 2024-03-13T14:49:44.643 INFO:tasks.workunit.client.0.smithi176.stdout:1/354: dread d2/f21 [0,4194304] 0 2024-03-13T14:49:44.647 INFO:tasks.workunit.client.0.smithi176.stdout:2/399: truncate d1/ff 1880229 0 2024-03-13T14:49:44.648 INFO:tasks.workunit.client.1.smithi196.stdout:3/507: truncate db/d19/f2f 2587375 0 2024-03-13T14:49:44.650 INFO:tasks.workunit.client.0.smithi176.stdout:9/436: creat d0/db/d89/d7a/d8b/f8f x:0 0 0 2024-03-13T14:49:44.657 INFO:tasks.workunit.client.1.smithi196.stdout:5/597: dwrite d4/db/d14/d37/fb5 [0,4194304] 0 2024-03-13T14:49:44.657 INFO:tasks.workunit.client.1.smithi196.stdout:1/535: dread d1/d38/f6a [0,4194304] 0 2024-03-13T14:49:44.657 INFO:tasks.workunit.client.1.smithi196.stdout:7/629: mkdir d0/d4d/dd1 0 2024-03-13T14:49:44.658 INFO:tasks.workunit.client.0.smithi176.stdout:5/356: write d5/d48/d49/f4d [654226,2478] 0 2024-03-13T14:49:44.660 INFO:tasks.workunit.client.0.smithi176.stdout:2/400: dread d1/dc/d2d/d4b/f63 [0,4194304] 0 2024-03-13T14:49:44.662 INFO:tasks.workunit.client.0.smithi176.stdout:7/358: creat d1/d3f/d4b/f6b x:0 0 0 2024-03-13T14:49:44.662 INFO:tasks.workunit.client.1.smithi196.stdout:2/409: creat d0/d6/d4b/f8a x:0 0 0 2024-03-13T14:49:44.663 INFO:tasks.workunit.client.0.smithi176.stdout:4/398: fsync d7/d8/d2f/f58 0 2024-03-13T14:49:44.664 INFO:tasks.workunit.client.1.smithi196.stdout:3/508: read db/d37/d64/f74 [2711174,123188] 0 2024-03-13T14:49:44.666 INFO:tasks.workunit.client.1.smithi196.stdout:3/509: dread - db/d37/d64/f9a zero size 2024-03-13T14:49:44.669 INFO:tasks.workunit.client.0.smithi176.stdout:9/437: mkdir d0/db/d89/d90 0 2024-03-13T14:49:44.669 INFO:tasks.workunit.client.0.smithi176.stdout:0/345: creat d2/d5/dc/d60/d61/f66 x:0 0 0 2024-03-13T14:49:44.670 INFO:tasks.workunit.client.1.smithi196.stdout:5/598: mkdir d4/db/d14/d19/d1c/dbc 0 2024-03-13T14:49:44.671 INFO:tasks.workunit.client.1.smithi196.stdout:1/536: symlink d1/d3f/d5b/d74/lb7 0 2024-03-13T14:49:44.672 INFO:tasks.workunit.client.0.smithi176.stdout:8/323: dwrite d0/f52 [0,4194304] 0 2024-03-13T14:49:44.673 INFO:tasks.workunit.client.1.smithi196.stdout:1/537: fsync d1/d9/d33/fa5 0 2024-03-13T14:49:44.674 INFO:tasks.workunit.client.0.smithi176.stdout:7/359: dread d1/d1c/f5e [0,4194304] 0 2024-03-13T14:49:44.674 INFO:tasks.workunit.client.0.smithi176.stdout:2/401: unlink d1/f71 0 2024-03-13T14:49:44.675 INFO:tasks.workunit.client.1.smithi196.stdout:1/538: chown d1/d9/d33/d4b/d66/d7b 54966391 1 2024-03-13T14:49:44.675 INFO:tasks.workunit.client.1.smithi196.stdout:7/630: mkdir d0/d6/d33/d57/d8a/dd2 0 2024-03-13T14:49:44.678 INFO:tasks.workunit.client.1.smithi196.stdout:8/458: rename d0/d10/d12/d6f to d0/d16/d3d/d9c 0 2024-03-13T14:49:44.679 INFO:tasks.workunit.client.0.smithi176.stdout:1/355: mknod d2/c6b 0 2024-03-13T14:49:44.682 INFO:tasks.workunit.client.0.smithi176.stdout:2/402: write d1/dc/d2d/d59/d80/d5c/f5f [69428,126377] 0 2024-03-13T14:49:44.683 INFO:tasks.workunit.client.0.smithi176.stdout:7/360: write d1/d3f/f4d [564244,3772] 0 2024-03-13T14:49:44.686 INFO:tasks.workunit.client.1.smithi196.stdout:3/510: symlink db/d19/d22/d3a/d53/laa 0 2024-03-13T14:49:44.687 INFO:tasks.workunit.client.0.smithi176.stdout:7/361: fdatasync d1/d28/f2a 0 2024-03-13T14:49:44.687 INFO:tasks.workunit.client.0.smithi176.stdout:6/365: dread d0/d9/f50 [0,4194304] 0 2024-03-13T14:49:44.689 INFO:tasks.workunit.client.0.smithi176.stdout:7/362: chown d1/d3f/l47 62129 1 2024-03-13T14:49:44.690 INFO:tasks.workunit.client.1.smithi196.stdout:3/511: dread - db/d15/d44/d79/f80 zero size 2024-03-13T14:49:44.694 INFO:tasks.workunit.client.1.smithi196.stdout:8/459: creat d0/d16/d3d/d9c/f9d x:0 0 0 2024-03-13T14:49:44.695 INFO:tasks.workunit.client.0.smithi176.stdout:8/324: unlink d0/db/d39/f4c 0 2024-03-13T14:49:44.702 INFO:tasks.workunit.client.0.smithi176.stdout:4/399: rename d7/d10/l87 to d7/d8/d2f/l93 0 2024-03-13T14:49:44.703 INFO:tasks.workunit.client.0.smithi176.stdout:3/386: dread d4/d13/f19 [4194304,4194304] 0 2024-03-13T14:49:44.706 INFO:tasks.workunit.client.0.smithi176.stdout:0/346: dread d2/d5/f46 [0,4194304] 0 2024-03-13T14:49:44.707 INFO:tasks.workunit.client.0.smithi176.stdout:9/438: dread d0/db/d89/d32/d58/d67/f7e [0,4194304] 0 2024-03-13T14:49:44.708 INFO:tasks.workunit.client.0.smithi176.stdout:2/403: mknod d1/dc/d2d/d90/c93 0 2024-03-13T14:49:44.712 INFO:tasks.workunit.client.1.smithi196.stdout:1/539: link d1/d9/d33/f82 d1/d3f/d5b/fb8 0 2024-03-13T14:49:44.713 INFO:tasks.workunit.client.1.smithi196.stdout:1/540: write d1/d38/fb2 [264598,49307] 0 2024-03-13T14:49:44.715 INFO:tasks.workunit.client.1.smithi196.stdout:0/541: sync 2024-03-13T14:49:44.715 INFO:tasks.workunit.client.1.smithi196.stdout:4/455: sync 2024-03-13T14:49:44.717 INFO:tasks.workunit.client.0.smithi176.stdout:6/366: dwrite d0/d10/d19/f63 [0,4194304] 0 2024-03-13T14:49:44.717 INFO:tasks.workunit.client.0.smithi176.stdout:3/387: mknod d4/d28/d3a/d5b/c7b 0 2024-03-13T14:49:44.718 INFO:tasks.workunit.client.1.smithi196.stdout:4/456: write d1/d3/d7/f1d [2020695,78108] 0 2024-03-13T14:49:44.718 INFO:tasks.workunit.client.1.smithi196.stdout:0/542: chown d8/da/d25/d26/d35/d3b/d50/d5d/d7e/l98 3813 1 2024-03-13T14:49:44.719 INFO:tasks.workunit.client.0.smithi176.stdout:6/367: chown d0 15865615 1 2024-03-13T14:49:44.721 INFO:tasks.workunit.client.0.smithi176.stdout:4/400: mkdir d7/d8/d15/d94 0 2024-03-13T14:49:44.725 INFO:tasks.workunit.client.0.smithi176.stdout:9/439: mknod d0/d1e/d1f/d68/c91 0 2024-03-13T14:49:44.728 INFO:tasks.workunit.client.0.smithi176.stdout:8/325: truncate d0/db/dc/f35 5200376 0 2024-03-13T14:49:44.729 INFO:tasks.workunit.client.0.smithi176.stdout:7/363: dwrite d1/d1c/d26/d41/d2b/d40/d64/f60 [0,4194304] 0 2024-03-13T14:49:44.729 INFO:tasks.workunit.client.0.smithi176.stdout:3/388: creat d4/d28/d3a/f7c x:0 0 0 2024-03-13T14:49:44.735 INFO:tasks.workunit.client.0.smithi176.stdout:5/357: sync 2024-03-13T14:49:44.739 INFO:tasks.workunit.client.0.smithi176.stdout:6/368: mkdir d0/d10/d19/d27/d53/d76 0 2024-03-13T14:49:44.740 INFO:tasks.workunit.client.0.smithi176.stdout:8/326: stat d0/db/c38 0 2024-03-13T14:49:44.741 INFO:tasks.workunit.client.0.smithi176.stdout:2/404: dwrite d1/dc/d2d/d59/d80/d77/f84 [0,4194304] 0 2024-03-13T14:49:44.742 INFO:tasks.workunit.client.0.smithi176.stdout:9/440: creat d0/db/d89/d32/d58/d5f/f92 x:0 0 0 2024-03-13T14:49:44.743 INFO:tasks.workunit.client.0.smithi176.stdout:6/369: dread - d0/d10/d19/d27/d2b/d2e/f52 zero size 2024-03-13T14:49:44.747 INFO:tasks.workunit.client.0.smithi176.stdout:1/356: rename d2/de/d10/c5e to d2/de/d3f/c6c 0 2024-03-13T14:49:44.749 INFO:tasks.workunit.client.1.smithi196.stdout:2/410: truncate d0/d7/f3f 2517320 0 2024-03-13T14:49:44.754 INFO:tasks.workunit.client.0.smithi176.stdout:4/401: mkdir d7/d8/d95 0 2024-03-13T14:49:44.757 INFO:tasks.workunit.client.1.smithi196.stdout:6/623: sync 2024-03-13T14:49:44.760 INFO:tasks.workunit.client.0.smithi176.stdout:5/358: mkdir d5/d2c/d40/d76 0 2024-03-13T14:49:44.763 INFO:tasks.workunit.client.1.smithi196.stdout:9/437: dread d7/d9/d17/f51 [0,4194304] 0 2024-03-13T14:49:44.766 INFO:tasks.workunit.client.0.smithi176.stdout:6/370: unlink d0/d9/c45 0 2024-03-13T14:49:44.767 INFO:tasks.workunit.client.1.smithi196.stdout:3/512: dwrite db/d4c/f61 [0,4194304] 0 2024-03-13T14:49:44.767 INFO:tasks.workunit.client.1.smithi196.stdout:8/460: write d0/d16/d3d/d8b/f8e [494119,52425] 0 2024-03-13T14:49:44.768 INFO:tasks.workunit.client.0.smithi176.stdout:3/389: mknod d4/d13/d2d/c7d 0 2024-03-13T14:49:44.770 INFO:tasks.workunit.client.1.smithi196.stdout:8/461: fdatasync d0/d16/d3d/d8b/d3f/f64 0 2024-03-13T14:49:44.771 INFO:tasks.workunit.client.1.smithi196.stdout:9/438: dread d7/f20 [0,4194304] 0 2024-03-13T14:49:44.771 INFO:tasks.workunit.client.0.smithi176.stdout:8/327: dwrite d0/d24/f31 [0,4194304] 0 2024-03-13T14:49:44.772 INFO:tasks.workunit.client.0.smithi176.stdout:6/371: write d0/d10/d19/d27/d53/f56 [389323,91765] 0 2024-03-13T14:49:44.773 INFO:tasks.workunit.client.1.smithi196.stdout:8/462: stat d0/d10/d12/d1e/f83 0 2024-03-13T14:49:44.776 INFO:tasks.workunit.client.0.smithi176.stdout:3/390: truncate d4/d28/d31/f75 889026 0 2024-03-13T14:49:44.777 INFO:tasks.workunit.client.0.smithi176.stdout:4/402: mkdir d7/d8/d33/d96 0 2024-03-13T14:49:44.783 INFO:tasks.workunit.client.0.smithi176.stdout:4/403: dread - d7/d10/f8a zero size 2024-03-13T14:49:44.783 INFO:tasks.workunit.client.0.smithi176.stdout:3/391: chown d4/d13/f57 390469081 1 2024-03-13T14:49:44.783 INFO:tasks.workunit.client.0.smithi176.stdout:5/359: rmdir d5/d2c/d40/d6a 39 2024-03-13T14:49:44.784 INFO:tasks.workunit.client.1.smithi196.stdout:7/631: link d0/f1c d0/d78/d81/fd3 0 2024-03-13T14:49:44.788 INFO:tasks.workunit.client.1.smithi196.stdout:5/599: getdents d4/db/d14/d19/d1c/d2d 0 2024-03-13T14:49:44.795 INFO:tasks.workunit.client.0.smithi176.stdout:3/392: rmdir d4/d13/d16 39 2024-03-13T14:49:44.798 INFO:tasks.workunit.client.0.smithi176.stdout:5/360: mknod d5/d2e/c77 0 2024-03-13T14:49:44.798 INFO:tasks.workunit.client.0.smithi176.stdout:9/441: dread d0/d1e/d1f/f2c [0,4194304] 0 2024-03-13T14:49:44.801 INFO:tasks.workunit.client.0.smithi176.stdout:0/347: rename d2/l30 to d2/d5/dc/l67 0 2024-03-13T14:49:44.803 INFO:tasks.workunit.client.1.smithi196.stdout:6/624: unlink d6/d7/df/d27/d39/d44/d74/f88 0 2024-03-13T14:49:44.804 INFO:tasks.workunit.client.1.smithi196.stdout:2/411: dread d0/d7/d66/d67/f85 [0,4194304] 0 2024-03-13T14:49:44.806 INFO:tasks.workunit.client.0.smithi176.stdout:4/404: mknod d7/d8/c97 0 2024-03-13T14:49:44.807 INFO:tasks.workunit.client.1.smithi196.stdout:2/412: truncate d0/d6/d4b/f64 305323 0 2024-03-13T14:49:44.808 INFO:tasks.workunit.client.0.smithi176.stdout:0/348: chown d2/d5/dc/d60/d61/f66 86 1 2024-03-13T14:49:44.810 INFO:tasks.workunit.client.1.smithi196.stdout:3/513: mknod db/d37/cab 0 2024-03-13T14:49:44.811 INFO:tasks.workunit.client.0.smithi176.stdout:3/393: truncate d4/d13/d2d/d50/f3c 1609767 0 2024-03-13T14:49:44.812 INFO:tasks.workunit.client.0.smithi176.stdout:5/361: symlink d5/d31/l78 0 2024-03-13T14:49:44.815 INFO:tasks.workunit.client.0.smithi176.stdout:7/364: dwrite d1/d1c/f5e [0,4194304] 0 2024-03-13T14:49:44.816 INFO:tasks.workunit.client.0.smithi176.stdout:7/365: fdatasync d1/d28/f2d 0 2024-03-13T14:49:44.819 INFO:tasks.workunit.client.1.smithi196.stdout:7/632: creat d0/d9/d1e/d58/d61/fd4 x:0 0 0 2024-03-13T14:49:44.821 INFO:tasks.workunit.client.0.smithi176.stdout:6/372: rename d0/d10/d19/d27/d53 to d0/de/d38/d77 0 2024-03-13T14:49:44.824 INFO:tasks.workunit.client.0.smithi176.stdout:8/328: write d0/f3 [4230426,117756] 0 2024-03-13T14:49:44.825 INFO:tasks.workunit.client.0.smithi176.stdout:3/394: creat d4/f7e x:0 0 0 2024-03-13T14:49:44.827 INFO:tasks.workunit.client.0.smithi176.stdout:8/329: readlink d0/d24/l5a 0 2024-03-13T14:49:44.828 INFO:tasks.workunit.client.0.smithi176.stdout:3/395: fsync d4/d28/d3a/f7c 0 2024-03-13T14:49:44.833 INFO:tasks.workunit.client.0.smithi176.stdout:3/396: chown d4/f7e 2091008459 1 2024-03-13T14:49:44.836 INFO:tasks.workunit.client.1.smithi196.stdout:2/413: mknod d0/d6/d1d/c8b 0 2024-03-13T14:49:44.836 INFO:tasks.workunit.client.1.smithi196.stdout:1/541: dwrite d1/d9/d33/d4b/f1d [0,4194304] 0 2024-03-13T14:49:44.837 INFO:tasks.workunit.client.0.smithi176.stdout:7/366: symlink d1/d1c/d4a/l6c 0 2024-03-13T14:49:44.837 INFO:tasks.workunit.client.0.smithi176.stdout:0/349: dwrite d2/db/d31/f62 [0,4194304] 0 2024-03-13T14:49:44.844 INFO:tasks.workunit.client.0.smithi176.stdout:7/367: write d1/d7/f3d [305363,125159] 0 2024-03-13T14:49:44.846 INFO:tasks.workunit.client.0.smithi176.stdout:1/357: dwrite d2/de/d10/d16/f27 [4194304,4194304] 0 2024-03-13T14:49:44.847 INFO:tasks.workunit.client.1.smithi196.stdout:3/514: mknod db/d19/d22/d3a/d6a/cac 0 2024-03-13T14:49:44.848 INFO:tasks.workunit.client.1.smithi196.stdout:8/463: symlink d0/d10/d79/l9e 0 2024-03-13T14:49:44.850 INFO:tasks.workunit.client.1.smithi196.stdout:5/600: write d4/db/d3c/f84 [2365154,33906] 0 2024-03-13T14:49:44.851 INFO:tasks.workunit.client.0.smithi176.stdout:7/368: chown d1/d1c/d26/d41/d2b/d40/l49 124254 1 2024-03-13T14:49:44.851 INFO:tasks.workunit.client.1.smithi196.stdout:4/457: creat d1/d3/d7/d35/d76/f94 x:0 0 0 2024-03-13T14:49:44.853 INFO:tasks.workunit.client.0.smithi176.stdout:9/442: link d0/d17/c28 d0/db/d89/d4d/c93 0 2024-03-13T14:49:44.853 INFO:tasks.workunit.client.0.smithi176.stdout:5/362: getdents d5/d2c/d40/d76 0 2024-03-13T14:49:44.854 INFO:tasks.workunit.client.0.smithi176.stdout:0/350: creat d2/f68 x:0 0 0 2024-03-13T14:49:44.856 INFO:tasks.workunit.client.0.smithi176.stdout:4/405: chown d7/d2d/f2e 18114 1 2024-03-13T14:49:44.858 INFO:tasks.workunit.client.0.smithi176.stdout:4/406: readlink d7/d10/d2c/l7a 0 2024-03-13T14:49:44.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.867+0000 7ff277200700 1 -- 172.21.15.176:0/4041693848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 msgr2=0x7ff278103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:44.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.867+0000 7ff277200700 1 --2- 172.21.15.176:0/4041693848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff278103050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff268009910 tx=0x7ff268009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:44.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.869+0000 7ff277200700 1 -- 172.21.15.176:0/4041693848 shutdown_connections 2024-03-13T14:49:44.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.869+0000 7ff277200700 1 --2- 172.21.15.176:0/4041693848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff278103e30 0x7ff278104290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.869+0000 7ff277200700 1 --2- 172.21.15.176:0/4041693848 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff278103050 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.869+0000 7ff277200700 1 -- 172.21.15.176:0/4041693848 >> 172.21.15.176:0/4041693848 conn(0x7ff2780fe1b0 msgr2=0x7ff278100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:44.860 INFO:tasks.workunit.client.1.smithi196.stdout:0/543: write d8/da/d25/d26/d35/d3b/d50/f6d [729412,28823] 0 2024-03-13T14:49:44.861 INFO:tasks.workunit.client.0.smithi176.stdout:6/373: read d0/d9/f37 [3211038,11663] 0 2024-03-13T14:49:44.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.871+0000 7ff277200700 1 -- 172.21.15.176:0/4041693848 shutdown_connections 2024-03-13T14:49:44.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.873+0000 7ff277200700 1 -- 172.21.15.176:0/4041693848 wait complete. 2024-03-13T14:49:44.863 INFO:tasks.workunit.client.0.smithi176.stdout:5/363: chown d5/d22/d15/c2a 838118 1 2024-03-13T14:49:44.863 INFO:tasks.workunit.client.1.smithi196.stdout:9/439: creat d7/d24/d28/d6c/f8f x:0 0 0 2024-03-13T14:49:44.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.876+0000 7ff277200700 1 Processor -- start 2024-03-13T14:49:44.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.876+0000 7ff277200700 1 -- start start 2024-03-13T14:49:44.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.877+0000 7ff277200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff27819c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:44.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.877+0000 7ff277200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff278103e30 0x7ff27819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:44.866 INFO:tasks.workunit.client.0.smithi176.stdout:8/330: creat d0/d5/d11/d47/f63 x:0 0 0 2024-03-13T14:49:44.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.877+0000 7ff277200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27819d320 con 0x7ff278102c30 2024-03-13T14:49:44.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.877+0000 7ff277200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27819d460 con 0x7ff278103e30 2024-03-13T14:49:44.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.879+0000 7ff275e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff27819c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:44.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.879+0000 7ff275e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff27819c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:43270/0 (socket says 172.21.15.176:43270) 2024-03-13T14:49:44.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.879+0000 7ff275e00700 1 -- 172.21.15.176:0/4219607131 learned_addr learned my addr 172.21.15.176:0/4219607131 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:44.869 INFO:tasks.workunit.client.1.smithi196.stdout:8/464: mknod d0/d10/d12/d17/d42/c9f 0 2024-03-13T14:49:44.870 INFO:tasks.workunit.client.1.smithi196.stdout:4/458: readlink d1/d3/d7/d35/d3e/l7b 0 2024-03-13T14:49:44.870 INFO:tasks.workunit.client.1.smithi196.stdout:2/414: dwrite d0/d6/d1d/f51 [4194304,4194304] 0 2024-03-13T14:49:44.871 INFO:tasks.workunit.client.0.smithi176.stdout:0/351: creat d2/f69 x:0 0 0 2024-03-13T14:49:44.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.883+0000 7ff275e00700 1 -- 172.21.15.176:0/4219607131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff278103e30 msgr2=0x7ff27819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:44.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.883+0000 7ff275e00700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff278103e30 0x7ff27819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:44.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.883+0000 7ff275e00700 1 -- 172.21.15.176:0/4219607131 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2680095f0 con 0x7ff278102c30 2024-03-13T14:49:44.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.884+0000 7ff275e00700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff27819c7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff26800bd70 tx=0x7ff26800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:44.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.884+0000 7ff26ea00700 1 -- 172.21.15.176:0/4219607131 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26801d070 con 0x7ff278102c30 2024-03-13T14:49:44.873 INFO:tasks.workunit.client.1.smithi196.stdout:8/465: dread - d0/d10/f77 zero size 2024-03-13T14:49:44.873 INFO:tasks.workunit.client.1.smithi196.stdout:5/601: rename d4/db/d14/d19/d1c/d2d/d82/d8b/d9b to d4/db/d14/d19/d1c/d2d/d82/d8b/dbd 0 2024-03-13T14:49:44.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.885+0000 7ff26ea00700 1 -- 172.21.15.176:0/4219607131 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff26800f460 con 0x7ff278102c30 2024-03-13T14:49:44.874 INFO:tasks.workunit.client.0.smithi176.stdout:0/352: write d2/d5/d1e/f4e [2020982,76174] 0 2024-03-13T14:49:44.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.885+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2781a1eb0 con 0x7ff278102c30 2024-03-13T14:49:44.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.886+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2781a2350 con 0x7ff278102c30 2024-03-13T14:49:44.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.886+0000 7ff26ea00700 1 -- 172.21.15.176:0/4219607131 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff268021620 con 0x7ff278102c30 2024-03-13T14:49:44.875 INFO:tasks.workunit.client.1.smithi196.stdout:8/466: chown d0/d16/d3d/d9c 1 1 2024-03-13T14:49:44.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.888+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff258005320 con 0x7ff278102c30 2024-03-13T14:49:44.878 INFO:tasks.workunit.client.0.smithi176.stdout:1/358: dwrite d2/d29/f39 [0,4194304] 0 2024-03-13T14:49:44.878 INFO:tasks.workunit.client.1.smithi196.stdout:6/625: write d6/d7/df/d27/f64 [177834,85334] 0 2024-03-13T14:49:44.879 INFO:tasks.workunit.client.0.smithi176.stdout:2/405: sync 2024-03-13T14:49:44.880 INFO:tasks.workunit.client.1.smithi196.stdout:0/544: mknod d8/da/d25/d26/d48/d82/cb1 0 2024-03-13T14:49:44.884 INFO:tasks.workunit.client.1.smithi196.stdout:6/626: chown d6/d57/c85 18 1 2024-03-13T14:49:44.884 INFO:tasks.workunit.client.1.smithi196.stdout:2/415: read d0/d6/f14 [1672118,115977] 0 2024-03-13T14:49:44.885 INFO:tasks.workunit.client.1.smithi196.stdout:9/440: truncate d7/de/d4b/f6a 1023861 0 2024-03-13T14:49:44.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.897+0000 7ff26ea00700 1 -- 172.21.15.176:0/4219607131 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7ff26800fa40 con 0x7ff278102c30 2024-03-13T14:49:44.886 INFO:tasks.workunit.client.1.smithi196.stdout:1/542: dwrite d1/f37 [0,4194304] 0 2024-03-13T14:49:44.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.899+0000 7ff26ea00700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff25c06ddf0 0x7ff25c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:44.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.900+0000 7ff26ea00700 1 -- 172.21.15.176:0/4219607131 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff26808f230 con 0x7ff278102c30 2024-03-13T14:49:44.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.901+0000 7ff275400700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff25c06ddf0 0x7ff25c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:44.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.903+0000 7ff275400700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff25c06ddf0 0x7ff25c0702b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff264005950 tx=0x7ff2640058e0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:44.895 INFO:tasks.workunit.client.0.smithi176.stdout:8/331: symlink d0/db/d39/d56/l64 0 2024-03-13T14:49:44.896 INFO:tasks.workunit.client.1.smithi196.stdout:9/441: chown d7/d9/f11 12109 1 2024-03-13T14:49:44.897 INFO:tasks.workunit.client.0.smithi176.stdout:8/332: fsync d0/d24/f31 0 2024-03-13T14:49:44.898 INFO:tasks.workunit.client.1.smithi196.stdout:5/602: mknod d4/db/d14/d19/d1c/d25/d38/cbe 0 2024-03-13T14:49:44.899 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:44.910+0000 7ff26ea00700 1 -- 172.21.15.176:0/4219607131 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7ff26805c470 con 0x7ff278102c30 2024-03-13T14:49:44.900 INFO:tasks.workunit.client.0.smithi176.stdout:6/374: creat d0/d10/d19/d27/d2b/d2e/d3e/f78 x:0 0 0 2024-03-13T14:49:44.903 INFO:tasks.workunit.client.1.smithi196.stdout:4/459: rmdir d1/d3/d7/d35/d76 39 2024-03-13T14:49:44.905 INFO:tasks.workunit.client.0.smithi176.stdout:3/397: write d4/d28/d31/f4b [246632,5212] 0 2024-03-13T14:49:44.905 INFO:tasks.workunit.client.0.smithi176.stdout:9/443: creat d0/db/d89/d32/d76/d82/f94 x:0 0 0 2024-03-13T14:49:44.906 INFO:tasks.workunit.client.1.smithi196.stdout:8/467: rename d0/c18 to d0/d63/ca0 0 2024-03-13T14:49:44.911 INFO:tasks.workunit.client.1.smithi196.stdout:8/468: write d0/d10/f77 [45184,109723] 0 2024-03-13T14:49:44.911 INFO:tasks.workunit.client.1.smithi196.stdout:6/627: creat d6/d2a/dc/da3/fc3 x:0 0 0 2024-03-13T14:49:44.912 INFO:tasks.workunit.client.0.smithi176.stdout:0/353: rmdir d2/d5/dc/d60/d61 39 2024-03-13T14:49:44.915 INFO:tasks.workunit.client.0.smithi176.stdout:4/407: creat d7/d8/d33/d96/f98 x:0 0 0 2024-03-13T14:49:44.916 INFO:tasks.workunit.client.1.smithi196.stdout:9/442: creat d7/de/d4b/d1d/d22/d4d/f90 x:0 0 0 2024-03-13T14:49:44.916 INFO:tasks.workunit.client.1.smithi196.stdout:6/628: stat d6/d7/dd/faf 0 2024-03-13T14:49:44.917 INFO:tasks.workunit.client.1.smithi196.stdout:2/416: dwrite d0/d7/d9/d5a/f7c [0,4194304] 0 2024-03-13T14:49:44.919 INFO:tasks.workunit.client.0.smithi176.stdout:5/364: creat d5/d2c/d40/d6a/f79 x:0 0 0 2024-03-13T14:49:44.921 INFO:tasks.workunit.client.0.smithi176.stdout:8/333: dwrite d0/db/dc/d40/f48 [0,4194304] 0 2024-03-13T14:49:44.921 INFO:tasks.workunit.client.1.smithi196.stdout:8/469: stat d0/d10/d70/d8d/c95 0 2024-03-13T14:49:44.926 INFO:tasks.workunit.client.0.smithi176.stdout:6/375: symlink d0/de/l79 0 2024-03-13T14:49:44.927 INFO:tasks.workunit.client.1.smithi196.stdout:6/629: fdatasync d6/d9/daa/fc2 0 2024-03-13T14:49:44.928 INFO:tasks.workunit.client.1.smithi196.stdout:4/460: chown d1/d3/d7/d35/l67 154876 1 2024-03-13T14:49:44.928 INFO:tasks.workunit.client.1.smithi196.stdout:0/545: dread d8/f9 [0,4194304] 0 2024-03-13T14:49:44.930 INFO:tasks.workunit.client.1.smithi196.stdout:9/443: symlink d7/de/d16/d3c/l91 0 2024-03-13T14:49:44.931 INFO:tasks.workunit.client.1.smithi196.stdout:6/630: stat d6/d6f 0 2024-03-13T14:49:44.935 INFO:tasks.workunit.client.1.smithi196.stdout:6/631: chown d6/d9/f7b 21470 1 2024-03-13T14:49:44.936 INFO:tasks.workunit.client.1.smithi196.stdout:0/546: chown d8/da/d25/d26/d48/d82/f8b 166656290 1 2024-03-13T14:49:44.936 INFO:tasks.workunit.client.1.smithi196.stdout:9/444: chown d7/de/d4b/d1d/f75 3002 1 2024-03-13T14:49:44.937 INFO:tasks.workunit.client.1.smithi196.stdout:7/633: dwrite d0/d6/d33/f40 [0,4194304] 0 2024-03-13T14:49:44.943 INFO:tasks.workunit.client.0.smithi176.stdout:5/365: mknod d5/d48/d49/c7a 0 2024-03-13T14:49:44.943 INFO:tasks.workunit.client.0.smithi176.stdout:6/376: creat d0/d10/d19/d27/d2b/d2e/f7a x:0 0 0 2024-03-13T14:49:44.946 INFO:tasks.workunit.client.0.smithi176.stdout:8/334: chown d0/d5/f17 353818143 1 2024-03-13T14:49:44.947 INFO:tasks.workunit.client.0.smithi176.stdout:1/359: creat d2/de/d10/f6d x:0 0 0 2024-03-13T14:49:44.948 INFO:tasks.workunit.client.0.smithi176.stdout:6/377: fsync d0/d10/d19/d27/f61 0 2024-03-13T14:49:44.948 INFO:tasks.workunit.client.1.smithi196.stdout:6/632: mknod d6/d7/cc4 0 2024-03-13T14:49:44.952 INFO:tasks.workunit.client.0.smithi176.stdout:3/398: dwrite d4/d13/d16/d3d/f54 [0,4194304] 0 2024-03-13T14:49:44.955 INFO:tasks.workunit.client.1.smithi196.stdout:6/633: truncate d6/d2a/dc/da3/fc3 279570 0 2024-03-13T14:49:44.956 INFO:tasks.workunit.client.0.smithi176.stdout:9/444: rename d0/d1e/f4e to d0/db/d89/d32/f95 0 2024-03-13T14:49:44.958 INFO:tasks.workunit.client.0.smithi176.stdout:4/408: link d7/d8/d45/f76 d7/d8/d45/f99 0 2024-03-13T14:49:44.961 INFO:tasks.workunit.client.0.smithi176.stdout:9/445: dread - d0/d1e/d6f/f7b zero size 2024-03-13T14:49:44.965 INFO:tasks.workunit.client.1.smithi196.stdout:6/634: dread d6/d2a/f62 [0,4194304] 0 2024-03-13T14:49:44.966 INFO:tasks.workunit.client.1.smithi196.stdout:9/445: rmdir d7/de/d4b 39 2024-03-13T14:49:44.967 INFO:tasks.workunit.client.0.smithi176.stdout:9/446: write d0/db/d89/d7a/d8b/f8f [579514,19408] 0 2024-03-13T14:49:44.969 INFO:tasks.workunit.client.0.smithi176.stdout:8/335: rmdir d0/db 39 2024-03-13T14:49:44.974 INFO:tasks.workunit.client.1.smithi196.stdout:8/470: dread d0/f51 [0,4194304] 0 2024-03-13T14:49:44.975 INFO:tasks.workunit.client.0.smithi176.stdout:7/369: truncate d1/d1c/d26/d41/f3e 463898 0 2024-03-13T14:49:44.976 INFO:tasks.workunit.client.1.smithi196.stdout:1/543: dwrite d1/d38/f30 [0,4194304] 0 2024-03-13T14:49:44.981 INFO:tasks.workunit.client.0.smithi176.stdout:5/366: truncate d5/f8 4224784 0 2024-03-13T14:49:44.986 INFO:tasks.workunit.client.1.smithi196.stdout:5/603: dwrite d4/f8 [0,4194304] 0 2024-03-13T14:49:44.989 INFO:tasks.workunit.client.1.smithi196.stdout:0/547: creat d8/da/d25/d26/d35/d3b/d5f/d33/fb2 x:0 0 0 2024-03-13T14:49:44.990 INFO:tasks.workunit.client.0.smithi176.stdout:5/367: truncate d5/d22/d15/f27 1775345 0 2024-03-13T14:49:44.990 INFO:tasks.workunit.client.1.smithi196.stdout:4/461: dread d1/d1e/f33 [0,4194304] 0 2024-03-13T14:49:44.999 INFO:tasks.workunit.client.1.smithi196.stdout:2/417: dread d0/d6/d1d/f27 [0,4194304] 0 2024-03-13T14:49:44.999 INFO:tasks.workunit.client.1.smithi196.stdout:4/462: stat d1/d3/d7/c18 0 2024-03-13T14:49:44.999 INFO:tasks.workunit.client.1.smithi196.stdout:2/418: dread - d0/d6/d4b/f7d zero size 2024-03-13T14:49:45.000 INFO:tasks.workunit.client.0.smithi176.stdout:2/406: dwrite d1/ff [0,4194304] 0 2024-03-13T14:49:45.002 INFO:tasks.workunit.client.0.smithi176.stdout:6/378: fsync d0/d10/d19/d27/d2b/d2e/f7a 0 2024-03-13T14:49:45.006 INFO:tasks.workunit.client.1.smithi196.stdout:2/419: dread d0/f78 [0,4194304] 0 2024-03-13T14:49:45.007 INFO:tasks.workunit.client.0.smithi176.stdout:2/407: fdatasync d1/dc/d2d/d59/f8a 0 2024-03-13T14:49:45.008 INFO:tasks.workunit.client.0.smithi176.stdout:7/370: dwrite d1/d3f/d4b/f6b [0,4194304] 0 2024-03-13T14:49:45.011 INFO:tasks.workunit.client.0.smithi176.stdout:6/379: read d0/d10/d19/d27/f61 [925653,42334] 0 2024-03-13T14:49:45.012 INFO:tasks.workunit.client.1.smithi196.stdout:6/635: dread d6/d2a/f93 [0,4194304] 0 2024-03-13T14:49:45.012 INFO:tasks.workunit.client.1.smithi196.stdout:0/548: truncate d8/da/f10 5095161 0 2024-03-13T14:49:45.012 INFO:tasks.workunit.client.1.smithi196.stdout:5/604: truncate d4/db/d14/d19/d26/faa 795218 0 2024-03-13T14:49:45.018 INFO:tasks.workunit.client.0.smithi176.stdout:8/336: dwrite d0/d24/f32 [4194304,4194304] 0 2024-03-13T14:49:45.019 INFO:tasks.workunit.client.1.smithi196.stdout:4/463: mknod d1/d3/d4c/d72/d7d/c95 0 2024-03-13T14:49:45.024 INFO:tasks.workunit.client.0.smithi176.stdout:9/447: truncate d0/db/d89/d32/d58/d67/f7e 1478557 0 2024-03-13T14:49:45.024 INFO:tasks.workunit.client.0.smithi176.stdout:1/360: creat d2/de/d10/d16/d59/f6e x:0 0 0 2024-03-13T14:49:45.026 INFO:tasks.workunit.client.0.smithi176.stdout:6/380: read d0/de/f12 [2751744,59676] 0 2024-03-13T14:49:45.030 INFO:tasks.workunit.client.1.smithi196.stdout:0/549: mkdir d8/da/d25/d26/d48/d82/db3 0 2024-03-13T14:49:45.031 INFO:tasks.workunit.client.1.smithi196.stdout:6/636: fdatasync d6/d9/d17/d19/f22 0 2024-03-13T14:49:45.035 INFO:tasks.workunit.client.0.smithi176.stdout:3/399: link d4/d13/d2d/c7d d4/c7f 0 2024-03-13T14:49:45.038 INFO:tasks.workunit.client.0.smithi176.stdout:2/408: dwrite d1/dc/d2d/d59/d80/d77/f84 [4194304,4194304] 0 2024-03-13T14:49:45.039 INFO:tasks.workunit.client.0.smithi176.stdout:5/368: mkdir d5/d48/d4c/d7b 0 2024-03-13T14:49:45.040 INFO:tasks.workunit.client.1.smithi196.stdout:7/634: dread d0/d9/d1e/d3f/f82 [0,4194304] 0 2024-03-13T14:49:45.040 INFO:tasks.workunit.client.0.smithi176.stdout:0/354: rename d2/d5/d28/c36 to d2/d5/dc/d60/c6a 0 2024-03-13T14:49:45.042 INFO:tasks.workunit.client.1.smithi196.stdout:4/464: rename d1/d3/c28 to d1/d3/d4c/d72/d7d/d80/d55/c96 0 2024-03-13T14:49:45.042 INFO:tasks.workunit.client.0.smithi176.stdout:7/371: creat d1/d28/d3a/d1b/d18/f6d x:0 0 0 2024-03-13T14:49:45.042 INFO:tasks.workunit.client.0.smithi176.stdout:2/409: read - d1/dc/d2d/d38/d5d/f79 zero size 2024-03-13T14:49:45.044 INFO:tasks.workunit.client.0.smithi176.stdout:9/448: unlink d0/db/d89/d32/d58/d5f/f92 0 2024-03-13T14:49:45.046 INFO:tasks.workunit.client.0.smithi176.stdout:6/381: creat d0/d10/d19/d27/d2b/d2e/f7b x:0 0 0 2024-03-13T14:49:45.046 INFO:tasks.workunit.client.0.smithi176.stdout:2/410: chown d1/d2a/d33/l47 52718 1 2024-03-13T14:49:45.046 INFO:tasks.workunit.client.0.smithi176.stdout:8/337: creat d0/db/d39/d56/f65 x:0 0 0 2024-03-13T14:49:45.048 INFO:tasks.workunit.client.1.smithi196.stdout:9/446: dread d7/de/d4b/f43 [0,4194304] 0 2024-03-13T14:49:45.048 INFO:tasks.workunit.client.0.smithi176.stdout:0/355: truncate d2/d5/dc/d59/f5a 788396 0 2024-03-13T14:49:45.051 INFO:tasks.workunit.client.0.smithi176.stdout:3/400: mkdir d4/d13/d16/d80 0 2024-03-13T14:49:45.051 INFO:tasks.workunit.client.0.smithi176.stdout:8/338: write d0/f30 [1326608,42687] 0 2024-03-13T14:49:45.052 INFO:tasks.workunit.client.1.smithi196.stdout:5/605: getdents d4/db/d14/d19/d1c/d25/d38 0 2024-03-13T14:49:45.052 INFO:tasks.workunit.client.1.smithi196.stdout:6/637: chown d6/d2a/d23/d28/f3f 896 1 2024-03-13T14:49:45.053 INFO:tasks.workunit.client.1.smithi196.stdout:1/544: dwrite d1/d9/d4e/d55/f6f [0,4194304] 0 2024-03-13T14:49:45.054 INFO:tasks.workunit.client.0.smithi176.stdout:4/409: truncate d7/d8/f39 3253652 0 2024-03-13T14:49:45.056 INFO:tasks.workunit.client.0.smithi176.stdout:8/339: chown d0/d5/d11/d47 29586 1 2024-03-13T14:49:45.057 INFO:tasks.workunit.client.0.smithi176.stdout:9/449: readlink d0/db/d89/d32/d58/d5f/l8c 0 2024-03-13T14:49:45.057 INFO:tasks.workunit.client.1.smithi196.stdout:4/465: stat d1/d3/d7/d35/f52 0 2024-03-13T14:49:45.059 INFO:tasks.workunit.client.0.smithi176.stdout:4/410: write d7/d10/d2c/f77 [267111,63829] 0 2024-03-13T14:49:45.062 INFO:tasks.workunit.client.0.smithi176.stdout:4/411: read - d7/d75/f89 zero size 2024-03-13T14:49:45.063 INFO:tasks.workunit.client.1.smithi196.stdout:8/471: dwrite d0/d16/d3d/d8b/f38 [0,4194304] 0 2024-03-13T14:49:45.065 INFO:tasks.workunit.client.0.smithi176.stdout:7/372: dwrite d1/d28/d3a/f14 [0,4194304] 0 2024-03-13T14:49:45.070 INFO:tasks.workunit.client.1.smithi196.stdout:9/447: symlink d7/d9/d17/l92 0 2024-03-13T14:49:45.072 INFO:tasks.workunit.client.0.smithi176.stdout:1/361: creat d2/de/d3f/f6f x:0 0 0 2024-03-13T14:49:45.075 INFO:tasks.workunit.client.1.smithi196.stdout:0/550: getdents d8/da/d25/d26/d35/d3b/d50/d5d/da5 0 2024-03-13T14:49:45.075 INFO:tasks.workunit.client.1.smithi196.stdout:7/635: dread d0/d9/d31/d50/f75 [0,4194304] 0 2024-03-13T14:49:45.077 INFO:tasks.workunit.client.0.smithi176.stdout:2/411: unlink d1/dc/d2d/d59/f86 0 2024-03-13T14:49:45.080 INFO:tasks.workunit.client.1.smithi196.stdout:0/551: chown d8/da/d25/d26/d54/d88/ca4 144410725 1 2024-03-13T14:49:45.081 INFO:tasks.workunit.client.0.smithi176.stdout:0/356: mknod d2/db/d31/c6b 0 2024-03-13T14:49:45.083 INFO:tasks.workunit.client.1.smithi196.stdout:1/545: mkdir d1/d9/d4e/d55/d63/db9 0 2024-03-13T14:49:45.085 INFO:tasks.workunit.client.1.smithi196.stdout:2/420: truncate d0/f77 4108381 0 2024-03-13T14:49:45.086 INFO:tasks.workunit.client.0.smithi176.stdout:9/450: readlink d0/l55 0 2024-03-13T14:49:45.087 INFO:tasks.workunit.client.0.smithi176.stdout:5/369: link d5/d2e/d5f/f6f d5/d2e/d5f/f7c 0 2024-03-13T14:49:45.090 INFO:tasks.workunit.client.1.smithi196.stdout:8/472: stat d0/d10/d70/c73 0 2024-03-13T14:49:45.090 INFO:tasks.workunit.client.1.smithi196.stdout:0/552: write d8/da/d25/d26/d35/d3b/d50/f6d [900661,13341] 0 2024-03-13T14:49:45.090 INFO:tasks.workunit.client.1.smithi196.stdout:8/473: stat d0/d10/d12/d1e/f5e 0 2024-03-13T14:49:45.091 INFO:tasks.workunit.client.0.smithi176.stdout:9/451: dread d0/db/f49 [0,4194304] 0 2024-03-13T14:49:45.091 INFO:tasks.workunit.client.1.smithi196.stdout:6/638: mknod d6/d57/d91/cc5 0 2024-03-13T14:49:45.091 INFO:tasks.workunit.client.0.smithi176.stdout:4/412: mknod d7/d8/d33/c9a 0 2024-03-13T14:49:45.094 INFO:tasks.workunit.client.1.smithi196.stdout:8/474: write d0/d10/d12/d1e/f9b [1022093,33402] 0 2024-03-13T14:49:45.095 INFO:tasks.workunit.client.1.smithi196.stdout:1/546: truncate d1/d9/d33/d4b/d66/d7b/fa9 832370 0 2024-03-13T14:49:45.096 INFO:tasks.workunit.client.0.smithi176.stdout:6/382: mkdir d0/de/d38/d77/d76/d7c 0 2024-03-13T14:49:45.096 INFO:tasks.workunit.client.0.smithi176.stdout:7/373: rmdir d1/d28/d3a/d1b 39 2024-03-13T14:49:45.096 INFO:tasks.workunit.client.0.smithi176.stdout:4/413: chown d7/d8/d33/l41 72499138 1 2024-03-13T14:49:45.099 INFO:tasks.workunit.client.1.smithi196.stdout:1/547: readlink d1/d9/d4e/d55/d63/l84 0 2024-03-13T14:49:45.100 INFO:tasks.workunit.client.0.smithi176.stdout:2/412: dread d1/dc/d2d/f2f [0,4194304] 0 2024-03-13T14:49:45.102 INFO:tasks.workunit.client.0.smithi176.stdout:1/362: rename d2/fc to d2/d29/f70 0 2024-03-13T14:49:45.104 INFO:tasks.workunit.client.1.smithi196.stdout:9/448: mkdir d7/de/d4b/d1d/d93 0 2024-03-13T14:49:45.108 INFO:tasks.workunit.client.1.smithi196.stdout:4/466: rename d1/l2d to d1/d3/d7/d35/l97 0 2024-03-13T14:49:45.111 INFO:tasks.workunit.client.1.smithi196.stdout:0/553: chown d8/da/d25/d26/d35/d3b/d50/d55/c9c 0 1 2024-03-13T14:49:45.112 INFO:tasks.workunit.client.1.smithi196.stdout:0/554: chown d8/da/d74/f8f 51121164 1 2024-03-13T14:49:45.113 INFO:tasks.workunit.client.0.smithi176.stdout:5/370: dwrite d5/d31/d47/f5c [0,4194304] 0 2024-03-13T14:49:45.113 INFO:tasks.workunit.client.0.smithi176.stdout:3/401: mkdir d4/d13/d81 0 2024-03-13T14:49:45.119 INFO:tasks.workunit.client.0.smithi176.stdout:8/340: mknod d0/db/c66 0 2024-03-13T14:49:45.122 INFO:tasks.workunit.client.1.smithi196.stdout:9/449: mkdir d7/de/d74/d94 0 2024-03-13T14:49:45.123 INFO:tasks.workunit.client.0.smithi176.stdout:9/452: truncate d0/d1e/f2e 2698867 0 2024-03-13T14:49:45.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:45 smithi176 ceph-mon[29064]: pgmap v11: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 29 MiB/s rd, 142 MiB/s wr, 262 op/s 2024-03-13T14:49:45.123 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:45 smithi176 ceph-mon[29064]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:45.125 INFO:tasks.workunit.client.0.smithi176.stdout:7/374: dwrite d1/d1c/d4a/f62 [0,4194304] 0 2024-03-13T14:49:45.128 INFO:tasks.workunit.client.0.smithi176.stdout:4/414: stat d7/d8/cb 0 2024-03-13T14:49:45.129 INFO:tasks.workunit.client.0.smithi176.stdout:7/375: fsync d1/d1c/d26/d41/d2b/f67 0 2024-03-13T14:49:45.130 INFO:tasks.workunit.client.0.smithi176.stdout:4/415: chown d7/fe 115722802 1 2024-03-13T14:49:45.133 INFO:tasks.workunit.client.1.smithi196.stdout:0/555: chown d8/da/d25/d26/d35/d3b/d5f/d17/f51 3629756 1 2024-03-13T14:49:45.134 INFO:tasks.workunit.client.0.smithi176.stdout:4/416: read d7/fe [876679,13551] 0 2024-03-13T14:49:45.137 INFO:tasks.workunit.client.1.smithi196.stdout:6/639: dread d6/d9/d17/d19/fa8 [0,4194304] 0 2024-03-13T14:49:45.138 INFO:tasks.workunit.client.1.smithi196.stdout:0/556: fsync d8/da/d25/d26/d35/d3b/d5f/d17/f4d 0 2024-03-13T14:49:45.139 INFO:tasks.workunit.client.0.smithi176.stdout:1/363: unlink d2/f9 0 2024-03-13T14:49:45.140 INFO:tasks.workunit.client.1.smithi196.stdout:5/606: write d4/db/d14/d19/d1c/d25/d38/d47/f64 [680108,66121] 0 2024-03-13T14:49:45.141 INFO:tasks.workunit.client.1.smithi196.stdout:0/557: fsync d8/da/d25/d26/d35/d3b/d5f/d33/fb2 0 2024-03-13T14:49:45.147 INFO:tasks.workunit.client.1.smithi196.stdout:1/548: symlink d1/lba 0 2024-03-13T14:49:45.149 INFO:tasks.workunit.client.1.smithi196.stdout:2/421: write d0/d6/f14 [185885,32394] 0 2024-03-13T14:49:45.149 INFO:tasks.workunit.client.0.smithi176.stdout:6/383: dread d0/d10/d19/d27/f4d [0,4194304] 0 2024-03-13T14:49:45.150 INFO:tasks.workunit.client.1.smithi196.stdout:9/450: mknod d7/de/d4b/d1d/d35/c95 0 2024-03-13T14:49:45.151 INFO:tasks.workunit.client.0.smithi176.stdout:0/357: mknod d2/d5/dc/d60/d61/c6c 0 2024-03-13T14:49:45.152 INFO:tasks.workunit.client.0.smithi176.stdout:6/384: chown d0/de/f49 1082 1 2024-03-13T14:49:45.153 INFO:tasks.workunit.client.0.smithi176.stdout:0/358: write d2/db/d31/f51 [259142,2884] 0 2024-03-13T14:49:45.155 INFO:tasks.workunit.client.0.smithi176.stdout:6/385: dread - d0/d10/d19/d27/d2b/f65 zero size 2024-03-13T14:49:45.157 INFO:tasks.workunit.client.1.smithi196.stdout:6/640: rmdir d6/d7/df/d27/d39 39 2024-03-13T14:49:45.158 INFO:tasks.workunit.client.0.smithi176.stdout:6/386: stat d0/d10/d19/d27/d2b/d2e/c35 0 2024-03-13T14:49:45.160 INFO:tasks.workunit.client.1.smithi196.stdout:2/422: dread d0/d7/f21 [0,4194304] 0 2024-03-13T14:49:45.162 INFO:tasks.workunit.client.1.smithi196.stdout:2/423: write d0/d6/d4b/d5f/f81 [552234,97148] 0 2024-03-13T14:49:45.164 INFO:tasks.workunit.client.0.smithi176.stdout:4/417: mkdir d7/d2d/d9b 0 2024-03-13T14:49:45.165 INFO:tasks.workunit.client.1.smithi196.stdout:2/424: read d0/d6/d1d/f7f [2674948,25488] 0 2024-03-13T14:49:45.166 INFO:tasks.workunit.client.1.smithi196.stdout:2/425: chown d0/d7/d9 30495132 1 2024-03-13T14:49:45.167 INFO:tasks.workunit.client.0.smithi176.stdout:8/341: dread d0/f3 [0,4194304] 0 2024-03-13T14:49:45.169 INFO:tasks.workunit.client.1.smithi196.stdout:4/467: creat d1/d3/f98 x:0 0 0 2024-03-13T14:49:45.170 INFO:tasks.workunit.client.0.smithi176.stdout:1/364: mkdir d2/d30/d63/d22/d71 0 2024-03-13T14:49:45.171 INFO:tasks.workunit.client.0.smithi176.stdout:3/402: truncate d4/f1d 190245 0 2024-03-13T14:49:45.172 INFO:tasks.workunit.client.0.smithi176.stdout:2/413: write d1/dc/d2d/d4b/f63 [12928,45763] 0 2024-03-13T14:49:45.174 INFO:tasks.workunit.client.1.smithi196.stdout:9/451: rename d7/d24/d28/l64 to d7/de/d74/l96 0 2024-03-13T14:49:45.176 INFO:tasks.workunit.client.1.smithi196.stdout:6/641: truncate d6/d57/fb0 2642382 0 2024-03-13T14:49:45.176 INFO:tasks.workunit.client.0.smithi176.stdout:5/371: dread d5/d22/f1e [0,4194304] 0 2024-03-13T14:49:45.178 INFO:tasks.workunit.client.0.smithi176.stdout:9/453: mknod d0/d40/d53/d6d/c96 0 2024-03-13T14:49:45.178 INFO:tasks.workunit.client.1.smithi196.stdout:8/475: link d0/c3 d0/d16/d3d/ca1 0 2024-03-13T14:49:45.184 INFO:tasks.workunit.client.1.smithi196.stdout:1/549: truncate d1/d3f/d5b/fb8 310095 0 2024-03-13T14:49:45.185 INFO:tasks.workunit.client.1.smithi196.stdout:1/550: chown d1/d28/d34/d67/f72 55 1 2024-03-13T14:49:45.185 INFO:tasks.workunit.client.1.smithi196.stdout:6/642: dread d6/d9/daa/fc2 [0,4194304] 0 2024-03-13T14:49:45.186 INFO:tasks.workunit.client.1.smithi196.stdout:4/468: fdatasync d1/d3/d4c/d72/d7d/d80/f86 0 2024-03-13T14:49:45.190 INFO:tasks.workunit.client.0.smithi176.stdout:8/342: creat d0/d24/d29/f67 x:0 0 0 2024-03-13T14:49:45.191 INFO:tasks.workunit.client.1.smithi196.stdout:9/452: creat d7/de/d4b/f97 x:0 0 0 2024-03-13T14:49:45.192 INFO:tasks.workunit.client.0.smithi176.stdout:8/343: chown d0/db/dc/l2d 3514 1 2024-03-13T14:49:45.196 INFO:tasks.workunit.client.1.smithi196.stdout:9/453: dread d7/d9/f77 [0,4194304] 0 2024-03-13T14:49:45.198 INFO:tasks.workunit.client.0.smithi176.stdout:2/414: creat d1/dc/d2d/d4b/d69/f94 x:0 0 0 2024-03-13T14:49:45.201 INFO:tasks.workunit.client.0.smithi176.stdout:2/415: chown d1/dc/d1c/f37 2 1 2024-03-13T14:49:45.201 INFO:tasks.workunit.client.0.smithi176.stdout:9/454: mkdir d0/d40/d53/d6d/d97 0 2024-03-13T14:49:45.202 INFO:tasks.workunit.client.0.smithi176.stdout:9/455: stat d0/d40/c4b 0 2024-03-13T14:49:45.203 INFO:tasks.workunit.client.0.smithi176.stdout:0/359: rename d2/d5/d1e/f38 to d2/d5/d1e/f6d 0 2024-03-13T14:49:45.204 INFO:tasks.workunit.client.0.smithi176.stdout:0/360: dread - d2/f50 zero size 2024-03-13T14:49:45.208 INFO:tasks.workunit.client.0.smithi176.stdout:7/376: creat d1/d1c/d26/d41/d2b/d40/f6e x:0 0 0 2024-03-13T14:49:45.211 INFO:tasks.workunit.client.1.smithi196.stdout:4/469: fsync d1/d3/d4c/d72/d7d/d80/d55/f5e 0 2024-03-13T14:49:45.212 INFO:tasks.workunit.client.0.smithi176.stdout:7/377: dread d1/d7/f3d [0,4194304] 0 2024-03-13T14:49:45.215 INFO:tasks.workunit.client.1.smithi196.stdout:5/607: getdents d4/db/d14/d19/d1c/d25 0 2024-03-13T14:49:45.218 INFO:tasks.workunit.client.1.smithi196.stdout:1/551: dwrite d1/d28/d34/f8e [4194304,4194304] 0 2024-03-13T14:49:45.219 INFO:tasks.workunit.client.1.smithi196.stdout:8/476: mknod d0/d16/ca2 0 2024-03-13T14:49:45.219 INFO:tasks.workunit.client.1.smithi196.stdout:1/552: chown d1/d9/f99 1513578 1 2024-03-13T14:49:45.219 INFO:tasks.workunit.client.0.smithi176.stdout:8/344: creat d0/d5/d11/f68 x:0 0 0 2024-03-13T14:49:45.222 INFO:tasks.workunit.client.0.smithi176.stdout:8/345: dread - d0/db/d39/d56/f5e zero size 2024-03-13T14:49:45.223 INFO:tasks.workunit.client.1.smithi196.stdout:6/643: creat d6/d7/df/d27/d39/d44/fc6 x:0 0 0 2024-03-13T14:49:45.224 INFO:tasks.workunit.client.0.smithi176.stdout:8/346: readlink d0/d5/l3a 0 2024-03-13T14:49:45.226 INFO:tasks.workunit.client.0.smithi176.stdout:9/456: mknod d0/d17/c98 0 2024-03-13T14:49:45.226 INFO:tasks.workunit.client.0.smithi176.stdout:8/347: fsync d0/d5/f4d 0 2024-03-13T14:49:45.227 INFO:tasks.workunit.client.0.smithi176.stdout:3/403: dwrite d4/d13/f61 [0,4194304] 0 2024-03-13T14:49:45.229 INFO:tasks.workunit.client.0.smithi176.stdout:8/348: dread - d0/d24/f57 zero size 2024-03-13T14:49:45.230 INFO:tasks.workunit.client.1.smithi196.stdout:5/608: rmdir d4/db/d14/d19/d1c/d2d 39 2024-03-13T14:49:45.231 INFO:tasks.workunit.client.0.smithi176.stdout:9/457: write d0/db/d89/d7a/d8b/f8f [1085189,101875] 0 2024-03-13T14:49:45.232 INFO:tasks.workunit.client.0.smithi176.stdout:2/416: symlink d1/dc/d2d/d59/d80/d5c/l95 0 2024-03-13T14:49:45.232 INFO:tasks.workunit.client.1.smithi196.stdout:5/609: write d4/db/d4a/fa3 [37556,119194] 0 2024-03-13T14:49:45.233 INFO:tasks.workunit.client.0.smithi176.stdout:4/418: rmdir d7/d8/d95 0 2024-03-13T14:49:45.235 INFO:tasks.workunit.client.0.smithi176.stdout:9/458: chown d0 1 1 2024-03-13T14:49:45.237 INFO:tasks.workunit.client.1.smithi196.stdout:6/644: rename d6/d2a/dc/d1c/fbb to d6/d9/d11/fc7 0 2024-03-13T14:49:45.238 INFO:tasks.workunit.client.0.smithi176.stdout:4/419: truncate d7/d8/d45/f99 473505 0 2024-03-13T14:49:45.239 INFO:tasks.workunit.client.1.smithi196.stdout:6/645: fdatasync d6/d2a/d23/f4c 0 2024-03-13T14:49:45.240 INFO:tasks.workunit.client.0.smithi176.stdout:1/365: rename d2/de/d3f/d19/f56 to d2/de/d10/f72 0 2024-03-13T14:49:45.245 INFO:tasks.workunit.client.0.smithi176.stdout:3/404: mkdir d4/d28/d31/d45/d82 0 2024-03-13T14:49:45.245 INFO:tasks.workunit.client.0.smithi176.stdout:6/387: getdents d0/de/d38/d77 0 2024-03-13T14:49:45.248 INFO:tasks.workunit.client.1.smithi196.stdout:9/454: creat d7/d24/f98 x:0 0 0 2024-03-13T14:49:45.248 INFO:tasks.workunit.client.1.smithi196.stdout:3/515: sync 2024-03-13T14:49:45.249 INFO:tasks.workunit.client.0.smithi176.stdout:3/405: dread - d4/d13/d2d/f51 zero size 2024-03-13T14:49:45.249 INFO:tasks.workunit.client.1.smithi196.stdout:1/553: mkdir d1/d9/d4e/d55/d63/db9/dbb 0 2024-03-13T14:49:45.249 INFO:tasks.workunit.client.1.smithi196.stdout:4/470: dread d1/f22 [0,4194304] 0 2024-03-13T14:49:45.256 INFO:tasks.workunit.client.0.smithi176.stdout:8/349: dwrite d0/d24/f57 [0,4194304] 0 2024-03-13T14:49:45.256 INFO:tasks.workunit.client.0.smithi176.stdout:4/420: symlink d7/d8/d33/l9c 0 2024-03-13T14:49:45.256 INFO:tasks.workunit.client.1.smithi196.stdout:6/646: dread d6/d9/d17/d19/d4e/f7c [0,4194304] 0 2024-03-13T14:49:45.257 INFO:tasks.workunit.client.1.smithi196.stdout:9/455: dread - d7/de/d4b/d1d/d22/d4d/f6b zero size 2024-03-13T14:49:45.259 INFO:tasks.workunit.client.0.smithi176.stdout:1/366: dread d2/de/d10/d16/f27 [4194304,4194304] 0 2024-03-13T14:49:45.261 INFO:tasks.workunit.client.1.smithi196.stdout:5/610: creat d4/db/d14/d19/d1c/d2d/d82/d8b/fbf x:0 0 0 2024-03-13T14:49:45.263 INFO:tasks.workunit.client.0.smithi176.stdout:0/361: creat d2/d5/dc/d60/f6e x:0 0 0 2024-03-13T14:49:45.263 INFO:tasks.workunit.client.1.smithi196.stdout:1/554: creat d1/d9/d4e/d55/d63/da6/fbc x:0 0 0 2024-03-13T14:49:45.266 INFO:tasks.workunit.client.0.smithi176.stdout:0/362: chown d2/db/d31/f62 262294 1 2024-03-13T14:49:45.268 INFO:tasks.workunit.client.1.smithi196.stdout:9/456: creat d7/de/d16/f99 x:0 0 0 2024-03-13T14:49:45.270 INFO:tasks.workunit.client.1.smithi196.stdout:4/471: symlink d1/d8f/d91/l99 0 2024-03-13T14:49:45.270 INFO:tasks.workunit.client.0.smithi176.stdout:6/388: dwrite d0/d10/d19/d27/d2b/f65 [0,4194304] 0 2024-03-13T14:49:45.276 INFO:tasks.workunit.client.1.smithi196.stdout:1/555: dread d1/d28/f35 [0,4194304] 0 2024-03-13T14:49:45.276 INFO:tasks.workunit.client.1.smithi196.stdout:8/477: truncate d0/f56 203813 0 2024-03-13T14:49:45.277 INFO:tasks.workunit.client.0.smithi176.stdout:4/421: dread d7/d10/d46/d4a/d5d/f78 [0,4194304] 0 2024-03-13T14:49:45.279 INFO:tasks.workunit.client.1.smithi196.stdout:3/516: creat db/d19/d22/d3a/fad x:0 0 0 2024-03-13T14:49:45.281 INFO:tasks.workunit.client.0.smithi176.stdout:4/422: read d7/d8/d33/f6d [1168196,91670] 0 2024-03-13T14:49:45.282 INFO:tasks.workunit.client.1.smithi196.stdout:9/457: rename d7/d24/d28/l36 to d7/d9/d17/l9a 0 2024-03-13T14:49:45.290 INFO:tasks.workunit.client.0.smithi176.stdout:5/372: rename d5/f8 to d5/d22/d15/f7d 0 2024-03-13T14:49:45.294 INFO:tasks.workunit.client.0.smithi176.stdout:5/373: fdatasync d5/d48/d49/f4d 0 2024-03-13T14:49:45.294 INFO:tasks.workunit.client.0.smithi176.stdout:8/350: creat d0/db/dc/f69 x:0 0 0 2024-03-13T14:49:45.297 INFO:tasks.workunit.client.1.smithi196.stdout:8/478: dread d0/d10/d12/d1e/f9b [0,4194304] 0 2024-03-13T14:49:45.298 INFO:tasks.workunit.client.1.smithi196.stdout:5/611: creat d4/db/d3c/fc0 x:0 0 0 2024-03-13T14:49:45.298 INFO:tasks.workunit.client.1.smithi196.stdout:9/458: dread d7/de/d4b/f59 [0,4194304] 0 2024-03-13T14:49:45.302 INFO:tasks.workunit.client.1.smithi196.stdout:1/556: dwrite d1/d9/d4e/d55/d90/fb1 [0,4194304] 0 2024-03-13T14:49:45.302 INFO:tasks.workunit.client.1.smithi196.stdout:4/472: rename d1/d3/d7/d35/d76 to d1/d3/d7/d35/d76/d9a 22 2024-03-13T14:49:45.303 INFO:tasks.workunit.client.0.smithi176.stdout:2/417: creat d1/f96 x:0 0 0 2024-03-13T14:49:45.306 INFO:tasks.workunit.client.0.smithi176.stdout:0/363: dread d2/d5/dc/d20/f48 [0,4194304] 0 2024-03-13T14:49:45.306 INFO:tasks.workunit.client.1.smithi196.stdout:3/517: dwrite db/f70 [0,4194304] 0 2024-03-13T14:49:45.307 INFO:tasks.workunit.client.0.smithi176.stdout:0/364: readlink d2/d5/l17 0 2024-03-13T14:49:45.309 INFO:tasks.workunit.client.0.smithi176.stdout:6/389: mkdir d0/d10/d19/d7d 0 2024-03-13T14:49:45.310 INFO:tasks.workunit.client.1.smithi196.stdout:8/479: truncate d0/d16/d3d/d9c/f9d 833968 0 2024-03-13T14:49:45.311 INFO:tasks.workunit.client.0.smithi176.stdout:9/459: truncate d0/d1e/d1f/f41 73787 0 2024-03-13T14:49:45.312 INFO:tasks.workunit.client.0.smithi176.stdout:5/374: dwrite d5/f63 [0,4194304] 0 2024-03-13T14:49:45.313 INFO:tasks.workunit.client.0.smithi176.stdout:5/375: stat d5/d2b/c5d 0 2024-03-13T14:49:45.316 INFO:tasks.workunit.client.0.smithi176.stdout:3/406: mkdir d4/d13/d83 0 2024-03-13T14:49:45.318 INFO:tasks.workunit.client.1.smithi196.stdout:5/612: truncate d4/db/d14/d19/d1c/d25/d38/d47/fb8 887491 0 2024-03-13T14:49:45.319 INFO:tasks.workunit.client.0.smithi176.stdout:7/378: getdents d1/d1c/d26/d41 0 2024-03-13T14:49:45.327 INFO:tasks.workunit.client.1.smithi196.stdout:9/459: creat d7/de/d16/f9b x:0 0 0 2024-03-13T14:49:45.331 INFO:tasks.workunit.client.1.smithi196.stdout:4/473: symlink d1/d1e/d43/l9b 0 2024-03-13T14:49:45.331 INFO:tasks.workunit.client.1.smithi196.stdout:9/460: dread - d7/de/d4b/d1d/d22/d4d/f6b zero size 2024-03-13T14:49:45.331 INFO:tasks.workunit.client.0.smithi176.stdout:0/365: dwrite d2/db/d31/f44 [0,4194304] 0 2024-03-13T14:49:45.332 INFO:tasks.workunit.client.0.smithi176.stdout:9/460: dwrite d0/d1e/d6f/f7b [0,4194304] 0 2024-03-13T14:49:45.337 INFO:tasks.workunit.client.1.smithi196.stdout:5/613: dread d4/db/d14/d19/f50 [4194304,4194304] 0 2024-03-13T14:49:45.337 INFO:tasks.workunit.client.0.smithi176.stdout:0/366: write d2/f6 [8727302,92463] 0 2024-03-13T14:49:45.337 INFO:tasks.workunit.client.0.smithi176.stdout:8/351: creat d0/d5/d11/d47/f6a x:0 0 0 2024-03-13T14:49:45.344 INFO:tasks.workunit.client.0.smithi176.stdout:9/461: dread d0/d40/d53/f65 [0,4194304] 0 2024-03-13T14:49:45.352 INFO:tasks.workunit.client.0.smithi176.stdout:5/376: mkdir d5/d2e/d7e 0 2024-03-13T14:49:45.353 INFO:tasks.workunit.client.1.smithi196.stdout:8/480: write d0/d10/d12/f33 [1404954,47099] 0 2024-03-13T14:49:45.355 INFO:tasks.workunit.client.1.smithi196.stdout:3/518: fsync db/d15/f41 0 2024-03-13T14:49:45.355 INFO:tasks.workunit.client.0.smithi176.stdout:4/423: getdents d7/d2d/d9b 0 2024-03-13T14:49:45.356 INFO:tasks.workunit.client.0.smithi176.stdout:3/407: rmdir d4/d28/d3a/d5b 39 2024-03-13T14:49:45.357 INFO:tasks.workunit.client.1.smithi196.stdout:5/614: fdatasync d4/db/d14/d19/d1c/f3f 0 2024-03-13T14:49:45.362 INFO:tasks.workunit.client.1.smithi196.stdout:5/615: write d4/db/d14/d19/d1c/d25/d38/d47/f64 [1759569,60123] 0 2024-03-13T14:49:45.364 INFO:tasks.workunit.client.1.smithi196.stdout:5/616: readlink d4/db/d4a/l99 0 2024-03-13T14:49:45.365 INFO:tasks.workunit.client.1.smithi196.stdout:3/519: mkdir db/d19/dae 0 2024-03-13T14:49:45.365 INFO:tasks.workunit.client.0.smithi176.stdout:5/377: rename d5/f37 to d5/d31/d47/f7f 0 2024-03-13T14:49:45.365 INFO:tasks.workunit.client.0.smithi176.stdout:8/352: dread d0/d24/f32 [0,4194304] 0 2024-03-13T14:49:45.367 INFO:tasks.workunit.client.1.smithi196.stdout:1/557: truncate d1/d38/f25 1119369 0 2024-03-13T14:49:45.368 INFO:tasks.workunit.client.0.smithi176.stdout:3/408: creat d4/d13/d16/f84 x:0 0 0 2024-03-13T14:49:45.369 INFO:tasks.workunit.client.1.smithi196.stdout:8/481: getdents d0/d10/d12/d1e/d2d 0 2024-03-13T14:49:45.369 INFO:tasks.workunit.client.1.smithi196.stdout:9/461: write d7/d24/f3f [1041449,35856] 0 2024-03-13T14:49:45.374 INFO:tasks.workunit.client.0.smithi176.stdout:2/418: creat d1/dc/d2d/f97 x:0 0 0 2024-03-13T14:49:45.375 INFO:tasks.workunit.client.1.smithi196.stdout:3/520: truncate db/d19/d22/d3a/d53/f59 878047 0 2024-03-13T14:49:45.377 INFO:tasks.workunit.client.1.smithi196.stdout:3/521: dread - db/d19/d22/d3a/f8f zero size 2024-03-13T14:49:45.378 INFO:tasks.workunit.client.0.smithi176.stdout:3/409: chown d4/d13/d2d/d50/f46 0 1 2024-03-13T14:49:45.378 INFO:tasks.workunit.client.0.smithi176.stdout:0/367: dwrite d2/d5/dc/d60/d61/f66 [0,4194304] 0 2024-03-13T14:49:45.381 INFO:tasks.workunit.client.0.smithi176.stdout:1/367: dwrite d2/d30/f65 [0,4194304] 0 2024-03-13T14:49:45.381 INFO:tasks.workunit.client.0.smithi176.stdout:4/424: mkdir d7/d10/d2c/d9d 0 2024-03-13T14:49:45.382 INFO:tasks.workunit.client.1.smithi196.stdout:9/462: creat d7/d9/d17/f9c x:0 0 0 2024-03-13T14:49:45.384 INFO:tasks.workunit.client.0.smithi176.stdout:8/353: creat d0/db/dc/f6b x:0 0 0 2024-03-13T14:49:45.387 INFO:tasks.workunit.client.0.smithi176.stdout:4/425: chown d7/d10/c17 13689392 1 2024-03-13T14:49:45.387 INFO:tasks.workunit.client.0.smithi176.stdout:6/390: dwrite d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:45.387 INFO:tasks.workunit.client.0.smithi176.stdout:5/378: rename d5/d48/d49 to d5/d31/d47/d80 0 2024-03-13T14:49:45.388 INFO:tasks.workunit.client.0.smithi176.stdout:0/368: read d2/db/d31/f51 [19058,44146] 0 2024-03-13T14:49:45.389 INFO:tasks.workunit.client.0.smithi176.stdout:1/368: chown d2/ca 1 1 2024-03-13T14:49:45.396 INFO:tasks.workunit.client.1.smithi196.stdout:1/558: write d1/f26 [266534,8926] 0 2024-03-13T14:49:45.396 INFO:tasks.workunit.client.1.smithi196.stdout:3/522: symlink db/d19/d22/laf 0 2024-03-13T14:49:45.397 INFO:tasks.workunit.client.0.smithi176.stdout:5/379: dread - d5/d2e/f34 zero size 2024-03-13T14:49:45.397 INFO:tasks.workunit.client.0.smithi176.stdout:0/369: write d2/d5/dc/d20/f34 [4736000,51068] 0 2024-03-13T14:49:45.398 INFO:tasks.workunit.client.1.smithi196.stdout:4/474: dread d1/f29 [0,4194304] 0 2024-03-13T14:49:45.399 INFO:tasks.workunit.client.0.smithi176.stdout:1/369: truncate d2/de/d10/f72 4212294 0 2024-03-13T14:49:45.399 INFO:tasks.workunit.client.0.smithi176.stdout:4/426: dread - d7/d10/d46/f8f zero size 2024-03-13T14:49:45.399 INFO:tasks.workunit.client.1.smithi196.stdout:3/523: chown db/f2c 1707 1 2024-03-13T14:49:45.404 INFO:tasks.workunit.client.0.smithi176.stdout:2/419: dwrite d1/dc/d2d/d4b/d69/f94 [0,4194304] 0 2024-03-13T14:49:45.406 INFO:tasks.workunit.client.0.smithi176.stdout:9/462: link d0/cf d0/db/d89/d32/d58/d67/c99 0 2024-03-13T14:49:45.407 INFO:tasks.workunit.client.1.smithi196.stdout:9/463: dwrite d7/de/d4b/f97 [0,4194304] 0 2024-03-13T14:49:45.407 INFO:tasks.workunit.client.1.smithi196.stdout:1/559: symlink d1/d94/lbd 0 2024-03-13T14:49:45.410 INFO:tasks.workunit.client.1.smithi196.stdout:3/524: truncate db/d19/d22/f98 1166687 0 2024-03-13T14:49:45.411 INFO:tasks.workunit.client.1.smithi196.stdout:1/560: dread - d1/d9/d33/fa5 zero size 2024-03-13T14:49:45.412 INFO:tasks.workunit.client.1.smithi196.stdout:3/525: fdatasync db/d19/d22/f98 0 2024-03-13T14:49:45.416 INFO:tasks.workunit.client.1.smithi196.stdout:5/617: link d4/db/d14/d19/d1c/d2d/d82/d8b/dbd/ca9 d4/db/d3c/cc1 0 2024-03-13T14:49:45.421 INFO:tasks.workunit.client.1.smithi196.stdout:4/475: rename d1/d1e/d5c/f74 to d1/d3/d7/d35/d3e/d56/f9c 0 2024-03-13T14:49:45.423 INFO:tasks.workunit.client.0.smithi176.stdout:1/370: stat d2/de/d10/f4f 0 2024-03-13T14:49:45.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.436+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff258005cc0 con 0x7ff278102c30 2024-03-13T14:49:45.426 INFO:tasks.workunit.client.0.smithi176.stdout:2/420: symlink d1/d2a/d33/l98 0 2024-03-13T14:49:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:45 smithi196 ceph-mon[37486]: pgmap v11: 65 pgs: 65 active+clean; 1.4 GiB data, 4.1 GiB used, 532 GiB / 536 GiB avail; 29 MiB/s rd, 142 MiB/s wr, 262 op/s 2024-03-13T14:49:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:45 smithi196 ceph-mon[37486]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:45.428 INFO:tasks.workunit.client.0.smithi176.stdout:2/421: chown d1/dc/d2d/d4b 947083368 1 2024-03-13T14:49:45.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.441+0000 7ff26ea00700 1 -- 172.21.15.176:0/4219607131 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff26805c000 con 0x7ff278102c30 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:49:45.433 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:49:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:49:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:49:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:49:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:49:45.434 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:49:45.434 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:49:45.434 INFO:tasks.workunit.client.1.smithi196.stdout:3/526: creat db/d15/d44/fb0 x:0 0 0 2024-03-13T14:49:45.436 INFO:tasks.workunit.client.1.smithi196.stdout:5/618: creat d4/d1b/fc2 x:0 0 0 2024-03-13T14:49:45.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff25c06ddf0 msgr2=0x7ff25c0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:45.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff25c06ddf0 0x7ff25c0702b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff264005950 tx=0x7ff2640058e0 comp rx=0 tx=0).stop 2024-03-13T14:49:45.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 msgr2=0x7ff27819c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:45.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff27819c7c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff26800bd70 tx=0x7ff26800be50 comp rx=0 tx=0).stop 2024-03-13T14:49:45.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 shutdown_connections 2024-03-13T14:49:45.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff25c06ddf0 0x7ff25c0702b0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff264005950 tx=0x7ff2640058e0 comp rx=0 tx=0).stop 2024-03-13T14:49:45.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff278102c30 0x7ff27819c7c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:45.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 --2- 172.21.15.176:0/4219607131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff278103e30 0x7ff27819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:45.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.447+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 >> 172.21.15.176:0/4219607131 conn(0x7ff2780fe1b0 msgr2=0x7ff278107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:45.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.448+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 shutdown_connections 2024-03-13T14:49:45.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.448+0000 7ff277200700 1 -- 172.21.15.176:0/4219607131 wait complete. 2024-03-13T14:49:45.437 INFO:tasks.workunit.client.0.smithi176.stdout:8/354: creat d0/f6c x:0 0 0 2024-03-13T14:49:45.438 INFO:tasks.workunit.client.1.smithi196.stdout:8/482: dwrite d0/d10/d12/d1e/f34 [0,4194304] 0 2024-03-13T14:49:45.439 INFO:tasks.workunit.client.0.smithi176.stdout:2/422: dread d1/dc/d2d/d4b/f63 [0,4194304] 0 2024-03-13T14:49:45.440 INFO:tasks.workunit.client.1.smithi196.stdout:8/483: write d0/d10/f11 [3141073,68584] 0 2024-03-13T14:49:45.446 INFO:tasks.workunit.client.0.smithi176.stdout:1/371: mkdir d2/d29/d73 0 2024-03-13T14:49:45.447 INFO:tasks.workunit.client.1.smithi196.stdout:3/527: readlink db/l16 0 2024-03-13T14:49:45.449 INFO:tasks.workunit.client.1.smithi196.stdout:3/528: readlink db/d19/d22/d3a/d6a/l77 0 2024-03-13T14:49:45.451 INFO:tasks.workunit.client.1.smithi196.stdout:5/619: dread - d4/db/d14/d37/f4d zero size 2024-03-13T14:49:45.454 INFO:tasks.workunit.client.1.smithi196.stdout:9/464: rename d7/f62 to d7/d9/d83/f9d 0 2024-03-13T14:49:45.456 INFO:tasks.workunit.client.1.smithi196.stdout:8/484: mknod d0/d1c/d29/d47/ca3 0 2024-03-13T14:49:45.458 INFO:tasks.workunit.client.0.smithi176.stdout:1/372: write d2/de/d3f/f6f [886596,85089] 0 2024-03-13T14:49:45.459 INFO:tasks.workunit.client.1.smithi196.stdout:2/426: sync 2024-03-13T14:49:45.459 INFO:tasks.workunit.client.1.smithi196.stdout:0/558: sync 2024-03-13T14:49:45.459 INFO:tasks.workunit.client.1.smithi196.stdout:7/636: sync 2024-03-13T14:49:45.459 INFO:tasks.workunit.client.1.smithi196.stdout:6/647: sync 2024-03-13T14:49:45.460 INFO:tasks.workunit.client.0.smithi176.stdout:1/373: read - d2/de/f53 zero size 2024-03-13T14:49:45.463 INFO:tasks.workunit.client.1.smithi196.stdout:5/620: read d4/db/d14/d19/d26/d68/f72 [1246657,75315] 0 2024-03-13T14:49:45.464 INFO:tasks.workunit.client.1.smithi196.stdout:1/561: rename d1 to d1/dbe 22 2024-03-13T14:49:45.464 INFO:tasks.workunit.client.0.smithi176.stdout:5/380: creat d5/f81 x:0 0 0 2024-03-13T14:49:45.464 INFO:tasks.workunit.client.0.smithi176.stdout:1/374: readlink d2/de/l44 0 2024-03-13T14:49:45.464 INFO:tasks.workunit.client.0.smithi176.stdout:3/410: truncate d4/d28/d4f/f53 1791430 0 2024-03-13T14:49:45.465 INFO:tasks.workunit.client.0.smithi176.stdout:0/370: unlink d2/d5/d28/f4b 0 2024-03-13T14:49:45.466 INFO:tasks.workunit.client.1.smithi196.stdout:5/621: dread - d4/db/d14/d19/fac zero size 2024-03-13T14:49:45.466 INFO:tasks.workunit.client.1.smithi196.stdout:4/476: dwrite d1/d3/d7/f1a [8388608,4194304] 0 2024-03-13T14:49:45.469 INFO:tasks.workunit.client.1.smithi196.stdout:3/529: fdatasync db/d15/f1a 0 2024-03-13T14:49:45.471 INFO:tasks.workunit.client.1.smithi196.stdout:8/485: unlink d0/d1c/d29/d47/l66 0 2024-03-13T14:49:45.473 INFO:tasks.workunit.client.0.smithi176.stdout:4/427: dwrite d7/d10/d46/f8f [0,4194304] 0 2024-03-13T14:49:45.474 INFO:tasks.workunit.client.1.smithi196.stdout:3/530: truncate db/d19/d22/d3a/d6a/f89 580 0 2024-03-13T14:49:45.474 INFO:tasks.workunit.client.0.smithi176.stdout:2/423: dwrite d1/dc/d2d/d4b/d69/f94 [0,4194304] 0 2024-03-13T14:49:45.479 INFO:tasks.workunit.client.1.smithi196.stdout:9/465: read d7/d9/d83/f9d [231147,53218] 0 2024-03-13T14:49:45.479 INFO:tasks.workunit.client.1.smithi196.stdout:7/637: creat d0/d78/fd5 x:0 0 0 2024-03-13T14:49:45.481 INFO:tasks.workunit.client.1.smithi196.stdout:6/648: mkdir d6/d7/df/d27/d39/d44/dc8 0 2024-03-13T14:49:45.482 INFO:tasks.workunit.client.1.smithi196.stdout:0/559: dwrite d8/da/d25/d26/d35/d3b/d5f/d33/f94 [4194304,4194304] 0 2024-03-13T14:49:45.483 INFO:tasks.workunit.client.1.smithi196.stdout:6/649: chown d6/d7/cc4 2741 1 2024-03-13T14:49:45.494 INFO:tasks.workunit.client.0.smithi176.stdout:9/463: dwrite d0/d1e/f2e [0,4194304] 0 2024-03-13T14:49:45.495 INFO:tasks.workunit.client.1.smithi196.stdout:4/477: symlink d1/d3/d4c/d72/d7d/d80/d55/l9d 0 2024-03-13T14:49:45.499 INFO:tasks.workunit.client.1.smithi196.stdout:9/466: creat d7/d71/f9e x:0 0 0 2024-03-13T14:49:45.501 INFO:tasks.workunit.client.1.smithi196.stdout:0/560: dread d8/da/d25/d26/d54/d88/f84 [0,4194304] 0 2024-03-13T14:49:45.502 INFO:tasks.workunit.client.0.smithi176.stdout:6/391: getdents d0/d10/d19/d27/d2b/d2e 0 2024-03-13T14:49:45.503 INFO:tasks.workunit.client.0.smithi176.stdout:8/355: mknod d0/db/d58/c6d 0 2024-03-13T14:49:45.504 INFO:tasks.workunit.client.0.smithi176.stdout:5/381: mknod d5/d2b/c82 0 2024-03-13T14:49:45.505 INFO:tasks.workunit.client.0.smithi176.stdout:8/356: stat d0/d24/f57 0 2024-03-13T14:49:45.509 INFO:tasks.workunit.client.0.smithi176.stdout:7/379: sync 2024-03-13T14:49:45.509 INFO:tasks.workunit.client.0.smithi176.stdout:1/375: symlink d2/de/d3f/d19/l74 0 2024-03-13T14:49:45.514 INFO:tasks.workunit.client.0.smithi176.stdout:2/424: fdatasync d1/dc/d2d/d4b/d69/f94 0 2024-03-13T14:49:45.515 INFO:tasks.workunit.client.1.smithi196.stdout:4/478: mknod d1/d1e/d43/c9e 0 2024-03-13T14:49:45.517 INFO:tasks.workunit.client.0.smithi176.stdout:4/428: creat d7/d75/f9e x:0 0 0 2024-03-13T14:49:45.520 INFO:tasks.workunit.client.1.smithi196.stdout:3/531: symlink db/d15/lb1 0 2024-03-13T14:49:45.520 INFO:tasks.workunit.client.0.smithi176.stdout:9/464: mkdir d0/db/d89/d32/d58/d5f/d84/d9a 0 2024-03-13T14:49:45.521 INFO:tasks.workunit.client.1.smithi196.stdout:3/532: stat db/d19/d50 0 2024-03-13T14:49:45.521 INFO:tasks.workunit.client.0.smithi176.stdout:9/465: stat d0/d40/f78 0 2024-03-13T14:49:45.524 INFO:tasks.workunit.client.0.smithi176.stdout:6/392: rmdir d0 39 2024-03-13T14:49:45.525 INFO:tasks.workunit.client.1.smithi196.stdout:7/638: mkdir d0/d4d/dd1/dd6 0 2024-03-13T14:49:45.528 INFO:tasks.workunit.client.0.smithi176.stdout:5/382: rename d5/d2c/d40/c6d to d5/d2c/d74/c83 0 2024-03-13T14:49:45.529 INFO:tasks.workunit.client.0.smithi176.stdout:3/411: dwrite d4/d28/d3a/f6d [0,4194304] 0 2024-03-13T14:49:45.529 INFO:tasks.workunit.client.1.smithi196.stdout:1/562: creat d1/d9/d4e/d55/fbf x:0 0 0 2024-03-13T14:49:45.533 INFO:tasks.workunit.client.1.smithi196.stdout:4/479: dwrite d1/d3/d7/d35/d76/d58/f87 [0,4194304] 0 2024-03-13T14:49:45.534 INFO:tasks.workunit.client.1.smithi196.stdout:5/622: creat d4/db/d3c/fc3 x:0 0 0 2024-03-13T14:49:45.537 INFO:tasks.workunit.client.1.smithi196.stdout:2/427: getdents d0/d50 0 2024-03-13T14:49:45.541 INFO:tasks.workunit.client.0.smithi176.stdout:1/376: symlink d2/de/d10/d16/d4a/l75 0 2024-03-13T14:49:45.541 INFO:tasks.workunit.client.1.smithi196.stdout:8/486: link d0/d16/d3d/d8b/d3f/c49 d0/d16/d3d/d9c/ca4 0 2024-03-13T14:49:45.547 INFO:tasks.workunit.client.0.smithi176.stdout:1/377: read d2/d30/f65 [920570,40209] 0 2024-03-13T14:49:45.547 INFO:tasks.workunit.client.0.smithi176.stdout:2/425: fdatasync d1/dc/d1c/f55 0 2024-03-13T14:49:45.547 INFO:tasks.workunit.client.0.smithi176.stdout:9/466: unlink d0/db/d89/d4d/d5d/c77 0 2024-03-13T14:49:45.550 INFO:tasks.workunit.client.1.smithi196.stdout:6/650: dwrite d6/d7/df/d27/d39/d44/d74/f87 [0,4194304] 0 2024-03-13T14:49:45.551 INFO:tasks.workunit.client.0.smithi176.stdout:3/412: dwrite d4/d13/d16/d3d/f5d [0,4194304] 0 2024-03-13T14:49:45.551 INFO:tasks.workunit.client.0.smithi176.stdout:2/426: write d1/d2a/f82 [367514,76088] 0 2024-03-13T14:49:45.552 INFO:tasks.workunit.client.0.smithi176.stdout:8/357: symlink d0/d24/l6e 0 2024-03-13T14:49:45.553 INFO:tasks.workunit.client.0.smithi176.stdout:2/427: stat d1/dc/d2d/d90/c93 0 2024-03-13T14:49:45.554 INFO:tasks.workunit.client.0.smithi176.stdout:4/429: rename d7/d10/f56 to d7/d8/d45/f9f 0 2024-03-13T14:49:45.557 INFO:tasks.workunit.client.0.smithi176.stdout:5/383: creat d5/d22/d15/f84 x:0 0 0 2024-03-13T14:49:45.559 INFO:tasks.workunit.client.0.smithi176.stdout:5/384: write d5/f81 [354649,19799] 0 2024-03-13T14:49:45.561 INFO:tasks.workunit.client.1.smithi196.stdout:1/563: creat d1/d9/d33/d4b/d66/fc0 x:0 0 0 2024-03-13T14:49:45.562 INFO:tasks.workunit.client.1.smithi196.stdout:5/623: mknod d4/db/d4a/cc4 0 2024-03-13T14:49:45.565 INFO:tasks.workunit.client.1.smithi196.stdout:5/624: write d4/f8 [8747861,53205] 0 2024-03-13T14:49:45.566 INFO:tasks.workunit.client.0.smithi176.stdout:6/393: write d0/f69 [3588658,44931] 0 2024-03-13T14:49:45.566 INFO:tasks.workunit.client.1.smithi196.stdout:3/533: getdents db/d19/d22/d30/da9 0 2024-03-13T14:49:45.568 INFO:tasks.workunit.client.1.smithi196.stdout:5/625: readlink d4/db/d3c/l69 0 2024-03-13T14:49:45.569 INFO:tasks.workunit.client.0.smithi176.stdout:6/394: stat d0/d10/d19/d27/d2b/d2e/f7b 0 2024-03-13T14:49:45.573 INFO:tasks.workunit.client.1.smithi196.stdout:0/561: link d8/da/d25/l8a d8/da/d25/d26/lb4 0 2024-03-13T14:49:45.579 INFO:tasks.workunit.client.0.smithi176.stdout:3/413: unlink d4/d28/d34/f7a 0 2024-03-13T14:49:45.583 INFO:tasks.workunit.client.1.smithi196.stdout:8/487: mkdir d0/d10/da5 0 2024-03-13T14:49:45.584 INFO:tasks.workunit.client.1.smithi196.stdout:3/534: creat db/d19/d4a/fb2 x:0 0 0 2024-03-13T14:49:45.584 INFO:tasks.workunit.client.0.smithi176.stdout:2/428: mkdir d1/dc/d2d/d90/d99 0 2024-03-13T14:49:45.586 INFO:tasks.workunit.client.0.smithi176.stdout:2/429: stat d1/dc/d2d/d59/d80/d5c/f5f 0 2024-03-13T14:49:45.587 INFO:tasks.workunit.client.1.smithi196.stdout:6/651: symlink d6/d9/lc9 0 2024-03-13T14:49:45.588 INFO:tasks.workunit.client.1.smithi196.stdout:8/488: chown d0/d10/d12/l39 109477 1 2024-03-13T14:49:45.592 INFO:tasks.workunit.client.1.smithi196.stdout:6/652: read - d6/d7/dd/faf zero size 2024-03-13T14:49:45.592 INFO:tasks.workunit.client.1.smithi196.stdout:7/639: dwrite d0/d6/f21 [0,4194304] 0 2024-03-13T14:49:45.593 INFO:tasks.workunit.client.0.smithi176.stdout:0/371: getdents d2/d5/dc/d60/d61 0 2024-03-13T14:49:45.593 INFO:tasks.workunit.client.1.smithi196.stdout:3/535: dread - db/d19/d22/d30/f9d zero size 2024-03-13T14:49:45.594 INFO:tasks.workunit.client.1.smithi196.stdout:3/536: readlink l6 0 2024-03-13T14:49:45.594 INFO:tasks.workunit.client.0.smithi176.stdout:0/372: chown d2/d5/dc/d59 2837168 1 2024-03-13T14:49:45.595 INFO:tasks.workunit.client.1.smithi196.stdout:7/640: chown d0/d6/db4 289 1 2024-03-13T14:49:45.597 INFO:tasks.workunit.client.0.smithi176.stdout:9/467: dread d0/d40/f47 [0,4194304] 0 2024-03-13T14:49:45.598 INFO:tasks.workunit.client.1.smithi196.stdout:2/428: dread d0/d7/f3f [0,4194304] 0 2024-03-13T14:49:45.598 INFO:tasks.workunit.client.1.smithi196.stdout:4/480: dread d1/d3/d7/f5d [0,4194304] 0 2024-03-13T14:49:45.602 INFO:tasks.workunit.client.0.smithi176.stdout:6/395: mkdir d0/de/d38/d7e 0 2024-03-13T14:49:45.602 INFO:tasks.workunit.client.0.smithi176.stdout:1/378: write d2/d30/d63/d22/f2b [1222959,65770] 0 2024-03-13T14:49:45.602 INFO:tasks.workunit.client.1.smithi196.stdout:7/641: stat d0/d9/d31/c73 0 2024-03-13T14:49:45.603 INFO:tasks.workunit.client.1.smithi196.stdout:6/653: mknod d6/d7/df/d27/d39/d44/d74/cca 0 2024-03-13T14:49:45.605 INFO:tasks.workunit.client.0.smithi176.stdout:7/380: rename d1/d28/d3a/d1b/l1d to d1/d1c/d26/l6f 0 2024-03-13T14:49:45.605 INFO:tasks.workunit.client.0.smithi176.stdout:6/396: write d0/d9/f55 [2345457,14677] 0 2024-03-13T14:49:45.606 INFO:tasks.workunit.client.1.smithi196.stdout:3/537: chown db/f36 4717 1 2024-03-13T14:49:45.606 INFO:tasks.workunit.client.0.smithi176.stdout:4/430: fsync d7/d8/f42 0 2024-03-13T14:49:45.607 INFO:tasks.workunit.client.1.smithi196.stdout:2/429: truncate d0/d7/f87 712294 0 2024-03-13T14:49:45.609 INFO:tasks.workunit.client.0.smithi176.stdout:4/431: dread - d7/d8/d33/d96/f98 zero size 2024-03-13T14:49:45.610 INFO:tasks.workunit.client.1.smithi196.stdout:4/481: write d1/d3/d7/d35/d76/f94 [689127,19715] 0 2024-03-13T14:49:45.611 INFO:tasks.workunit.client.0.smithi176.stdout:0/373: symlink d2/d5/d1e/l6f 0 2024-03-13T14:49:45.612 INFO:tasks.workunit.client.0.smithi176.stdout:8/358: creat d0/d24/f6f x:0 0 0 2024-03-13T14:49:45.613 INFO:tasks.workunit.client.1.smithi196.stdout:8/489: write d0/d10/d12/d1e/f6a [792459,127850] 0 2024-03-13T14:49:45.615 INFO:tasks.workunit.client.1.smithi196.stdout:4/482: read d1/f29 [3945466,78975] 0 2024-03-13T14:49:45.615 INFO:tasks.workunit.client.0.smithi176.stdout:4/432: read d7/d8/d33/f6d [2515238,121471] 0 2024-03-13T14:49:45.616 INFO:tasks.workunit.client.0.smithi176.stdout:3/414: symlink d4/d28/d5a/l85 0 2024-03-13T14:49:45.617 INFO:tasks.workunit.client.1.smithi196.stdout:7/642: dread d0/d9/d31/d50/f75 [0,4194304] 0 2024-03-13T14:49:45.621 INFO:tasks.workunit.client.1.smithi196.stdout:1/564: dwrite d1/d38/f3b [0,4194304] 0 2024-03-13T14:49:45.626 INFO:tasks.workunit.client.1.smithi196.stdout:0/562: dread d8/da/fc [0,4194304] 0 2024-03-13T14:49:45.633 INFO:tasks.workunit.client.1.smithi196.stdout:2/430: mknod d0/d6/c8c 0 2024-03-13T14:49:45.640 INFO:tasks.workunit.client.1.smithi196.stdout:7/643: creat d0/d9/d1e/d3f/fd7 x:0 0 0 2024-03-13T14:49:45.641 INFO:tasks.workunit.client.1.smithi196.stdout:1/565: dread d1/f26 [0,4194304] 0 2024-03-13T14:49:45.641 INFO:tasks.workunit.client.0.smithi176.stdout:3/415: rename d4/fa to d4/d28/f86 0 2024-03-13T14:49:45.641 INFO:tasks.workunit.client.0.smithi176.stdout:4/433: dwrite d7/d10/d46/d4a/f79 [0,4194304] 0 2024-03-13T14:49:45.642 INFO:tasks.workunit.client.1.smithi196.stdout:6/654: rmdir d6/d7/df/db1 0 2024-03-13T14:49:45.643 INFO:tasks.workunit.client.0.smithi176.stdout:9/468: creat d0/db/d89/d32/d58/d5f/d84/d9a/f9b x:0 0 0 2024-03-13T14:49:45.646 INFO:tasks.workunit.client.1.smithi196.stdout:2/431: symlink d0/d7/d66/l8d 0 2024-03-13T14:49:45.648 INFO:tasks.workunit.client.0.smithi176.stdout:6/397: creat d0/de/d38/d7e/f7f x:0 0 0 2024-03-13T14:49:45.648 INFO:tasks.workunit.client.1.smithi196.stdout:3/538: creat db/d19/d22/d3a/fb3 x:0 0 0 2024-03-13T14:49:45.648 INFO:tasks.workunit.client.1.smithi196.stdout:2/432: read d0/d6/d34/f41 [603803,79650] 0 2024-03-13T14:49:45.649 INFO:tasks.workunit.client.0.smithi176.stdout:6/398: stat d0/de/f49 0 2024-03-13T14:49:45.651 INFO:tasks.workunit.client.0.smithi176.stdout:5/385: getdents d5/d48 0 2024-03-13T14:49:45.658 INFO:tasks.workunit.client.0.smithi176.stdout:0/374: symlink d2/d5/d28/l70 0 2024-03-13T14:49:45.659 INFO:tasks.workunit.client.1.smithi196.stdout:1/566: creat d1/d9/d6c/fc1 x:0 0 0 2024-03-13T14:49:45.659 INFO:tasks.workunit.client.0.smithi176.stdout:1/379: creat d2/f76 x:0 0 0 2024-03-13T14:49:45.662 INFO:tasks.workunit.client.1.smithi196.stdout:4/483: rename d1/d8f/d91 to d1/d3/d7/d35/d76/d58/d69/d9f 0 2024-03-13T14:49:45.663 INFO:tasks.workunit.client.0.smithi176.stdout:1/380: write d2/de/d10/d16/d47/f5c [1018341,13573] 0 2024-03-13T14:49:45.668 INFO:tasks.workunit.client.0.smithi176.stdout:2/430: getdents d1/d2a 0 2024-03-13T14:49:45.669 INFO:tasks.workunit.client.1.smithi196.stdout:8/490: dread d0/d16/d3d/d8b/f8e [0,4194304] 0 2024-03-13T14:49:45.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.682+0000 7fc0b7400700 1 -- 172.21.15.176:0/1867508632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0073770 msgr2=0x7fc0b0073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:45.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.682+0000 7fc0b7400700 1 --2- 172.21.15.176:0/1867508632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0073770 0x7fc0b0073bf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc098009910 tx=0x7fc098009c20 comp rx=0 tx=0).stop 2024-03-13T14:49:45.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.682+0000 7fc0b7400700 1 -- 172.21.15.176:0/1867508632 shutdown_connections 2024-03-13T14:49:45.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.682+0000 7fc0b7400700 1 --2- 172.21.15.176:0/1867508632 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0073770 0x7fc0b0073bf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:45.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.682+0000 7fc0b7400700 1 --2- 172.21.15.176:0/1867508632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0b0074d40 0x7fc0b00731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:45.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.682+0000 7fc0b7400700 1 -- 172.21.15.176:0/1867508632 >> 172.21.15.176:0/1867508632 conn(0x7fc0b00fc110 msgr2=0x7fc0b00fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:45.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.683+0000 7fc0b7400700 1 -- 172.21.15.176:0/1867508632 shutdown_connections 2024-03-13T14:49:45.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.683+0000 7fc0b7400700 1 -- 172.21.15.176:0/1867508632 wait complete. 2024-03-13T14:49:45.671 INFO:tasks.workunit.client.0.smithi176.stdout:2/431: chown d1/dc/d1c/c87 106 1 2024-03-13T14:49:45.672 INFO:tasks.workunit.client.1.smithi196.stdout:7/644: dwrite d0/d9/d1e/d58/d72/da9/d47/f4e [0,4194304] 0 2024-03-13T14:49:45.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.684+0000 7fc0b7400700 1 Processor -- start 2024-03-13T14:49:45.673 INFO:tasks.workunit.client.0.smithi176.stdout:4/434: mknod d7/d10/d46/d4a/d5d/ca0 0 2024-03-13T14:49:45.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.685+0000 7fc0b7400700 1 -- start start 2024-03-13T14:49:45.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.685+0000 7fc0b7400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0b0073770 0x7fc0b0071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:45.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.685+0000 7fc0b7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0074d40 0x7fc0b00722d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:45.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.685+0000 7fc0b7400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b00728f0 con 0x7fc0b0074d40 2024-03-13T14:49:45.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.685+0000 7fc0b7400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b0072a30 con 0x7fc0b0073770 2024-03-13T14:49:45.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.686+0000 7fc0b4a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0b0073770 0x7fc0b0071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:45.674 INFO:tasks.workunit.client.1.smithi196.stdout:3/539: readlink db/d19/d22/d3a/d6a/d8d/la8 0 2024-03-13T14:49:45.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.686+0000 7fc0b4a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0b0073770 0x7fc0b0071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:41508/0 (socket says 172.21.15.176:41508) 2024-03-13T14:49:45.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.686+0000 7fc0b4a00700 1 -- 172.21.15.176:0/2208284318 learned_addr learned my addr 172.21.15.176:0/2208284318 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:49:45.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.686+0000 7fc0a7e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0074d40 0x7fc0b00722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:45.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.686+0000 7fc0a7e00700 1 -- 172.21.15.176:0/2208284318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0b0073770 msgr2=0x7fc0b0071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:45.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.686+0000 7fc0a7e00700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0b0073770 0x7fc0b0071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:45.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.686+0000 7fc0a7e00700 1 -- 172.21.15.176:0/2208284318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980095f0 con 0x7fc0b0074d40 2024-03-13T14:49:45.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.687+0000 7fc0a7e00700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0074d40 0x7fc0b00722d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc09800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:45.676 INFO:tasks.workunit.client.1.smithi196.stdout:3/540: write db/d19/f62 [424141,80317] 0 2024-03-13T14:49:45.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.688+0000 7fc0ae000700 1 -- 172.21.15.176:0/2208284318 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc09801d070 con 0x7fc0b0074d40 2024-03-13T14:49:45.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.688+0000 7fc0ae000700 1 -- 172.21.15.176:0/2208284318 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc09800f460 con 0x7fc0b0074d40 2024-03-13T14:49:45.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.688+0000 7fc0ae000700 1 -- 172.21.15.176:0/2208284318 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc098021640 con 0x7fc0b0074d40 2024-03-13T14:49:45.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.689+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b01a6440 con 0x7fc0b0074d40 2024-03-13T14:49:45.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.689+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b01a6930 con 0x7fc0b0074d40 2024-03-13T14:49:45.679 INFO:tasks.workunit.client.0.smithi176.stdout:6/399: creat d0/de/d38/f80 x:0 0 0 2024-03-13T14:49:45.683 INFO:tasks.workunit.client.1.smithi196.stdout:4/484: dwrite d1/f54 [0,4194304] 0 2024-03-13T14:49:45.683 INFO:tasks.workunit.client.1.smithi196.stdout:9/467: sync 2024-03-13T14:49:45.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.693+0000 7fc0ae000700 1 -- 172.21.15.176:0/2208284318 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fc09800fa90 con 0x7fc0b0074d40 2024-03-13T14:49:45.686 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.697+0000 7fc0ae000700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc09406dd90 0x7fc094070250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:49:45.686 INFO:tasks.workunit.client.1.smithi196.stdout:4/485: dread - d1/d1e/d43/f85 zero size 2024-03-13T14:49:45.687 INFO:tasks.workunit.client.0.smithi176.stdout:0/375: dwrite d2/f6 [4194304,4194304] 0 2024-03-13T14:49:45.687 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.699+0000 7fc0b4a00700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc09406dd90 0x7fc094070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:49:45.687 INFO:tasks.workunit.client.1.smithi196.stdout:9/468: chown d7/de/d4b/d1d/d22/d4d/f5c 2539364 1 2024-03-13T14:49:45.688 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.695+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b010b900 con 0x7fc0b0074d40 2024-03-13T14:49:45.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.701+0000 7fc0ae000700 1 -- 172.21.15.176:0/2208284318 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc09808ea70 con 0x7fc0b0074d40 2024-03-13T14:49:45.690 INFO:tasks.workunit.client.1.smithi196.stdout:0/563: getdents d8/da/d25/d26/d35/d3b/d6c 0 2024-03-13T14:49:45.690 INFO:tasks.workunit.client.1.smithi196.stdout:9/469: write d7/d24/f3f [1597175,43456] 0 2024-03-13T14:49:45.691 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.703+0000 7fc0b4a00700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc09406dd90 0x7fc094070250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009550 tx=0x7fc0a0009110 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:49:45.692 INFO:tasks.workunit.client.1.smithi196.stdout:2/433: dwrite d0/d6/d1d/f7f [4194304,4194304] 0 2024-03-13T14:49:45.694 INFO:tasks.workunit.client.0.smithi176.stdout:3/416: mkdir d4/d28/d31/d45/d82/d87 0 2024-03-13T14:49:45.695 INFO:tasks.workunit.client.1.smithi196.stdout:0/564: write d8/da/d74/f8e [784988,63134] 0 2024-03-13T14:49:45.698 INFO:tasks.workunit.client.0.smithi176.stdout:4/435: rename d7/d75 to d7/d10/d46/d4a/d5d/da1 0 2024-03-13T14:49:45.698 INFO:tasks.workunit.client.0.smithi176.stdout:3/417: chown d4/d28 6760 1 2024-03-13T14:49:45.699 INFO:tasks.workunit.client.0.smithi176.stdout:5/386: write d5/d2e/d5f/f6f [376302,11227] 0 2024-03-13T14:49:45.699 INFO:tasks.workunit.client.1.smithi196.stdout:3/541: creat db/d15/d44/d51/fb4 x:0 0 0 2024-03-13T14:49:45.701 INFO:tasks.workunit.client.1.smithi196.stdout:8/491: unlink d0/d10/d12/f3e 0 2024-03-13T14:49:45.703 INFO:tasks.workunit.client.1.smithi196.stdout:4/486: creat d1/d3/d4c/d72/fa0 x:0 0 0 2024-03-13T14:49:45.705 INFO:tasks.workunit.client.0.smithi176.stdout:7/381: getdents d1/d1c/d26/d41/d2b/d40/d64 0 2024-03-13T14:49:45.706 INFO:tasks.workunit.client.1.smithi196.stdout:1/567: rmdir d1/d9/d33/d4b/d66/db6 0 2024-03-13T14:49:45.708 INFO:tasks.workunit.client.0.smithi176.stdout:2/432: dread d1/dc/d2d/d59/d80/d77/f84 [0,4194304] 0 2024-03-13T14:49:45.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:45.722+0000 7fc0ae000700 1 -- 172.21.15.176:0/2208284318 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fc09805bdf0 con 0x7fc0b0074d40 2024-03-13T14:49:45.715 INFO:tasks.workunit.client.0.smithi176.stdout:8/359: getdents d0/db 0 2024-03-13T14:49:45.715 INFO:tasks.workunit.client.0.smithi176.stdout:1/381: creat d2/de/d10/d16/d40/f77 x:0 0 0 2024-03-13T14:49:45.715 INFO:tasks.workunit.client.0.smithi176.stdout:0/376: creat d2/d5/d1e/f71 x:0 0 0 2024-03-13T14:49:45.718 INFO:tasks.workunit.client.0.smithi176.stdout:3/418: creat d4/d28/d34/d38/f88 x:0 0 0 2024-03-13T14:49:45.721 INFO:tasks.workunit.client.0.smithi176.stdout:5/387: creat d5/d31/d47/f85 x:0 0 0 2024-03-13T14:49:45.721 INFO:tasks.workunit.client.0.smithi176.stdout:6/400: dwrite d0/f6c [0,4194304] 0 2024-03-13T14:49:45.722 INFO:tasks.workunit.client.0.smithi176.stdout:4/436: unlink d7/d8/d15/l31 0 2024-03-13T14:49:45.725 INFO:tasks.workunit.client.1.smithi196.stdout:6/655: truncate d6/d7/f8d 1830067 0 2024-03-13T14:49:45.726 INFO:tasks.workunit.client.0.smithi176.stdout:2/433: mknod d1/dc/d2d/d4f/c9a 0 2024-03-13T14:49:45.726 INFO:tasks.workunit.client.0.smithi176.stdout:3/419: fsync d4/d13/d2d/d50/f46 0 2024-03-13T14:49:45.727 INFO:tasks.workunit.client.1.smithi196.stdout:8/492: unlink d0/d16/d3d/d8b/d3f/f93 0 2024-03-13T14:49:45.729 INFO:tasks.workunit.client.1.smithi196.stdout:0/565: dwrite d8/da/fb0 [0,4194304] 0 2024-03-13T14:49:45.732 INFO:tasks.workunit.client.0.smithi176.stdout:2/434: chown d1/dc/d2d/d38/d5d 73 1 2024-03-13T14:49:45.732 INFO:tasks.workunit.client.0.smithi176.stdout:6/401: write d0/d10/d19/d27/f4d [3829527,51797] 0 2024-03-13T14:49:45.732 INFO:tasks.workunit.client.0.smithi176.stdout:3/420: fsync d4/d28/d31/f62 0 2024-03-13T14:49:45.736 INFO:tasks.workunit.client.0.smithi176.stdout:8/360: mkdir d0/d24/d29/d70 0 2024-03-13T14:49:45.737 INFO:tasks.workunit.client.1.smithi196.stdout:4/487: dwrite d1/d3/d7/f1d [0,4194304] 0 2024-03-13T14:49:45.740 INFO:tasks.workunit.client.0.smithi176.stdout:5/388: mkdir d5/d31/d86 0 2024-03-13T14:49:45.750 INFO:tasks.workunit.client.0.smithi176.stdout:9/469: dread d0/f54 [0,4194304] 0 2024-03-13T14:49:45.752 INFO:tasks.workunit.client.0.smithi176.stdout:9/470: chown d0/db/d89/d32/d58/d67 10 1 2024-03-13T14:49:45.760 INFO:tasks.workunit.client.0.smithi176.stdout:4/437: link d7/d10/d46/d4a/d5d/da1/f89 d7/d8/d15/d23/fa2 0 2024-03-13T14:49:45.762 INFO:tasks.workunit.client.0.smithi176.stdout:7/382: rename d1/l4e to d1/d1c/d26/d41/d2b/d40/l70 0 2024-03-13T14:49:45.764 INFO:tasks.workunit.client.0.smithi176.stdout:2/435: mknod d1/dc/d2d/d59/d80/d7e/c9b 0 2024-03-13T14:49:45.766 INFO:tasks.workunit.client.0.smithi176.stdout:2/436: chown d1/dc/d40/l85 20984 1 2024-03-13T14:49:45.771 INFO:tasks.workunit.client.0.smithi176.stdout:3/421: truncate d4/d13/d2d/d50/f3c 1791158 0 2024-03-13T14:49:45.772 INFO:tasks.workunit.client.0.smithi176.stdout:3/422: chown d4/d13/f48 631993 1 2024-03-13T14:49:45.773 INFO:tasks.workunit.client.0.smithi176.stdout:5/389: symlink d5/d48/d4c/d7b/l87 0 2024-03-13T14:49:45.781 INFO:tasks.workunit.client.0.smithi176.stdout:4/438: mknod d7/d2d/d9b/ca3 0 2024-03-13T14:49:45.783 INFO:tasks.workunit.client.0.smithi176.stdout:8/361: truncate d0/f30 2333270 0 2024-03-13T14:49:45.784 INFO:tasks.workunit.client.0.smithi176.stdout:7/383: mknod d1/d3f/d65/c71 0 2024-03-13T14:49:45.785 INFO:tasks.workunit.client.0.smithi176.stdout:2/437: dread d1/dc/d1c/f4c [0,4194304] 0 2024-03-13T14:49:45.787 INFO:tasks.workunit.client.0.smithi176.stdout:8/362: dread - d0/d24/f6f zero size 2024-03-13T14:49:45.787 INFO:tasks.workunit.client.0.smithi176.stdout:1/382: rename d2/de/f41 to d2/de/d3f/d50/f78 0 2024-03-13T14:49:45.789 INFO:tasks.workunit.client.0.smithi176.stdout:4/439: readlink d7/d8/d2f/l93 0 2024-03-13T14:49:45.790 INFO:tasks.workunit.client.1.smithi196.stdout:5/626: sync 2024-03-13T14:49:45.791 INFO:tasks.workunit.client.0.smithi176.stdout:7/384: write d1/d28/d3a/d1b/f38 [268816,3818] 0 2024-03-13T14:49:45.792 INFO:tasks.workunit.client.1.smithi196.stdout:7/645: link d0/d9/d1e/d3f/f82 d0/d4d/d74/fd8 0 2024-03-13T14:49:45.792 INFO:tasks.workunit.client.0.smithi176.stdout:5/390: dwrite d5/d31/d47/d80/f62 [0,4194304] 0 2024-03-13T14:49:45.795 INFO:tasks.workunit.client.1.smithi196.stdout:3/542: creat db/d19/d4a/d9b/fb5 x:0 0 0 2024-03-13T14:49:45.797 INFO:tasks.workunit.client.0.smithi176.stdout:9/471: dwrite d0/d17/f19 [4194304,4194304] 0 2024-03-13T14:49:45.797 INFO:tasks.workunit.client.0.smithi176.stdout:5/391: truncate d5/d22/d15/f2d 4772306 0 2024-03-13T14:49:45.800 INFO:tasks.workunit.client.1.smithi196.stdout:7/646: dread d0/d6/d33/f4c [0,4194304] 0 2024-03-13T14:49:45.802 INFO:tasks.workunit.client.1.smithi196.stdout:6/656: mkdir d6/d7/df/dcb 0 2024-03-13T14:49:45.822 INFO:tasks.workunit.client.1.smithi196.stdout:5/627: mkdir d4/db/d14/d19/d1c/d2d/dc5 0 2024-03-13T14:49:45.822 INFO:tasks.workunit.client.1.smithi196.stdout:1/568: dread d1/d3f/d5b/f69 [0,4194304] 0 2024-03-13T14:49:45.828 INFO:tasks.workunit.client.0.smithi176.stdout:6/402: rename d0/de/d54/l73 to d0/d9/l81 0 2024-03-13T14:49:45.828 INFO:tasks.workunit.client.1.smithi196.stdout:2/434: write d0/d7/d9/f48 [595749,55201] 0 2024-03-13T14:49:45.829 INFO:tasks.workunit.client.0.smithi176.stdout:9/472: mkdir d0/db/d89/d32/d58/d9c 0 2024-03-13T14:49:45.829 INFO:tasks.workunit.client.1.smithi196.stdout:7/647: creat d0/d4d/d74/fd9 x:0 0 0 2024-03-13T14:49:45.830 INFO:tasks.workunit.client.0.smithi176.stdout:8/363: mknod d0/d24/c71 0 2024-03-13T14:49:45.834 INFO:tasks.workunit.client.1.smithi196.stdout:0/566: symlink d8/da/d25/d26/d35/d3b/lb5 0 2024-03-13T14:49:45.839 INFO:tasks.workunit.client.0.smithi176.stdout:3/423: rename d4/d28/d3a to d4/d13/d81/d89 0 2024-03-13T14:49:45.843 INFO:tasks.workunit.client.1.smithi196.stdout:5/628: unlink d4/db/d14/d19/d1c/d2d/d82/f83 0 2024-03-13T14:49:45.843 INFO:tasks.workunit.client.1.smithi196.stdout:4/488: dwrite d1/d3/d7/d35/d76/f32 [0,4194304] 0 2024-03-13T14:49:45.847 INFO:tasks.workunit.client.0.smithi176.stdout:1/383: dread d2/de/d3f/d50/f42 [0,4194304] 0 2024-03-13T14:49:45.847 INFO:tasks.workunit.client.0.smithi176.stdout:7/385: dwrite d1/d1c/f5e [4194304,4194304] 0 2024-03-13T14:49:45.847 INFO:tasks.workunit.client.0.smithi176.stdout:2/438: dwrite d1/dc/d2d/f2f [0,4194304] 0 2024-03-13T14:49:45.847 INFO:tasks.workunit.client.0.smithi176.stdout:8/364: creat d0/d24/d29/f72 x:0 0 0 2024-03-13T14:49:45.847 INFO:tasks.workunit.client.0.smithi176.stdout:7/386: fsync d1/d3f/d4b/f6b 0 2024-03-13T14:49:45.848 INFO:tasks.workunit.client.1.smithi196.stdout:3/543: truncate db/d19/d22/d3a/d53/f59 702568 0 2024-03-13T14:49:45.853 INFO:tasks.workunit.client.0.smithi176.stdout:2/439: write d1/dc/d2d/d4b/d69/f94 [3733963,17819] 0 2024-03-13T14:49:45.856 INFO:tasks.workunit.client.0.smithi176.stdout:3/424: mknod d4/d28/d31/d45/c8a 0 2024-03-13T14:49:45.856 INFO:tasks.workunit.client.1.smithi196.stdout:0/567: truncate d8/da/d25/f2f 1759969 0 2024-03-13T14:49:45.862 INFO:tasks.workunit.client.0.smithi176.stdout:5/392: getdents d5/d48/d4c 0 2024-03-13T14:49:45.862 INFO:tasks.workunit.client.0.smithi176.stdout:0/377: sync 2024-03-13T14:49:45.863 INFO:tasks.workunit.client.0.smithi176.stdout:9/473: dwrite d0/db/d89/d4d/d5d/f7d [0,4194304] 0 2024-03-13T14:49:45.864 INFO:tasks.workunit.client.1.smithi196.stdout:4/489: mknod d1/d3/d4c/d72/d7d/d80/ca1 0 2024-03-13T14:49:45.867 INFO:tasks.workunit.client.0.smithi176.stdout:0/378: write d2/d5/dc/f49 [4564538,19512] 0 2024-03-13T14:49:45.869 INFO:tasks.workunit.client.1.smithi196.stdout:9/470: sync 2024-03-13T14:49:45.869 INFO:tasks.workunit.client.0.smithi176.stdout:6/403: dwrite d0/de/d54/f5e [0,4194304] 0 2024-03-13T14:49:45.871 INFO:tasks.workunit.client.1.smithi196.stdout:4/490: read d1/d1e/f7f [1557367,123447] 0 2024-03-13T14:49:45.872 INFO:tasks.workunit.client.1.smithi196.stdout:1/569: link d1/d9/d33/d4b/d66/d7b/fb4 d1/d9/d4e/d55/d63/db9/dbb/fc2 0 2024-03-13T14:49:45.872 INFO:tasks.workunit.client.1.smithi196.stdout:9/471: dread - d7/de/d16/f99 zero size 2024-03-13T14:49:45.884 INFO:tasks.workunit.client.1.smithi196.stdout:2/435: mkdir d0/d7/d8e 0 2024-03-13T14:49:45.886 INFO:tasks.workunit.client.0.smithi176.stdout:3/425: symlink d4/d13/d16/l8b 0 2024-03-13T14:49:45.890 INFO:tasks.workunit.client.1.smithi196.stdout:6/657: creat d6/fcc x:0 0 0 2024-03-13T14:49:45.895 INFO:tasks.workunit.client.1.smithi196.stdout:7/648: creat d0/d6/d33/d57/d8a/d9f/fda x:0 0 0 2024-03-13T14:49:45.901 INFO:tasks.workunit.client.0.smithi176.stdout:1/384: rename d2/c2f to d2/de/d10/d16/d40/c79 0 2024-03-13T14:49:45.904 INFO:tasks.workunit.client.0.smithi176.stdout:1/385: truncate d2/d30/f4c 54256 0 2024-03-13T14:49:45.905 INFO:tasks.workunit.client.0.smithi176.stdout:5/393: creat d5/d48/d4c/f88 x:0 0 0 2024-03-13T14:49:45.905 INFO:tasks.workunit.client.0.smithi176.stdout:9/474: symlink d0/db/d89/d32/d58/l9d 0 2024-03-13T14:49:45.908 INFO:tasks.workunit.client.0.smithi176.stdout:5/394: dread - d5/d2c/d40/d6a/f79 zero size 2024-03-13T14:49:45.908 INFO:tasks.workunit.client.0.smithi176.stdout:6/404: dread d0/d10/d19/d27/d2b/f3a [0,4194304] 0 2024-03-13T14:49:45.909 INFO:tasks.workunit.client.0.smithi176.stdout:1/386: write d2/d30/f4c [437707,97054] 0 2024-03-13T14:49:45.909 INFO:tasks.workunit.client.0.smithi176.stdout:7/387: symlink d1/d28/d3a/d1b/l72 0 2024-03-13T14:49:45.911 INFO:tasks.workunit.client.1.smithi196.stdout:5/629: dwrite d4/db/d14/d19/d1c/d25/f49 [0,4194304] 0 2024-03-13T14:49:45.913 INFO:tasks.workunit.client.1.smithi196.stdout:9/472: dread d7/de/d4b/f7c [0,4194304] 0 2024-03-13T14:49:45.914 INFO:tasks.workunit.client.0.smithi176.stdout:1/387: chown d2/de/d3f/d19/f32 3298860 1 2024-03-13T14:49:45.915 INFO:tasks.workunit.client.0.smithi176.stdout:7/388: read d1/d1c/f56 [1986916,97045] 0 2024-03-13T14:49:45.923 INFO:tasks.workunit.client.0.smithi176.stdout:1/388: truncate d2/de/d10/f72 4289489 0 2024-03-13T14:49:45.923 INFO:tasks.workunit.client.0.smithi176.stdout:7/389: write d1/d28/d3a/d1b/d18/f52 [4418750,28639] 0 2024-03-13T14:49:45.923 INFO:tasks.workunit.client.0.smithi176.stdout:2/440: dread d1/dc/d2d/d38/d5d/f62 [0,4194304] 0 2024-03-13T14:49:45.924 INFO:tasks.workunit.client.1.smithi196.stdout:7/649: dwrite d0/d6/faa [0,4194304] 0 2024-03-13T14:49:45.924 INFO:tasks.workunit.client.1.smithi196.stdout:6/658: mknod d6/d9/daa/ccd 0 2024-03-13T14:49:45.924 INFO:tasks.workunit.client.1.smithi196.stdout:9/473: read d7/f12 [139374,1943] 0 2024-03-13T14:49:45.924 INFO:tasks.workunit.client.1.smithi196.stdout:9/474: chown d7/d9/d17/f9c 30767007 1 2024-03-13T14:49:45.924 INFO:tasks.workunit.client.1.smithi196.stdout:0/568: symlink d8/da/d25/d26/d35/d3b/d50/d5d/da5/lb6 0 2024-03-13T14:49:45.925 INFO:tasks.workunit.client.0.smithi176.stdout:2/441: dread - d1/dc/d2d/d38/d5d/f79 zero size 2024-03-13T14:49:45.928 INFO:tasks.workunit.client.1.smithi196.stdout:0/569: write d8/da/d25/d26/d35/d3b/d50/d5d/f57 [3577476,293] 0 2024-03-13T14:49:45.930 INFO:tasks.workunit.client.0.smithi176.stdout:6/405: dwrite d0/d10/d19/f63 [4194304,4194304] 0 2024-03-13T14:49:45.932 INFO:tasks.workunit.client.1.smithi196.stdout:3/544: dwrite db/f2d [4194304,4194304] 0 2024-03-13T14:49:45.932 INFO:tasks.workunit.client.0.smithi176.stdout:9/475: symlink d0/d40/d53/d6d/l9e 0 2024-03-13T14:49:45.938 INFO:tasks.workunit.client.0.smithi176.stdout:6/406: write d0/f6c [1389880,54346] 0 2024-03-13T14:49:45.939 INFO:tasks.workunit.client.0.smithi176.stdout:5/395: mkdir d5/d2c/d40/d6a/d89 0 2024-03-13T14:49:45.943 INFO:tasks.workunit.client.0.smithi176.stdout:8/365: link d0/db/l3e d0/d5/d11/d47/l73 0 2024-03-13T14:49:45.946 INFO:tasks.workunit.client.0.smithi176.stdout:3/426: creat d4/d13/d16/d80/f8c x:0 0 0 2024-03-13T14:49:45.947 INFO:tasks.workunit.client.1.smithi196.stdout:7/650: mknod d0/d4d/d74/cdb 0 2024-03-13T14:49:45.948 INFO:tasks.workunit.client.1.smithi196.stdout:6/659: symlink d6/d57/d91/lce 0 2024-03-13T14:49:45.950 INFO:tasks.workunit.client.1.smithi196.stdout:8/493: sync 2024-03-13T14:49:45.950 INFO:tasks.workunit.client.1.smithi196.stdout:9/475: creat d7/d9/d83/f9f x:0 0 0 2024-03-13T14:49:45.952 INFO:tasks.workunit.client.1.smithi196.stdout:9/476: stat d7/f69 0 2024-03-13T14:49:45.954 INFO:tasks.workunit.client.1.smithi196.stdout:0/570: mkdir d8/da/d25/d26/d48/d82/db7 0 2024-03-13T14:49:45.954 INFO:tasks.workunit.client.0.smithi176.stdout:9/476: read - d0/d1e/d1f/d56/f5b zero size 2024-03-13T14:49:45.955 INFO:tasks.workunit.client.1.smithi196.stdout:5/630: read d4/db/d14/d19/d26/f3a [592228,19645] 0 2024-03-13T14:49:45.956 INFO:tasks.workunit.client.1.smithi196.stdout:1/570: rename d1/d9/f99 to d1/d9/d4e/d55/fc3 0 2024-03-13T14:49:45.959 INFO:tasks.workunit.client.1.smithi196.stdout:0/571: chown d8/da/d25/d26/d54/da9/fab 679278843 1 2024-03-13T14:49:45.960 INFO:tasks.workunit.client.0.smithi176.stdout:5/396: rmdir d5/d22/d15 39 2024-03-13T14:49:45.963 INFO:tasks.workunit.client.0.smithi176.stdout:6/407: creat d0/de/d38/d77/f82 x:0 0 0 2024-03-13T14:49:45.969 INFO:tasks.workunit.client.1.smithi196.stdout:6/660: symlink d6/d7/df/d27/d39/d44/lcf 0 2024-03-13T14:49:45.971 INFO:tasks.workunit.client.1.smithi196.stdout:6/661: chown d6/d9/d11/f5e 43913 1 2024-03-13T14:49:45.973 INFO:tasks.workunit.client.1.smithi196.stdout:3/545: rename db/f70 to db/d15/d44/fb6 0 2024-03-13T14:49:45.974 INFO:tasks.workunit.client.0.smithi176.stdout:0/379: getdents d2/db/d35 0 2024-03-13T14:49:45.974 INFO:tasks.workunit.client.1.smithi196.stdout:9/477: truncate d7/de/d4b/d1d/d22/d4d/f58 1720992 0 2024-03-13T14:49:45.976 INFO:tasks.workunit.client.0.smithi176.stdout:0/380: chown d2/d5/d1e/l58 34419 1 2024-03-13T14:49:45.976 INFO:tasks.workunit.client.0.smithi176.stdout:3/427: dread d4/d13/d2d/d50/f3c [0,4194304] 0 2024-03-13T14:49:45.977 INFO:tasks.workunit.client.1.smithi196.stdout:3/546: chown db/d19/d22/f3c 1 1 2024-03-13T14:49:45.977 INFO:tasks.workunit.client.1.smithi196.stdout:2/436: link d0/d6/d34/d5b/c82 d0/d6/d4b/c8f 0 2024-03-13T14:49:45.978 INFO:tasks.workunit.client.0.smithi176.stdout:0/381: chown d2/d5/dc/d60/d61/f66 2570207 1 2024-03-13T14:49:45.978 INFO:tasks.workunit.client.0.smithi176.stdout:5/397: mkdir d5/d2c/d8a 0 2024-03-13T14:49:45.979 INFO:tasks.workunit.client.0.smithi176.stdout:5/398: readlink d5/d31/l78 0 2024-03-13T14:49:45.980 INFO:tasks.workunit.client.1.smithi196.stdout:2/437: fdatasync d0/d6/d1d/f7a 0 2024-03-13T14:49:45.980 INFO:tasks.workunit.client.1.smithi196.stdout:4/491: dwrite d1/d3/d7/d35/f40 [0,4194304] 0 2024-03-13T14:49:45.981 INFO:tasks.workunit.client.0.smithi176.stdout:0/382: dread d2/d5/d1e/f42 [0,4194304] 0 2024-03-13T14:49:45.982 INFO:tasks.workunit.client.0.smithi176.stdout:5/399: read d5/d48/d4c/f65 [3563341,114282] 0 2024-03-13T14:49:45.983 INFO:tasks.workunit.client.0.smithi176.stdout:6/408: symlink d0/d10/d19/d27/d2b/d2e/d3e/l83 0 2024-03-13T14:49:45.984 INFO:tasks.workunit.client.1.smithi196.stdout:4/492: read d1/d3/f4e [3087845,39494] 0 2024-03-13T14:49:45.988 INFO:tasks.workunit.client.0.smithi176.stdout:7/390: link d1/c59 d1/d1c/d4a/c73 0 2024-03-13T14:49:45.990 INFO:tasks.workunit.client.0.smithi176.stdout:1/389: creat d2/de/d3f/d50/f7a x:0 0 0 2024-03-13T14:49:45.991 INFO:tasks.workunit.client.0.smithi176.stdout:8/366: write d0/d5/d3b/f3c [837326,23477] 0 2024-03-13T14:49:45.992 INFO:tasks.workunit.client.1.smithi196.stdout:7/651: write d0/d9/d31/d77/d93/d94/f9e [1023800,87608] 0 2024-03-13T14:49:45.993 INFO:tasks.workunit.client.0.smithi176.stdout:9/477: mkdir d0/d1e/d8d/d9f 0 2024-03-13T14:49:45.995 INFO:tasks.workunit.client.0.smithi176.stdout:1/390: truncate d2/d30/f4c 821665 0 2024-03-13T14:49:45.997 INFO:tasks.workunit.client.1.smithi196.stdout:5/631: rename d4/db/d14/d19/l35 to d4/db/d14/d19/d1c/d25/d38/d47/lc6 0 2024-03-13T14:49:46.001 INFO:tasks.workunit.client.1.smithi196.stdout:3/547: rmdir db/d15/d44/d51 39 2024-03-13T14:49:46.002 INFO:tasks.workunit.client.1.smithi196.stdout:9/478: truncate d7/d9/f77 322206 0 2024-03-13T14:49:46.002 INFO:tasks.workunit.client.1.smithi196.stdout:1/571: creat d1/d9/d4e/d55/d63/db9/dbb/fc4 x:0 0 0 2024-03-13T14:49:46.004 INFO:tasks.workunit.client.0.smithi176.stdout:7/391: fsync d1/d1c/d26/d41/f3e 0 2024-03-13T14:49:46.006 INFO:tasks.workunit.client.0.smithi176.stdout:7/392: read - d1/d1c/d26/d41/f5a zero size 2024-03-13T14:49:46.007 INFO:tasks.workunit.client.0.smithi176.stdout:9/478: truncate d0/db/f3f 71324 0 2024-03-13T14:49:46.011 INFO:tasks.workunit.client.1.smithi196.stdout:0/572: write d8/da/d25/d26/f60 [203702,34849] 0 2024-03-13T14:49:46.014 INFO:tasks.workunit.client.0.smithi176.stdout:5/400: mknod d5/d22/d15/c8b 0 2024-03-13T14:49:46.014 INFO:tasks.workunit.client.1.smithi196.stdout:0/573: write d8/da/d25/d26/d54/da9/fab [298954,3415] 0 2024-03-13T14:49:46.014 INFO:tasks.workunit.client.1.smithi196.stdout:7/652: dread - d0/d9/d1e/d58/d72/f7c zero size 2024-03-13T14:49:46.016 INFO:tasks.workunit.client.0.smithi176.stdout:0/383: rename d2/d5/d28/f29 to d2/d5/d1e/f72 0 2024-03-13T14:49:46.019 INFO:tasks.workunit.client.0.smithi176.stdout:3/428: write d4/d13/d2d/d50/f3c [1049860,43417] 0 2024-03-13T14:49:46.019 INFO:tasks.workunit.client.0.smithi176.stdout:0/384: stat d2/d5/dc/d20/c32 0 2024-03-13T14:49:46.020 INFO:tasks.workunit.client.0.smithi176.stdout:3/429: read - d4/f39 zero size 2024-03-13T14:49:46.021 INFO:tasks.workunit.client.1.smithi196.stdout:1/572: fsync d1/d28/f5a 0 2024-03-13T14:49:46.022 INFO:tasks.workunit.client.0.smithi176.stdout:9/479: stat d0/la 0 2024-03-13T14:49:46.023 INFO:tasks.workunit.client.1.smithi196.stdout:9/479: mknod d7/d24/d28/ca0 0 2024-03-13T14:49:46.025 INFO:tasks.workunit.client.1.smithi196.stdout:6/662: dwrite d6/d9/d11/f5e [0,4194304] 0 2024-03-13T14:49:46.029 INFO:tasks.workunit.client.0.smithi176.stdout:6/409: rename d0/d9/f37 to d0/de/d38/d7e/f84 0 2024-03-13T14:49:46.030 INFO:tasks.workunit.client.0.smithi176.stdout:4/440: sync 2024-03-13T14:49:46.030 INFO:tasks.workunit.client.1.smithi196.stdout:7/653: unlink d0/d4d/f63 0 2024-03-13T14:49:46.031 INFO:tasks.workunit.client.1.smithi196.stdout:4/493: rename d1/d3/d7/d35/d3e/d56/c93 to d1/d3/d4c/d72/ca2 0 2024-03-13T14:49:46.034 INFO:tasks.workunit.client.1.smithi196.stdout:1/573: fdatasync f0 0 2024-03-13T14:49:46.036 INFO:tasks.workunit.client.0.smithi176.stdout:6/410: chown d0/d10/d19/d27/d2b/d2e/d3e/l83 4633374 1 2024-03-13T14:49:46.036 INFO:tasks.workunit.client.0.smithi176.stdout:0/385: creat d2/db/d39/f73 x:0 0 0 2024-03-13T14:49:46.038 INFO:tasks.workunit.client.0.smithi176.stdout:5/401: creat d5/d75/f8c x:0 0 0 2024-03-13T14:49:46.039 INFO:tasks.workunit.client.1.smithi196.stdout:8/494: read d0/d1c/d29/d47/f52 [407049,85885] 0 2024-03-13T14:49:46.045 INFO:tasks.workunit.client.1.smithi196.stdout:3/548: rename db/d19/d22/d30 to db/d19/d50/db7 0 2024-03-13T14:49:46.047 INFO:tasks.workunit.client.1.smithi196.stdout:3/549: chown db/d19/d50/db7/f9d 1562505949 1 2024-03-13T14:49:46.048 INFO:tasks.workunit.client.0.smithi176.stdout:1/391: rename d2/de/d10/d16/l46 to d2/de/d3f/d3e/l7b 0 2024-03-13T14:49:46.048 INFO:tasks.workunit.client.1.smithi196.stdout:4/494: dread d1/d3/d4c/d72/d7d/d80/d55/f5e [0,4194304] 0 2024-03-13T14:49:46.051 INFO:tasks.workunit.client.1.smithi196.stdout:1/574: fsync f0 0 2024-03-13T14:49:46.051 INFO:tasks.workunit.client.0.smithi176.stdout:6/411: creat d0/de/d38/d77/d76/f85 x:0 0 0 2024-03-13T14:49:46.053 INFO:tasks.workunit.client.1.smithi196.stdout:1/575: dread - d1/d9/d4e/d55/d63/da6/fb0 zero size 2024-03-13T14:49:46.055 INFO:tasks.workunit.client.0.smithi176.stdout:7/393: write d1/f3c [2167908,88309] 0 2024-03-13T14:49:46.057 INFO:tasks.workunit.client.1.smithi196.stdout:7/654: mkdir d0/d6/ddc 0 2024-03-13T14:49:46.058 INFO:tasks.workunit.client.0.smithi176.stdout:4/441: truncate d7/d8/d2f/f37 561357 0 2024-03-13T14:49:46.060 INFO:tasks.workunit.client.1.smithi196.stdout:2/438: write d0/d7/d9/d42/f80 [312806,103466] 0 2024-03-13T14:49:46.064 INFO:tasks.workunit.client.0.smithi176.stdout:0/386: dwrite d2/d5/dc/d60/f6e [0,4194304] 0 2024-03-13T14:49:46.064 INFO:tasks.workunit.client.0.smithi176.stdout:7/394: write d1/d28/f2a [3852555,71081] 0 2024-03-13T14:49:46.070 INFO:tasks.workunit.client.1.smithi196.stdout:3/550: rename db/d37/l72 to db/lb8 0 2024-03-13T14:49:46.073 INFO:tasks.workunit.client.0.smithi176.stdout:9/480: dwrite d0/f5e [0,4194304] 0 2024-03-13T14:49:46.073 INFO:tasks.workunit.client.1.smithi196.stdout:4/495: mknod d1/d3/d4c/d72/d7d/d80/ca3 0 2024-03-13T14:49:46.077 INFO:tasks.workunit.client.0.smithi176.stdout:6/412: creat d0/d10/d19/d7d/f86 x:0 0 0 2024-03-13T14:49:46.079 INFO:tasks.workunit.client.1.smithi196.stdout:1/576: creat d1/d9/d4e/fc5 x:0 0 0 2024-03-13T14:49:46.079 INFO:tasks.workunit.client.0.smithi176.stdout:0/387: rmdir d2/d5/d1e 39 2024-03-13T14:49:46.082 INFO:tasks.workunit.client.1.smithi196.stdout:1/577: read d1/d9/d4e/d55/d90/fb1 [2803384,20360] 0 2024-03-13T14:49:46.082 INFO:tasks.workunit.client.0.smithi176.stdout:3/430: write d4/d28/f86 [2754908,22355] 0 2024-03-13T14:49:46.084 INFO:tasks.workunit.client.0.smithi176.stdout:4/442: unlink d7/d10/d46/d4a/d5d/ca0 0 2024-03-13T14:49:46.090 INFO:tasks.workunit.client.1.smithi196.stdout:6/663: write d6/d9/d17/d19/f2b [14041,124333] 0 2024-03-13T14:49:46.090 INFO:tasks.workunit.client.1.smithi196.stdout:1/578: truncate d1/d3f/d5b/d74/f8f 913076 0 2024-03-13T14:49:46.092 INFO:tasks.workunit.client.1.smithi196.stdout:9/480: dwrite d7/de/d4b/f1a [4194304,4194304] 0 2024-03-13T14:49:46.099 INFO:tasks.workunit.client.0.smithi176.stdout:9/481: write d0/d1e/d1f/f38 [474721,108725] 0 2024-03-13T14:49:46.099 INFO:tasks.workunit.client.0.smithi176.stdout:6/413: creat d0/d10/d19/d7d/f87 x:0 0 0 2024-03-13T14:49:46.100 INFO:tasks.workunit.client.1.smithi196.stdout:4/496: write d1/d3/d7/d35/f37 [2171064,66082] 0 2024-03-13T14:49:46.101 INFO:tasks.workunit.client.0.smithi176.stdout:7/395: dwrite d1/d7/f10 [4194304,4194304] 0 2024-03-13T14:49:46.106 INFO:tasks.workunit.client.0.smithi176.stdout:4/443: creat d7/d2d/fa4 x:0 0 0 2024-03-13T14:49:46.109 INFO:tasks.workunit.client.1.smithi196.stdout:7/655: dread d0/d6/d33/f39 [4194304,4194304] 0 2024-03-13T14:49:46.111 INFO:tasks.workunit.client.1.smithi196.stdout:8/495: dwrite d0/d1c/d29/f3c [0,4194304] 0 2024-03-13T14:49:46.112 INFO:tasks.workunit.client.0.smithi176.stdout:3/431: creat d4/d28/d31/d45/d82/d87/f8d x:0 0 0 2024-03-13T14:49:46.115 INFO:tasks.workunit.client.1.smithi196.stdout:8/496: truncate d0/d10/d12/f33 4992581 0 2024-03-13T14:49:46.122 INFO:tasks.workunit.client.0.smithi176.stdout:3/432: readlink d4/d28/d5a/l85 0 2024-03-13T14:49:46.125 INFO:tasks.workunit.client.0.smithi176.stdout:1/392: dwrite d2/d29/f3c [0,4194304] 0 2024-03-13T14:49:46.125 INFO:tasks.workunit.client.1.smithi196.stdout:9/481: unlink d7/c33 0 2024-03-13T14:49:46.126 INFO:tasks.workunit.client.0.smithi176.stdout:9/482: mkdir d0/d40/d53/da0 0 2024-03-13T14:49:46.127 INFO:tasks.workunit.client.0.smithi176.stdout:4/444: dread d7/d10/d2c/f66 [0,4194304] 0 2024-03-13T14:49:46.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.139+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0b0108130 con 0x7fc09406dd90 2024-03-13T14:49:46.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.142+0000 7fc0ae000700 1 -- 172.21.15.176:0/2208284318 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fc0b0108130 con 0x7fc09406dd90 2024-03-13T14:49:46.130 INFO:tasks.workunit.client.1.smithi196.stdout:3/551: write db/d19/d22/f31 [908773,97631] 0 2024-03-13T14:49:46.130 INFO:tasks.workunit.client.1.smithi196.stdout:6/664: mknod d6/d9/d17/d19/cd0 0 2024-03-13T14:49:46.132 INFO:tasks.workunit.client.1.smithi196.stdout:8/497: fdatasync d0/d10/d12/d17/f57 0 2024-03-13T14:49:46.132 INFO:tasks.workunit.client.1.smithi196.stdout:6/665: chown d6/d9/d17/d19/f22 24176 1 2024-03-13T14:49:46.132 INFO:tasks.workunit.client.0.smithi176.stdout:6/414: chown d0/l58 8537155 1 2024-03-13T14:49:46.133 INFO:tasks.workunit.client.0.smithi176.stdout:5/402: dwrite d5/d22/d15/f7d [0,4194304] 0 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:49:46.134 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "2/2 daemons upgraded", 2024-03-13T14:49:46.135 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "", 2024-03-13T14:49:46.135 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:49:46.135 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:49:46.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc09406dd90 msgr2=0x7fc094070250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc09406dd90 0x7fc094070250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009550 tx=0x7fc0a0009110 comp rx=0 tx=0).stop 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0074d40 msgr2=0x7fc0b00722d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0074d40 0x7fc0b00722d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc09800be20 comp rx=0 tx=0).stop 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 shutdown_connections 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc09406dd90 0x7fc094070250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0b0073770 0x7fc0b0071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 --2- 172.21.15.176:0/2208284318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc0b0074d40 0x7fc0b00722d0 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc098000c00 tx=0x7fc09800be20 comp rx=0 tx=0).stop 2024-03-13T14:49:46.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.146+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 >> 172.21.15.176:0/2208284318 conn(0x7fc0b00fc110 msgr2=0x7fc0b0106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:49:46.136 INFO:tasks.workunit.client.0.smithi176.stdout:1/393: truncate d2/de/d10/d16/d47/f5a 74023 0 2024-03-13T14:49:46.137 INFO:tasks.workunit.client.1.smithi196.stdout:4/497: creat d1/d3/d7/fa4 x:0 0 0 2024-03-13T14:49:46.138 INFO:tasks.workunit.client.0.smithi176.stdout:6/415: chown d0/de/c28 6113 1 2024-03-13T14:49:46.138 INFO:tasks.workunit.client.1.smithi196.stdout:6/666: fsync d6/d9/d17/d19/fa8 0 2024-03-13T14:49:46.139 INFO:tasks.workunit.client.1.smithi196.stdout:8/498: creat d0/d1c/d29/fa6 x:0 0 0 2024-03-13T14:49:46.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.152+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 shutdown_connections 2024-03-13T14:49:46.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:49:46.152+0000 7fc0b7400700 1 -- 172.21.15.176:0/2208284318 wait complete. 2024-03-13T14:49:46.142 INFO:tasks.workunit.client.0.smithi176.stdout:7/396: dwrite d1/d28/d3a/f5c [0,4194304] 0 2024-03-13T14:49:46.142 INFO:tasks.workunit.client.0.smithi176.stdout:3/433: symlink d4/d28/d31/d45/d82/l8e 0 2024-03-13T14:49:46.148 INFO:tasks.workunit.client.0.smithi176.stdout:4/445: creat d7/d8/d45/fa5 x:0 0 0 2024-03-13T14:49:46.149 INFO:tasks.workunit.client.1.smithi196.stdout:1/579: write d1/d3f/f80 [559997,45953] 0 2024-03-13T14:49:46.152 INFO:tasks.workunit.client.0.smithi176.stdout:4/446: dread - d7/d8/d15/d23/f8b zero size 2024-03-13T14:49:46.152 INFO:tasks.workunit.client.0.smithi176.stdout:4/447: readlink d7/d10/l14 0 2024-03-13T14:49:46.153 INFO:tasks.workunit.client.0.smithi176.stdout:2/442: sync 2024-03-13T14:49:46.154 INFO:tasks.workunit.client.1.smithi196.stdout:2/439: dwrite d0/f77 [4194304,4194304] 0 2024-03-13T14:49:46.158 INFO:tasks.workunit.client.1.smithi196.stdout:4/498: unlink d1/d3/d4c/d72/d7d/d80/d55/c71 0 2024-03-13T14:49:46.158 INFO:tasks.workunit.client.0.smithi176.stdout:7/397: unlink d1/d28/d3a/d1b/d18/f52 0 2024-03-13T14:49:46.158 INFO:tasks.workunit.client.0.smithi176.stdout:2/443: chown d1/l3 14 1 2024-03-13T14:49:46.161 INFO:tasks.workunit.client.0.smithi176.stdout:4/448: stat d7/d10/d46/d4a/d5d/da1/f9e 0 2024-03-13T14:49:46.161 INFO:tasks.workunit.client.1.smithi196.stdout:9/482: dwrite d7/d24/f3a [0,4194304] 0 2024-03-13T14:49:46.166 INFO:tasks.workunit.client.0.smithi176.stdout:2/444: mknod d1/d2a/c9c 0 2024-03-13T14:49:46.167 INFO:tasks.workunit.client.0.smithi176.stdout:9/483: dread d0/d1e/d1f/f2c [0,4194304] 0 2024-03-13T14:49:46.167 INFO:tasks.workunit.client.1.smithi196.stdout:1/580: write d1/f5 [1223869,1277] 0 2024-03-13T14:49:46.168 INFO:tasks.workunit.client.1.smithi196.stdout:4/499: creat d1/d3/d4c/fa5 x:0 0 0 2024-03-13T14:49:46.171 INFO:tasks.workunit.client.1.smithi196.stdout:1/581: chown d1/d38/f6a 3365480 1 2024-03-13T14:49:46.173 INFO:tasks.workunit.client.0.smithi176.stdout:2/445: creat d1/d2a/d33/f9d x:0 0 0 2024-03-13T14:49:46.174 INFO:tasks.workunit.client.1.smithi196.stdout:1/582: chown d1/d28/d34/d67 75669 1 2024-03-13T14:49:46.174 INFO:tasks.workunit.client.0.smithi176.stdout:2/446: stat d1/c19 0 2024-03-13T14:49:46.175 INFO:tasks.workunit.client.0.smithi176.stdout:4/449: getdents d7/d8/d15/d94 0 2024-03-13T14:49:46.177 INFO:tasks.workunit.client.1.smithi196.stdout:4/500: mknod d1/d3/d7/d35/d76/d58/d69/ca6 0 2024-03-13T14:49:46.178 INFO:tasks.workunit.client.1.smithi196.stdout:9/483: getdents d7/de/d4b/d1d/d22/d4d 0 2024-03-13T14:49:46.181 INFO:tasks.workunit.client.0.smithi176.stdout:5/403: dread d5/d2b/f30 [0,4194304] 0 2024-03-13T14:49:46.182 INFO:tasks.workunit.client.1.smithi196.stdout:1/583: creat d1/d38/fc6 x:0 0 0 2024-03-13T14:49:46.185 INFO:tasks.workunit.client.1.smithi196.stdout:2/440: dwrite d0/d6/d1d/f27 [0,4194304] 0 2024-03-13T14:49:46.185 INFO:tasks.workunit.client.1.smithi196.stdout:8/499: dwrite d0/f51 [4194304,4194304] 0 2024-03-13T14:49:46.185 INFO:tasks.workunit.client.0.smithi176.stdout:7/398: dread d1/d1c/f5e [4194304,4194304] 0 2024-03-13T14:49:46.186 INFO:tasks.workunit.client.0.smithi176.stdout:3/434: dwrite d4/d28/f49 [0,4194304] 0 2024-03-13T14:49:46.187 INFO:tasks.workunit.client.0.smithi176.stdout:2/447: mkdir d1/dc/d61/d9e 0 2024-03-13T14:49:46.188 INFO:tasks.workunit.client.0.smithi176.stdout:2/448: dread - d1/dc/d2d/f97 zero size 2024-03-13T14:49:46.189 INFO:tasks.workunit.client.1.smithi196.stdout:9/484: fdatasync d7/de/d4b/f59 0 2024-03-13T14:49:46.189 INFO:tasks.workunit.client.1.smithi196.stdout:2/441: readlink d0/d6/d4b/d5f/l7b 0 2024-03-13T14:49:46.190 INFO:tasks.workunit.client.0.smithi176.stdout:6/416: truncate d0/d9/f55 2198269 0 2024-03-13T14:49:46.196 INFO:tasks.workunit.client.0.smithi176.stdout:9/484: truncate d0/d1e/f25 504256 0 2024-03-13T14:49:46.198 INFO:tasks.workunit.client.1.smithi196.stdout:1/584: rmdir d1/d9/d4e/d55/d63 39 2024-03-13T14:49:46.201 INFO:tasks.workunit.client.1.smithi196.stdout:6/667: dread d6/d9/d11/f13 [0,4194304] 0 2024-03-13T14:49:46.201 INFO:tasks.workunit.client.0.smithi176.stdout:3/435: read d4/d28/d31/f75 [448658,9847] 0 2024-03-13T14:49:46.203 INFO:tasks.workunit.client.0.smithi176.stdout:4/450: creat d7/d8/d15/d94/fa6 x:0 0 0 2024-03-13T14:49:46.209 INFO:tasks.workunit.client.1.smithi196.stdout:9/485: unlink d7/d24/f3f 0 2024-03-13T14:49:46.212 INFO:tasks.workunit.client.0.smithi176.stdout:2/449: symlink d1/dc/d2d/d38/d5d/l9f 0 2024-03-13T14:49:46.212 INFO:tasks.workunit.client.1.smithi196.stdout:2/442: chown d0/d7/d9/d42/c54 94 1 2024-03-13T14:49:46.217 INFO:tasks.workunit.client.0.smithi176.stdout:5/404: symlink d5/d22/l8d 0 2024-03-13T14:49:46.217 INFO:tasks.workunit.client.0.smithi176.stdout:6/417: dwrite d0/d10/d19/d27/d2b/f65 [0,4194304] 0 2024-03-13T14:49:46.218 INFO:tasks.workunit.client.0.smithi176.stdout:5/405: chown d5/d31/d47 1 1 2024-03-13T14:49:46.220 INFO:tasks.workunit.client.0.smithi176.stdout:6/418: fdatasync d0/d10/d19/d27/f68 0 2024-03-13T14:49:46.221 INFO:tasks.workunit.client.0.smithi176.stdout:5/406: write d5/f55 [610376,108427] 0 2024-03-13T14:49:46.222 INFO:tasks.workunit.client.0.smithi176.stdout:3/436: dwrite d4/d13/f73 [0,4194304] 0 2024-03-13T14:49:46.223 INFO:tasks.workunit.client.0.smithi176.stdout:6/419: write d0/de/f2c [5126670,246] 0 2024-03-13T14:49:46.224 INFO:tasks.workunit.client.1.smithi196.stdout:4/501: getdents d1/d1e/d43 0 2024-03-13T14:49:46.225 INFO:tasks.workunit.client.0.smithi176.stdout:2/450: rmdir d1/dc/d40 39 2024-03-13T14:49:46.227 INFO:tasks.workunit.client.0.smithi176.stdout:9/485: symlink d0/db/d89/d32/d58/d9c/la1 0 2024-03-13T14:49:46.228 INFO:tasks.workunit.client.1.smithi196.stdout:1/585: creat d1/d9/d33/d4b/d88/fc7 x:0 0 0 2024-03-13T14:49:46.230 INFO:tasks.workunit.client.0.smithi176.stdout:9/486: chown d0/db/d89/d32/d58/d5f/c87 332 1 2024-03-13T14:49:46.232 INFO:tasks.workunit.client.1.smithi196.stdout:1/586: write d1/d28/d34/f8e [7377652,51659] 0 2024-03-13T14:49:46.233 INFO:tasks.workunit.client.1.smithi196.stdout:9/486: rename d7/de/l30 to d7/d71/la1 0 2024-03-13T14:49:46.234 INFO:tasks.workunit.client.1.smithi196.stdout:2/443: dwrite d0/d7/d9/d42/f80 [0,4194304] 0 2024-03-13T14:49:46.236 INFO:tasks.workunit.client.1.smithi196.stdout:2/444: readlink d0/d6/d34/l63 0 2024-03-13T14:49:46.241 INFO:tasks.workunit.client.0.smithi176.stdout:6/420: dwrite d0/de/f57 [0,4194304] 0 2024-03-13T14:49:46.243 INFO:tasks.workunit.client.0.smithi176.stdout:8/367: sync 2024-03-13T14:49:46.244 INFO:tasks.workunit.client.1.smithi196.stdout:4/502: creat d1/d3/d4c/d72/d7d/d80/fa7 x:0 0 0 2024-03-13T14:49:46.245 INFO:tasks.workunit.client.0.smithi176.stdout:0/388: sync 2024-03-13T14:49:46.245 INFO:tasks.workunit.client.0.smithi176.stdout:0/389: dread - d2/d5/f64 zero size 2024-03-13T14:49:46.247 INFO:tasks.workunit.client.0.smithi176.stdout:7/399: rename d1/d3f/d4b/c4f to d1/d28/c74 0 2024-03-13T14:49:46.249 INFO:tasks.workunit.client.0.smithi176.stdout:7/400: chown d1/d28/d3a/d1b/l66 94 1 2024-03-13T14:49:46.250 INFO:tasks.workunit.client.0.smithi176.stdout:7/401: truncate d1/f61 770373 0 2024-03-13T14:49:46.250 INFO:tasks.workunit.client.0.smithi176.stdout:3/437: mknod d4/d13/d5e/c8f 0 2024-03-13T14:49:46.252 INFO:tasks.workunit.client.0.smithi176.stdout:3/438: readlink d4/d13/d81/d89/l40 0 2024-03-13T14:49:46.254 INFO:tasks.workunit.client.1.smithi196.stdout:8/500: dwrite d0/d10/d12/d17/f28 [0,4194304] 0 2024-03-13T14:49:46.260 INFO:tasks.workunit.client.0.smithi176.stdout:9/487: symlink d0/d40/d53/la2 0 2024-03-13T14:49:46.261 INFO:tasks.workunit.client.1.smithi196.stdout:6/668: getdents d6/d57 0 2024-03-13T14:49:46.264 INFO:tasks.workunit.client.0.smithi176.stdout:6/421: mkdir d0/de/d38/d88 0 2024-03-13T14:49:46.265 INFO:tasks.workunit.client.1.smithi196.stdout:5/632: sync 2024-03-13T14:49:46.268 INFO:tasks.workunit.client.1.smithi196.stdout:4/503: rename d1/d1e/c26 to d1/d8f/ca8 0 2024-03-13T14:49:46.268 INFO:tasks.workunit.client.1.smithi196.stdout:6/669: readlink d6/d2a/dc/d1c/da7/d9e/la5 0 2024-03-13T14:49:46.274 INFO:tasks.workunit.client.1.smithi196.stdout:8/501: mknod d0/d10/d70/ca7 0 2024-03-13T14:49:46.274 INFO:tasks.workunit.client.1.smithi196.stdout:1/587: dread f0 [0,4194304] 0 2024-03-13T14:49:46.278 INFO:tasks.workunit.client.1.smithi196.stdout:2/445: mkdir d0/d6/d90 0 2024-03-13T14:49:46.285 INFO:tasks.workunit.client.0.smithi176.stdout:0/390: rename d2/d5/dc/d60 to d2/d74 0 2024-03-13T14:49:46.286 INFO:tasks.workunit.client.0.smithi176.stdout:7/402: dread d1/d28/d3a/d1b/f38 [0,4194304] 0 2024-03-13T14:49:46.290 INFO:tasks.workunit.client.0.smithi176.stdout:2/451: unlink d1/dc/d2d/d59/d80/d5c/l88 0 2024-03-13T14:49:46.291 INFO:tasks.workunit.client.1.smithi196.stdout:6/670: read d6/fa2 [2515375,845] 0 2024-03-13T14:49:46.291 INFO:tasks.workunit.client.0.smithi176.stdout:2/452: chown d1/dc/d61 25 1 2024-03-13T14:49:46.292 INFO:tasks.workunit.client.1.smithi196.stdout:9/487: dread d7/d24/f2b [0,4194304] 0 2024-03-13T14:49:46.296 INFO:tasks.workunit.client.0.smithi176.stdout:9/488: creat d0/d17/fa3 x:0 0 0 2024-03-13T14:49:46.297 INFO:tasks.workunit.client.0.smithi176.stdout:4/451: getdents d7/d10/d46/d55 0 2024-03-13T14:49:46.299 INFO:tasks.workunit.client.1.smithi196.stdout:2/446: dwrite d0/d7/d40/f46 [0,4194304] 0 2024-03-13T14:49:46.299 INFO:tasks.workunit.client.1.smithi196.stdout:8/502: rename d0/d10/d12/f50 to d0/d16/d3d/d7f/fa8 0 2024-03-13T14:49:46.301 INFO:tasks.workunit.client.1.smithi196.stdout:8/503: chown d0/d10/d12/d1e/l8f 85431750 1 2024-03-13T14:49:46.304 INFO:tasks.workunit.client.0.smithi176.stdout:8/368: mknod d0/d5/c74 0 2024-03-13T14:49:46.307 INFO:tasks.workunit.client.1.smithi196.stdout:6/671: creat d6/d2a/dc/d1c/da7/fd1 x:0 0 0 2024-03-13T14:49:46.309 INFO:tasks.workunit.client.0.smithi176.stdout:3/439: rename d4/d13/d2d/d50/f46 to d4/d28/d4f/f90 0 2024-03-13T14:49:46.310 INFO:tasks.workunit.client.1.smithi196.stdout:6/672: chown d6/l96 255154350 1 2024-03-13T14:49:46.310 INFO:tasks.workunit.client.1.smithi196.stdout:0/574: sync 2024-03-13T14:49:46.310 INFO:tasks.workunit.client.1.smithi196.stdout:7/656: sync 2024-03-13T14:49:46.311 INFO:tasks.workunit.client.0.smithi176.stdout:4/452: dwrite d7/d8/d33/f7f [0,4194304] 0 2024-03-13T14:49:46.313 INFO:tasks.workunit.client.1.smithi196.stdout:1/588: mkdir d1/d9/d4e/d55/d63/da6/dc8 0 2024-03-13T14:49:46.314 INFO:tasks.workunit.client.0.smithi176.stdout:4/453: chown d7/d10/d46/d4a/d5d/da1/f9e 0 1 2024-03-13T14:49:46.315 INFO:tasks.workunit.client.0.smithi176.stdout:9/489: mknod d0/db/d89/d32/d76/ca4 0 2024-03-13T14:49:46.316 INFO:tasks.workunit.client.0.smithi176.stdout:2/453: write d1/dc/d2d/d59/d8c/f8f [554599,28722] 0 2024-03-13T14:49:46.318 INFO:tasks.workunit.client.1.smithi196.stdout:2/447: chown d0/d6/c45 3452085 1 2024-03-13T14:49:46.320 INFO:tasks.workunit.client.0.smithi176.stdout:2/454: stat d1/dc/d2d/d38/l41 0 2024-03-13T14:49:46.321 INFO:tasks.workunit.client.0.smithi176.stdout:8/369: dwrite d0/f52 [0,4194304] 0 2024-03-13T14:49:46.322 INFO:tasks.workunit.client.1.smithi196.stdout:8/504: mkdir d0/d16/d3d/d9c/d87/d8c/da9 0 2024-03-13T14:49:46.323 INFO:tasks.workunit.client.0.smithi176.stdout:6/422: link d0/d10/d19/d27/c46 d0/de/d54/c89 0 2024-03-13T14:49:46.323 INFO:tasks.workunit.client.0.smithi176.stdout:9/490: readlink d0/l57 0 2024-03-13T14:49:46.324 INFO:tasks.workunit.client.0.smithi176.stdout:3/440: symlink d4/d28/d31/l91 0 2024-03-13T14:49:46.324 INFO:tasks.workunit.client.1.smithi196.stdout:0/575: chown d8/da/d25/d26/d35/d3b/d5f/d17/c39 437064 1 2024-03-13T14:49:46.324 INFO:tasks.workunit.client.1.smithi196.stdout:5/633: dwrite d4/db/f24 [0,4194304] 0 2024-03-13T14:49:46.325 INFO:tasks.workunit.client.1.smithi196.stdout:1/589: mknod d1/d9/d4e/d55/d63/cc9 0 2024-03-13T14:49:46.325 INFO:tasks.workunit.client.0.smithi176.stdout:6/423: dread - d0/d10/d19/d7d/f86 zero size 2024-03-13T14:49:46.328 INFO:tasks.workunit.client.0.smithi176.stdout:3/441: truncate d4/d28/d34/d38/f88 296517 0 2024-03-13T14:49:46.329 INFO:tasks.workunit.client.0.smithi176.stdout:3/442: fdatasync d4/d28/d34/f3f 0 2024-03-13T14:49:46.332 INFO:tasks.workunit.client.0.smithi176.stdout:8/370: getdents d0/db/d53 0 2024-03-13T14:49:46.335 INFO:tasks.workunit.client.0.smithi176.stdout:8/371: chown d0/d24/l5a 278424687 1 2024-03-13T14:49:46.337 INFO:tasks.workunit.client.0.smithi176.stdout:8/372: fsync d0/f5c 0 2024-03-13T14:49:46.337 INFO:tasks.workunit.client.1.smithi196.stdout:8/505: chown d0/d10/d12/d17/c97 61294769 1 2024-03-13T14:49:46.337 INFO:tasks.workunit.client.1.smithi196.stdout:4/504: dwrite d1/d3/d7/d35/d76/f38 [0,4194304] 0 2024-03-13T14:49:46.340 INFO:tasks.workunit.client.1.smithi196.stdout:1/590: unlink d1/d9/d33/d4b/f44 0 2024-03-13T14:49:46.340 INFO:tasks.workunit.client.1.smithi196.stdout:5/634: fdatasync d4/db/d14/d19/d1c/d25/f8e 0 2024-03-13T14:49:46.344 INFO:tasks.workunit.client.1.smithi196.stdout:4/505: readlink d1/d1e/d43/l9b 0 2024-03-13T14:49:46.344 INFO:tasks.workunit.client.1.smithi196.stdout:6/673: truncate d6/d9/f1f 2757387 0 2024-03-13T14:49:46.347 INFO:tasks.workunit.client.1.smithi196.stdout:4/506: chown d1/d1e/d5c 28 1 2024-03-13T14:49:46.348 INFO:tasks.workunit.client.0.smithi176.stdout:8/373: symlink d0/db/d39/d56/l75 0 2024-03-13T14:49:46.348 INFO:tasks.workunit.client.1.smithi196.stdout:1/591: chown d1/d9/d33/d4b/d66/d7b/fa9 548717679 1 2024-03-13T14:49:46.353 INFO:tasks.workunit.client.1.smithi196.stdout:8/506: symlink d0/d16/d3d/d9c/d87/laa 0 2024-03-13T14:49:46.355 INFO:tasks.workunit.client.1.smithi196.stdout:2/448: rename d0/d6/c53 to d0/d7/d9/d42/c91 0 2024-03-13T14:49:46.356 INFO:tasks.workunit.client.0.smithi176.stdout:8/374: creat d0/d5/d11/f76 x:0 0 0 2024-03-13T14:49:46.360 INFO:tasks.workunit.client.1.smithi196.stdout:1/592: mknod d1/d3f/cca 0 2024-03-13T14:49:46.363 INFO:tasks.workunit.client.1.smithi196.stdout:4/507: mknod d1/d3/d7/d35/d76/d58/d69/ca9 0 2024-03-13T14:49:46.367 INFO:tasks.workunit.client.1.smithi196.stdout:0/576: getdents d8/da/d25/d26/d54 0 2024-03-13T14:49:46.367 INFO:tasks.workunit.client.0.smithi176.stdout:4/454: dread d7/d8/d15/f2b [0,4194304] 0 2024-03-13T14:49:46.367 INFO:tasks.workunit.client.0.smithi176.stdout:8/375: read d0/db/f3f [2280056,20705] 0 2024-03-13T14:49:46.368 INFO:tasks.workunit.client.0.smithi176.stdout:4/455: dread - d7/d8/d15/f81 zero size 2024-03-13T14:49:46.368 INFO:tasks.workunit.client.1.smithi196.stdout:3/552: sync 2024-03-13T14:49:46.370 INFO:tasks.workunit.client.1.smithi196.stdout:6/674: mkdir d6/dd2 0 2024-03-13T14:49:46.372 INFO:tasks.workunit.client.0.smithi176.stdout:8/376: creat d0/db/dc/d40/f77 x:0 0 0 2024-03-13T14:49:46.372 INFO:tasks.workunit.client.1.smithi196.stdout:1/593: symlink d1/d28/lcb 0 2024-03-13T14:49:46.372 INFO:tasks.workunit.client.1.smithi196.stdout:9/488: dread d7/de/d4b/d1d/f23 [0,4194304] 0 2024-03-13T14:49:46.376 INFO:tasks.workunit.client.1.smithi196.stdout:7/657: write d0/d9/f5a [8034975,47496] 0 2024-03-13T14:49:46.378 INFO:tasks.workunit.client.0.smithi176.stdout:4/456: write d7/d8/d45/f9f [388536,57799] 0 2024-03-13T14:49:46.378 INFO:tasks.workunit.client.0.smithi176.stdout:0/391: truncate d2/d5/dc/f49 3492859 0 2024-03-13T14:49:46.381 INFO:tasks.workunit.client.0.smithi176.stdout:7/403: dwrite d1/f1f [0,4194304] 0 2024-03-13T14:49:46.382 INFO:tasks.workunit.client.1.smithi196.stdout:5/635: dread d4/db/d14/d19/d26/f3a [0,4194304] 0 2024-03-13T14:49:46.382 INFO:tasks.workunit.client.1.smithi196.stdout:1/594: truncate d1/d28/f56 207262 0 2024-03-13T14:49:46.384 INFO:tasks.workunit.client.0.smithi176.stdout:8/377: symlink d0/l78 0 2024-03-13T14:49:46.384 INFO:tasks.workunit.client.0.smithi176.stdout:6/424: write d0/de/d38/d7e/f84 [2143536,67083] 0 2024-03-13T14:49:46.384 INFO:tasks.workunit.client.0.smithi176.stdout:2/455: write d1/dc/d1c/f4c [1871419,98885] 0 2024-03-13T14:49:46.385 INFO:tasks.workunit.client.1.smithi196.stdout:5/636: write d4/db/d4a/faf [385543,101442] 0 2024-03-13T14:49:46.385 INFO:tasks.workunit.client.1.smithi196.stdout:1/595: write d1/d9/d4e/d55/d63/f73 [748290,11729] 0 2024-03-13T14:49:46.386 INFO:tasks.workunit.client.0.smithi176.stdout:3/443: write d4/f78 [366041,71665] 0 2024-03-13T14:49:46.386 INFO:tasks.workunit.client.1.smithi196.stdout:4/508: symlink d1/d3/d7/laa 0 2024-03-13T14:49:46.387 INFO:tasks.workunit.client.1.smithi196.stdout:4/509: readlink d1/d1e/l3c 0 2024-03-13T14:49:46.388 INFO:tasks.workunit.client.0.smithi176.stdout:1/394: sync 2024-03-13T14:49:46.389 INFO:tasks.workunit.client.0.smithi176.stdout:4/457: mkdir d7/d2d/da7 0 2024-03-13T14:49:46.390 INFO:tasks.workunit.client.0.smithi176.stdout:0/392: creat d2/f75 x:0 0 0 2024-03-13T14:49:46.390 INFO:tasks.workunit.client.0.smithi176.stdout:1/395: stat d2/de/d10/d16/l4e 0 2024-03-13T14:49:46.391 INFO:tasks.workunit.client.0.smithi176.stdout:4/458: chown d7/d8/d33/f6d 0 1 2024-03-13T14:49:46.393 INFO:tasks.workunit.client.0.smithi176.stdout:4/459: readlink d7/d10/d46/l52 0 2024-03-13T14:49:46.394 INFO:tasks.workunit.client.0.smithi176.stdout:7/404: chown d1/d1c/d4a/c73 402 1 2024-03-13T14:49:46.401 INFO:tasks.workunit.client.1.smithi196.stdout:7/658: fsync d0/d6/d33/f80 0 2024-03-13T14:49:46.403 INFO:tasks.workunit.client.0.smithi176.stdout:6/425: unlink d0/de/d38/l5a 0 2024-03-13T14:49:46.406 INFO:tasks.workunit.client.0.smithi176.stdout:3/444: stat d4/f1d 0 2024-03-13T14:49:46.407 INFO:tasks.workunit.client.0.smithi176.stdout:0/393: symlink d2/l76 0 2024-03-13T14:49:46.408 INFO:tasks.workunit.client.1.smithi196.stdout:9/489: mknod d7/de/d74/d94/ca2 0 2024-03-13T14:49:46.410 INFO:tasks.workunit.client.0.smithi176.stdout:4/460: creat d7/d8/d15/d23/fa8 x:0 0 0 2024-03-13T14:49:46.411 INFO:tasks.workunit.client.1.smithi196.stdout:9/490: write d7/d24/f3a [3910718,89729] 0 2024-03-13T14:49:46.414 INFO:tasks.workunit.client.0.smithi176.stdout:8/378: dwrite d0/d24/f32 [0,4194304] 0 2024-03-13T14:49:46.414 INFO:tasks.workunit.client.1.smithi196.stdout:4/510: mknod d1/d3/d4c/d72/d7d/d80/d55/cab 0 2024-03-13T14:49:46.416 INFO:tasks.workunit.client.1.smithi196.stdout:4/511: chown d1/d3/d7/laa 0 1 2024-03-13T14:49:46.416 INFO:tasks.workunit.client.0.smithi176.stdout:8/379: fsync d0/db/dc/d40/f4f 0 2024-03-13T14:49:46.417 INFO:tasks.workunit.client.1.smithi196.stdout:8/507: rename d0/d16/d3d/ca1 to d0/d10/d12/cab 0 2024-03-13T14:49:46.420 INFO:tasks.workunit.client.1.smithi196.stdout:8/508: write d0/d16/d3d/d8b/f91 [144961,70491] 0 2024-03-13T14:49:46.424 INFO:tasks.workunit.client.0.smithi176.stdout:1/396: mkdir d2/de/d3f/d50/d7c 0 2024-03-13T14:49:46.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:46 smithi196 ceph-mon[37486]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:46.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:46 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/4219607131' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:49:46.429 INFO:tasks.workunit.client.1.smithi196.stdout:3/553: write db/d19/f60 [471605,83126] 0 2024-03-13T14:49:46.430 INFO:tasks.workunit.client.0.smithi176.stdout:6/426: dwrite d0/d10/d19/d27/f61 [0,4194304] 0 2024-03-13T14:49:46.431 INFO:tasks.workunit.client.1.smithi196.stdout:2/449: getdents d0 0 2024-03-13T14:49:46.432 INFO:tasks.workunit.client.0.smithi176.stdout:6/427: readlink d0/de/l36 0 2024-03-13T14:49:46.433 INFO:tasks.workunit.client.1.smithi196.stdout:2/450: chown d0/d6/f84 77587608 1 2024-03-13T14:49:46.435 INFO:tasks.workunit.client.1.smithi196.stdout:8/509: dread d0/d16/d3d/d8b/d3f/f76 [0,4194304] 0 2024-03-13T14:49:46.435 INFO:tasks.workunit.client.0.smithi176.stdout:3/445: fsync d4/d28/d4f/f53 0 2024-03-13T14:49:46.436 INFO:tasks.workunit.client.0.smithi176.stdout:2/456: dread d1/dc/d61/f6d [0,4194304] 0 2024-03-13T14:49:46.438 INFO:tasks.workunit.client.0.smithi176.stdout:8/380: rename d0/f2 to d0/db/d39/d56/f79 0 2024-03-13T14:49:46.440 INFO:tasks.workunit.client.0.smithi176.stdout:1/397: dread d2/d30/f4c [0,4194304] 0 2024-03-13T14:49:46.443 INFO:tasks.workunit.client.0.smithi176.stdout:1/398: dread - d2/de/d10/d16/d47/f54 zero size 2024-03-13T14:49:46.444 INFO:tasks.workunit.client.0.smithi176.stdout:0/394: mknod d2/d5/dc/c77 0 2024-03-13T14:49:46.445 INFO:tasks.workunit.client.0.smithi176.stdout:0/395: fsync d2/db/d31/f44 0 2024-03-13T14:49:46.447 INFO:tasks.workunit.client.1.smithi196.stdout:6/675: rename d6/d6f/c78 to d6/d9/daa/cd3 0 2024-03-13T14:49:46.448 INFO:tasks.workunit.client.0.smithi176.stdout:5/407: sync 2024-03-13T14:49:46.450 INFO:tasks.workunit.client.0.smithi176.stdout:0/396: truncate d2/db/d31/f57 403806 0 2024-03-13T14:49:46.450 INFO:tasks.workunit.client.1.smithi196.stdout:5/637: dwrite d4/db/d14/d19/d1c/d25/d38/f3e [0,4194304] 0 2024-03-13T14:49:46.450 INFO:tasks.workunit.client.1.smithi196.stdout:1/596: dread d1/d28/d34/f3e [0,4194304] 0 2024-03-13T14:49:46.451 INFO:tasks.workunit.client.0.smithi176.stdout:7/405: creat d1/d28/f75 x:0 0 0 2024-03-13T14:49:46.452 INFO:tasks.workunit.client.1.smithi196.stdout:0/577: truncate d8/da/d25/d26/d35/d3b/d5f/d33/d44/f86 3619652 0 2024-03-13T14:49:46.452 INFO:tasks.workunit.client.0.smithi176.stdout:4/461: mkdir d7/da9 0 2024-03-13T14:49:46.453 INFO:tasks.workunit.client.0.smithi176.stdout:7/406: stat d1/d7/f10 0 2024-03-13T14:49:46.454 INFO:tasks.workunit.client.0.smithi176.stdout:6/428: fsync d0/f8 0 2024-03-13T14:49:46.461 INFO:tasks.workunit.client.1.smithi196.stdout:9/491: symlink d7/de/d4b/d1d/d93/la3 0 2024-03-13T14:49:46.462 INFO:tasks.workunit.client.0.smithi176.stdout:3/446: stat d4/c7f 0 2024-03-13T14:49:46.463 INFO:tasks.workunit.client.0.smithi176.stdout:8/381: fdatasync d0/f3 0 2024-03-13T14:49:46.464 INFO:tasks.workunit.client.0.smithi176.stdout:2/457: dwrite d1/dc/d2d/f97 [0,4194304] 0 2024-03-13T14:49:46.465 INFO:tasks.workunit.client.1.smithi196.stdout:9/492: write d7/de/d4b/d1d/f66 [4918220,50391] 0 2024-03-13T14:49:46.465 INFO:tasks.workunit.client.1.smithi196.stdout:8/510: rename d0/d10/d12/d1e/c67 to d0/d16/d3d/d8b/d3f/cac 0 2024-03-13T14:49:46.466 INFO:tasks.workunit.client.0.smithi176.stdout:3/447: chown d4/d13/d16/l8b 877880023 1 2024-03-13T14:49:46.468 INFO:tasks.workunit.client.0.smithi176.stdout:4/462: symlink d7/d2d/laa 0 2024-03-13T14:49:46.468 INFO:tasks.workunit.client.0.smithi176.stdout:0/397: stat d2/db/c27 0 2024-03-13T14:49:46.468 INFO:tasks.workunit.client.1.smithi196.stdout:6/676: creat d6/d7/df/d27/d39/d44/d74/fd4 x:0 0 0 2024-03-13T14:49:46.469 INFO:tasks.workunit.client.1.smithi196.stdout:9/493: chown d7/d24/d28/f3e 47899 1 2024-03-13T14:49:46.473 INFO:tasks.workunit.client.1.smithi196.stdout:0/578: symlink d8/da/d25/d26/d35/d3b/d50/d5d/da5/lb8 0 2024-03-13T14:49:46.476 INFO:tasks.workunit.client.1.smithi196.stdout:0/579: write d8/da/f13 [4487491,88482] 0 2024-03-13T14:49:46.478 INFO:tasks.workunit.client.1.smithi196.stdout:6/677: dread d6/d9/d17/d19/d4e/f7c [0,4194304] 0 2024-03-13T14:49:46.479 INFO:tasks.workunit.client.1.smithi196.stdout:6/678: chown d6/d2a/dc/d1c/d34 28683 1 2024-03-13T14:49:46.482 INFO:tasks.workunit.client.1.smithi196.stdout:5/638: mkdir d4/db/d14/d19/da7/dc7 0 2024-03-13T14:49:46.486 INFO:tasks.workunit.client.0.smithi176.stdout:3/448: dwrite d4/f78 [0,4194304] 0 2024-03-13T14:49:46.487 INFO:tasks.workunit.client.0.smithi176.stdout:9/491: sync 2024-03-13T14:49:46.489 INFO:tasks.workunit.client.0.smithi176.stdout:9/492: fdatasync d0/db/d89/d32/d58/f83 0 2024-03-13T14:49:46.491 INFO:tasks.workunit.client.1.smithi196.stdout:1/597: rename d1/d9/d6c/d79 to d1/d9/d4e/dcc 0 2024-03-13T14:49:46.492 INFO:tasks.workunit.client.1.smithi196.stdout:9/494: dwrite d7/de/d4b/d1d/f66 [0,4194304] 0 2024-03-13T14:49:46.498 INFO:tasks.workunit.client.1.smithi196.stdout:2/451: write d0/d7/d40/f36 [795454,12581] 0 2024-03-13T14:49:46.500 INFO:tasks.workunit.client.0.smithi176.stdout:6/429: dread d0/d10/f3f [4194304,4194304] 0 2024-03-13T14:49:46.502 INFO:tasks.workunit.client.1.smithi196.stdout:7/659: getdents d0/d9/d31 0 2024-03-13T14:49:46.507 INFO:tasks.workunit.client.1.smithi196.stdout:0/580: rename d8/da/d25/d26/d35/d3b/d50/d5d/da5/lb8 to d8/da/d25/lb9 0 2024-03-13T14:49:46.509 INFO:tasks.workunit.client.1.smithi196.stdout:4/512: truncate d1/d3/d4c/f83 387080 0 2024-03-13T14:49:46.511 INFO:tasks.workunit.client.0.smithi176.stdout:5/408: unlink d5/d2b/l44 0 2024-03-13T14:49:46.514 INFO:tasks.workunit.client.0.smithi176.stdout:7/407: rename d1/d28/d3a/d1b/d18 to d1/d3f/d76 0 2024-03-13T14:49:46.519 INFO:tasks.workunit.client.1.smithi196.stdout:9/495: fdatasync d7/de/d4b/d1d/d22/d4d/f5c 0 2024-03-13T14:49:46.519 INFO:tasks.workunit.client.0.smithi176.stdout:8/382: creat d0/d5/d11/d47/f7a x:0 0 0 2024-03-13T14:49:46.521 INFO:tasks.workunit.client.0.smithi176.stdout:8/383: write d0/f3 [1940034,41885] 0 2024-03-13T14:49:46.523 INFO:tasks.workunit.client.1.smithi196.stdout:3/554: getdents db/d15 0 2024-03-13T14:49:46.525 INFO:tasks.workunit.client.0.smithi176.stdout:8/384: read d0/db/dc/d40/f48 [82599,79683] 0 2024-03-13T14:49:46.527 INFO:tasks.workunit.client.1.smithi196.stdout:7/660: chown d0/d78/lcc 292766055 1 2024-03-13T14:49:46.528 INFO:tasks.workunit.client.1.smithi196.stdout:6/679: unlink d6/d9/daa/cd3 0 2024-03-13T14:49:46.528 INFO:tasks.workunit.client.0.smithi176.stdout:5/409: dwrite d5/d2e/d43/f50 [0,4194304] 0 2024-03-13T14:49:46.528 INFO:tasks.workunit.client.0.smithi176.stdout:8/385: dread - d0/db/dc/f69 zero size 2024-03-13T14:49:46.530 INFO:tasks.workunit.client.1.smithi196.stdout:0/581: dwrite d8/da/d25/d26/d35/d3b/d50/f6d [0,4194304] 0 2024-03-13T14:49:46.530 INFO:tasks.workunit.client.1.smithi196.stdout:1/598: dwrite d1/d38/f16 [0,4194304] 0 2024-03-13T14:49:46.531 INFO:tasks.workunit.client.0.smithi176.stdout:5/410: write d5/d22/d15/f67 [988656,23016] 0 2024-03-13T14:49:46.531 INFO:tasks.workunit.client.1.smithi196.stdout:7/661: dread - d0/d9/d31/d77/d93/fc6 zero size 2024-03-13T14:49:46.536 INFO:tasks.workunit.client.1.smithi196.stdout:4/513: creat d1/d3/d4c/d72/d7d/d80/fac x:0 0 0 2024-03-13T14:49:46.536 INFO:tasks.workunit.client.0.smithi176.stdout:9/493: creat d0/db/d89/d32/d76/d82/fa5 x:0 0 0 2024-03-13T14:49:46.537 INFO:tasks.workunit.client.0.smithi176.stdout:1/399: link d2/de/d3f/d19/c43 d2/de/d3f/d6a/c7d 0 2024-03-13T14:49:46.537 INFO:tasks.workunit.client.1.smithi196.stdout:1/599: write d1/d38/fc6 [41885,2043] 0 2024-03-13T14:49:46.538 INFO:tasks.workunit.client.0.smithi176.stdout:8/386: read d0/db/fd [5214202,6119] 0 2024-03-13T14:49:46.539 INFO:tasks.workunit.client.0.smithi176.stdout:1/400: write d2/f76 [657568,17807] 0 2024-03-13T14:49:46.539 INFO:tasks.workunit.client.0.smithi176.stdout:9/494: write d0/d1e/d1f/d68/f85 [562676,37310] 0 2024-03-13T14:49:46.541 INFO:tasks.workunit.client.1.smithi196.stdout:4/514: chown d1/d3/d7/f5d 1 1 2024-03-13T14:49:46.542 INFO:tasks.workunit.client.0.smithi176.stdout:8/387: dread - d0/d24/d29/f72 zero size 2024-03-13T14:49:46.546 INFO:tasks.workunit.client.1.smithi196.stdout:2/452: mknod d0/d6/d90/c92 0 2024-03-13T14:49:46.549 INFO:tasks.workunit.client.1.smithi196.stdout:9/496: mknod d7/de/d74/d94/ca4 0 2024-03-13T14:49:46.552 INFO:tasks.workunit.client.0.smithi176.stdout:1/401: dread - d2/de/d3f/d3e/f5b zero size 2024-03-13T14:49:46.552 INFO:tasks.workunit.client.0.smithi176.stdout:7/408: creat d1/d1c/f77 x:0 0 0 2024-03-13T14:49:46.553 INFO:tasks.workunit.client.1.smithi196.stdout:9/497: dread - d7/de/d16/f9b zero size 2024-03-13T14:49:46.554 INFO:tasks.workunit.client.1.smithi196.stdout:7/662: dwrite d0/f56 [4194304,4194304] 0 2024-03-13T14:49:46.555 INFO:tasks.workunit.client.1.smithi196.stdout:8/511: truncate d0/d10/d12/d1e/f6a 542445 0 2024-03-13T14:49:46.559 INFO:tasks.workunit.client.0.smithi176.stdout:2/458: mkdir d1/dc/d40/da0 0 2024-03-13T14:49:46.560 INFO:tasks.workunit.client.0.smithi176.stdout:0/398: unlink d2/d5/d1e/c1d 0 2024-03-13T14:49:46.562 INFO:tasks.workunit.client.0.smithi176.stdout:4/463: mkdir d7/dab 0 2024-03-13T14:49:46.564 INFO:tasks.workunit.client.1.smithi196.stdout:1/600: dwrite d1/d9/d4e/d55/d63/da6/fb0 [0,4194304] 0 2024-03-13T14:49:46.565 INFO:tasks.workunit.client.0.smithi176.stdout:5/411: fdatasync d5/d2e/d43/f50 0 2024-03-13T14:49:46.569 INFO:tasks.workunit.client.1.smithi196.stdout:2/453: fdatasync d0/d6/d1d/f23 0 2024-03-13T14:49:46.574 INFO:tasks.workunit.client.0.smithi176.stdout:8/388: dwrite d0/db/dc/d40/f62 [0,4194304] 0 2024-03-13T14:49:46.575 INFO:tasks.workunit.client.1.smithi196.stdout:9/498: read d7/de/d4b/f6a [812776,105415] 0 2024-03-13T14:49:46.575 INFO:tasks.workunit.client.0.smithi176.stdout:9/495: symlink d0/db/d89/d4d/d6e/la6 0 2024-03-13T14:49:46.579 INFO:tasks.workunit.client.0.smithi176.stdout:6/430: rename d0/de/d54/c89 to d0/de/d38/d70/c8a 0 2024-03-13T14:49:46.580 INFO:tasks.workunit.client.1.smithi196.stdout:9/499: dread - d7/de/d16/f9b zero size 2024-03-13T14:49:46.580 INFO:tasks.workunit.client.0.smithi176.stdout:1/402: dread - d2/de/d10/f4f zero size 2024-03-13T14:49:46.580 INFO:tasks.workunit.client.0.smithi176.stdout:8/389: write d0/db/f1a [6090287,9348] 0 2024-03-13T14:49:46.583 INFO:tasks.workunit.client.1.smithi196.stdout:3/555: link db/d37/d64/f90 db/d19/d50/db7/da9/fb9 0 2024-03-13T14:49:46.584 INFO:tasks.workunit.client.1.smithi196.stdout:8/512: dwrite d0/d1c/d29/d47/f80 [0,4194304] 0 2024-03-13T14:49:46.587 INFO:tasks.workunit.client.1.smithi196.stdout:3/556: write db/f4e [750106,36073] 0 2024-03-13T14:49:46.590 INFO:tasks.workunit.client.1.smithi196.stdout:1/601: write d1/f26 [1010067,75862] 0 2024-03-13T14:49:46.592 INFO:tasks.workunit.client.1.smithi196.stdout:2/454: creat d0/d7/d9/d61/f93 x:0 0 0 2024-03-13T14:49:46.604 INFO:tasks.workunit.client.0.smithi176.stdout:9/496: dwrite d0/d1e/f2e [0,4194304] 0 2024-03-13T14:49:46.607 INFO:tasks.workunit.client.1.smithi196.stdout:7/663: write d0/d9/d1e/d58/d72/f66 [1567470,19351] 0 2024-03-13T14:49:46.610 INFO:tasks.workunit.client.1.smithi196.stdout:9/500: rename d7/de/l68 to d7/de/d16/d70/la5 0 2024-03-13T14:49:46.610 INFO:tasks.workunit.client.1.smithi196.stdout:7/664: dread - d0/d4d/d74/fd9 zero size 2024-03-13T14:49:46.611 INFO:tasks.workunit.client.0.smithi176.stdout:0/399: rename d2/db/d35/c2f to d2/c78 0 2024-03-13T14:49:46.612 INFO:tasks.workunit.client.1.smithi196.stdout:6/680: write d6/d9/f1f [1541532,72429] 0 2024-03-13T14:49:46.613 INFO:tasks.workunit.client.1.smithi196.stdout:7/665: chown d0/d78/caf 7966 1 2024-03-13T14:49:46.614 INFO:tasks.workunit.client.1.smithi196.stdout:8/513: stat d0/f56 0 2024-03-13T14:49:46.615 INFO:tasks.workunit.client.1.smithi196.stdout:6/681: chown d6/d9/d17/d19/l43 10 1 2024-03-13T14:49:46.616 INFO:tasks.workunit.client.1.smithi196.stdout:8/514: chown d0/d16/d3d/d8b/f38 102 1 2024-03-13T14:49:46.617 INFO:tasks.workunit.client.0.smithi176.stdout:0/400: dread d2/db/d39/f47 [0,4194304] 0 2024-03-13T14:49:46.618 INFO:tasks.workunit.client.1.smithi196.stdout:3/557: creat db/d15/d44/d79/fba x:0 0 0 2024-03-13T14:49:46.622 INFO:tasks.workunit.client.0.smithi176.stdout:1/403: mknod d2/de/c7e 0 2024-03-13T14:49:46.623 INFO:tasks.workunit.client.0.smithi176.stdout:6/431: creat d0/de/d38/f8b x:0 0 0 2024-03-13T14:49:46.623 INFO:tasks.workunit.client.1.smithi196.stdout:3/558: chown db/d19/d50/db7/da9 1904255694 1 2024-03-13T14:49:46.623 INFO:tasks.workunit.client.1.smithi196.stdout:1/602: symlink d1/d9/d4e/d55/d63/da6/lcd 0 2024-03-13T14:49:46.624 INFO:tasks.workunit.client.0.smithi176.stdout:3/449: link d4/d13/c18 d4/d13/c92 0 2024-03-13T14:49:46.624 INFO:tasks.workunit.client.1.smithi196.stdout:1/603: chown d1/d3f/d5b/l61 117 1 2024-03-13T14:49:46.626 INFO:tasks.workunit.client.1.smithi196.stdout:1/604: chown d1/d3f/d5b/f60 247077349 1 2024-03-13T14:49:46.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:46 smithi176 ceph-mon[29064]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:46.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:46 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/4219607131' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:49:46.628 INFO:tasks.workunit.client.1.smithi196.stdout:0/582: fsync d8/da/d25/d26/d35/d3b/d5f/d33/d44/f86 0 2024-03-13T14:49:46.629 INFO:tasks.workunit.client.0.smithi176.stdout:4/464: getdents d7/da9 0 2024-03-13T14:49:46.633 INFO:tasks.workunit.client.0.smithi176.stdout:2/459: mknod d1/dc/d2d/d59/d80/d5c/d78/ca1 0 2024-03-13T14:49:46.633 INFO:tasks.workunit.client.0.smithi176.stdout:4/465: write d7/d10/d46/d4a/d5d/da1/f8c [463160,76539] 0 2024-03-13T14:49:46.633 INFO:tasks.workunit.client.0.smithi176.stdout:0/401: dwrite d2/d5/f64 [0,4194304] 0 2024-03-13T14:49:46.644 INFO:tasks.workunit.client.1.smithi196.stdout:7/666: unlink d0/c3a 0 2024-03-13T14:49:46.649 INFO:tasks.workunit.client.0.smithi176.stdout:2/460: dread d1/dc/d2d/f97 [0,4194304] 0 2024-03-13T14:49:46.649 INFO:tasks.workunit.client.1.smithi196.stdout:6/682: creat d6/d7/dd/dbc/fd5 x:0 0 0 2024-03-13T14:49:46.649 INFO:tasks.workunit.client.1.smithi196.stdout:8/515: dread d0/d16/d3d/d8b/f38 [0,4194304] 0 2024-03-13T14:49:46.650 INFO:tasks.workunit.client.0.smithi176.stdout:8/390: dread d0/db/d39/d56/f79 [0,4194304] 0 2024-03-13T14:49:46.652 INFO:tasks.workunit.client.0.smithi176.stdout:9/497: mkdir d0/d1e/d8d/da7 0 2024-03-13T14:49:46.653 INFO:tasks.workunit.client.1.smithi196.stdout:3/559: mkdir db/d4c/dbb 0 2024-03-13T14:49:46.654 INFO:tasks.workunit.client.0.smithi176.stdout:2/461: read d1/dc/d2d/d38/f5e [3217934,57889] 0 2024-03-13T14:49:46.655 INFO:tasks.workunit.client.0.smithi176.stdout:6/432: unlink d0/de/d38/c48 0 2024-03-13T14:49:46.659 INFO:tasks.workunit.client.0.smithi176.stdout:3/450: rename d4/d28/d31/f4b to d4/d13/d16/d3d/f93 0 2024-03-13T14:49:46.661 INFO:tasks.workunit.client.1.smithi196.stdout:4/515: link d1/d3/d4c/d72/d7d/d80/d55/c96 d1/d3/d7/d35/d3e/d56/cad 0 2024-03-13T14:49:46.662 INFO:tasks.workunit.client.1.smithi196.stdout:5/639: sync 2024-03-13T14:49:46.663 INFO:tasks.workunit.client.0.smithi176.stdout:7/409: sync 2024-03-13T14:49:46.665 INFO:tasks.workunit.client.1.smithi196.stdout:0/583: rename d8/da/d25/d26/d54/da9 to d8/da/d25/d26/d48/dba 0 2024-03-13T14:49:46.668 INFO:tasks.workunit.client.1.smithi196.stdout:5/640: dread d4/db/d14/d19/d1c/d25/d38/d47/fb8 [0,4194304] 0 2024-03-13T14:49:46.669 INFO:tasks.workunit.client.1.smithi196.stdout:9/501: write d7/de/d4b/f59 [379212,34137] 0 2024-03-13T14:49:46.670 INFO:tasks.workunit.client.0.smithi176.stdout:4/466: creat d7/d8/d45/fac x:0 0 0 2024-03-13T14:49:46.670 INFO:tasks.workunit.client.0.smithi176.stdout:7/410: read d1/d3f/d4b/f58 [901434,41430] 0 2024-03-13T14:49:46.670 INFO:tasks.workunit.client.0.smithi176.stdout:5/412: dwrite d5/f24 [0,4194304] 0 2024-03-13T14:49:46.671 INFO:tasks.workunit.client.1.smithi196.stdout:9/502: fdatasync d7/de/d16/f9b 0 2024-03-13T14:49:46.673 INFO:tasks.workunit.client.0.smithi176.stdout:6/433: dwrite d0/d10/d19/d27/f4d [0,4194304] 0 2024-03-13T14:49:46.674 INFO:tasks.workunit.client.1.smithi196.stdout:6/683: mknod d6/d57/d91/cd6 0 2024-03-13T14:49:46.675 INFO:tasks.workunit.client.0.smithi176.stdout:2/462: dwrite d1/dc/d2d/f2f [0,4194304] 0 2024-03-13T14:49:46.677 INFO:tasks.workunit.client.1.smithi196.stdout:8/516: rmdir d0/d10/d70/d8d 39 2024-03-13T14:49:46.681 INFO:tasks.workunit.client.0.smithi176.stdout:1/404: symlink d2/d29/d73/l7f 0 2024-03-13T14:49:46.682 INFO:tasks.workunit.client.1.smithi196.stdout:3/560: chown db/d37/c7c 14 1 2024-03-13T14:49:46.685 INFO:tasks.workunit.client.1.smithi196.stdout:3/561: write db/f4e [927652,68623] 0 2024-03-13T14:49:46.686 INFO:tasks.workunit.client.0.smithi176.stdout:1/405: dread - d2/de/d10/d16/d59/f6e zero size 2024-03-13T14:49:46.690 INFO:tasks.workunit.client.0.smithi176.stdout:1/406: write d2/de/d10/d16/d40/f77 [114997,105425] 0 2024-03-13T14:49:46.698 INFO:tasks.workunit.client.1.smithi196.stdout:7/667: write d0/d6/d33/f62 [417091,42642] 0 2024-03-13T14:49:46.699 INFO:tasks.workunit.client.0.smithi176.stdout:4/467: dwrite d7/d8/d15/f81 [0,4194304] 0 2024-03-13T14:49:46.699 INFO:tasks.workunit.client.0.smithi176.stdout:9/498: dread d0/db/d89/d32/f95 [0,4194304] 0 2024-03-13T14:49:46.699 INFO:tasks.workunit.client.0.smithi176.stdout:8/391: write d0/db/f19 [3685347,63999] 0 2024-03-13T14:49:46.701 INFO:tasks.workunit.client.0.smithi176.stdout:8/392: chown d0/db 44 1 2024-03-13T14:49:46.702 INFO:tasks.workunit.client.1.smithi196.stdout:0/584: rmdir d8/da/d74 39 2024-03-13T14:49:46.704 INFO:tasks.workunit.client.0.smithi176.stdout:8/393: truncate d0/d5/d11/d47/f7a 466397 0 2024-03-13T14:49:46.705 INFO:tasks.workunit.client.1.smithi196.stdout:0/585: chown d8/da/d25/d26/d35/d3b/d59/l97 14969432 1 2024-03-13T14:49:46.707 INFO:tasks.workunit.client.1.smithi196.stdout:0/586: chown c4 948260461 1 2024-03-13T14:49:46.708 INFO:tasks.workunit.client.0.smithi176.stdout:7/411: mknod d1/d1c/c78 0 2024-03-13T14:49:46.710 INFO:tasks.workunit.client.0.smithi176.stdout:5/413: rename d5/c52 to d5/d2c/c8e 0 2024-03-13T14:49:46.713 INFO:tasks.workunit.client.0.smithi176.stdout:6/434: mkdir d0/d9/d8c 0 2024-03-13T14:49:46.715 INFO:tasks.workunit.client.1.smithi196.stdout:8/517: write d0/d16/d3d/d8b/f8e [3212823,104950] 0 2024-03-13T14:49:46.716 INFO:tasks.workunit.client.1.smithi196.stdout:1/605: write d1/d9/d33/f82 [300383,94466] 0 2024-03-13T14:49:46.717 INFO:tasks.workunit.client.1.smithi196.stdout:8/518: readlink d0/d10/l62 0 2024-03-13T14:49:46.719 INFO:tasks.workunit.client.1.smithi196.stdout:2/455: link d0/d7/d9/d42/f80 d0/d7/f94 0 2024-03-13T14:49:46.719 INFO:tasks.workunit.client.1.smithi196.stdout:1/606: readlink d1/d9/d4e/d55/d63/da6/lac 0 2024-03-13T14:49:46.720 INFO:tasks.workunit.client.1.smithi196.stdout:2/456: readlink d0/d7/l11 0 2024-03-13T14:49:46.722 INFO:tasks.workunit.client.1.smithi196.stdout:4/516: write d1/d3/d4c/f59 [1899559,40173] 0 2024-03-13T14:49:46.722 INFO:tasks.workunit.client.1.smithi196.stdout:3/562: creat db/fbc x:0 0 0 2024-03-13T14:49:46.722 INFO:tasks.workunit.client.1.smithi196.stdout:2/457: chown d0/d6/d90 582857 1 2024-03-13T14:49:46.723 INFO:tasks.workunit.client.1.smithi196.stdout:8/519: read d0/d10/d12/f33 [2632482,27353] 0 2024-03-13T14:49:46.725 INFO:tasks.workunit.client.0.smithi176.stdout:5/414: dwrite d5/d31/f61 [0,4194304] 0 2024-03-13T14:49:46.726 INFO:tasks.workunit.client.0.smithi176.stdout:5/415: fsync d5/d2e/f54 0 2024-03-13T14:49:46.728 INFO:tasks.workunit.client.0.smithi176.stdout:5/416: write d5/d2c/f3d [826691,104198] 0 2024-03-13T14:49:46.740 INFO:tasks.workunit.client.1.smithi196.stdout:5/641: dwrite d4/db/d3c/d6a/d93/f9e [0,4194304] 0 2024-03-13T14:49:46.742 INFO:tasks.workunit.client.1.smithi196.stdout:5/642: chown d4/db/d14 416859893 1 2024-03-13T14:49:46.749 INFO:tasks.workunit.client.1.smithi196.stdout:2/458: dwrite d0/f77 [0,4194304] 0 2024-03-13T14:49:46.771 INFO:tasks.workunit.client.1.smithi196.stdout:5/643: dwrite d4/db/d14/fb4 [0,4194304] 0 2024-03-13T14:49:46.776 INFO:tasks.workunit.client.1.smithi196.stdout:5/644: chown d4/db/d14/d19/d1c/d25/d38/d47/da0/d7d 14 1 2024-03-13T14:49:46.827 INFO:tasks.workunit.client.0.smithi176.stdout:1/407: chown d2/de/d3f/d50/c36 32 1 2024-03-13T14:49:46.834 INFO:tasks.workunit.client.1.smithi196.stdout:9/503: rename d7/de/d16/d70/la5 to d7/de/d74/la6 0 2024-03-13T14:49:46.837 INFO:tasks.workunit.client.1.smithi196.stdout:9/504: chown d7/de/d74/c86 0 1 2024-03-13T14:49:46.838 INFO:tasks.workunit.client.1.smithi196.stdout:6/684: symlink d6/d7/ld7 0 2024-03-13T14:49:46.839 INFO:tasks.workunit.client.0.smithi176.stdout:9/499: chown d0/le 7 1 2024-03-13T14:49:46.840 INFO:tasks.workunit.client.1.smithi196.stdout:6/685: chown d6/d2a/lb8 281508 1 2024-03-13T14:49:46.840 INFO:tasks.workunit.client.1.smithi196.stdout:9/505: write d7/d24/d28/d6c/f8f [221012,120579] 0 2024-03-13T14:49:46.845 INFO:tasks.workunit.client.1.smithi196.stdout:0/587: mknod d8/da/d25/d26/d35/d3b/d5f/d33/d44/cbb 0 2024-03-13T14:49:46.861 INFO:tasks.workunit.client.1.smithi196.stdout:4/517: symlink d1/d8f/lae 0 2024-03-13T14:49:46.874 INFO:tasks.workunit.client.0.smithi176.stdout:3/451: creat d4/d13/d81/d89/f94 x:0 0 0 2024-03-13T14:49:46.876 INFO:tasks.workunit.client.0.smithi176.stdout:3/452: dread - d4/d13/d81/d89/f94 zero size 2024-03-13T14:49:46.878 INFO:tasks.workunit.client.0.smithi176.stdout:3/453: read d4/d13/d81/d89/f27 [273890,26646] 0 2024-03-13T14:49:46.887 INFO:tasks.workunit.client.0.smithi176.stdout:3/454: creat d4/d28/d34/d38/f95 x:0 0 0 2024-03-13T14:49:46.895 INFO:tasks.workunit.client.0.smithi176.stdout:9/500: getdents d0/db/d89/d32/d58/d5f/d84/d9a 0 2024-03-13T14:49:46.898 INFO:tasks.workunit.client.0.smithi176.stdout:3/455: symlink d4/d13/d83/l96 0 2024-03-13T14:49:46.899 INFO:tasks.workunit.client.0.smithi176.stdout:6/435: mkdir d0/de/d8d 0 2024-03-13T14:49:46.900 INFO:tasks.workunit.client.0.smithi176.stdout:3/456: dread - d4/d13/d81/d89/f94 zero size 2024-03-13T14:49:46.903 INFO:tasks.workunit.client.0.smithi176.stdout:3/457: creat d4/f97 x:0 0 0 2024-03-13T14:49:46.905 INFO:tasks.workunit.client.0.smithi176.stdout:5/417: rmdir d5/d2e/d43 39 2024-03-13T14:49:46.907 INFO:tasks.workunit.client.1.smithi196.stdout:1/607: creat d1/d9/d4e/d55/d63/db9/dbb/fce x:0 0 0 2024-03-13T14:49:46.907 INFO:tasks.workunit.client.0.smithi176.stdout:9/501: dread d0/db/d89/d32/d58/d5f/f75 [0,4194304] 0 2024-03-13T14:49:46.907 INFO:tasks.workunit.client.0.smithi176.stdout:1/408: rename d2/de/d10/d16/l38 to d2/de/d10/d16/d40/l80 0 2024-03-13T14:49:46.908 INFO:tasks.workunit.client.1.smithi196.stdout:1/608: fdatasync d1/d9/d33/d4b/d66/f7c 0 2024-03-13T14:49:46.909 INFO:tasks.workunit.client.0.smithi176.stdout:3/458: mknod d4/d13/d5e/c98 0 2024-03-13T14:49:46.910 INFO:tasks.workunit.client.1.smithi196.stdout:3/563: mkdir db/d19/d22/d3a/d68/dbd 0 2024-03-13T14:49:46.915 INFO:tasks.workunit.client.0.smithi176.stdout:6/436: rename d0/d10/d19/d7d/f87 to d0/de/d38/d70/f8e 0 2024-03-13T14:49:46.918 INFO:tasks.workunit.client.0.smithi176.stdout:6/437: write d0/d10/d19/d27/f4a [4373047,63539] 0 2024-03-13T14:49:46.922 INFO:tasks.workunit.client.0.smithi176.stdout:6/438: write d0/de/d38/f8b [127057,40802] 0 2024-03-13T14:49:46.929 INFO:tasks.workunit.client.0.smithi176.stdout:1/409: truncate d2/de/d3f/d50/f42 348797 0 2024-03-13T14:49:46.930 INFO:tasks.workunit.client.0.smithi176.stdout:1/410: chown d2/d30/f4c 1 1 2024-03-13T14:49:46.933 INFO:tasks.workunit.client.0.smithi176.stdout:9/502: dwrite d0/d40/f47 [0,4194304] 0 2024-03-13T14:49:46.936 INFO:tasks.workunit.client.0.smithi176.stdout:9/503: write d0/d40/f47 [3095481,50200] 0 2024-03-13T14:49:46.936 INFO:tasks.workunit.client.1.smithi196.stdout:4/518: rmdir d1/d3/d7 39 2024-03-13T14:49:46.937 INFO:tasks.workunit.client.0.smithi176.stdout:3/459: dread d4/d13/d81/d89/f32 [0,4194304] 0 2024-03-13T14:49:46.937 INFO:tasks.workunit.client.1.smithi196.stdout:9/506: dread d7/d24/d28/f82 [0,4194304] 0 2024-03-13T14:49:46.939 INFO:tasks.workunit.client.0.smithi176.stdout:5/418: creat d5/d2c/d40/f8f x:0 0 0 2024-03-13T14:49:46.941 INFO:tasks.workunit.client.0.smithi176.stdout:6/439: mknod d0/de/d38/d77/d76/c8f 0 2024-03-13T14:49:46.943 INFO:tasks.workunit.client.0.smithi176.stdout:1/411: creat d2/de/d10/d16/d4a/d5f/f81 x:0 0 0 2024-03-13T14:49:46.945 INFO:tasks.workunit.client.0.smithi176.stdout:6/440: write d0/d10/d19/f1a [4457094,64480] 0 2024-03-13T14:49:46.953 INFO:tasks.workunit.client.0.smithi176.stdout:5/419: dwrite d5/d22/d15/f2d [0,4194304] 0 2024-03-13T14:49:46.957 INFO:tasks.workunit.client.0.smithi176.stdout:1/412: creat d2/d30/d63/d22/f82 x:0 0 0 2024-03-13T14:49:46.959 INFO:tasks.workunit.client.1.smithi196.stdout:9/507: mkdir d7/de/d4b/d1d/d93/da7 0 2024-03-13T14:49:46.964 INFO:tasks.workunit.client.1.smithi196.stdout:3/564: dwrite db/d19/fa2 [0,4194304] 0 2024-03-13T14:49:46.967 INFO:tasks.workunit.client.0.smithi176.stdout:5/420: dwrite d5/d22/d15/f67 [0,4194304] 0 2024-03-13T14:49:46.979 INFO:tasks.workunit.client.0.smithi176.stdout:9/504: link d0/le d0/d40/la8 0 2024-03-13T14:49:46.980 INFO:tasks.workunit.client.1.smithi196.stdout:9/508: rename d7/d24/f2b to d7/de/d74/d94/fa8 0 2024-03-13T14:49:46.986 INFO:tasks.workunit.client.0.smithi176.stdout:5/421: rmdir d5/d22 39 2024-03-13T14:49:46.986 INFO:tasks.workunit.client.1.smithi196.stdout:1/609: creat d1/fcf x:0 0 0 2024-03-13T14:49:46.988 INFO:tasks.workunit.client.1.smithi196.stdout:5/645: getdents d4/db/d14/d19/d1c/d25/d38/d47 0 2024-03-13T14:49:46.990 INFO:tasks.workunit.client.0.smithi176.stdout:3/460: creat d4/d28/d31/f99 x:0 0 0 2024-03-13T14:49:46.991 INFO:tasks.workunit.client.1.smithi196.stdout:1/610: dread f0 [4194304,4194304] 0 2024-03-13T14:49:46.991 INFO:tasks.workunit.client.0.smithi176.stdout:3/461: chown d4/d13/f57 7 1 2024-03-13T14:49:46.993 INFO:tasks.workunit.client.0.smithi176.stdout:3/462: write d4/d13/d81/d89/f2b [1445263,25930] 0 2024-03-13T14:49:46.995 INFO:tasks.workunit.client.1.smithi196.stdout:3/565: write db/d19/d50/f58 [534412,88697] 0 2024-03-13T14:49:46.996 INFO:tasks.workunit.client.0.smithi176.stdout:1/413: mkdir d2/de/d3f/d50/d83 0 2024-03-13T14:49:46.999 INFO:tasks.workunit.client.1.smithi196.stdout:0/588: getdents d8 0 2024-03-13T14:49:47.000 INFO:tasks.workunit.client.1.smithi196.stdout:9/509: dwrite d7/de/d16/f60 [0,4194304] 0 2024-03-13T14:49:47.005 INFO:tasks.workunit.client.0.smithi176.stdout:9/505: rename d0/d17/l43 to d0/d1e/la9 0 2024-03-13T14:49:47.009 INFO:tasks.workunit.client.0.smithi176.stdout:5/422: creat d5/d31/d47/d80/f90 x:0 0 0 2024-03-13T14:49:47.012 INFO:tasks.workunit.client.1.smithi196.stdout:1/611: symlink d1/d9/d33/d4b/d88/ld0 0 2024-03-13T14:49:47.012 INFO:tasks.workunit.client.0.smithi176.stdout:3/463: dwrite d4/d13/d81/d89/f94 [0,4194304] 0 2024-03-13T14:49:47.016 INFO:tasks.workunit.client.1.smithi196.stdout:3/566: creat db/d19/d50/db7/fbe x:0 0 0 2024-03-13T14:49:47.016 INFO:tasks.workunit.client.0.smithi176.stdout:8/394: write d0/db/dc/f2a [931943,67151] 0 2024-03-13T14:49:47.016 INFO:tasks.workunit.client.0.smithi176.stdout:3/464: fsync d4/d13/d2d/d50/f3c 0 2024-03-13T14:49:47.018 INFO:tasks.workunit.client.0.smithi176.stdout:6/441: getdents d0/de/d38/d7e 0 2024-03-13T14:49:47.018 INFO:tasks.workunit.client.1.smithi196.stdout:7/668: rmdir d0 39 2024-03-13T14:49:47.019 INFO:tasks.workunit.client.0.smithi176.stdout:8/395: read d0/d5/d11/f1b [709719,92902] 0 2024-03-13T14:49:47.024 INFO:tasks.workunit.client.1.smithi196.stdout:9/510: unlink d7/d24/d28/f3e 0 2024-03-13T14:49:47.025 INFO:tasks.workunit.client.0.smithi176.stdout:9/506: rename d0/d40/c51 to d0/d40/d53/caa 0 2024-03-13T14:49:47.030 INFO:tasks.workunit.client.0.smithi176.stdout:1/414: creat d2/de/d10/d16/d4a/d5f/d66/f84 x:0 0 0 2024-03-13T14:49:47.036 INFO:tasks.workunit.client.1.smithi196.stdout:0/589: dwrite d8/da/d74/fae [0,4194304] 0 2024-03-13T14:49:47.037 INFO:tasks.workunit.client.0.smithi176.stdout:2/463: truncate d1/dc/d2d/d38/d5d/f62 2208494 0 2024-03-13T14:49:47.038 INFO:tasks.workunit.client.1.smithi196.stdout:7/669: chown d0/d9/d1e/d3f/f60 7 1 2024-03-13T14:49:47.038 INFO:tasks.workunit.client.1.smithi196.stdout:9/511: creat d7/de/d16/d3c/fa9 x:0 0 0 2024-03-13T14:49:47.039 INFO:tasks.workunit.client.0.smithi176.stdout:2/464: readlink d1/d2a/l2b 0 2024-03-13T14:49:47.040 INFO:tasks.workunit.client.0.smithi176.stdout:6/442: dwrite d0/d10/d19/f1a [4194304,4194304] 0 2024-03-13T14:49:47.040 INFO:tasks.workunit.client.0.smithi176.stdout:3/465: write d4/d13/d16/d3d/f93 [1164426,71866] 0 2024-03-13T14:49:47.041 INFO:tasks.workunit.client.0.smithi176.stdout:7/412: dwrite d1/d28/d3a/f20 [0,4194304] 0 2024-03-13T14:49:47.047 INFO:tasks.workunit.client.1.smithi196.stdout:0/590: chown d8/da/d25/d26/d54/d88/c93 1797507631 1 2024-03-13T14:49:47.051 INFO:tasks.workunit.client.0.smithi176.stdout:5/423: dread d5/d22/d15/f2d [0,4194304] 0 2024-03-13T14:49:47.055 INFO:tasks.workunit.client.0.smithi176.stdout:1/415: rmdir d2/de/d3f/d3e 39 2024-03-13T14:49:47.057 INFO:tasks.workunit.client.1.smithi196.stdout:0/591: symlink d8/d37/lbc 0 2024-03-13T14:49:47.059 INFO:tasks.workunit.client.1.smithi196.stdout:9/512: dread d7/de/d4b/f7c [0,4194304] 0 2024-03-13T14:49:47.061 INFO:tasks.workunit.client.0.smithi176.stdout:3/466: creat d4/d13/d5e/f9a x:0 0 0 2024-03-13T14:49:47.064 INFO:tasks.workunit.client.0.smithi176.stdout:3/467: read - d4/d28/d31/f62 zero size 2024-03-13T14:49:47.064 INFO:tasks.workunit.client.0.smithi176.stdout:2/465: dwrite d1/d2a/f82 [0,4194304] 0 2024-03-13T14:49:47.066 INFO:tasks.workunit.client.1.smithi196.stdout:9/513: unlink d7/d9/f44 0 2024-03-13T14:49:47.066 INFO:tasks.workunit.client.0.smithi176.stdout:3/468: dread - d4/d28/d31/d45/d82/d87/f8d zero size 2024-03-13T14:49:47.070 INFO:tasks.workunit.client.0.smithi176.stdout:2/466: read d1/dc/d2d/d59/d8c/f8f [619595,103596] 0 2024-03-13T14:49:47.072 INFO:tasks.workunit.client.0.smithi176.stdout:5/424: mknod d5/d48/d4c/d7b/c91 0 2024-03-13T14:49:47.072 INFO:tasks.workunit.client.1.smithi196.stdout:9/514: symlink d7/de/d16/d3c/laa 0 2024-03-13T14:49:47.073 INFO:tasks.workunit.client.0.smithi176.stdout:2/467: write d1/dc/f16 [3048650,74661] 0 2024-03-13T14:49:47.079 INFO:tasks.workunit.client.1.smithi196.stdout:7/670: creat d0/d9/d31/fdd x:0 0 0 2024-03-13T14:49:47.080 INFO:tasks.workunit.client.0.smithi176.stdout:3/469: mknod d4/d13/d83/c9b 0 2024-03-13T14:49:47.081 INFO:tasks.workunit.client.1.smithi196.stdout:0/592: dread d8/da/d25/f2f [0,4194304] 0 2024-03-13T14:49:47.082 INFO:tasks.workunit.client.1.smithi196.stdout:7/671: readlink d0/d6/d33/d57/d8a/lbc 0 2024-03-13T14:49:47.084 INFO:tasks.workunit.client.0.smithi176.stdout:7/413: getdents d1/d1c/d26/d41 0 2024-03-13T14:49:47.085 INFO:tasks.workunit.client.0.smithi176.stdout:0/402: truncate d2/d5/d1e/f52 1841814 0 2024-03-13T14:49:47.087 INFO:tasks.workunit.client.0.smithi176.stdout:9/507: link d0/db/d89/d32/d58/d9c/la1 d0/lab 0 2024-03-13T14:49:47.087 INFO:tasks.workunit.client.0.smithi176.stdout:1/416: dwrite d2/de/d3f/d50/f78 [0,4194304] 0 2024-03-13T14:49:47.090 INFO:tasks.workunit.client.1.smithi196.stdout:0/593: mknod d8/da/d25/d26/d35/d3b/d50/d5d/cbd 0 2024-03-13T14:49:47.090 INFO:tasks.workunit.client.0.smithi176.stdout:1/417: truncate d2/de/d10/f6d 687729 0 2024-03-13T14:49:47.093 INFO:tasks.workunit.client.1.smithi196.stdout:9/515: rename d7/de/d4b/d1d/l4f to d7/de/lab 0 2024-03-13T14:49:47.093 INFO:tasks.workunit.client.1.smithi196.stdout:0/594: chown d8/da/d25/f5a 347715853 1 2024-03-13T14:49:47.095 INFO:tasks.workunit.client.1.smithi196.stdout:6/686: sync 2024-03-13T14:49:47.096 INFO:tasks.workunit.client.0.smithi176.stdout:2/468: truncate d1/d2a/f73 1334085 0 2024-03-13T14:49:47.096 INFO:tasks.workunit.client.0.smithi176.stdout:9/508: dread d0/d40/d53/f65 [0,4194304] 0 2024-03-13T14:49:47.097 INFO:tasks.workunit.client.1.smithi196.stdout:7/672: creat d0/d4d/dd1/dd6/fde x:0 0 0 2024-03-13T14:49:47.098 INFO:tasks.workunit.client.0.smithi176.stdout:7/414: creat d1/d1c/f79 x:0 0 0 2024-03-13T14:49:47.100 INFO:tasks.workunit.client.0.smithi176.stdout:0/403: write d2/d5/d28/f56 [24463,23987] 0 2024-03-13T14:49:47.104 INFO:tasks.workunit.client.0.smithi176.stdout:1/418: fsync d2/de/d3f/d19/f32 0 2024-03-13T14:49:47.105 INFO:tasks.workunit.client.1.smithi196.stdout:0/595: mkdir d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe 0 2024-03-13T14:49:47.107 INFO:tasks.workunit.client.1.smithi196.stdout:0/596: chown d8/da/d25/d26/d48/d82/db7 47782 1 2024-03-13T14:49:47.107 INFO:tasks.workunit.client.1.smithi196.stdout:9/516: rename d7/d9/d17/l9a to d7/de/d4b/d1d/d22/lac 0 2024-03-13T14:49:47.109 INFO:tasks.workunit.client.0.smithi176.stdout:9/509: creat d0/d1e/d6f/fac x:0 0 0 2024-03-13T14:49:47.110 INFO:tasks.workunit.client.1.smithi196.stdout:9/517: truncate d7/d24/d28/d6c/f8f 418695 0 2024-03-13T14:49:47.111 INFO:tasks.workunit.client.1.smithi196.stdout:7/673: mkdir d0/d9/d1e/d58/d72/da9/da3/ddf 0 2024-03-13T14:49:47.112 INFO:tasks.workunit.client.1.smithi196.stdout:6/687: symlink d6/d2a/dc/d1c/da7/d9f/ld8 0 2024-03-13T14:49:47.115 INFO:tasks.workunit.client.0.smithi176.stdout:7/415: dread d1/d7/f3d [0,4194304] 0 2024-03-13T14:49:47.115 INFO:tasks.workunit.client.0.smithi176.stdout:2/469: rename d1/dc/d1c/f36 to d1/d2a/d33/fa2 0 2024-03-13T14:49:47.118 INFO:tasks.workunit.client.0.smithi176.stdout:3/470: creat d4/d13/d81/d89/f9c x:0 0 0 2024-03-13T14:49:47.118 INFO:tasks.workunit.client.1.smithi196.stdout:9/518: creat d7/de/d74/fad x:0 0 0 2024-03-13T14:49:47.119 INFO:tasks.workunit.client.0.smithi176.stdout:0/404: mkdir d2/d5/dc/d59/d79 0 2024-03-13T14:49:47.121 INFO:tasks.workunit.client.0.smithi176.stdout:9/510: creat d0/d40/d53/fad x:0 0 0 2024-03-13T14:49:47.122 INFO:tasks.workunit.client.0.smithi176.stdout:6/443: fsync d0/de/d38/d70/f8e 0 2024-03-13T14:49:47.122 INFO:tasks.workunit.client.1.smithi196.stdout:9/519: read d7/de/f40 [108631,86359] 0 2024-03-13T14:49:47.123 INFO:tasks.workunit.client.1.smithi196.stdout:6/688: symlink d6/d9/d17/d19/d4e/d66/ld9 0 2024-03-13T14:49:47.123 INFO:tasks.workunit.client.0.smithi176.stdout:6/444: chown d0/d9/l20 27926 1 2024-03-13T14:49:47.125 INFO:tasks.workunit.client.1.smithi196.stdout:2/459: write d0/d7/f94 [1347858,52021] 0 2024-03-13T14:49:47.128 INFO:tasks.workunit.client.0.smithi176.stdout:1/419: rename d2/d29/d73/l7f to d2/de/d10/d16/d4a/l85 0 2024-03-13T14:49:47.131 INFO:tasks.workunit.client.1.smithi196.stdout:7/674: mkdir d0/d9/d31/d77/dcf/de0 0 2024-03-13T14:49:47.135 INFO:tasks.workunit.client.0.smithi176.stdout:0/405: creat d2/db/d39/f7a x:0 0 0 2024-03-13T14:49:47.137 INFO:tasks.workunit.client.0.smithi176.stdout:6/445: rename d0/d10/d19/d27/d2b/d2e/c5d to d0/d10/d19/c90 0 2024-03-13T14:49:47.137 INFO:tasks.workunit.client.0.smithi176.stdout:2/470: dwrite d1/dc/f34 [0,4194304] 0 2024-03-13T14:49:47.138 INFO:tasks.workunit.client.1.smithi196.stdout:0/597: rmdir d8/da/d25/d26/d35/d3b/d5f/d17/daa 0 2024-03-13T14:49:47.143 INFO:tasks.workunit.client.0.smithi176.stdout:6/446: dread - d0/d10/d19/d27/d2b/d2e/f52 zero size 2024-03-13T14:49:47.146 INFO:tasks.workunit.client.1.smithi196.stdout:8/520: unlink d0/d10/d12/f33 0 2024-03-13T14:49:47.146 INFO:tasks.workunit.client.0.smithi176.stdout:3/471: dwrite d4/f97 [0,4194304] 0 2024-03-13T14:49:47.146 INFO:tasks.workunit.client.0.smithi176.stdout:0/406: fdatasync d2/db/f25 0 2024-03-13T14:49:47.149 INFO:tasks.workunit.client.0.smithi176.stdout:1/420: mkdir d2/de/d3f/d3e/d86 0 2024-03-13T14:49:47.152 INFO:tasks.workunit.client.0.smithi176.stdout:2/471: mkdir d1/dc/d2d/d59/d80/d7e/da3 0 2024-03-13T14:49:47.152 INFO:tasks.workunit.client.0.smithi176.stdout:1/421: dread - d2/d30/d63/d22/f82 zero size 2024-03-13T14:49:47.156 INFO:tasks.workunit.client.0.smithi176.stdout:6/447: write d0/d10/d19/d27/d2b/f65 [3852018,130028] 0 2024-03-13T14:49:47.156 INFO:tasks.workunit.client.0.smithi176.stdout:2/472: chown d1/d2a/l43 4952099 1 2024-03-13T14:49:47.157 INFO:tasks.workunit.client.1.smithi196.stdout:6/689: rename d6/d7/f8d to d6/d9/d17/d19/d5f/fda 0 2024-03-13T14:49:47.161 INFO:tasks.workunit.client.0.smithi176.stdout:2/473: fsync d1/dc/d2d/d59/f8a 0 2024-03-13T14:49:47.161 INFO:tasks.workunit.client.0.smithi176.stdout:3/472: mknod d4/c9d 0 2024-03-13T14:49:47.162 INFO:tasks.workunit.client.1.smithi196.stdout:0/598: mknod d8/da/d25/d26/cbf 0 2024-03-13T14:49:47.162 INFO:tasks.workunit.client.0.smithi176.stdout:3/473: read - d4/d13/d5e/f9a zero size 2024-03-13T14:49:47.163 INFO:tasks.workunit.client.1.smithi196.stdout:2/460: truncate d0/d7/d9/f18 3621983 0 2024-03-13T14:49:47.164 INFO:tasks.workunit.client.0.smithi176.stdout:6/448: mknod d0/de/d54/c91 0 2024-03-13T14:49:47.165 INFO:tasks.workunit.client.1.smithi196.stdout:0/599: chown d8/da/d25/d26/d35/d3b/d50/d5d/l15 3 1 2024-03-13T14:49:47.166 INFO:tasks.workunit.client.1.smithi196.stdout:8/521: creat d0/d16/d3d/d8b/d3f/fad x:0 0 0 2024-03-13T14:49:47.168 INFO:tasks.workunit.client.0.smithi176.stdout:1/422: rename d2/de/d10/d16/d4a/d5f/f69 to d2/d30/d63/d22/d71/f87 0 2024-03-13T14:49:47.171 INFO:tasks.workunit.client.0.smithi176.stdout:7/416: dread d1/d1c/d26/d41/d2b/f37 [0,4194304] 0 2024-03-13T14:49:47.171 INFO:tasks.workunit.client.1.smithi196.stdout:6/690: mknod d6/d2a/dc/d1c/d34/cdb 0 2024-03-13T14:49:47.172 INFO:tasks.workunit.client.0.smithi176.stdout:7/417: fsync d1/d3f/d76/f6d 0 2024-03-13T14:49:47.173 INFO:tasks.workunit.client.0.smithi176.stdout:2/474: creat d1/dc/d1c/fa4 x:0 0 0 2024-03-13T14:49:47.174 INFO:tasks.workunit.client.1.smithi196.stdout:0/600: symlink d8/d37/lc0 0 2024-03-13T14:49:47.174 INFO:tasks.workunit.client.1.smithi196.stdout:2/461: dread d0/d7/f3f [0,4194304] 0 2024-03-13T14:49:47.176 INFO:tasks.workunit.client.1.smithi196.stdout:4/519: write d1/d3/d7/d35/d3e/f6b [881141,52916] 0 2024-03-13T14:49:47.176 INFO:tasks.workunit.client.1.smithi196.stdout:8/522: unlink d0/l4f 0 2024-03-13T14:49:47.178 INFO:tasks.workunit.client.0.smithi176.stdout:3/474: dread d4/d13/d81/d89/f2b [0,4194304] 0 2024-03-13T14:49:47.180 INFO:tasks.workunit.client.0.smithi176.stdout:6/449: mknod d0/d10/d19/d27/d2b/d2e/c92 0 2024-03-13T14:49:47.183 INFO:tasks.workunit.client.1.smithi196.stdout:2/462: creat d0/d6/d34/d5b/f95 x:0 0 0 2024-03-13T14:49:47.184 INFO:tasks.workunit.client.0.smithi176.stdout:3/475: fsync d4/f1d 0 2024-03-13T14:49:47.186 INFO:tasks.workunit.client.1.smithi196.stdout:4/520: mkdir d1/d3/d4c/d72/daf 0 2024-03-13T14:49:47.187 INFO:tasks.workunit.client.0.smithi176.stdout:3/476: chown d4/d28/d31/f75 1203 1 2024-03-13T14:49:47.188 INFO:tasks.workunit.client.0.smithi176.stdout:3/477: chown d4/d13/f61 185 1 2024-03-13T14:49:47.188 INFO:tasks.workunit.client.1.smithi196.stdout:7/675: rename d0/d9/d1e/d58/d72/da9/d47/f4e to d0/d6/fe1 0 2024-03-13T14:49:47.189 INFO:tasks.workunit.client.0.smithi176.stdout:3/478: fdatasync f2 0 2024-03-13T14:49:47.191 INFO:tasks.workunit.client.0.smithi176.stdout:7/418: mkdir d1/d3f/d4b/d63/d7a 0 2024-03-13T14:49:47.191 INFO:tasks.workunit.client.1.smithi196.stdout:3/567: mknod db/cbf 0 2024-03-13T14:49:47.192 INFO:tasks.workunit.client.1.smithi196.stdout:3/568: fsync db/f2d 0 2024-03-13T14:49:47.193 INFO:tasks.workunit.client.0.smithi176.stdout:4/468: symlink d7/lad 0 2024-03-13T14:49:47.195 INFO:tasks.workunit.client.1.smithi196.stdout:0/601: mknod d8/da/d25/d26/d35/d3b/cc1 0 2024-03-13T14:49:47.199 INFO:tasks.workunit.client.0.smithi176.stdout:7/419: mkdir d1/d1c/d26/d41/d2b/d7b 0 2024-03-13T14:49:47.204 INFO:tasks.workunit.client.0.smithi176.stdout:3/479: dwrite d4/d28/d31/d45/d82/d87/f8d [0,4194304] 0 2024-03-13T14:49:47.206 INFO:tasks.workunit.client.1.smithi196.stdout:5/646: truncate d4/db/d14/fb4 257377 0 2024-03-13T14:49:47.208 INFO:tasks.workunit.client.1.smithi196.stdout:6/691: link d6/d2a/dc/d1c/d34/cc0 d6/d2a/dc/d1c/da7/cdc 0 2024-03-13T14:49:47.217 INFO:tasks.workunit.client.1.smithi196.stdout:2/463: mkdir d0/d6/d4b/d96 0 2024-03-13T14:49:47.218 INFO:tasks.workunit.client.0.smithi176.stdout:4/469: dread d7/d10/d46/f8f [0,4194304] 0 2024-03-13T14:49:47.219 INFO:tasks.workunit.client.1.smithi196.stdout:1/612: dwrite d1/f2a [0,4194304] 0 2024-03-13T14:49:47.221 INFO:tasks.workunit.client.1.smithi196.stdout:4/521: symlink d1/d3/d7/d35/lb0 0 2024-03-13T14:49:47.222 INFO:tasks.workunit.client.1.smithi196.stdout:8/523: link d0/d16/d71/c82 d0/d16/d71/cae 0 2024-03-13T14:49:47.222 INFO:tasks.workunit.client.0.smithi176.stdout:8/396: dwrite d0/db/d39/d56/f79 [0,4194304] 0 2024-03-13T14:49:47.224 INFO:tasks.workunit.client.1.smithi196.stdout:7/676: link d0/d9/d31/d50/f75 d0/d4d/dd1/dd6/fe2 0 2024-03-13T14:49:47.227 INFO:tasks.workunit.client.0.smithi176.stdout:8/397: dread d0/d5/d11/f1b [0,4194304] 0 2024-03-13T14:49:47.227 INFO:tasks.workunit.client.1.smithi196.stdout:3/569: mkdir db/d19/d50/db7/da9/dc0 0 2024-03-13T14:49:47.228 INFO:tasks.workunit.client.1.smithi196.stdout:4/522: write d1/d3/d4c/d72/d7d/d80/fac [279266,99978] 0 2024-03-13T14:49:47.235 INFO:tasks.workunit.client.1.smithi196.stdout:5/647: dwrite d4/db/d14/d19/d1c/d2d/d82/d8b/fbf [0,4194304] 0 2024-03-13T14:49:47.235 INFO:tasks.workunit.client.1.smithi196.stdout:2/464: rename d0/d6/f14 to d0/d7/d9/d5a/f97 0 2024-03-13T14:49:47.235 INFO:tasks.workunit.client.0.smithi176.stdout:4/470: creat d7/d2d/da7/fae x:0 0 0 2024-03-13T14:49:47.235 INFO:tasks.workunit.client.0.smithi176.stdout:4/471: read - d7/d2d/fa4 zero size 2024-03-13T14:49:47.243 INFO:tasks.workunit.client.0.smithi176.stdout:8/398: mkdir d0/d24/d7b 0 2024-03-13T14:49:47.244 INFO:tasks.workunit.client.0.smithi176.stdout:8/399: stat d0/f3 0 2024-03-13T14:49:47.250 INFO:tasks.workunit.client.0.smithi176.stdout:4/472: dread d7/d8/d15/f81 [0,4194304] 0 2024-03-13T14:49:47.251 INFO:tasks.workunit.client.1.smithi196.stdout:4/523: mkdir d1/d3/d7/d35/d3e/db1 0 2024-03-13T14:49:47.252 INFO:tasks.workunit.client.0.smithi176.stdout:4/473: stat d7/d10/d46/f4c 0 2024-03-13T14:49:47.254 INFO:tasks.workunit.client.0.smithi176.stdout:2/475: rename d1/d2a/f73 to d1/dc/fa5 0 2024-03-13T14:49:47.254 INFO:tasks.workunit.client.0.smithi176.stdout:6/450: rename d0 to d0/de/d54/d93 22 2024-03-13T14:49:47.255 INFO:tasks.workunit.client.1.smithi196.stdout:6/692: dread d6/d2a/dc/d1c/da7/fb5 [0,4194304] 0 2024-03-13T14:49:47.256 INFO:tasks.workunit.client.0.smithi176.stdout:2/476: dread - d1/f96 zero size 2024-03-13T14:49:47.256 INFO:tasks.workunit.client.1.smithi196.stdout:1/613: rename d1/d3f/d5b/c57 to d1/d38/cd1 0 2024-03-13T14:49:47.259 INFO:tasks.workunit.client.1.smithi196.stdout:4/524: dread d1/d3/d4c/d72/d7d/d80/fac [0,4194304] 0 2024-03-13T14:49:47.259 INFO:tasks.workunit.client.1.smithi196.stdout:6/693: chown d6/d9/d17/d19/f2b 0 1 2024-03-13T14:49:47.260 INFO:tasks.workunit.client.1.smithi196.stdout:4/525: fsync d1/d3/d4c/f59 0 2024-03-13T14:49:47.262 INFO:tasks.workunit.client.1.smithi196.stdout:7/677: symlink d0/d6/db4/dca/le3 0 2024-03-13T14:49:47.265 INFO:tasks.workunit.client.0.smithi176.stdout:3/480: rename f1 to d4/d28/d34/f9e 0 2024-03-13T14:49:47.266 INFO:tasks.workunit.client.0.smithi176.stdout:3/481: chown d4/d28/d5a 0 1 2024-03-13T14:49:47.267 INFO:tasks.workunit.client.1.smithi196.stdout:3/570: mkdir db/d19/dc1 0 2024-03-13T14:49:47.268 INFO:tasks.workunit.client.0.smithi176.stdout:3/482: write d4/d13/d16/d80/f8c [570692,49214] 0 2024-03-13T14:49:47.272 INFO:tasks.workunit.client.1.smithi196.stdout:2/465: mkdir d0/d7/d9/d42/d98 0 2024-03-13T14:49:47.274 INFO:tasks.workunit.client.1.smithi196.stdout:1/614: mkdir d1/d9/d33/dd2 0 2024-03-13T14:49:47.278 INFO:tasks.workunit.client.0.smithi176.stdout:2/477: link d1/d2a/d33/f5b d1/dc/d40/da0/fa6 0 2024-03-13T14:49:47.279 INFO:tasks.workunit.client.0.smithi176.stdout:3/483: rmdir d4/d13/d81 39 2024-03-13T14:49:47.279 INFO:tasks.workunit.client.0.smithi176.stdout:2/478: write d1/dc/f16 [4995265,113394] 0 2024-03-13T14:49:47.282 INFO:tasks.workunit.client.0.smithi176.stdout:5/425: sync 2024-03-13T14:49:47.283 INFO:tasks.workunit.client.0.smithi176.stdout:2/479: write d1/dc/f34 [837737,111409] 0 2024-03-13T14:49:47.283 INFO:tasks.workunit.client.0.smithi176.stdout:8/400: link d0/db/c66 d0/d24/c7c 0 2024-03-13T14:49:47.283 INFO:tasks.workunit.client.0.smithi176.stdout:3/484: truncate d4/d13/d16/f84 518727 0 2024-03-13T14:49:47.284 INFO:tasks.workunit.client.0.smithi176.stdout:2/480: fsync d1/dc/f16 0 2024-03-13T14:49:47.284 INFO:tasks.workunit.client.1.smithi196.stdout:4/526: fsync d1/d1e/d5c/f5f 0 2024-03-13T14:49:47.286 INFO:tasks.workunit.client.0.smithi176.stdout:5/426: chown d5/d2c/f3d 254948 1 2024-03-13T14:49:47.287 INFO:tasks.workunit.client.0.smithi176.stdout:2/481: chown d1/dc/d2d/d59/d80/d76 1386 1 2024-03-13T14:49:47.288 INFO:tasks.workunit.client.1.smithi196.stdout:3/571: unlink db/d19/d22/d3a/d68/d8e/f96 0 2024-03-13T14:49:47.290 INFO:tasks.workunit.client.1.smithi196.stdout:9/520: truncate d7/d9/f11 1377542 0 2024-03-13T14:49:47.298 INFO:tasks.workunit.client.0.smithi176.stdout:8/401: dwrite d0/db/d39/d56/f79 [0,4194304] 0 2024-03-13T14:49:47.299 INFO:tasks.workunit.client.0.smithi176.stdout:8/402: chown d0/db/d39/d56/l64 0 1 2024-03-13T14:49:47.300 INFO:tasks.workunit.client.0.smithi176.stdout:4/474: getdents d7/d8/d45 0 2024-03-13T14:49:47.301 INFO:tasks.workunit.client.0.smithi176.stdout:8/403: chown d0/d24 2 1 2024-03-13T14:49:47.302 INFO:tasks.workunit.client.0.smithi176.stdout:4/475: chown d7/d8/d33/l71 1833 1 2024-03-13T14:49:47.303 INFO:tasks.workunit.client.0.smithi176.stdout:8/404: stat d0/d5/d11/d47/f63 0 2024-03-13T14:49:47.303 INFO:tasks.workunit.client.1.smithi196.stdout:4/527: mknod d1/d3/d4c/d72/d7d/cb2 0 2024-03-13T14:49:47.304 INFO:tasks.workunit.client.1.smithi196.stdout:4/528: dread - d1/d3/f98 zero size 2024-03-13T14:49:47.306 INFO:tasks.workunit.client.0.smithi176.stdout:4/476: truncate d7/d10/d46/d55/f68 4928740 0 2024-03-13T14:49:47.309 INFO:tasks.workunit.client.0.smithi176.stdout:2/482: creat d1/d2a/d33/fa7 x:0 0 0 2024-03-13T14:49:47.311 INFO:tasks.workunit.client.0.smithi176.stdout:3/485: mknod d4/d13/d81/c9f 0 2024-03-13T14:49:47.312 INFO:tasks.workunit.client.1.smithi196.stdout:9/521: rename d7/de/d16/f2a to d7/de/d4b/d1d/d93/da7/fae 0 2024-03-13T14:49:47.314 INFO:tasks.workunit.client.0.smithi176.stdout:6/451: rename d0/d10/l1d to d0/d10/d19/d27/l94 0 2024-03-13T14:49:47.315 INFO:tasks.workunit.client.1.smithi196.stdout:1/615: truncate d1/d9/f20 5375252 0 2024-03-13T14:49:47.315 INFO:tasks.workunit.client.0.smithi176.stdout:3/486: dread d4/d13/d16/d80/f8c [0,4194304] 0 2024-03-13T14:49:47.316 INFO:tasks.workunit.client.0.smithi176.stdout:6/452: chown d0/d9/l1c 25929 1 2024-03-13T14:49:47.317 INFO:tasks.workunit.client.1.smithi196.stdout:7/678: link d0/d6/l7 d0/d9/d31/le4 0 2024-03-13T14:49:47.319 INFO:tasks.workunit.client.1.smithi196.stdout:7/679: stat d0/d9/d31/d77/d93/lcd 0 2024-03-13T14:49:47.319 INFO:tasks.workunit.client.0.smithi176.stdout:5/427: link d5/f55 d5/d48/d4c/f92 0 2024-03-13T14:49:47.320 INFO:tasks.workunit.client.1.smithi196.stdout:4/529: symlink d1/d3/d7/d35/d76/d58/d69/d6c/lb3 0 2024-03-13T14:49:47.322 INFO:tasks.workunit.client.0.smithi176.stdout:4/477: fdatasync d7/d10/f2a 0 2024-03-13T14:49:47.327 INFO:tasks.workunit.client.0.smithi176.stdout:3/487: dread d4/f78 [0,4194304] 0 2024-03-13T14:49:47.330 INFO:tasks.workunit.client.1.smithi196.stdout:1/616: dread d1/d9/d4e/d55/d63/da6/fb0 [0,4194304] 0 2024-03-13T14:49:47.331 INFO:tasks.workunit.client.1.smithi196.stdout:8/524: rmdir d0 39 2024-03-13T14:49:47.336 INFO:tasks.workunit.client.1.smithi196.stdout:7/680: creat d0/d4d/dd1/dd6/fe5 x:0 0 0 2024-03-13T14:49:47.337 INFO:tasks.workunit.client.0.smithi176.stdout:5/428: dwrite d5/d2c/f3d [0,4194304] 0 2024-03-13T14:49:47.339 INFO:tasks.workunit.client.1.smithi196.stdout:4/530: mknod d1/d3/d7/d35/d3e/cb4 0 2024-03-13T14:49:47.339 INFO:tasks.workunit.client.1.smithi196.stdout:7/681: chown d0/d9/d1e/d58/d72/f84 214 1 2024-03-13T14:49:47.339 INFO:tasks.workunit.client.1.smithi196.stdout:0/602: write d8/d37/f73 [1490702,44810] 0 2024-03-13T14:49:47.340 INFO:tasks.workunit.client.1.smithi196.stdout:7/682: dread - d0/f6b zero size 2024-03-13T14:49:47.343 INFO:tasks.workunit.client.1.smithi196.stdout:2/466: getdents d0/d6/d34 0 2024-03-13T14:49:47.344 INFO:tasks.workunit.client.0.smithi176.stdout:2/483: symlink d1/dc/d40/da0/la8 0 2024-03-13T14:49:47.344 INFO:tasks.workunit.client.0.smithi176.stdout:7/420: truncate d1/d28/d3a/f5c 3360675 0 2024-03-13T14:49:47.344 INFO:tasks.workunit.client.0.smithi176.stdout:7/421: stat d1/d3f/d4b/f58 0 2024-03-13T14:49:47.344 INFO:tasks.workunit.client.0.smithi176.stdout:2/484: rename d1/dc/d61 to d1/dc/d61/d9e/da9 22 2024-03-13T14:49:47.345 INFO:tasks.workunit.client.0.smithi176.stdout:3/488: dwrite d4/d13/d5e/f9a [0,4194304] 0 2024-03-13T14:49:47.346 INFO:tasks.workunit.client.1.smithi196.stdout:9/522: rename d7/de/f50 to d7/d71/faf 0 2024-03-13T14:49:47.349 INFO:tasks.workunit.client.1.smithi196.stdout:1/617: creat d1/d3f/d5b/d74/fd3 x:0 0 0 2024-03-13T14:49:47.350 INFO:tasks.workunit.client.0.smithi176.stdout:8/405: dread d0/db/fd [0,4194304] 0 2024-03-13T14:49:47.357 INFO:tasks.workunit.client.0.smithi176.stdout:6/453: creat d0/de/d8d/f95 x:0 0 0 2024-03-13T14:49:47.358 INFO:tasks.workunit.client.0.smithi176.stdout:6/454: read - d0/de/d38/d77/f82 zero size 2024-03-13T14:49:47.363 INFO:tasks.workunit.client.1.smithi196.stdout:2/467: dwrite d0/d6/f84 [0,4194304] 0 2024-03-13T14:49:47.368 INFO:tasks.workunit.client.1.smithi196.stdout:4/531: rename d1/d3/d7/d35/d76/f45 to d1/d1e/d5c/fb5 0 2024-03-13T14:49:47.370 INFO:tasks.workunit.client.1.smithi196.stdout:4/532: fdatasync d1/d3/d7/f1a 0 2024-03-13T14:49:47.371 INFO:tasks.workunit.client.1.smithi196.stdout:0/603: symlink d8/da/d25/d26/d54/lc2 0 2024-03-13T14:49:47.372 INFO:tasks.workunit.client.1.smithi196.stdout:1/618: dwrite d1/d9/d6c/fc1 [0,4194304] 0 2024-03-13T14:49:47.375 INFO:tasks.workunit.client.0.smithi176.stdout:4/478: truncate d7/d8/d15/f2b 1084574 0 2024-03-13T14:49:47.378 INFO:tasks.workunit.client.1.smithi196.stdout:5/648: write d4/db/f66 [420287,23760] 0 2024-03-13T14:49:47.381 INFO:tasks.workunit.client.1.smithi196.stdout:5/649: chown d4/db/d3c/d6a/d91 13 1 2024-03-13T14:49:47.383 INFO:tasks.workunit.client.0.smithi176.stdout:2/485: mkdir d1/dc/d61/daa 0 2024-03-13T14:49:47.385 INFO:tasks.workunit.client.0.smithi176.stdout:3/489: creat d4/d13/d5e/fa0 x:0 0 0 2024-03-13T14:49:47.386 INFO:tasks.workunit.client.1.smithi196.stdout:2/468: dread d0/d7/f94 [0,4194304] 0 2024-03-13T14:49:47.387 INFO:tasks.workunit.client.0.smithi176.stdout:3/490: write d4/d28/d34/f3f [923505,1290] 0 2024-03-13T14:49:47.388 INFO:tasks.workunit.client.1.smithi196.stdout:9/523: rename d7/de/d4b/d1d/d22/d4d/f58 to d7/d9/d83/fb0 0 2024-03-13T14:49:47.389 INFO:tasks.workunit.client.0.smithi176.stdout:3/491: dread - d4/d28/d34/f6e zero size 2024-03-13T14:49:47.390 INFO:tasks.workunit.client.0.smithi176.stdout:6/455: rmdir d0/d10/d19/d27/d2b 39 2024-03-13T14:49:47.397 INFO:tasks.workunit.client.1.smithi196.stdout:8/525: symlink d0/d16/d71/d9a/laf 0 2024-03-13T14:49:47.400 INFO:tasks.workunit.client.1.smithi196.stdout:4/533: creat d1/d3/d7/d35/d3e/d56/fb6 x:0 0 0 2024-03-13T14:49:47.404 INFO:tasks.workunit.client.1.smithi196.stdout:2/469: dwrite d0/d6/d1d/f51 [0,4194304] 0 2024-03-13T14:49:47.404 INFO:tasks.workunit.client.1.smithi196.stdout:6/694: dwrite d6/d9/d17/d19/d4e/f8a [0,4194304] 0 2024-03-13T14:49:47.407 INFO:tasks.workunit.client.0.smithi176.stdout:7/422: mkdir d1/d28/d7c 0 2024-03-13T14:49:47.408 INFO:tasks.workunit.client.1.smithi196.stdout:0/604: mkdir d8/da/d25/d26/d35/d3b/d5f/d17/dc3 0 2024-03-13T14:49:47.408 INFO:tasks.workunit.client.1.smithi196.stdout:1/619: mkdir d1/d3f/d5b/d74/dd4 0 2024-03-13T14:49:47.412 INFO:tasks.workunit.client.0.smithi176.stdout:4/479: unlink d7/d10/d2c/c60 0 2024-03-13T14:49:47.413 INFO:tasks.workunit.client.0.smithi176.stdout:4/480: readlink d7/d10/d2c/l7e 0 2024-03-13T14:49:47.416 INFO:tasks.workunit.client.0.smithi176.stdout:9/511: sync 2024-03-13T14:49:47.418 INFO:tasks.workunit.client.0.smithi176.stdout:9/512: write d0/d17/f3a [1268866,81513] 0 2024-03-13T14:49:47.420 INFO:tasks.workunit.client.0.smithi176.stdout:3/492: rename d4/f7e to d4/d13/d83/fa1 0 2024-03-13T14:49:47.421 INFO:tasks.workunit.client.1.smithi196.stdout:5/650: dread d4/db/d14/d19/d1c/d25/d38/d47/da0/f5c [0,4194304] 0 2024-03-13T14:49:47.424 INFO:tasks.workunit.client.0.smithi176.stdout:7/423: rmdir d1/d1c/d26/d41/d2b/d40 39 2024-03-13T14:49:47.426 INFO:tasks.workunit.client.1.smithi196.stdout:9/524: write d7/de/d4b/f43 [952843,97420] 0 2024-03-13T14:49:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:47 smithi196 ceph-mon[37486]: pgmap v12: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 52 MiB/s rd, 213 MiB/s wr, 406 op/s 2024-03-13T14:49:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:47 smithi196 ceph-mon[37486]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:47.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:47 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:47.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:47 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:47.429 INFO:tasks.workunit.client.0.smithi176.stdout:8/406: creat d0/d24/f7d x:0 0 0 2024-03-13T14:49:47.429 INFO:tasks.workunit.client.1.smithi196.stdout:3/572: dwrite db/f2c [0,4194304] 0 2024-03-13T14:49:47.436 INFO:tasks.workunit.client.1.smithi196.stdout:2/470: rename d0/f32 to d0/d7/d9/d42/d98/f99 0 2024-03-13T14:49:47.437 INFO:tasks.workunit.client.0.smithi176.stdout:2/486: symlink d1/dc/d2d/d59/d80/d7e/da3/lab 0 2024-03-13T14:49:47.439 INFO:tasks.workunit.client.1.smithi196.stdout:7/683: getdents d0/d9/d31/d77 0 2024-03-13T14:49:47.442 INFO:tasks.workunit.client.1.smithi196.stdout:8/526: dread d0/d16/d3d/d8b/f38 [0,4194304] 0 2024-03-13T14:49:47.443 INFO:tasks.workunit.client.1.smithi196.stdout:7/684: write d0/d9/d1e/d58/d72/f66 [1602058,98114] 0 2024-03-13T14:49:47.443 INFO:tasks.workunit.client.0.smithi176.stdout:6/456: symlink d0/de/d38/d77/d76/d7c/l96 0 2024-03-13T14:49:47.447 INFO:tasks.workunit.client.1.smithi196.stdout:0/605: symlink d8/da/d25/d26/d35/d3b/d50/d5d/da5/lc4 0 2024-03-13T14:49:47.448 INFO:tasks.workunit.client.0.smithi176.stdout:7/424: chown d1/d1c/d26/d41/d2b/d40/d64 4 1 2024-03-13T14:49:47.449 INFO:tasks.workunit.client.1.smithi196.stdout:9/525: symlink d7/de/d4b/d1d/d22/d4d/lb1 0 2024-03-13T14:49:47.450 INFO:tasks.workunit.client.0.smithi176.stdout:7/425: write d1/f3c [2593707,124994] 0 2024-03-13T14:49:47.453 INFO:tasks.workunit.client.1.smithi196.stdout:3/573: dread - db/d37/d64/f57 zero size 2024-03-13T14:49:47.457 INFO:tasks.workunit.client.1.smithi196.stdout:3/574: write db/d4c/f61 [1650360,822] 0 2024-03-13T14:49:47.459 INFO:tasks.workunit.client.0.smithi176.stdout:6/457: chown d0/d10/d19/d27/d2b/d2e/f62 62533 1 2024-03-13T14:49:47.461 INFO:tasks.workunit.client.1.smithi196.stdout:3/575: write db/d71/fa5 [576687,129819] 0 2024-03-13T14:49:47.464 INFO:tasks.workunit.client.0.smithi176.stdout:5/429: write d5/d2b/f46 [818308,118599] 0 2024-03-13T14:49:47.472 INFO:tasks.workunit.client.0.smithi176.stdout:8/407: mknod d0/c7e 0 2024-03-13T14:49:47.473 INFO:tasks.workunit.client.0.smithi176.stdout:6/458: dread d0/d10/d19/d27/f61 [0,4194304] 0 2024-03-13T14:49:47.474 INFO:tasks.workunit.client.0.smithi176.stdout:8/408: read d0/d24/f57 [213988,14606] 0 2024-03-13T14:49:47.475 INFO:tasks.workunit.client.1.smithi196.stdout:5/651: rename d4/db to d4/d1b/dc8 0 2024-03-13T14:49:47.476 INFO:tasks.workunit.client.1.smithi196.stdout:5/652: chown d4/l6c 13716624 1 2024-03-13T14:49:47.477 INFO:tasks.workunit.client.1.smithi196.stdout:1/620: creat d1/d3f/d5b/d74/db5/fd5 x:0 0 0 2024-03-13T14:49:47.479 INFO:tasks.workunit.client.1.smithi196.stdout:5/653: write d4/d1b/dc8/d4a/faf [1275173,52110] 0 2024-03-13T14:49:47.480 INFO:tasks.workunit.client.0.smithi176.stdout:4/481: rmdir d7/da9 0 2024-03-13T14:49:47.481 INFO:tasks.workunit.client.1.smithi196.stdout:9/526: dread d7/de/d4b/f19 [0,4194304] 0 2024-03-13T14:49:47.482 INFO:tasks.workunit.client.1.smithi196.stdout:5/654: chown d4/d1b/dc8/d14/d19/l20 6998053 1 2024-03-13T14:49:47.486 INFO:tasks.workunit.client.0.smithi176.stdout:3/493: link d4/f78 d4/d28/fa2 0 2024-03-13T14:49:47.487 INFO:tasks.workunit.client.1.smithi196.stdout:8/527: symlink d0/d16/d71/lb0 0 2024-03-13T14:49:47.487 INFO:tasks.workunit.client.0.smithi176.stdout:3/494: readlink d4/l15 0 2024-03-13T14:49:47.491 INFO:tasks.workunit.client.1.smithi196.stdout:7/685: unlink d0/d9/d1e/la4 0 2024-03-13T14:49:47.492 INFO:tasks.workunit.client.0.smithi176.stdout:6/459: dread d0/f69 [0,4194304] 0 2024-03-13T14:49:47.493 INFO:tasks.workunit.client.0.smithi176.stdout:2/487: chown d1/f3c 0 1 2024-03-13T14:49:47.494 INFO:tasks.workunit.client.1.smithi196.stdout:4/534: creat d1/d1e/fb7 x:0 0 0 2024-03-13T14:49:47.494 INFO:tasks.workunit.client.0.smithi176.stdout:2/488: chown d1/dc/f16 15921 1 2024-03-13T14:49:47.495 INFO:tasks.workunit.client.1.smithi196.stdout:1/621: dwrite d1/d9/d33/d4b/f1b [4194304,4194304] 0 2024-03-13T14:49:47.495 INFO:tasks.workunit.client.0.smithi176.stdout:4/482: dwrite d7/d8/d15/d23/f47 [0,4194304] 0 2024-03-13T14:49:47.495 INFO:tasks.workunit.client.0.smithi176.stdout:2/489: fsync d1/dc/f34 0 2024-03-13T14:49:47.505 INFO:tasks.workunit.client.1.smithi196.stdout:0/606: rename d8/da/d25/d26/d35/d3b/f47 to d8/da/d25/d26/d54/d88/fc5 0 2024-03-13T14:49:47.515 INFO:tasks.workunit.client.1.smithi196.stdout:5/655: dread - d4/d1b/dc8/d14/d19/d26/f92 zero size 2024-03-13T14:49:47.517 INFO:tasks.workunit.client.1.smithi196.stdout:5/656: fdatasync d4/d1b/dc8/d14/d19/d26/f43 0 2024-03-13T14:49:47.518 INFO:tasks.workunit.client.1.smithi196.stdout:8/528: mkdir d0/d10/d12/d1e/db1 0 2024-03-13T14:49:47.522 INFO:tasks.workunit.client.0.smithi176.stdout:5/430: unlink d5/d22/d15/f27 0 2024-03-13T14:49:47.524 INFO:tasks.workunit.client.1.smithi196.stdout:7/686: fsync d0/d6/d33/d57/f95 0 2024-03-13T14:49:47.526 INFO:tasks.workunit.client.1.smithi196.stdout:4/535: creat d1/d3/d4c/fb8 x:0 0 0 2024-03-13T14:49:47.528 INFO:tasks.workunit.client.1.smithi196.stdout:6/695: getdents d6/d7/dd/dbc 0 2024-03-13T14:49:47.535 INFO:tasks.workunit.client.1.smithi196.stdout:2/471: fdatasync d0/d6/d34/f62 0 2024-03-13T14:49:47.536 INFO:tasks.workunit.client.1.smithi196.stdout:2/472: chown d0/d6/d4b 0 1 2024-03-13T14:49:47.537 INFO:tasks.workunit.client.1.smithi196.stdout:0/607: creat d8/da/d25/d26/d35/d3b/d6c/fc6 x:0 0 0 2024-03-13T14:49:47.539 INFO:tasks.workunit.client.0.smithi176.stdout:0/407: sync 2024-03-13T14:49:47.539 INFO:tasks.workunit.client.0.smithi176.stdout:1/423: sync 2024-03-13T14:49:47.542 INFO:tasks.workunit.client.0.smithi176.stdout:6/460: rename d0/de/d38/d77/f56 to d0/de/d38/d70/f97 0 2024-03-13T14:49:47.545 INFO:tasks.workunit.client.0.smithi176.stdout:4/483: creat d7/d10/d46/faf x:0 0 0 2024-03-13T14:49:47.547 INFO:tasks.workunit.client.0.smithi176.stdout:2/490: creat d1/dc/d2d/d4f/fac x:0 0 0 2024-03-13T14:49:47.547 INFO:tasks.workunit.client.1.smithi196.stdout:7/687: mknod d0/d6/db4/ce6 0 2024-03-13T14:49:47.549 INFO:tasks.workunit.client.1.smithi196.stdout:6/696: creat d6/d7/df/fdd x:0 0 0 2024-03-13T14:49:47.552 INFO:tasks.workunit.client.0.smithi176.stdout:0/408: dwrite d2/f69 [0,4194304] 0 2024-03-13T14:49:47.553 INFO:tasks.workunit.client.1.smithi196.stdout:6/697: write d6/d7/df/d27/f64 [304728,72851] 0 2024-03-13T14:49:47.554 INFO:tasks.workunit.client.0.smithi176.stdout:5/431: symlink d5/d31/l93 0 2024-03-13T14:49:47.555 INFO:tasks.workunit.client.0.smithi176.stdout:5/432: chown d5/f24 7908891 1 2024-03-13T14:49:47.555 INFO:tasks.workunit.client.1.smithi196.stdout:6/698: chown l5 235888 1 2024-03-13T14:49:47.557 INFO:tasks.workunit.client.1.smithi196.stdout:6/699: write d6/d2a/d23/fb6 [11952,118876] 0 2024-03-13T14:49:47.558 INFO:tasks.workunit.client.0.smithi176.stdout:8/409: write d0/db/f25 [1653063,100109] 0 2024-03-13T14:49:47.565 INFO:tasks.workunit.client.1.smithi196.stdout:0/608: rmdir d8/da/d25/d26/d54/d88 39 2024-03-13T14:49:47.566 INFO:tasks.workunit.client.0.smithi176.stdout:9/513: sync 2024-03-13T14:49:47.568 INFO:tasks.workunit.client.1.smithi196.stdout:8/529: mkdir d0/d10/da5/db2 0 2024-03-13T14:49:47.571 INFO:tasks.workunit.client.1.smithi196.stdout:5/657: symlink d4/d1b/dc8/d14/d19/d1c/d2d/dc5/lc9 0 2024-03-13T14:49:47.575 INFO:tasks.workunit.client.1.smithi196.stdout:9/527: dwrite d7/de/d74/d94/fa8 [0,4194304] 0 2024-03-13T14:49:47.576 INFO:tasks.workunit.client.1.smithi196.stdout:7/688: rename d0/d6/d33/d57/d8a/dd2 to d0/d9/d31/d77/dcf/de0/de7 0 2024-03-13T14:49:47.578 INFO:tasks.workunit.client.1.smithi196.stdout:9/528: write d7/d9/d17/f9c [963011,20582] 0 2024-03-13T14:49:47.578 INFO:tasks.workunit.client.0.smithi176.stdout:6/461: chown d0/de/d38/d6b/l71 0 1 2024-03-13T14:49:47.581 INFO:tasks.workunit.client.1.smithi196.stdout:9/529: write d7/d24/f98 [628874,30114] 0 2024-03-13T14:49:47.588 INFO:tasks.workunit.client.0.smithi176.stdout:1/424: dwrite d2/d30/d63/d22/d71/f87 [0,4194304] 0 2024-03-13T14:49:47.589 INFO:tasks.workunit.client.1.smithi196.stdout:1/622: creat d1/d9/d33/d4b/fd6 x:0 0 0 2024-03-13T14:49:47.590 INFO:tasks.workunit.client.0.smithi176.stdout:3/495: truncate d4/d28/d34/f9e 3935049 0 2024-03-13T14:49:47.591 INFO:tasks.workunit.client.0.smithi176.stdout:2/491: creat d1/dc/d2d/d59/d8c/fad x:0 0 0 2024-03-13T14:49:47.591 INFO:tasks.workunit.client.0.smithi176.stdout:3/496: chown d4/d13/d2d 352 1 2024-03-13T14:49:47.593 INFO:tasks.workunit.client.1.smithi196.stdout:8/530: dwrite d0/d16/d3d/d8b/d3f/fad [0,4194304] 0 2024-03-13T14:49:47.594 INFO:tasks.workunit.client.0.smithi176.stdout:3/497: fsync d4/d28/d34/d38/f95 0 2024-03-13T14:49:47.595 INFO:tasks.workunit.client.0.smithi176.stdout:0/409: symlink d2/d74/d61/l7b 0 2024-03-13T14:49:47.597 INFO:tasks.workunit.client.1.smithi196.stdout:6/700: creat d6/d7/df/d27/d39/fde x:0 0 0 2024-03-13T14:49:47.598 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:47 smithi176 ceph-mon[29064]: pgmap v12: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 52 MiB/s rd, 213 MiB/s wr, 406 op/s 2024-03-13T14:49:47.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:47 smithi176 ceph-mon[29064]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:49:47.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:47 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:47.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:47 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:47.599 INFO:tasks.workunit.client.0.smithi176.stdout:4/484: write d7/d8/f42 [268202,42495] 0 2024-03-13T14:49:47.601 INFO:tasks.workunit.client.1.smithi196.stdout:6/701: chown d6/d7/dd/d60 1 1 2024-03-13T14:49:47.603 INFO:tasks.workunit.client.1.smithi196.stdout:2/473: write d0/d7/d9/d42/d98/f99 [1605651,111855] 0 2024-03-13T14:49:47.606 INFO:tasks.workunit.client.1.smithi196.stdout:6/702: readlink d6/d57/d91/lce 0 2024-03-13T14:49:47.607 INFO:tasks.workunit.client.0.smithi176.stdout:8/410: creat d0/d5/d11/f7f x:0 0 0 2024-03-13T14:49:47.609 INFO:tasks.workunit.client.0.smithi176.stdout:7/426: link d1/d1c/c78 d1/d1c/d26/d41/d2b/d40/c7d 0 2024-03-13T14:49:47.610 INFO:tasks.workunit.client.1.smithi196.stdout:1/623: dread d1/f2a [0,4194304] 0 2024-03-13T14:49:47.613 INFO:tasks.workunit.client.1.smithi196.stdout:0/609: dread d8/da/d25/d26/d48/d82/f8b [0,4194304] 0 2024-03-13T14:49:47.613 INFO:tasks.workunit.client.1.smithi196.stdout:1/624: chown d1/d3f/d5b/d74/f8f 236 1 2024-03-13T14:49:47.615 INFO:tasks.workunit.client.1.smithi196.stdout:1/625: readlink d1/d28/d34/d67/l6b 0 2024-03-13T14:49:47.615 INFO:tasks.workunit.client.0.smithi176.stdout:6/462: creat d0/de/d38/d77/f98 x:0 0 0 2024-03-13T14:49:47.618 INFO:tasks.workunit.client.1.smithi196.stdout:8/531: dwrite d0/d16/d3d/d8b/d3f/fad [0,4194304] 0 2024-03-13T14:49:47.620 INFO:tasks.workunit.client.1.smithi196.stdout:8/532: dread - d0/d1c/d29/d47/f98 zero size 2024-03-13T14:49:47.623 INFO:tasks.workunit.client.0.smithi176.stdout:1/425: symlink d2/de/d10/d16/d4a/l88 0 2024-03-13T14:49:47.626 INFO:tasks.workunit.client.0.smithi176.stdout:2/492: unlink d1/dc/d2d/d59/d80/d5c/d78/ca1 0 2024-03-13T14:49:47.628 INFO:tasks.workunit.client.1.smithi196.stdout:7/689: mkdir d0/d6/d33/d57/dae/de8 0 2024-03-13T14:49:47.630 INFO:tasks.workunit.client.0.smithi176.stdout:3/498: mknod d4/d28/d31/d52/ca3 0 2024-03-13T14:49:47.630 INFO:tasks.workunit.client.0.smithi176.stdout:6/463: dread d0/de/d38/f8b [0,4194304] 0 2024-03-13T14:49:47.630 INFO:tasks.workunit.client.0.smithi176.stdout:1/426: chown d2/c20 21142 1 2024-03-13T14:49:47.630 INFO:tasks.workunit.client.1.smithi196.stdout:4/536: link d1/d3/d7/d35/d76/d58/d69/d6c/f7e d1/d3/d7/d35/d76/fb9 0 2024-03-13T14:49:47.632 INFO:tasks.workunit.client.0.smithi176.stdout:8/411: mkdir d0/db/d53/d80 0 2024-03-13T14:49:47.635 INFO:tasks.workunit.client.0.smithi176.stdout:1/427: fsync d2/d30/d63/d22/d71/f87 0 2024-03-13T14:49:47.635 INFO:tasks.workunit.client.1.smithi196.stdout:8/533: creat d0/d16/d3d/d7f/fb3 x:0 0 0 2024-03-13T14:49:47.637 INFO:tasks.workunit.client.1.smithi196.stdout:2/474: mkdir d0/d7/d9/d9a 0 2024-03-13T14:49:47.639 INFO:tasks.workunit.client.0.smithi176.stdout:5/433: dwrite d5/d48/f4e [0,4194304] 0 2024-03-13T14:49:47.643 INFO:tasks.workunit.client.1.smithi196.stdout:7/690: truncate d0/d6/d33/d57/f95 580230 0 2024-03-13T14:49:47.644 INFO:tasks.workunit.client.0.smithi176.stdout:3/499: mkdir d4/d28/d34/da4 0 2024-03-13T14:49:47.644 INFO:tasks.workunit.client.0.smithi176.stdout:4/485: mknod d7/d8/cb0 0 2024-03-13T14:49:47.645 INFO:tasks.workunit.client.0.smithi176.stdout:9/514: rename d0/f5e to d0/db/fae 0 2024-03-13T14:49:47.650 INFO:tasks.workunit.client.1.smithi196.stdout:8/534: rmdir d0/d10/d79 39 2024-03-13T14:49:47.650 INFO:tasks.workunit.client.1.smithi196.stdout:1/626: dread d1/d38/fb2 [0,4194304] 0 2024-03-13T14:49:47.650 INFO:tasks.workunit.client.1.smithi196.stdout:1/627: stat d1/d28/l9f 0 2024-03-13T14:49:47.650 INFO:tasks.workunit.client.1.smithi196.stdout:6/703: creat d6/d9/d17/d19/d4e/fdf x:0 0 0 2024-03-13T14:49:47.650 INFO:tasks.workunit.client.1.smithi196.stdout:2/475: dread d0/d6/d4b/f79 [0,4194304] 0 2024-03-13T14:49:47.651 INFO:tasks.workunit.client.0.smithi176.stdout:9/515: chown d0/d1e/d1f/f2c 60240121 1 2024-03-13T14:49:47.655 INFO:tasks.workunit.client.0.smithi176.stdout:7/427: truncate d1/d3f/d4b/f6b 2961562 0 2024-03-13T14:49:47.655 INFO:tasks.workunit.client.1.smithi196.stdout:7/691: symlink d0/d9/d1e/d58/d61/da0/le9 0 2024-03-13T14:49:47.657 INFO:tasks.workunit.client.0.smithi176.stdout:8/412: creat d0/d5/d3b/f81 x:0 0 0 2024-03-13T14:49:47.657 INFO:tasks.workunit.client.0.smithi176.stdout:7/428: truncate d1/f61 1162271 0 2024-03-13T14:49:47.661 INFO:tasks.workunit.client.1.smithi196.stdout:7/692: truncate d0/d6/d33/f39 8642303 0 2024-03-13T14:49:47.661 INFO:tasks.workunit.client.1.smithi196.stdout:0/610: getdents d8/da/d25/d26/d35/d3b/d50/d5d/d7e 0 2024-03-13T14:49:47.664 INFO:tasks.workunit.client.0.smithi176.stdout:5/434: rmdir d5/d2c/d74 39 2024-03-13T14:49:47.668 INFO:tasks.workunit.client.0.smithi176.stdout:5/435: dread - d5/d2c/d40/d6a/f79 zero size 2024-03-13T14:49:47.672 INFO:tasks.workunit.client.1.smithi196.stdout:1/628: dwrite d1/d38/fc6 [0,4194304] 0 2024-03-13T14:49:47.672 INFO:tasks.workunit.client.1.smithi196.stdout:6/704: creat d6/d57/d91/fe0 x:0 0 0 2024-03-13T14:49:47.674 INFO:tasks.workunit.client.0.smithi176.stdout:3/500: creat d4/d13/d83/fa5 x:0 0 0 2024-03-13T14:49:47.676 INFO:tasks.workunit.client.0.smithi176.stdout:8/413: dwrite d0/d5/f17 [0,4194304] 0 2024-03-13T14:49:47.687 INFO:tasks.workunit.client.1.smithi196.stdout:4/537: dread d1/d3/d7/d35/d76/f32 [0,4194304] 0 2024-03-13T14:49:47.687 INFO:tasks.workunit.client.1.smithi196.stdout:4/538: stat d1/d3/d4c/d72 0 2024-03-13T14:49:47.690 INFO:tasks.workunit.client.1.smithi196.stdout:1/629: dwrite d1/d28/f32 [0,4194304] 0 2024-03-13T14:49:47.703 INFO:tasks.workunit.client.0.smithi176.stdout:3/501: dwrite d4/d28/d34/d38/f95 [0,4194304] 0 2024-03-13T14:49:47.704 INFO:tasks.workunit.client.0.smithi176.stdout:8/414: dwrite d0/d24/d29/f67 [0,4194304] 0 2024-03-13T14:49:47.705 INFO:tasks.workunit.client.0.smithi176.stdout:8/415: dread - d0/f5c zero size 2024-03-13T14:49:47.712 INFO:tasks.workunit.client.1.smithi196.stdout:6/705: dread d6/d57/f63 [0,4194304] 0 2024-03-13T14:49:47.720 INFO:tasks.workunit.client.0.smithi176.stdout:8/416: dwrite d0/d24/d29/f4e [4194304,4194304] 0 2024-03-13T14:49:47.731 INFO:tasks.workunit.client.0.smithi176.stdout:4/486: mknod d7/d8/d33/d96/cb1 0 2024-03-13T14:49:47.732 INFO:tasks.workunit.client.0.smithi176.stdout:2/493: write d1/f31 [1915904,73364] 0 2024-03-13T14:49:47.734 INFO:tasks.workunit.client.1.smithi196.stdout:0/611: chown d8/da/d25/d26/d54/d88/c80 17 1 2024-03-13T14:49:47.746 INFO:tasks.workunit.client.0.smithi176.stdout:8/417: fdatasync d0/d24/d29/f4e 0 2024-03-13T14:49:47.749 INFO:tasks.workunit.client.0.smithi176.stdout:8/418: write d0/d24/d29/f67 [969899,13497] 0 2024-03-13T14:49:47.752 INFO:tasks.workunit.client.0.smithi176.stdout:8/419: fdatasync d0/f5c 0 2024-03-13T14:49:47.753 INFO:tasks.workunit.client.0.smithi176.stdout:9/516: dwrite d0/d1e/f2e [4194304,4194304] 0 2024-03-13T14:49:47.753 INFO:tasks.workunit.client.0.smithi176.stdout:3/502: rename d4/d13/d83/l96 to d4/d13/d16/d80/la6 0 2024-03-13T14:49:47.755 INFO:tasks.workunit.client.0.smithi176.stdout:9/517: write d0/d40/d53/fad [527502,92094] 0 2024-03-13T14:49:47.757 INFO:tasks.workunit.client.0.smithi176.stdout:9/518: chown d0/db/d89/d32/d76 10687779 1 2024-03-13T14:49:47.758 INFO:tasks.workunit.client.0.smithi176.stdout:0/410: rename d2/d5/dc/l2b to d2/d5/dc/l7c 0 2024-03-13T14:49:47.759 INFO:tasks.workunit.client.0.smithi176.stdout:0/411: chown d2/db/c27 1502674 1 2024-03-13T14:49:47.764 INFO:tasks.workunit.client.0.smithi176.stdout:5/436: creat d5/d2e/d43/f94 x:0 0 0 2024-03-13T14:49:47.765 INFO:tasks.workunit.client.0.smithi176.stdout:6/464: write d0/d9/f55 [1862752,67036] 0 2024-03-13T14:49:47.773 INFO:tasks.workunit.client.0.smithi176.stdout:8/420: mkdir d0/d24/d29/d82 0 2024-03-13T14:49:47.775 INFO:tasks.workunit.client.0.smithi176.stdout:7/429: write d1/d3f/d4b/f6b [3468451,67953] 0 2024-03-13T14:49:47.776 INFO:tasks.workunit.client.1.smithi196.stdout:7/693: creat d0/d9/d1e/d58/d72/da9/da3/ddf/fea x:0 0 0 2024-03-13T14:49:47.778 INFO:tasks.workunit.client.0.smithi176.stdout:9/519: creat d0/d1e/d1f/d68/faf x:0 0 0 2024-03-13T14:49:47.780 INFO:tasks.workunit.client.1.smithi196.stdout:0/612: symlink d8/da/d25/d26/d35/d3b/d5f/d17/lc7 0 2024-03-13T14:49:47.781 INFO:tasks.workunit.client.0.smithi176.stdout:1/428: getdents d2/de/d10/d16/d40 0 2024-03-13T14:49:47.783 INFO:tasks.workunit.client.1.smithi196.stdout:8/535: link d0/d10/f21 d0/d10/d12/fb4 0 2024-03-13T14:49:47.785 INFO:tasks.workunit.client.0.smithi176.stdout:6/465: rmdir d0/de/d38 39 2024-03-13T14:49:47.786 INFO:tasks.workunit.client.0.smithi176.stdout:6/466: chown d0/de/d54 135402 1 2024-03-13T14:49:47.788 INFO:tasks.workunit.client.1.smithi196.stdout:1/630: creat d1/d3f/d5b/d74/dd4/fd7 x:0 0 0 2024-03-13T14:49:47.789 INFO:tasks.workunit.client.1.smithi196.stdout:1/631: dread - d1/d9/d4e/f70 zero size 2024-03-13T14:49:47.793 INFO:tasks.workunit.client.1.smithi196.stdout:0/613: dread d8/da/d25/d26/d35/d3b/d50/f6d [0,4194304] 0 2024-03-13T14:49:47.794 INFO:tasks.workunit.client.0.smithi176.stdout:2/494: creat d1/dc/fae x:0 0 0 2024-03-13T14:49:47.795 INFO:tasks.workunit.client.0.smithi176.stdout:2/495: chown d1/d2a/f75 58607409 1 2024-03-13T14:49:47.796 INFO:tasks.workunit.client.0.smithi176.stdout:2/496: readlink d1/l3e 0 2024-03-13T14:49:47.798 INFO:tasks.workunit.client.0.smithi176.stdout:0/412: mknod d2/d74/c7d 0 2024-03-13T14:49:47.799 INFO:tasks.workunit.client.0.smithi176.stdout:2/497: fdatasync d1/d2a/d33/f9d 0 2024-03-13T14:49:47.800 INFO:tasks.workunit.client.0.smithi176.stdout:7/430: dread d1/d3f/d4b/f58 [0,4194304] 0 2024-03-13T14:49:47.801 INFO:tasks.workunit.client.0.smithi176.stdout:6/467: fsync d0/de/d38/d7e/f84 0 2024-03-13T14:49:47.802 INFO:tasks.workunit.client.1.smithi196.stdout:8/536: rmdir d0/d16/d3d/d8b 39 2024-03-13T14:49:47.803 INFO:tasks.workunit.client.0.smithi176.stdout:2/498: truncate d1/dc/f34 4305701 0 2024-03-13T14:49:47.804 INFO:tasks.workunit.client.1.smithi196.stdout:4/539: creat d1/d3/d7/d35/d76/fba x:0 0 0 2024-03-13T14:49:47.805 INFO:tasks.workunit.client.1.smithi196.stdout:6/706: creat d6/d7/df/d27/fe1 x:0 0 0 2024-03-13T14:49:47.807 INFO:tasks.workunit.client.1.smithi196.stdout:1/632: symlink d1/d3f/d5b/d74/ld8 0 2024-03-13T14:49:47.808 INFO:tasks.workunit.client.0.smithi176.stdout:9/520: creat d0/db/d89/d90/fb0 x:0 0 0 2024-03-13T14:49:47.809 INFO:tasks.workunit.client.1.smithi196.stdout:2/476: getdents d0/d7/d56 0 2024-03-13T14:49:47.810 INFO:tasks.workunit.client.1.smithi196.stdout:1/633: fsync d1/d3f/d5b/faf 0 2024-03-13T14:49:47.811 INFO:tasks.workunit.client.0.smithi176.stdout:1/429: dread d2/de/d3f/d19/f32 [0,4194304] 0 2024-03-13T14:49:47.813 INFO:tasks.workunit.client.1.smithi196.stdout:2/477: chown d0/d7/d40/f46 3 1 2024-03-13T14:49:47.814 INFO:tasks.workunit.client.0.smithi176.stdout:5/437: dwrite d5/d22/d15/f67 [4194304,4194304] 0 2024-03-13T14:49:47.816 INFO:tasks.workunit.client.1.smithi196.stdout:4/540: mkdir d1/d3/d7/d35/d76/d58/d69/d9f/dbb 0 2024-03-13T14:49:47.816 INFO:tasks.workunit.client.0.smithi176.stdout:3/503: getdents d4/d13 0 2024-03-13T14:49:47.817 INFO:tasks.workunit.client.1.smithi196.stdout:0/614: dwrite d8/da/d25/d26/d35/d3b/d50/f6d [0,4194304] 0 2024-03-13T14:49:47.819 INFO:tasks.workunit.client.1.smithi196.stdout:7/694: dwrite d0/d9/d1e/f51 [0,4194304] 0 2024-03-13T14:49:47.820 INFO:tasks.workunit.client.0.smithi176.stdout:2/499: write d1/dc/d2d/f7a [2126113,96617] 0 2024-03-13T14:49:47.824 INFO:tasks.workunit.client.0.smithi176.stdout:8/421: dwrite d0/db/dc/f13 [0,4194304] 0 2024-03-13T14:49:47.826 INFO:tasks.workunit.client.0.smithi176.stdout:6/468: dwrite d0/d10/d19/d27/f4a [0,4194304] 0 2024-03-13T14:49:47.830 INFO:tasks.workunit.client.1.smithi196.stdout:2/478: truncate d0/d7/d9/d5a/f6f 577962 0 2024-03-13T14:49:47.831 INFO:tasks.workunit.client.1.smithi196.stdout:0/615: mknod d8/da/d25/d26/d35/d3b/d5f/d33/cc8 0 2024-03-13T14:49:47.831 INFO:tasks.workunit.client.0.smithi176.stdout:9/521: creat d0/db/d89/d32/d58/d5f/d84/fb1 x:0 0 0 2024-03-13T14:49:47.836 INFO:tasks.workunit.client.1.smithi196.stdout:4/541: rename d1/d8f to d1/d3/dbc 0 2024-03-13T14:49:47.840 INFO:tasks.workunit.client.0.smithi176.stdout:3/504: mknod d4/d28/d34/d38/ca7 0 2024-03-13T14:49:47.844 INFO:tasks.workunit.client.1.smithi196.stdout:4/542: write d1/d3/d4c/fb8 [930935,16213] 0 2024-03-13T14:49:47.844 INFO:tasks.workunit.client.1.smithi196.stdout:6/707: dwrite d6/d9/d17/d19/d4e/f8a [0,4194304] 0 2024-03-13T14:49:47.846 INFO:tasks.workunit.client.0.smithi176.stdout:6/469: unlink d0/d10/d19/f1a 0 2024-03-13T14:49:47.847 INFO:tasks.workunit.client.0.smithi176.stdout:7/431: dread d1/d1c/f56 [4194304,4194304] 0 2024-03-13T14:49:47.850 INFO:tasks.workunit.client.0.smithi176.stdout:5/438: creat d5/d2c/d40/d76/f95 x:0 0 0 2024-03-13T14:49:47.850 INFO:tasks.workunit.client.0.smithi176.stdout:7/432: dread - d1/d1c/d26/d41/f5a zero size 2024-03-13T14:49:47.852 INFO:tasks.workunit.client.0.smithi176.stdout:7/433: fdatasync d1/d28/d3a/f14 0 2024-03-13T14:49:47.853 INFO:tasks.workunit.client.1.smithi196.stdout:4/543: fdatasync d1/d3/d7/d35/d76/f94 0 2024-03-13T14:49:47.853 INFO:tasks.workunit.client.0.smithi176.stdout:5/439: stat d5/f81 0 2024-03-13T14:49:47.854 INFO:tasks.workunit.client.1.smithi196.stdout:1/634: dwrite d1/d9/d4e/d55/d63/da6/fbc [0,4194304] 0 2024-03-13T14:49:47.855 INFO:tasks.workunit.client.0.smithi176.stdout:1/430: creat d2/de/d3f/d50/f89 x:0 0 0 2024-03-13T14:49:47.856 INFO:tasks.workunit.client.0.smithi176.stdout:4/487: sync 2024-03-13T14:49:47.856 INFO:tasks.workunit.client.1.smithi196.stdout:6/708: rmdir d6/d9/d17 39 2024-03-13T14:49:47.862 INFO:tasks.workunit.client.0.smithi176.stdout:4/488: write d7/d10/d46/d4a/d5d/da1/f86 [159040,79580] 0 2024-03-13T14:49:47.864 INFO:tasks.workunit.client.1.smithi196.stdout:2/479: dread d0/d7/d40/f60 [0,4194304] 0 2024-03-13T14:49:47.864 INFO:tasks.workunit.client.0.smithi176.stdout:4/489: chown d7/d8/d45/f9f 126355460 1 2024-03-13T14:49:47.865 INFO:tasks.workunit.client.0.smithi176.stdout:9/522: symlink d0/db/d89/d32/d58/d67/lb2 0 2024-03-13T14:49:47.869 INFO:tasks.workunit.client.0.smithi176.stdout:6/470: mkdir d0/de/d54/d99 0 2024-03-13T14:49:47.871 INFO:tasks.workunit.client.0.smithi176.stdout:0/413: write d2/d5/d1e/f42 [1398935,127399] 0 2024-03-13T14:49:47.872 INFO:tasks.workunit.client.0.smithi176.stdout:9/523: dread - d0/db/d89/d32/d58/d5f/d84/fb1 zero size 2024-03-13T14:49:47.874 INFO:tasks.workunit.client.1.smithi196.stdout:6/709: stat d6/d9/d17/d19/fa8 0 2024-03-13T14:49:47.877 INFO:tasks.workunit.client.0.smithi176.stdout:2/500: dread d1/d2a/d33/fa2 [0,4194304] 0 2024-03-13T14:49:47.879 INFO:tasks.workunit.client.0.smithi176.stdout:5/440: dwrite d5/d2c/d40/d6a/f6e [0,4194304] 0 2024-03-13T14:49:47.880 INFO:tasks.workunit.client.1.smithi196.stdout:0/616: link d8/da/d25/d26/d35/d3b/d5f/d33/cc8 d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe/cc9 0 2024-03-13T14:49:47.884 INFO:tasks.workunit.client.1.smithi196.stdout:2/480: mknod d0/d7/d9/d9a/c9b 0 2024-03-13T14:49:47.884 INFO:tasks.workunit.client.0.smithi176.stdout:4/490: unlink d7/d10/d46/d4a/d5d/f8e 0 2024-03-13T14:49:47.885 INFO:tasks.workunit.client.1.smithi196.stdout:8/537: dwrite d0/d16/d3d/d8b/d48/f5a [0,4194304] 0 2024-03-13T14:49:47.885 INFO:tasks.workunit.client.0.smithi176.stdout:4/491: chown d7/c54 56074519 1 2024-03-13T14:49:47.885 INFO:tasks.workunit.client.0.smithi176.stdout:6/471: chown d0/d9/f6f 2358380 1 2024-03-13T14:49:47.888 INFO:tasks.workunit.client.1.smithi196.stdout:4/544: fdatasync d1/d3/d4c/fb8 0 2024-03-13T14:49:47.889 INFO:tasks.workunit.client.0.smithi176.stdout:8/422: rename d0/l3d to d0/l83 0 2024-03-13T14:49:47.890 INFO:tasks.workunit.client.1.smithi196.stdout:4/545: chown d1/d1e/d43/c4d 13107362 1 2024-03-13T14:49:47.891 INFO:tasks.workunit.client.0.smithi176.stdout:8/423: chown d0/f42 126284016 1 2024-03-13T14:49:47.892 INFO:tasks.workunit.client.0.smithi176.stdout:9/524: creat d0/db/d89/d32/d58/d9c/fb3 x:0 0 0 2024-03-13T14:49:47.894 INFO:tasks.workunit.client.1.smithi196.stdout:2/481: creat d0/d7/d40/f9c x:0 0 0 2024-03-13T14:49:47.905 INFO:tasks.workunit.client.1.smithi196.stdout:8/538: mknod d0/d10/d12/d17/cb5 0 2024-03-13T14:49:47.910 INFO:tasks.workunit.client.0.smithi176.stdout:6/472: mkdir d0/d10/d19/d27/d2b/d2e/d3e/d9a 0 2024-03-13T14:49:47.913 INFO:tasks.workunit.client.0.smithi176.stdout:3/505: dread d4/d13/d2d/f30 [0,4194304] 0 2024-03-13T14:49:47.913 INFO:tasks.workunit.client.1.smithi196.stdout:4/546: dread d1/d3/d4c/d72/d7d/d80/fac [0,4194304] 0 2024-03-13T14:49:47.914 INFO:tasks.workunit.client.0.smithi176.stdout:3/506: readlink d4/l22 0 2024-03-13T14:49:47.914 INFO:tasks.workunit.client.0.smithi176.stdout:7/434: write d1/d1c/d26/d41/d2b/d40/d64/f60 [1724379,97241] 0 2024-03-13T14:49:47.915 INFO:tasks.workunit.client.0.smithi176.stdout:9/525: creat d0/d1e/d1f/d68/fb4 x:0 0 0 2024-03-13T14:49:47.916 INFO:tasks.workunit.client.1.smithi196.stdout:8/539: rmdir d0/d10/d12 39 2024-03-13T14:49:47.919 INFO:tasks.workunit.client.0.smithi176.stdout:2/501: mkdir d1/dc/d2d/d59/d8b/daf 0 2024-03-13T14:49:47.921 INFO:tasks.workunit.client.1.smithi196.stdout:0/617: creat d8/da/d25/d26/d35/d3b/d5f/fca x:0 0 0 2024-03-13T14:49:47.921 INFO:tasks.workunit.client.0.smithi176.stdout:9/526: write d0/db/d89/d32/d58/d5f/f75 [8971331,72418] 0 2024-03-13T14:49:47.924 INFO:tasks.workunit.client.1.smithi196.stdout:1/635: dwrite d1/d38/f25 [0,4194304] 0 2024-03-13T14:49:47.924 INFO:tasks.workunit.client.0.smithi176.stdout:4/492: dwrite d7/d10/d46/f8f [0,4194304] 0 2024-03-13T14:49:47.925 INFO:tasks.workunit.client.1.smithi196.stdout:4/547: fsync d1/d3/d7/d35/f63 0 2024-03-13T14:49:47.925 INFO:tasks.workunit.client.1.smithi196.stdout:1/636: readlink d1/d28/l8b 0 2024-03-13T14:49:47.927 INFO:tasks.workunit.client.0.smithi176.stdout:6/473: dwrite d0/d10/d19/d27/d2b/f39 [0,4194304] 0 2024-03-13T14:49:47.928 INFO:tasks.workunit.client.0.smithi176.stdout:6/474: stat d0/d10/d19 0 2024-03-13T14:49:47.930 INFO:tasks.workunit.client.0.smithi176.stdout:4/493: fsync d7/d8/f42 0 2024-03-13T14:49:47.931 INFO:tasks.workunit.client.1.smithi196.stdout:6/710: truncate d6/d7/df/d27/d39/d44/d74/f87 3358047 0 2024-03-13T14:49:47.932 INFO:tasks.workunit.client.0.smithi176.stdout:5/441: mkdir d5/d2e/d7e/d96 0 2024-03-13T14:49:47.932 INFO:tasks.workunit.client.0.smithi176.stdout:4/494: chown d7/d8/d2f/c90 415220687 1 2024-03-13T14:49:47.933 INFO:tasks.workunit.client.0.smithi176.stdout:8/424: symlink d0/d24/d29/d70/l84 0 2024-03-13T14:49:47.934 INFO:tasks.workunit.client.1.smithi196.stdout:0/618: creat d8/da/d25/d26/d54/d88/fcb x:0 0 0 2024-03-13T14:49:47.938 INFO:tasks.workunit.client.0.smithi176.stdout:2/502: symlink d1/dc/d40/da0/lb0 0 2024-03-13T14:49:47.942 INFO:tasks.workunit.client.1.smithi196.stdout:2/482: link d0/d7/d9/d5a/f97 d0/d50/f9d 0 2024-03-13T14:49:47.942 INFO:tasks.workunit.client.0.smithi176.stdout:9/527: chown d0/l21 3430195 1 2024-03-13T14:49:47.946 INFO:tasks.workunit.client.0.smithi176.stdout:0/414: dwrite d2/db/f25 [0,4194304] 0 2024-03-13T14:49:47.948 INFO:tasks.workunit.client.0.smithi176.stdout:1/431: dwrite d2/de/d3f/d50/f42 [0,4194304] 0 2024-03-13T14:49:47.949 INFO:tasks.workunit.client.1.smithi196.stdout:6/711: truncate d6/d2a/dc/d1c/d34/fa9 882292 0 2024-03-13T14:49:47.952 INFO:tasks.workunit.client.1.smithi196.stdout:4/548: dwrite d1/d3/d7/f65 [0,4194304] 0 2024-03-13T14:49:47.954 INFO:tasks.workunit.client.0.smithi176.stdout:6/475: rename d0/de/d38/d70/f97 to d0/de/d38/d77/d76/d7c/f9b 0 2024-03-13T14:49:47.963 INFO:tasks.workunit.client.0.smithi176.stdout:3/507: symlink d4/d28/d34/da4/la8 0 2024-03-13T14:49:47.966 INFO:tasks.workunit.client.1.smithi196.stdout:2/483: dwrite d0/d6/f84 [4194304,4194304] 0 2024-03-13T14:49:47.967 INFO:tasks.workunit.client.1.smithi196.stdout:1/637: link d1/d9/d4e/d55/d63/db9/dbb/fc4 d1/d3f/d5b/d74/db5/fd9 0 2024-03-13T14:49:47.971 INFO:tasks.workunit.client.0.smithi176.stdout:7/435: link d1/d3f/d4b/f58 d1/d1c/d26/d41/d2b/d7b/f7e 0 2024-03-13T14:49:47.973 INFO:tasks.workunit.client.0.smithi176.stdout:7/436: chown d1/d28/d7c 0 1 2024-03-13T14:49:47.974 INFO:tasks.workunit.client.0.smithi176.stdout:6/476: dwrite d0/de/d8d/f95 [0,4194304] 0 2024-03-13T14:49:47.978 INFO:tasks.workunit.client.0.smithi176.stdout:4/495: creat d7/d10/d2c/fb2 x:0 0 0 2024-03-13T14:49:47.981 INFO:tasks.workunit.client.1.smithi196.stdout:8/540: dread d0/d16/d3d/d8b/f4b [0,4194304] 0 2024-03-13T14:49:47.983 INFO:tasks.workunit.client.1.smithi196.stdout:0/619: link d8/f1e d8/da/d25/d26/d35/d3b/d5f/d17/fcc 0 2024-03-13T14:49:47.984 INFO:tasks.workunit.client.0.smithi176.stdout:2/503: rmdir d1/dc/d2d/d38/d5d 39 2024-03-13T14:49:47.985 INFO:tasks.workunit.client.1.smithi196.stdout:0/620: write d8/d37/f73 [1979839,46964] 0 2024-03-13T14:49:47.989 INFO:tasks.workunit.client.0.smithi176.stdout:9/528: creat d0/d40/d53/fb5 x:0 0 0 2024-03-13T14:49:47.989 INFO:tasks.workunit.client.1.smithi196.stdout:1/638: read d1/d9/d4e/d55/d63/da6/fb0 [1935354,108972] 0 2024-03-13T14:49:47.990 INFO:tasks.workunit.client.0.smithi176.stdout:6/477: dwrite d0/de/d38/d77/f74 [0,4194304] 0 2024-03-13T14:49:47.992 INFO:tasks.workunit.client.0.smithi176.stdout:9/529: truncate d0/d1e/d1f/d68/f85 1016408 0 2024-03-13T14:49:47.994 INFO:tasks.workunit.client.0.smithi176.stdout:0/415: readlink d2/d5/d1e/l55 0 2024-03-13T14:49:47.998 INFO:tasks.workunit.client.0.smithi176.stdout:1/432: readlink d2/de/d10/l24 0 2024-03-13T14:49:47.999 INFO:tasks.workunit.client.1.smithi196.stdout:8/541: mknod d0/d16/d71/cb6 0 2024-03-13T14:49:48.003 INFO:tasks.workunit.client.0.smithi176.stdout:3/508: creat d4/d28/d34/da4/fa9 x:0 0 0 2024-03-13T14:49:48.006 INFO:tasks.workunit.client.1.smithi196.stdout:2/484: mkdir d0/d6/d4b/d96/d9e 0 2024-03-13T14:49:48.008 INFO:tasks.workunit.client.0.smithi176.stdout:7/437: truncate d1/d7/f3d 1385690 0 2024-03-13T14:49:48.009 INFO:tasks.workunit.client.1.smithi196.stdout:0/621: creat d8/da/d25/fcd x:0 0 0 2024-03-13T14:49:48.010 INFO:tasks.workunit.client.0.smithi176.stdout:8/425: symlink d0/d24/l85 0 2024-03-13T14:49:48.013 INFO:tasks.workunit.client.0.smithi176.stdout:3/509: dread d4/f97 [0,4194304] 0 2024-03-13T14:49:48.013 INFO:tasks.workunit.client.1.smithi196.stdout:6/712: rename d6/d7/dd/dbc to d6/d2a/de2 0 2024-03-13T14:49:48.014 INFO:tasks.workunit.client.0.smithi176.stdout:3/510: chown d4/c1e 40001424 1 2024-03-13T14:49:48.016 INFO:tasks.workunit.client.0.smithi176.stdout:4/496: mkdir d7/d2d/da7/db3 0 2024-03-13T14:49:48.024 INFO:tasks.workunit.client.1.smithi196.stdout:0/622: write d8/f9 [1800536,11174] 0 2024-03-13T14:49:48.024 INFO:tasks.workunit.client.0.smithi176.stdout:0/416: truncate d2/db/d31/f51 948272 0 2024-03-13T14:49:48.029 INFO:tasks.workunit.client.1.smithi196.stdout:1/639: symlink d1/d9/d33/d4b/lda 0 2024-03-13T14:49:48.031 INFO:tasks.workunit.client.0.smithi176.stdout:7/438: chown d1/d1c/c21 1 1 2024-03-13T14:49:48.031 INFO:tasks.workunit.client.0.smithi176.stdout:7/439: chown d1/d1c 5614513 1 2024-03-13T14:49:48.032 INFO:tasks.workunit.client.1.smithi196.stdout:4/549: getdents d1/d3/d4c/d72/d7d/d80 0 2024-03-13T14:49:48.034 INFO:tasks.workunit.client.0.smithi176.stdout:8/426: fdatasync d0/d5/d11/f26 0 2024-03-13T14:49:48.035 INFO:tasks.workunit.client.1.smithi196.stdout:2/485: dread d0/f1a [0,4194304] 0 2024-03-13T14:49:48.039 INFO:tasks.workunit.client.1.smithi196.stdout:0/623: rename d8/d37/ca8 to d8/da/d74/cce 0 2024-03-13T14:49:48.041 INFO:tasks.workunit.client.0.smithi176.stdout:3/511: mkdir d4/d13/d5e/daa 0 2024-03-13T14:49:48.042 INFO:tasks.workunit.client.1.smithi196.stdout:6/713: mkdir d6/d2a/dc/de3 0 2024-03-13T14:49:48.042 INFO:tasks.workunit.client.0.smithi176.stdout:3/512: readlink d4/d28/d34/da4/la8 0 2024-03-13T14:49:48.044 INFO:tasks.workunit.client.1.smithi196.stdout:6/714: dread - d6/d7/df/d27/fe1 zero size 2024-03-13T14:49:48.047 INFO:tasks.workunit.client.0.smithi176.stdout:6/478: symlink d0/d10/d19/d27/d2b/d2e/d3e/d9a/l9c 0 2024-03-13T14:49:48.049 INFO:tasks.workunit.client.0.smithi176.stdout:6/479: write d0/de/d38/f80 [557488,125475] 0 2024-03-13T14:49:48.049 INFO:tasks.workunit.client.0.smithi176.stdout:9/530: rename d0/d73/c7f to d0/db/d89/d4d/d6e/d81/cb6 0 2024-03-13T14:49:48.050 INFO:tasks.workunit.client.1.smithi196.stdout:2/486: dwrite d0/d6/d34/f7e [0,4194304] 0 2024-03-13T14:49:48.051 INFO:tasks.workunit.client.0.smithi176.stdout:9/531: stat d0/d1e/c34 0 2024-03-13T14:49:48.052 INFO:tasks.workunit.client.1.smithi196.stdout:2/487: fdatasync d0/d7/f89 0 2024-03-13T14:49:48.053 INFO:tasks.workunit.client.0.smithi176.stdout:0/417: dread d2/db/d39/f47 [0,4194304] 0 2024-03-13T14:49:48.054 INFO:tasks.workunit.client.0.smithi176.stdout:2/504: truncate d1/d2a/f82 1564697 0 2024-03-13T14:49:48.056 INFO:tasks.workunit.client.0.smithi176.stdout:0/418: truncate d2/d5/dc/d59/f5a 857205 0 2024-03-13T14:49:48.062 INFO:tasks.workunit.client.1.smithi196.stdout:8/542: getdents d0/d10/d12/d17/d42 0 2024-03-13T14:49:48.063 INFO:tasks.workunit.client.0.smithi176.stdout:8/427: mkdir d0/db/d58/d86 0 2024-03-13T14:49:48.063 INFO:tasks.workunit.client.0.smithi176.stdout:8/428: chown d0/db/dc/l18 234736 1 2024-03-13T14:49:48.063 INFO:tasks.workunit.client.0.smithi176.stdout:5/442: getdents d5/d2b 0 2024-03-13T14:49:48.064 INFO:tasks.workunit.client.0.smithi176.stdout:5/443: write d5/f14 [6208132,108630] 0 2024-03-13T14:49:48.067 INFO:tasks.workunit.client.1.smithi196.stdout:1/640: getdents d1/d9 0 2024-03-13T14:49:48.070 INFO:tasks.workunit.client.1.smithi196.stdout:4/550: getdents d1/d3/d4c 0 2024-03-13T14:49:48.070 INFO:tasks.workunit.client.0.smithi176.stdout:4/497: link d7/d10/d46/d4a/d5d/f69 d7/d8/d2f/fb4 0 2024-03-13T14:49:48.071 INFO:tasks.workunit.client.0.smithi176.stdout:3/513: rename d4/cf to d4/d28/d31/d52/cab 0 2024-03-13T14:49:48.071 INFO:tasks.workunit.client.1.smithi196.stdout:7/695: sync 2024-03-13T14:49:48.071 INFO:tasks.workunit.client.1.smithi196.stdout:5/658: sync 2024-03-13T14:49:48.071 INFO:tasks.workunit.client.1.smithi196.stdout:3/576: sync 2024-03-13T14:49:48.072 INFO:tasks.workunit.client.1.smithi196.stdout:9/530: sync 2024-03-13T14:49:48.072 INFO:tasks.workunit.client.0.smithi176.stdout:4/498: write d7/d10/d2c/f77 [884387,3184] 0 2024-03-13T14:49:48.073 INFO:tasks.workunit.client.1.smithi196.stdout:2/488: dwrite d0/d6/d4b/f7d [0,4194304] 0 2024-03-13T14:49:48.074 INFO:tasks.workunit.client.0.smithi176.stdout:6/480: unlink d0/de/d38/d7e/f7f 0 2024-03-13T14:49:48.076 INFO:tasks.workunit.client.1.smithi196.stdout:9/531: write d7/d24/f3a [1997624,126343] 0 2024-03-13T14:49:48.077 INFO:tasks.workunit.client.1.smithi196.stdout:2/489: truncate d0/d6/d4b/f64 812933 0 2024-03-13T14:49:48.079 INFO:tasks.workunit.client.0.smithi176.stdout:1/433: rmdir d2/de/d3f/d50/d7c 0 2024-03-13T14:49:48.082 INFO:tasks.workunit.client.1.smithi196.stdout:0/624: getdents d8/da/d25/d26/d54 0 2024-03-13T14:49:48.082 INFO:tasks.workunit.client.1.smithi196.stdout:1/641: rename d1/d28/l8b to d1/d9/d4e/ldb 0 2024-03-13T14:49:48.082 INFO:tasks.workunit.client.0.smithi176.stdout:7/440: symlink d1/d3f/d4b/d63/d7a/l7f 0 2024-03-13T14:49:48.084 INFO:tasks.workunit.client.0.smithi176.stdout:8/429: dwrite d0/d5/d3b/f41 [0,4194304] 0 2024-03-13T14:49:48.085 INFO:tasks.workunit.client.1.smithi196.stdout:1/642: write d1/d9/d33/d4b/d66/d7b/fa9 [802642,95594] 0 2024-03-13T14:49:48.089 INFO:tasks.workunit.client.1.smithi196.stdout:1/643: write d1/d38/f3b [1747999,8649] 0 2024-03-13T14:49:48.100 INFO:tasks.workunit.client.0.smithi176.stdout:5/444: chown d5/c11 532323466 1 2024-03-13T14:49:48.103 INFO:tasks.workunit.client.1.smithi196.stdout:2/490: dwrite d0/d6/d4b/f7d [0,4194304] 0 2024-03-13T14:49:48.104 INFO:tasks.workunit.client.0.smithi176.stdout:5/445: truncate d5/d31/d47/f85 379526 0 2024-03-13T14:49:48.104 INFO:tasks.workunit.client.0.smithi176.stdout:3/514: mknod d4/d13/cac 0 2024-03-13T14:49:48.105 INFO:tasks.workunit.client.0.smithi176.stdout:3/515: chown d4/d28/d31/d52/ca3 473 1 2024-03-13T14:49:48.106 INFO:tasks.workunit.client.1.smithi196.stdout:5/659: chown d4/d1b/dc8/d3c/f57 25577306 1 2024-03-13T14:49:48.110 INFO:tasks.workunit.client.1.smithi196.stdout:4/551: mknod d1/d3/d7/cbd 0 2024-03-13T14:49:48.110 INFO:tasks.workunit.client.1.smithi196.stdout:8/543: mknod d0/d10/da5/db2/cb7 0 2024-03-13T14:49:48.114 INFO:tasks.workunit.client.1.smithi196.stdout:6/715: link d6/d2a/dc/d1c/da7/d9e/la5 d6/d2a/dc/le4 0 2024-03-13T14:49:48.115 INFO:tasks.workunit.client.1.smithi196.stdout:4/552: chown d1/d3/c41 29326 1 2024-03-13T14:49:48.120 INFO:tasks.workunit.client.0.smithi176.stdout:2/505: mknod d1/dc/d2d/d38/d5d/cb1 0 2024-03-13T14:49:48.122 INFO:tasks.workunit.client.1.smithi196.stdout:3/577: dwrite db/d37/f76 [0,4194304] 0 2024-03-13T14:49:48.129 INFO:tasks.workunit.client.0.smithi176.stdout:1/434: creat d2/de/d10/d16/d4a/f8a x:0 0 0 2024-03-13T14:49:48.130 INFO:tasks.workunit.client.0.smithi176.stdout:1/435: fdatasync d2/d30/f3a 0 2024-03-13T14:49:48.131 INFO:tasks.workunit.client.1.smithi196.stdout:7/696: write d0/d9/d31/fb5 [995254,94831] 0 2024-03-13T14:49:48.131 INFO:tasks.workunit.client.1.smithi196.stdout:9/532: symlink d7/d24/lb2 0 2024-03-13T14:49:48.134 INFO:tasks.workunit.client.0.smithi176.stdout:7/441: symlink d1/d1c/l80 0 2024-03-13T14:49:48.136 INFO:tasks.workunit.client.0.smithi176.stdout:5/446: chown d5/ld 15409 1 2024-03-13T14:49:48.140 INFO:tasks.workunit.client.1.smithi196.stdout:2/491: symlink d0/d6/d90/l9f 0 2024-03-13T14:49:48.140 INFO:tasks.workunit.client.0.smithi176.stdout:4/499: mkdir d7/dab/db5 0 2024-03-13T14:49:48.145 INFO:tasks.workunit.client.1.smithi196.stdout:4/553: dwrite d1/d3/d4c/fb8 [0,4194304] 0 2024-03-13T14:49:48.145 INFO:tasks.workunit.client.1.smithi196.stdout:6/716: mknod d6/d2a/d23/ce5 0 2024-03-13T14:49:48.146 INFO:tasks.workunit.client.0.smithi176.stdout:9/532: dwrite d0/d1e/d1f/f41 [0,4194304] 0 2024-03-13T14:49:48.149 INFO:tasks.workunit.client.1.smithi196.stdout:4/554: chown d1/d3/d7/fa4 97 1 2024-03-13T14:49:48.150 INFO:tasks.workunit.client.0.smithi176.stdout:0/419: truncate d2/f69 623686 0 2024-03-13T14:49:48.155 INFO:tasks.workunit.client.1.smithi196.stdout:1/644: creat d1/d28/da3/fdc x:0 0 0 2024-03-13T14:49:48.156 INFO:tasks.workunit.client.0.smithi176.stdout:0/420: dread d2/db/d31/f57 [0,4194304] 0 2024-03-13T14:49:48.161 INFO:tasks.workunit.client.1.smithi196.stdout:2/492: write d0/d7/d9/f48 [242195,41274] 0 2024-03-13T14:49:48.162 INFO:tasks.workunit.client.0.smithi176.stdout:6/481: dwrite d0/de/d38/f8b [0,4194304] 0 2024-03-13T14:49:48.165 INFO:tasks.workunit.client.0.smithi176.stdout:6/482: write d0/d10/d19/d27/f4d [1931496,8518] 0 2024-03-13T14:49:48.166 INFO:tasks.workunit.client.0.smithi176.stdout:4/500: symlink d7/d10/d46/lb6 0 2024-03-13T14:49:48.168 INFO:tasks.workunit.client.0.smithi176.stdout:6/483: read d0/de/d38/f8b [1003563,102287] 0 2024-03-13T14:49:48.170 INFO:tasks.workunit.client.1.smithi196.stdout:0/625: link d8/da/d25/d26/d35/d3b/d50/d55/l8c d8/da/d25/d26/d54/lcf 0 2024-03-13T14:49:48.170 INFO:tasks.workunit.client.0.smithi176.stdout:2/506: mkdir d1/dc/d2d/d59/d8b/daf/db2 0 2024-03-13T14:49:48.172 INFO:tasks.workunit.client.0.smithi176.stdout:6/484: truncate d0/d10/d19/d27/f68 1313782 0 2024-03-13T14:49:48.174 INFO:tasks.workunit.client.0.smithi176.stdout:8/430: creat d0/d5/f87 x:0 0 0 2024-03-13T14:49:48.180 INFO:tasks.workunit.client.1.smithi196.stdout:1/645: creat d1/d9/d4e/fdd x:0 0 0 2024-03-13T14:49:48.180 INFO:tasks.workunit.client.1.smithi196.stdout:4/555: dread d1/d3/d4c/f83 [0,4194304] 0 2024-03-13T14:49:48.189 INFO:tasks.workunit.client.0.smithi176.stdout:5/447: dread f0 [8388608,4194304] 0 2024-03-13T14:49:48.192 INFO:tasks.workunit.client.0.smithi176.stdout:0/421: truncate d2/d5/d28/f54 640111 0 2024-03-13T14:49:48.193 INFO:tasks.workunit.client.0.smithi176.stdout:3/516: mkdir d4/d13/d16/dad 0 2024-03-13T14:49:48.199 INFO:tasks.workunit.client.1.smithi196.stdout:4/556: stat d1/d1e/c8b 0 2024-03-13T14:49:48.200 INFO:tasks.workunit.client.0.smithi176.stdout:4/501: symlink d7/d10/d46/d55/lb7 0 2024-03-13T14:49:48.202 INFO:tasks.workunit.client.1.smithi196.stdout:8/544: getdents d0/d16/d3d/d9c 0 2024-03-13T14:49:48.203 INFO:tasks.workunit.client.0.smithi176.stdout:1/436: write f1 [2177329,34384] 0 2024-03-13T14:49:48.205 INFO:tasks.workunit.client.0.smithi176.stdout:2/507: chown d1/dc/d40/l92 748 1 2024-03-13T14:49:48.206 INFO:tasks.workunit.client.0.smithi176.stdout:3/517: dwrite d4/d28/d34/da4/fa9 [0,4194304] 0 2024-03-13T14:49:48.209 INFO:tasks.workunit.client.1.smithi196.stdout:0/626: write d8/da/d25/d26/d35/d3b/f7d [521921,44721] 0 2024-03-13T14:49:48.209 INFO:tasks.workunit.client.0.smithi176.stdout:1/437: truncate d2/de/d10/d16/d4a/d5f/f81 1027973 0 2024-03-13T14:49:48.210 INFO:tasks.workunit.client.1.smithi196.stdout:7/697: dwrite d0/f56 [8388608,4194304] 0 2024-03-13T14:49:48.215 INFO:tasks.workunit.client.1.smithi196.stdout:8/545: chown d0/d16/d3d/d8b/d3f/f76 9569 1 2024-03-13T14:49:48.219 INFO:tasks.workunit.client.1.smithi196.stdout:6/717: dread d6/d7/df/d27/d39/d44/d74/f87 [0,4194304] 0 2024-03-13T14:49:48.220 INFO:tasks.workunit.client.0.smithi176.stdout:5/448: rmdir d5/d2c 39 2024-03-13T14:49:48.224 INFO:tasks.workunit.client.0.smithi176.stdout:2/508: chown d1/dc/d2d/d38/d5d/f62 159 1 2024-03-13T14:49:48.224 INFO:tasks.workunit.client.1.smithi196.stdout:5/660: dwrite d4/d1b/dc8/d14/fb4 [0,4194304] 0 2024-03-13T14:49:48.224 INFO:tasks.workunit.client.1.smithi196.stdout:1/646: mkdir d1/d9/d33/dd2/dde 0 2024-03-13T14:49:48.226 INFO:tasks.workunit.client.1.smithi196.stdout:2/493: creat d0/d7/d9/fa0 x:0 0 0 2024-03-13T14:49:48.228 INFO:tasks.workunit.client.0.smithi176.stdout:2/509: chown d1/dc/l42 8178 1 2024-03-13T14:49:48.230 INFO:tasks.workunit.client.1.smithi196.stdout:8/546: chown d0/d16/d3d/d8b/d3f 1471713233 1 2024-03-13T14:49:48.230 INFO:tasks.workunit.client.1.smithi196.stdout:9/533: dwrite f5 [4194304,4194304] 0 2024-03-13T14:49:48.230 INFO:tasks.workunit.client.1.smithi196.stdout:0/627: mkdir d8/da/d25/d26/d48/d82/dd0 0 2024-03-13T14:49:48.231 INFO:tasks.workunit.client.1.smithi196.stdout:3/578: dwrite db/d19/f28 [0,4194304] 0 2024-03-13T14:49:48.231 INFO:tasks.workunit.client.0.smithi176.stdout:1/438: rename d2/de/d3f/d50/c36 to d2/de/d10/d16/d47/c8b 0 2024-03-13T14:49:48.231 INFO:tasks.workunit.client.0.smithi176.stdout:7/442: getdents d1/d28/d3a/d1b 0 2024-03-13T14:49:48.231 INFO:tasks.workunit.client.0.smithi176.stdout:2/510: chown d1/dc/d2d/d59/d80/d5c/c7b 52931344 1 2024-03-13T14:49:48.232 INFO:tasks.workunit.client.0.smithi176.stdout:7/443: dread - d1/d1c/f77 zero size 2024-03-13T14:49:48.236 INFO:tasks.workunit.client.0.smithi176.stdout:0/422: symlink d2/d5/l7e 0 2024-03-13T14:49:48.237 INFO:tasks.workunit.client.0.smithi176.stdout:3/518: symlink d4/d13/d5e/lae 0 2024-03-13T14:49:48.237 INFO:tasks.workunit.client.0.smithi176.stdout:4/502: creat d7/d10/d2c/d9d/fb8 x:0 0 0 2024-03-13T14:49:48.238 INFO:tasks.workunit.client.0.smithi176.stdout:6/485: rmdir d0/d9/d8c 0 2024-03-13T14:49:48.239 INFO:tasks.workunit.client.1.smithi196.stdout:7/698: truncate d0/d9/f24 4363695 0 2024-03-13T14:49:48.240 INFO:tasks.workunit.client.0.smithi176.stdout:4/503: truncate d7/d10/f8a 262659 0 2024-03-13T14:49:48.247 INFO:tasks.workunit.client.1.smithi196.stdout:0/628: creat d8/da/d25/fd1 x:0 0 0 2024-03-13T14:49:48.247 INFO:tasks.workunit.client.1.smithi196.stdout:6/718: write d6/d2a/dc/d1c/d34/fbd [390365,73962] 0 2024-03-13T14:49:48.247 INFO:tasks.workunit.client.1.smithi196.stdout:3/579: creat db/d19/d22/fc2 x:0 0 0 2024-03-13T14:49:48.249 INFO:tasks.workunit.client.0.smithi176.stdout:1/439: mknod d2/de/c8c 0 2024-03-13T14:49:48.250 INFO:tasks.workunit.client.1.smithi196.stdout:0/629: chown d8/da/d25/d26/d35/d3b/d50/d5d 4368 1 2024-03-13T14:49:48.252 INFO:tasks.workunit.client.0.smithi176.stdout:7/444: mkdir d1/d3f/d81 0 2024-03-13T14:49:48.256 INFO:tasks.workunit.client.1.smithi196.stdout:1/647: rename d1/d9/f36 to d1/d28/d34/d67/fdf 0 2024-03-13T14:49:48.256 INFO:tasks.workunit.client.0.smithi176.stdout:8/431: dwrite d0/d5/d11/f26 [0,4194304] 0 2024-03-13T14:49:48.260 INFO:tasks.workunit.client.1.smithi196.stdout:5/661: creat d4/d1b/dc8/d14/d19/da7/fca x:0 0 0 2024-03-13T14:49:48.260 INFO:tasks.workunit.client.0.smithi176.stdout:0/423: creat d2/db/d31/f7f x:0 0 0 2024-03-13T14:49:48.261 INFO:tasks.workunit.client.0.smithi176.stdout:1/440: dread d2/d30/d63/d22/d71/f87 [0,4194304] 0 2024-03-13T14:49:48.262 INFO:tasks.workunit.client.0.smithi176.stdout:6/486: unlink d0/de/d38/f8b 0 2024-03-13T14:49:48.263 INFO:tasks.workunit.client.0.smithi176.stdout:1/441: write d2/d30/f65 [5141395,61714] 0 2024-03-13T14:49:48.264 INFO:tasks.workunit.client.1.smithi196.stdout:3/580: write db/d15/d44/fb6 [2837230,12181] 0 2024-03-13T14:49:48.265 INFO:tasks.workunit.client.1.smithi196.stdout:8/547: mkdir d0/d16/d3d/d9c/d87/d8c/da9/db8 0 2024-03-13T14:49:48.265 INFO:tasks.workunit.client.0.smithi176.stdout:1/442: write d2/de/d3f/d50/f42 [6235020,73842] 0 2024-03-13T14:49:48.266 INFO:tasks.workunit.client.0.smithi176.stdout:3/519: creat d4/d13/d16/d80/faf x:0 0 0 2024-03-13T14:49:48.271 INFO:tasks.workunit.client.0.smithi176.stdout:7/445: readlink d1/d1c/d26/l6f 0 2024-03-13T14:49:48.272 INFO:tasks.workunit.client.1.smithi196.stdout:4/557: dread d1/d3/d7/d35/f40 [0,4194304] 0 2024-03-13T14:49:48.272 INFO:tasks.workunit.client.1.smithi196.stdout:6/719: rename d6/d2a/f62 to d6/d7/df/fe6 0 2024-03-13T14:49:48.273 INFO:tasks.workunit.client.0.smithi176.stdout:5/449: mknod d5/d2c/d40/c97 0 2024-03-13T14:49:48.274 INFO:tasks.workunit.client.0.smithi176.stdout:0/424: rmdir d2/d74 39 2024-03-13T14:49:48.279 INFO:tasks.workunit.client.0.smithi176.stdout:6/487: unlink d0/d9/l20 0 2024-03-13T14:49:48.281 INFO:tasks.workunit.client.0.smithi176.stdout:4/504: mkdir d7/db9 0 2024-03-13T14:49:48.282 INFO:tasks.workunit.client.1.smithi196.stdout:4/558: unlink d1/d3/d7/cbd 0 2024-03-13T14:49:48.283 INFO:tasks.workunit.client.0.smithi176.stdout:4/505: truncate d7/d8/d45/f9f 1044208 0 2024-03-13T14:49:48.287 INFO:tasks.workunit.client.1.smithi196.stdout:6/720: mknod d6/d2a/dc/d1c/da7/ce7 0 2024-03-13T14:49:48.289 INFO:tasks.workunit.client.0.smithi176.stdout:3/520: chown d4/d28/d31/d52/c64 7669463 1 2024-03-13T14:49:48.298 INFO:tasks.workunit.client.1.smithi196.stdout:6/721: mkdir d6/d9/d17/da1/d50/de8 0 2024-03-13T14:49:48.301 INFO:tasks.workunit.client.0.smithi176.stdout:0/425: unlink d2/d5/dc/d20/c32 0 2024-03-13T14:49:48.303 INFO:tasks.workunit.client.0.smithi176.stdout:6/488: chown d0/d10/l32 2900070 1 2024-03-13T14:49:48.304 INFO:tasks.workunit.client.1.smithi196.stdout:4/559: mknod d1/cbe 0 2024-03-13T14:49:48.308 INFO:tasks.workunit.client.1.smithi196.stdout:5/662: dread d4/d1b/dc8/f24 [0,4194304] 0 2024-03-13T14:49:48.309 INFO:tasks.workunit.client.1.smithi196.stdout:1/648: write d1/d9/d33/d4b/d66/fa8 [3042557,85652] 0 2024-03-13T14:49:48.310 INFO:tasks.workunit.client.1.smithi196.stdout:2/494: truncate d0/d6/d1d/f27 3633013 0 2024-03-13T14:49:48.312 INFO:tasks.workunit.client.1.smithi196.stdout:3/581: getdents db/d37/d64 0 2024-03-13T14:49:48.313 INFO:tasks.workunit.client.1.smithi196.stdout:5/663: chown d4/d1b/dc8/d14/c1e 158939359 1 2024-03-13T14:49:48.314 INFO:tasks.workunit.client.0.smithi176.stdout:2/511: getdents d1 0 2024-03-13T14:49:48.315 INFO:tasks.workunit.client.1.smithi196.stdout:4/560: fdatasync d1/d3/d7/f49 0 2024-03-13T14:49:48.316 INFO:tasks.workunit.client.0.smithi176.stdout:2/512: readlink d1/dc/d40/l92 0 2024-03-13T14:49:48.317 INFO:tasks.workunit.client.1.smithi196.stdout:4/561: stat d1/d1e/l3c 0 2024-03-13T14:49:48.318 INFO:tasks.workunit.client.0.smithi176.stdout:5/450: symlink d5/d2c/d40/d6a/d89/l98 0 2024-03-13T14:49:48.319 INFO:tasks.workunit.client.0.smithi176.stdout:2/513: write d1/dc/d2d/d59/f8a [679706,35883] 0 2024-03-13T14:49:48.320 INFO:tasks.workunit.client.1.smithi196.stdout:5/664: readlink d4/d1b/dc8/d14/d19/d26/l65 0 2024-03-13T14:49:48.320 INFO:tasks.workunit.client.1.smithi196.stdout:0/630: dwrite d8/da/d25/d26/d35/d3b/d50/d55/f72 [0,4194304] 0 2024-03-13T14:49:48.323 INFO:tasks.workunit.client.1.smithi196.stdout:4/562: dread d1/f2e [0,4194304] 0 2024-03-13T14:49:48.323 INFO:tasks.workunit.client.1.smithi196.stdout:5/665: chown d4/d1b/dc8/d14/d19/d26/d68 3437578 1 2024-03-13T14:49:48.325 INFO:tasks.workunit.client.0.smithi176.stdout:4/506: dwrite d7/d8/f28 [0,4194304] 0 2024-03-13T14:49:48.328 INFO:tasks.workunit.client.0.smithi176.stdout:4/507: read - d7/d8/d2f/fb4 zero size 2024-03-13T14:49:48.328 INFO:tasks.workunit.client.0.smithi176.stdout:3/521: mknod d4/d13/d2d/d50/cb0 0 2024-03-13T14:49:48.331 INFO:tasks.workunit.client.0.smithi176.stdout:7/446: rename d1/d1c/d26/d41/l48 to d1/d1c/d26/d41/d2b/l82 0 2024-03-13T14:49:48.332 INFO:tasks.workunit.client.0.smithi176.stdout:7/447: readlink d1/d7/lb 0 2024-03-13T14:49:48.333 INFO:tasks.workunit.client.1.smithi196.stdout:2/495: dwrite d0/d7/d40/f9c [0,4194304] 0 2024-03-13T14:49:48.335 INFO:tasks.workunit.client.1.smithi196.stdout:6/722: link d6/d2a/f93 d6/fe9 0 2024-03-13T14:49:48.338 INFO:tasks.workunit.client.0.smithi176.stdout:6/489: mkdir d0/de/d38/d88/d9d 0 2024-03-13T14:49:48.339 INFO:tasks.workunit.client.0.smithi176.stdout:5/451: dread d5/d2e/f3f [0,4194304] 0 2024-03-13T14:49:48.341 INFO:tasks.workunit.client.0.smithi176.stdout:2/514: mknod d1/dc/d2d/d59/d80/d5c/d78/cb3 0 2024-03-13T14:49:48.343 INFO:tasks.workunit.client.1.smithi196.stdout:3/582: getdents db/d19/dae 0 2024-03-13T14:49:48.345 INFO:tasks.workunit.client.1.smithi196.stdout:0/631: fsync d8/da/d25/d26/d35/d3b/d5f/f2b 0 2024-03-13T14:49:48.345 INFO:tasks.workunit.client.0.smithi176.stdout:1/443: getdents d2/de 0 2024-03-13T14:49:48.346 INFO:tasks.workunit.client.1.smithi196.stdout:2/496: dread d0/d6/d4b/f79 [0,4194304] 0 2024-03-13T14:49:48.346 INFO:tasks.workunit.client.1.smithi196.stdout:6/723: rmdir d6/d9/d17/d19/d4e 39 2024-03-13T14:49:48.348 INFO:tasks.workunit.client.0.smithi176.stdout:2/515: truncate d1/dc/d2d/d38/d5d/f70 1099705 0 2024-03-13T14:49:48.349 INFO:tasks.workunit.client.0.smithi176.stdout:4/508: mkdir d7/d2d/d9b/dba 0 2024-03-13T14:49:48.350 INFO:tasks.workunit.client.0.smithi176.stdout:2/516: dread - d1/dc/d1c/fa4 zero size 2024-03-13T14:49:48.352 INFO:tasks.workunit.client.0.smithi176.stdout:1/444: chown d2/de/d10/d16/l4e 93580313 1 2024-03-13T14:49:48.352 INFO:tasks.workunit.client.1.smithi196.stdout:1/649: dread d1/f89 [0,4194304] 0 2024-03-13T14:49:48.354 INFO:tasks.workunit.client.1.smithi196.stdout:3/583: mkdir db/d37/dc3 0 2024-03-13T14:49:48.355 INFO:tasks.workunit.client.0.smithi176.stdout:4/509: read d7/f24 [4152753,103018] 0 2024-03-13T14:49:48.356 INFO:tasks.workunit.client.1.smithi196.stdout:1/650: chown d1/d38/f30 9 1 2024-03-13T14:49:48.358 INFO:tasks.workunit.client.1.smithi196.stdout:1/651: stat d1/d3f/d5b/d74/f65 0 2024-03-13T14:49:48.358 INFO:tasks.workunit.client.0.smithi176.stdout:4/510: write d7/d10/d46/f8f [2042900,72832] 0 2024-03-13T14:49:48.359 INFO:tasks.workunit.client.0.smithi176.stdout:6/490: rename d0/d10/d19/d27/d2b/d2e/f31 to d0/de/d38/d70/f9e 0 2024-03-13T14:49:48.360 INFO:tasks.workunit.client.0.smithi176.stdout:4/511: fsync d7/d10/d2c/fb2 0 2024-03-13T14:49:48.361 INFO:tasks.workunit.client.0.smithi176.stdout:5/452: dread d5/d22/d15/f67 [4194304,4194304] 0 2024-03-13T14:49:48.362 INFO:tasks.workunit.client.0.smithi176.stdout:3/522: symlink d4/d13/lb1 0 2024-03-13T14:49:48.368 INFO:tasks.workunit.client.0.smithi176.stdout:0/426: dwrite d2/d5/dc/f15 [0,4194304] 0 2024-03-13T14:49:48.370 INFO:tasks.workunit.client.0.smithi176.stdout:2/517: creat d1/dc/d2d/d4b/d69/fb4 x:0 0 0 2024-03-13T14:49:48.371 INFO:tasks.workunit.client.1.smithi196.stdout:3/584: fdatasync db/d19/d50/f63 0 2024-03-13T14:49:48.372 INFO:tasks.workunit.client.1.smithi196.stdout:4/563: write d1/d3/d7/d35/f52 [216056,87947] 0 2024-03-13T14:49:48.373 INFO:tasks.workunit.client.1.smithi196.stdout:3/585: fdatasync db/d19/fa2 0 2024-03-13T14:49:48.375 INFO:tasks.workunit.client.0.smithi176.stdout:1/445: dwrite d2/de/d10/d16/d4a/f8a [0,4194304] 0 2024-03-13T14:49:48.375 INFO:tasks.workunit.client.1.smithi196.stdout:2/497: mknod d0/d7/d8e/ca1 0 2024-03-13T14:49:48.377 INFO:tasks.workunit.client.0.smithi176.stdout:6/491: rmdir d0/de/d8d 39 2024-03-13T14:49:48.377 INFO:tasks.workunit.client.0.smithi176.stdout:1/446: read - d2/de/d10/d16/d47/f54 zero size 2024-03-13T14:49:48.378 INFO:tasks.workunit.client.0.smithi176.stdout:9/533: sync 2024-03-13T14:49:48.380 INFO:tasks.workunit.client.1.smithi196.stdout:5/666: getdents d4/d1b/dc8/d14/d37 0 2024-03-13T14:49:48.388 INFO:tasks.workunit.client.0.smithi176.stdout:7/448: truncate d1/d3f/f4d 31370 0 2024-03-13T14:49:48.389 INFO:tasks.workunit.client.1.smithi196.stdout:0/632: dwrite d8/da/d25/d26/d48/d82/f8b [4194304,4194304] 0 2024-03-13T14:49:48.390 INFO:tasks.workunit.client.0.smithi176.stdout:7/449: stat d1/d1c/d26/d41/d2b/d7b/f7e 0 2024-03-13T14:49:48.391 INFO:tasks.workunit.client.1.smithi196.stdout:6/724: dwrite d6/d7/f95 [0,4194304] 0 2024-03-13T14:49:48.394 INFO:tasks.workunit.client.0.smithi176.stdout:0/427: creat d2/d5/dc/d59/f80 x:0 0 0 2024-03-13T14:49:48.395 INFO:tasks.workunit.client.0.smithi176.stdout:4/512: dread d7/d8/d33/f6d [0,4194304] 0 2024-03-13T14:49:48.397 INFO:tasks.workunit.client.1.smithi196.stdout:0/633: write d8/da/d25/d26/d48/d82/f8b [1082072,76660] 0 2024-03-13T14:49:48.399 INFO:tasks.workunit.client.0.smithi176.stdout:2/518: mknod d1/dc/d40/cb5 0 2024-03-13T14:49:48.400 INFO:tasks.workunit.client.0.smithi176.stdout:2/519: stat d1/dc/d2d/d38/f46 0 2024-03-13T14:49:48.401 INFO:tasks.workunit.client.0.smithi176.stdout:5/453: write d5/d22/f29 [1357955,59055] 0 2024-03-13T14:49:48.401 INFO:tasks.workunit.client.1.smithi196.stdout:4/564: symlink d1/d3/d7/d35/d3e/lbf 0 2024-03-13T14:49:48.405 INFO:tasks.workunit.client.0.smithi176.stdout:5/454: dread d5/d22/d15/f2d [4194304,4194304] 0 2024-03-13T14:49:48.411 INFO:tasks.workunit.client.0.smithi176.stdout:6/492: rename d0/de/d38/d77 to d0/d10/d19/d9f 0 2024-03-13T14:49:48.412 INFO:tasks.workunit.client.1.smithi196.stdout:2/498: dwrite d0/d6/d1d/f51 [0,4194304] 0 2024-03-13T14:49:48.419 INFO:tasks.workunit.client.1.smithi196.stdout:5/667: fdatasync d4/d1b/dc8/d14/d19/f50 0 2024-03-13T14:49:48.419 INFO:tasks.workunit.client.1.smithi196.stdout:1/652: write d1/d9/f9a [521315,64718] 0 2024-03-13T14:49:48.420 INFO:tasks.workunit.client.1.smithi196.stdout:6/725: dwrite d6/d2a/dc/d1c/d34/fbd [0,4194304] 0 2024-03-13T14:49:48.421 INFO:tasks.workunit.client.0.smithi176.stdout:5/455: dwrite d5/d2e/d43/f94 [0,4194304] 0 2024-03-13T14:49:48.422 INFO:tasks.workunit.client.0.smithi176.stdout:5/456: dread - d5/d22/d15/f84 zero size 2024-03-13T14:49:48.423 INFO:tasks.workunit.client.0.smithi176.stdout:3/523: write d4/d13/d81/d89/f32 [2896717,36772] 0 2024-03-13T14:49:48.424 INFO:tasks.workunit.client.1.smithi196.stdout:5/668: write d4/d1b/fc2 [972917,69316] 0 2024-03-13T14:49:48.424 INFO:tasks.workunit.client.1.smithi196.stdout:6/726: write d6/d57/d91/fe0 [375335,44243] 0 2024-03-13T14:49:48.432 INFO:tasks.workunit.client.0.smithi176.stdout:7/450: unlink d1/d1c/d26/d41/d2b/f67 0 2024-03-13T14:49:48.432 INFO:tasks.workunit.client.0.smithi176.stdout:4/513: mkdir d7/d10/d2c/d9d/dbb 0 2024-03-13T14:49:48.439 INFO:tasks.workunit.client.0.smithi176.stdout:2/520: symlink d1/dc/d2d/d59/lb6 0 2024-03-13T14:49:48.440 INFO:tasks.workunit.client.0.smithi176.stdout:2/521: stat d1/dc/d2d/d59/lb6 0 2024-03-13T14:49:48.442 INFO:tasks.workunit.client.0.smithi176.stdout:2/522: chown d1/dc/d2d/d38/d5d/f62 14756043 1 2024-03-13T14:49:48.443 INFO:tasks.workunit.client.0.smithi176.stdout:9/534: rename d0/d40/d53/f65 to d0/db/d89/d32/d76/d82/fb7 0 2024-03-13T14:49:48.444 INFO:tasks.workunit.client.0.smithi176.stdout:2/523: stat d1/d2a/f75 0 2024-03-13T14:49:48.448 INFO:tasks.workunit.client.0.smithi176.stdout:6/493: symlink d0/de/d38/la0 0 2024-03-13T14:49:48.449 INFO:tasks.workunit.client.0.smithi176.stdout:9/535: dread d0/db/f49 [0,4194304] 0 2024-03-13T14:49:48.450 INFO:tasks.workunit.client.0.smithi176.stdout:5/457: unlink d5/d22/f13 0 2024-03-13T14:49:48.450 INFO:tasks.workunit.client.0.smithi176.stdout:3/524: fsync d4/d28/d4f/f90 0 2024-03-13T14:49:48.451 INFO:tasks.workunit.client.0.smithi176.stdout:5/458: readlink d5/d22/l8d 0 2024-03-13T14:49:48.455 INFO:tasks.workunit.client.1.smithi196.stdout:3/586: dwrite db/d19/f3f [0,4194304] 0 2024-03-13T14:49:48.457 INFO:tasks.workunit.client.1.smithi196.stdout:3/587: dread - db/d19/d4a/fb2 zero size 2024-03-13T14:49:48.459 INFO:tasks.workunit.client.1.smithi196.stdout:3/588: readlink db/d19/d22/d3a/d6a/d8d/la8 0 2024-03-13T14:49:48.461 INFO:tasks.workunit.client.0.smithi176.stdout:0/428: rename d2/d74/d61/l7b to d2/d74/d61/l81 0 2024-03-13T14:49:48.466 INFO:tasks.workunit.client.1.smithi196.stdout:6/727: creat d6/d9/d17/d19/d4e/d66/fea x:0 0 0 2024-03-13T14:49:48.467 INFO:tasks.workunit.client.1.smithi196.stdout:1/653: creat d1/d9/d33/d4b/fe0 x:0 0 0 2024-03-13T14:49:48.469 INFO:tasks.workunit.client.0.smithi176.stdout:4/514: creat d7/d8/d15/fbc x:0 0 0 2024-03-13T14:49:48.474 INFO:tasks.workunit.client.1.smithi196.stdout:6/728: write d6/d7/df/d27/d39/d44/d74/fac [61265,63302] 0 2024-03-13T14:49:48.475 INFO:tasks.workunit.client.0.smithi176.stdout:4/515: write d7/d8/f28 [4790674,54911] 0 2024-03-13T14:49:48.475 INFO:tasks.workunit.client.0.smithi176.stdout:4/516: chown d7/d8/d2f/c74 16028 1 2024-03-13T14:49:48.477 INFO:tasks.workunit.client.0.smithi176.stdout:7/451: rename d1/f3c to d1/d1c/d26/d41/d2b/d7b/f83 0 2024-03-13T14:49:48.480 INFO:tasks.workunit.client.0.smithi176.stdout:2/524: symlink d1/d2a/lb7 0 2024-03-13T14:49:48.482 INFO:tasks.workunit.client.0.smithi176.stdout:6/494: mkdir d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1 0 2024-03-13T14:49:48.484 INFO:tasks.workunit.client.1.smithi196.stdout:2/499: write d0/d7/f21 [505448,55615] 0 2024-03-13T14:49:48.487 INFO:tasks.workunit.client.1.smithi196.stdout:6/729: creat d6/d2a/de2/feb x:0 0 0 2024-03-13T14:49:48.488 INFO:tasks.workunit.client.1.smithi196.stdout:0/634: dwrite d8/da/d25/d26/d35/d3b/d5f/d17/fcc [0,4194304] 0 2024-03-13T14:49:48.489 INFO:tasks.workunit.client.0.smithi176.stdout:3/525: creat d4/d13/d16/d3d/fb2 x:0 0 0 2024-03-13T14:49:48.489 INFO:tasks.workunit.client.0.smithi176.stdout:4/517: dwrite d7/d8/d15/d23/fa8 [0,4194304] 0 2024-03-13T14:49:48.491 INFO:tasks.workunit.client.0.smithi176.stdout:3/526: fsync d4/d28/d34/f3f 0 2024-03-13T14:49:48.491 INFO:tasks.workunit.client.1.smithi196.stdout:0/635: chown d8/da/d25/d26/d35/d3b/d5f/d17 57229507 1 2024-03-13T14:49:48.492 INFO:tasks.workunit.client.0.smithi176.stdout:4/518: dread - d7/d8/d2f/fb4 zero size 2024-03-13T14:49:48.498 INFO:tasks.workunit.client.0.smithi176.stdout:4/519: read d7/d10/d46/f4c [165705,129879] 0 2024-03-13T14:49:48.504 INFO:tasks.workunit.client.1.smithi196.stdout:6/730: rename d6/d57/d82/fb7 to d6/d6f/fec 0 2024-03-13T14:49:48.504 INFO:tasks.workunit.client.1.smithi196.stdout:2/500: creat d0/d50/fa2 x:0 0 0 2024-03-13T14:49:48.506 INFO:tasks.workunit.client.1.smithi196.stdout:6/731: stat d6/d2a/de2/fd5 0 2024-03-13T14:49:48.507 INFO:tasks.workunit.client.1.smithi196.stdout:3/589: rmdir db/d37/dc3 0 2024-03-13T14:49:48.514 INFO:tasks.workunit.client.1.smithi196.stdout:2/501: chown d0/d7/d9/d42/c91 45 1 2024-03-13T14:49:48.515 INFO:tasks.workunit.client.1.smithi196.stdout:2/502: chown d0/d6 4 1 2024-03-13T14:49:48.517 INFO:tasks.workunit.client.1.smithi196.stdout:3/590: fsync db/d37/d64/f90 0 2024-03-13T14:49:48.518 INFO:tasks.workunit.client.1.smithi196.stdout:2/503: chown d0/d7/d9/d5a/f7c 145697 1 2024-03-13T14:49:48.519 INFO:tasks.workunit.client.1.smithi196.stdout:6/732: symlink d6/d7/df/d27/led 0 2024-03-13T14:49:48.519 INFO:tasks.workunit.client.1.smithi196.stdout:0/636: write d8/da/d25/d26/d54/fa7 [4956266,43809] 0 2024-03-13T14:49:48.520 INFO:tasks.workunit.client.0.smithi176.stdout:8/432: sync 2024-03-13T14:49:48.521 INFO:tasks.workunit.client.1.smithi196.stdout:1/654: dwrite d1/d9/d4e/d55/d63/da6/fb0 [0,4194304] 0 2024-03-13T14:49:48.533 INFO:tasks.workunit.client.1.smithi196.stdout:1/655: mknod d1/d9/d4e/d55/d63/db9/dbb/ce1 0 2024-03-13T14:49:48.534 INFO:tasks.workunit.client.0.smithi176.stdout:1/447: getdents d2/de/d3f/d19 0 2024-03-13T14:49:48.539 INFO:tasks.workunit.client.1.smithi196.stdout:3/591: dread db/f35 [0,4194304] 0 2024-03-13T14:49:48.540 INFO:tasks.workunit.client.0.smithi176.stdout:1/448: write d2/de/d10/d16/d40/f77 [742217,23767] 0 2024-03-13T14:49:48.541 INFO:tasks.workunit.client.1.smithi196.stdout:3/592: write db/f2d [4836096,31134] 0 2024-03-13T14:49:48.544 INFO:tasks.workunit.client.1.smithi196.stdout:0/637: rmdir d8/da/d25/d26/d48/d82/dd0 0 2024-03-13T14:49:48.544 INFO:tasks.workunit.client.0.smithi176.stdout:8/433: dwrite d0/db/f25 [0,4194304] 0 2024-03-13T14:49:48.547 INFO:tasks.workunit.client.0.smithi176.stdout:8/434: truncate d0/d5/d3b/f81 307560 0 2024-03-13T14:49:48.547 INFO:tasks.workunit.client.0.smithi176.stdout:9/536: unlink d0/db/d89/d4d/d6e/d81/cb6 0 2024-03-13T14:49:48.548 INFO:tasks.workunit.client.1.smithi196.stdout:2/504: link d0/d7/d9/l2b d0/d7/la3 0 2024-03-13T14:49:48.550 INFO:tasks.workunit.client.0.smithi176.stdout:9/537: write d0/d17/f3a [3535985,69988] 0 2024-03-13T14:49:48.550 INFO:tasks.workunit.client.1.smithi196.stdout:3/593: creat db/d19/d4a/fc4 x:0 0 0 2024-03-13T14:49:48.551 INFO:tasks.workunit.client.1.smithi196.stdout:6/733: write d6/d7/df/f9c [183043,2052] 0 2024-03-13T14:49:48.553 INFO:tasks.workunit.client.1.smithi196.stdout:1/656: dwrite d1/d3f/d5b/fb8 [0,4194304] 0 2024-03-13T14:49:48.554 INFO:tasks.workunit.client.1.smithi196.stdout:6/734: write d6/d2a/de2/fbe [238491,120900] 0 2024-03-13T14:49:48.557 INFO:tasks.workunit.client.1.smithi196.stdout:2/505: write d0/f77 [6589113,46205] 0 2024-03-13T14:49:48.565 INFO:tasks.workunit.client.0.smithi176.stdout:4/520: fsync d7/d8/d2f/fb4 0 2024-03-13T14:49:48.565 INFO:tasks.workunit.client.1.smithi196.stdout:6/735: rmdir d6/d9/d11 39 2024-03-13T14:49:48.566 INFO:tasks.workunit.client.0.smithi176.stdout:4/521: chown d7/d10/d2c/c6e 63 1 2024-03-13T14:49:48.567 INFO:tasks.workunit.client.1.smithi196.stdout:2/506: creat d0/d6/d90/fa4 x:0 0 0 2024-03-13T14:49:48.568 INFO:tasks.workunit.client.0.smithi176.stdout:5/459: truncate d5/d2e/d43/f50 1844925 0 2024-03-13T14:49:48.569 INFO:tasks.workunit.client.0.smithi176.stdout:4/522: chown d7/d2d/da7/db3 1622 1 2024-03-13T14:49:48.570 INFO:tasks.workunit.client.1.smithi196.stdout:1/657: symlink d1/d9/d4e/dcc/le2 0 2024-03-13T14:49:48.571 INFO:tasks.workunit.client.1.smithi196.stdout:6/736: mkdir d6/d9/d17/d19/d5f/dee 0 2024-03-13T14:49:48.573 INFO:tasks.workunit.client.1.smithi196.stdout:2/507: readlink d0/d7/la3 0 2024-03-13T14:49:48.577 INFO:tasks.workunit.client.0.smithi176.stdout:8/435: rmdir d0/d5/d3b 39 2024-03-13T14:49:48.577 INFO:tasks.workunit.client.0.smithi176.stdout:6/495: dread d0/d10/d19/f59 [0,4194304] 0 2024-03-13T14:49:48.579 INFO:tasks.workunit.client.0.smithi176.stdout:8/436: fdatasync d0/d5/f87 0 2024-03-13T14:49:48.580 INFO:tasks.workunit.client.0.smithi176.stdout:6/496: chown d0/d10/d19/d27/f4a 63698741 1 2024-03-13T14:49:48.583 INFO:tasks.workunit.client.0.smithi176.stdout:6/497: chown d0/d10/d19/d9f/d76/c8f 53145265 1 2024-03-13T14:49:48.584 INFO:tasks.workunit.client.1.smithi196.stdout:2/508: rename d0/d7/d9/fa0 to d0/d7/d8e/fa5 0 2024-03-13T14:49:48.586 INFO:tasks.workunit.client.0.smithi176.stdout:0/429: dwrite d2/f11 [0,4194304] 0 2024-03-13T14:49:48.588 INFO:tasks.workunit.client.1.smithi196.stdout:3/594: dwrite db/f36 [0,4194304] 0 2024-03-13T14:49:48.590 INFO:tasks.workunit.client.0.smithi176.stdout:3/527: link d4/d28/d31/f62 d4/d28/d31/d45/d82/fb3 0 2024-03-13T14:49:48.591 INFO:tasks.workunit.client.1.smithi196.stdout:3/595: write db/d19/d4a/fc4 [965261,43322] 0 2024-03-13T14:49:48.592 INFO:tasks.workunit.client.1.smithi196.stdout:6/737: mknod d6/d9/d17/d19/d4e/cef 0 2024-03-13T14:49:48.593 INFO:tasks.workunit.client.1.smithi196.stdout:3/596: fdatasync db/f2d 0 2024-03-13T14:49:48.594 INFO:tasks.workunit.client.0.smithi176.stdout:2/525: write d1/d2a/f82 [116777,95735] 0 2024-03-13T14:49:48.598 INFO:tasks.workunit.client.0.smithi176.stdout:6/498: dread d0/d10/d19/d9f/f74 [0,4194304] 0 2024-03-13T14:49:48.600 INFO:tasks.workunit.client.1.smithi196.stdout:1/658: write d1/d28/d34/f9c [25943,79455] 0 2024-03-13T14:49:48.603 INFO:tasks.workunit.client.0.smithi176.stdout:1/449: dwrite d2/de/d3f/f52 [4194304,4194304] 0 2024-03-13T14:49:48.606 INFO:tasks.workunit.client.0.smithi176.stdout:1/450: chown d2/lb 968865997 1 2024-03-13T14:49:48.614 INFO:tasks.workunit.client.0.smithi176.stdout:7/452: rename d1/d1c/d26/d41/d2b/d40/l70 to d1/d28/d3a/l84 0 2024-03-13T14:49:48.618 INFO:tasks.workunit.client.1.smithi196.stdout:6/738: dwrite d6/d9/d17/d19/d4e/fdf [0,4194304] 0 2024-03-13T14:49:48.621 INFO:tasks.workunit.client.0.smithi176.stdout:2/526: creat d1/dc/d61/daa/fb8 x:0 0 0 2024-03-13T14:49:48.625 INFO:tasks.workunit.client.1.smithi196.stdout:2/509: dread d0/d7/d66/d67/f85 [0,4194304] 0 2024-03-13T14:49:48.629 INFO:tasks.workunit.client.0.smithi176.stdout:9/538: write d0/db/f3f [310013,32899] 0 2024-03-13T14:49:48.631 INFO:tasks.workunit.client.0.smithi176.stdout:4/523: creat d7/d8/fbd x:0 0 0 2024-03-13T14:49:48.632 INFO:tasks.workunit.client.0.smithi176.stdout:5/460: link d5/d48/d4c/f65 d5/d31/d86/f99 0 2024-03-13T14:49:48.634 INFO:tasks.workunit.client.0.smithi176.stdout:5/461: stat d5/d2b/f30 0 2024-03-13T14:49:48.634 INFO:tasks.workunit.client.1.smithi196.stdout:1/659: truncate d1/d3f/d5b/d74/f8f 885556 0 2024-03-13T14:49:48.637 INFO:tasks.workunit.client.0.smithi176.stdout:0/430: creat d2/f82 x:0 0 0 2024-03-13T14:49:48.638 INFO:tasks.workunit.client.1.smithi196.stdout:1/660: chown d1/d9/d33/d4b/d66/d7b/fa9 111319776 1 2024-03-13T14:49:48.638 INFO:tasks.workunit.client.1.smithi196.stdout:3/597: dwrite db/f35 [0,4194304] 0 2024-03-13T14:49:48.639 INFO:tasks.workunit.client.0.smithi176.stdout:3/528: creat d4/d13/d81/d89/fb4 x:0 0 0 2024-03-13T14:49:48.640 INFO:tasks.workunit.client.0.smithi176.stdout:6/499: link d0/d10/d19/d27/d2b/d2e/c92 d0/de/d54/d99/ca2 0 2024-03-13T14:49:48.643 INFO:tasks.workunit.client.0.smithi176.stdout:8/437: rename d0/db/c66 to d0/d24/d29/c88 0 2024-03-13T14:49:48.646 INFO:tasks.workunit.client.0.smithi176.stdout:8/438: chown d0/db/d39/d56/f5e 1195963 1 2024-03-13T14:49:48.655 INFO:tasks.workunit.client.1.smithi196.stdout:6/739: write d6/d9/d17/da1/d50/f5a [3370047,74207] 0 2024-03-13T14:49:48.656 INFO:tasks.workunit.client.0.smithi176.stdout:0/431: readlink d2/d5/d1e/l45 0 2024-03-13T14:49:48.659 INFO:tasks.workunit.client.1.smithi196.stdout:6/740: chown d6/d7/df/d27 1 1 2024-03-13T14:49:48.660 INFO:tasks.workunit.client.0.smithi176.stdout:3/529: creat d4/d13/d81/fb5 x:0 0 0 2024-03-13T14:49:48.662 INFO:tasks.workunit.client.1.smithi196.stdout:9/534: sync 2024-03-13T14:49:48.664 INFO:tasks.workunit.client.0.smithi176.stdout:6/500: unlink d0/d10/d19/f59 0 2024-03-13T14:49:48.664 INFO:tasks.workunit.client.0.smithi176.stdout:9/539: dwrite d0/db/fae [0,4194304] 0 2024-03-13T14:49:48.665 INFO:tasks.workunit.client.1.smithi196.stdout:2/510: truncate d0/d7/f94 3773685 0 2024-03-13T14:49:48.665 INFO:tasks.workunit.client.1.smithi196.stdout:9/535: dread - d7/d9/d83/f9f zero size 2024-03-13T14:49:48.667 INFO:tasks.workunit.client.1.smithi196.stdout:6/741: mknod d6/d7/df/cf0 0 2024-03-13T14:49:48.667 INFO:tasks.workunit.client.1.smithi196.stdout:2/511: chown d0/d6/d1d/f51 0 1 2024-03-13T14:49:48.673 INFO:tasks.workunit.client.1.smithi196.stdout:1/661: getdents d1/d9/d33 0 2024-03-13T14:49:48.674 INFO:tasks.workunit.client.0.smithi176.stdout:8/439: dwrite d0/d24/d29/f67 [0,4194304] 0 2024-03-13T14:49:48.682 INFO:tasks.workunit.client.0.smithi176.stdout:1/451: getdents d2/de/d10/d16/d47 0 2024-03-13T14:49:48.684 INFO:tasks.workunit.client.0.smithi176.stdout:0/432: dwrite d2/d5/dc/d59/f80 [0,4194304] 0 2024-03-13T14:49:48.685 INFO:tasks.workunit.client.0.smithi176.stdout:9/540: dread d0/db/d89/d32/f62 [0,4194304] 0 2024-03-13T14:49:48.686 INFO:tasks.workunit.client.0.smithi176.stdout:3/530: mkdir d4/d13/d16/db6 0 2024-03-13T14:49:48.687 INFO:tasks.workunit.client.1.smithi196.stdout:9/536: dread d7/de/d4b/d1d/d22/f53 [0,4194304] 0 2024-03-13T14:49:48.687 INFO:tasks.workunit.client.1.smithi196.stdout:3/598: dwrite db/f49 [0,4194304] 0 2024-03-13T14:49:48.692 INFO:tasks.workunit.client.1.smithi196.stdout:2/512: rename d0/d6/d4b/c6b to d0/d7/ca6 0 2024-03-13T14:49:48.698 INFO:tasks.workunit.client.1.smithi196.stdout:2/513: truncate d0/d6/d90/fa4 581799 0 2024-03-13T14:49:48.700 INFO:tasks.workunit.client.0.smithi176.stdout:6/501: mkdir d0/de/d38/d6b/da3 0 2024-03-13T14:49:48.702 INFO:tasks.workunit.client.1.smithi196.stdout:9/537: rmdir d7/d9/d17 39 2024-03-13T14:49:48.703 INFO:tasks.workunit.client.0.smithi176.stdout:1/452: dread d2/d30/f65 [0,4194304] 0 2024-03-13T14:49:48.704 INFO:tasks.workunit.client.0.smithi176.stdout:4/524: write d7/d8/f39 [1622024,77033] 0 2024-03-13T14:49:48.704 INFO:tasks.workunit.client.1.smithi196.stdout:1/662: dwrite d1/d9/d4e/d55/fbf [0,4194304] 0 2024-03-13T14:49:48.709 INFO:tasks.workunit.client.0.smithi176.stdout:5/462: link d5/d31/d47/d80/f4d d5/f9a 0 2024-03-13T14:49:48.709 INFO:tasks.workunit.client.0.smithi176.stdout:6/502: chown d0/d10/f3f 1548544 1 2024-03-13T14:49:48.710 INFO:tasks.workunit.client.1.smithi196.stdout:9/538: chown d7/d9/c2f 5004449 1 2024-03-13T14:49:48.712 INFO:tasks.workunit.client.0.smithi176.stdout:2/527: getdents d1/dc 0 2024-03-13T14:49:48.713 INFO:tasks.workunit.client.1.smithi196.stdout:2/514: truncate d0/f78 486921 0 2024-03-13T14:49:48.716 INFO:tasks.workunit.client.0.smithi176.stdout:7/453: rename d1/d1c/d26/d41/f5a to d1/d1c/d26/d41/d2b/f85 0 2024-03-13T14:49:48.717 INFO:tasks.workunit.client.0.smithi176.stdout:3/531: chown d4/d13/f61 12574275 1 2024-03-13T14:49:48.721 INFO:tasks.workunit.client.1.smithi196.stdout:3/599: dwrite db/d15/d44/d79/fa7 [0,4194304] 0 2024-03-13T14:49:48.722 INFO:tasks.workunit.client.0.smithi176.stdout:7/454: chown d1/d3f/d4b/l68 3766 1 2024-03-13T14:49:48.723 INFO:tasks.workunit.client.0.smithi176.stdout:0/433: dwrite d2/f6 [8388608,4194304] 0 2024-03-13T14:49:48.726 INFO:tasks.workunit.client.0.smithi176.stdout:3/532: read d4/f1d [75701,4122] 0 2024-03-13T14:49:48.726 INFO:tasks.workunit.client.0.smithi176.stdout:4/525: fsync d7/d8/d15/d23/f35 0 2024-03-13T14:49:48.728 INFO:tasks.workunit.client.1.smithi196.stdout:2/515: creat d0/d6/d4b/d96/fa7 x:0 0 0 2024-03-13T14:49:48.729 INFO:tasks.workunit.client.0.smithi176.stdout:3/533: fdatasync d4/d13/d2d/d50/f3c 0 2024-03-13T14:49:48.731 INFO:tasks.workunit.client.1.smithi196.stdout:9/539: fdatasync d7/d9/f77 0 2024-03-13T14:49:48.734 INFO:tasks.workunit.client.0.smithi176.stdout:1/453: symlink d2/d29/l8d 0 2024-03-13T14:49:48.735 INFO:tasks.workunit.client.1.smithi196.stdout:9/540: dread - d7/d71/f9e zero size 2024-03-13T14:49:48.743 INFO:tasks.workunit.client.1.smithi196.stdout:3/600: write db/d37/d64/f74 [909783,129141] 0 2024-03-13T14:49:48.744 INFO:tasks.workunit.client.0.smithi176.stdout:9/541: dread d0/db/d89/d32/d58/d5f/f75 [0,4194304] 0 2024-03-13T14:49:48.746 INFO:tasks.workunit.client.0.smithi176.stdout:8/440: dwrite d0/d5/d11/f1b [0,4194304] 0 2024-03-13T14:49:48.746 INFO:tasks.workunit.client.0.smithi176.stdout:9/542: readlink d0/d17/l1a 0 2024-03-13T14:49:48.747 INFO:tasks.workunit.client.0.smithi176.stdout:9/543: chown d0/db/d89/d90/fb0 719 1 2024-03-13T14:49:48.751 INFO:tasks.workunit.client.1.smithi196.stdout:2/516: creat d0/d7/d9/d42/d98/fa8 x:0 0 0 2024-03-13T14:49:48.753 INFO:tasks.workunit.client.0.smithi176.stdout:5/463: rename d5/d31/l93 to d5/d75/l9b 0 2024-03-13T14:49:48.754 INFO:tasks.workunit.client.0.smithi176.stdout:7/455: mknod d1/d1c/d26/d41/d2b/d7b/c86 0 2024-03-13T14:49:48.755 INFO:tasks.workunit.client.1.smithi196.stdout:2/517: fsync d0/d6/d4b/f7d 0 2024-03-13T14:49:48.755 INFO:tasks.workunit.client.0.smithi176.stdout:4/526: dread - d7/d10/f85 zero size 2024-03-13T14:49:48.757 INFO:tasks.workunit.client.0.smithi176.stdout:4/527: write d7/d10/d46/faf [521141,89818] 0 2024-03-13T14:49:48.757 INFO:tasks.workunit.client.1.smithi196.stdout:1/663: dread f0 [0,4194304] 0 2024-03-13T14:49:48.757 INFO:tasks.workunit.client.0.smithi176.stdout:3/534: creat d4/d13/d16/d3d/fb7 x:0 0 0 2024-03-13T14:49:48.759 INFO:tasks.workunit.client.1.smithi196.stdout:1/664: readlink d1/lba 0 2024-03-13T14:49:48.764 INFO:tasks.workunit.client.1.smithi196.stdout:1/665: dread - d1/d28/da3/fdc zero size 2024-03-13T14:49:48.764 INFO:tasks.workunit.client.0.smithi176.stdout:9/544: mkdir d0/d73/db8 0 2024-03-13T14:49:48.765 INFO:tasks.workunit.client.1.smithi196.stdout:3/601: dwrite db/f35 [0,4194304] 0 2024-03-13T14:49:48.766 INFO:tasks.workunit.client.0.smithi176.stdout:9/545: read d0/d1e/d1f/f41 [2628496,26256] 0 2024-03-13T14:49:48.767 INFO:tasks.workunit.client.0.smithi176.stdout:0/434: symlink d2/db/l83 0 2024-03-13T14:49:48.770 INFO:tasks.workunit.client.0.smithi176.stdout:2/528: rename d1/ff to d1/dc/d2d/d59/d80/d7e/da3/fb9 0 2024-03-13T14:49:48.773 INFO:tasks.workunit.client.0.smithi176.stdout:7/456: mknod d1/d7/c87 0 2024-03-13T14:49:48.788 INFO:tasks.workunit.client.0.smithi176.stdout:2/529: mknod d1/dc/d61/cba 0 2024-03-13T14:49:48.790 INFO:tasks.workunit.client.0.smithi176.stdout:2/530: truncate d1/dc/d2d/d4b/d69/fb4 952542 0 2024-03-13T14:49:48.792 INFO:tasks.workunit.client.0.smithi176.stdout:2/531: write d1/dc/d2d/f2f [2900904,102271] 0 2024-03-13T14:49:48.794 INFO:tasks.workunit.client.1.smithi196.stdout:8/548: sync 2024-03-13T14:49:48.795 INFO:tasks.workunit.client.1.smithi196.stdout:5/669: sync 2024-03-13T14:49:48.795 INFO:tasks.workunit.client.1.smithi196.stdout:0/638: sync 2024-03-13T14:49:48.795 INFO:tasks.workunit.client.1.smithi196.stdout:4/565: sync 2024-03-13T14:49:48.795 INFO:tasks.workunit.client.1.smithi196.stdout:7/699: sync 2024-03-13T14:49:48.795 INFO:tasks.workunit.client.0.smithi176.stdout:4/528: mknod d7/d8/cbe 0 2024-03-13T14:49:48.797 INFO:tasks.workunit.client.0.smithi176.stdout:4/529: truncate d7/d8/d45/f76 776567 0 2024-03-13T14:49:48.800 INFO:tasks.workunit.client.1.smithi196.stdout:9/541: symlink d7/de/d4b/lb3 0 2024-03-13T14:49:48.803 INFO:tasks.workunit.client.0.smithi176.stdout:8/441: write d0/f30 [1364108,16534] 0 2024-03-13T14:49:48.805 INFO:tasks.workunit.client.0.smithi176.stdout:9/546: unlink d0/lab 0 2024-03-13T14:49:48.806 INFO:tasks.workunit.client.0.smithi176.stdout:3/535: dread d4/d28/d34/f9e [0,4194304] 0 2024-03-13T14:49:48.806 INFO:tasks.workunit.client.0.smithi176.stdout:8/442: chown d0/f5c 4853496 1 2024-03-13T14:49:48.807 INFO:tasks.workunit.client.0.smithi176.stdout:7/457: symlink d1/d28/d3a/l88 0 2024-03-13T14:49:48.808 INFO:tasks.workunit.client.1.smithi196.stdout:1/666: symlink d1/d28/le3 0 2024-03-13T14:49:48.810 INFO:tasks.workunit.client.1.smithi196.stdout:1/667: read - d1/d9/d4e/d55/fc3 zero size 2024-03-13T14:49:48.810 INFO:tasks.workunit.client.1.smithi196.stdout:0/639: dread d8/da/d25/d26/d35/d3b/d50/f6d [0,4194304] 0 2024-03-13T14:49:48.811 INFO:tasks.workunit.client.0.smithi176.stdout:4/530: dwrite d7/d2d/da7/fae [0,4194304] 0 2024-03-13T14:49:48.811 INFO:tasks.workunit.client.0.smithi176.stdout:2/532: truncate d1/f54 613152 0 2024-03-13T14:49:48.813 INFO:tasks.workunit.client.0.smithi176.stdout:5/464: truncate d5/d22/d15/f67 4502980 0 2024-03-13T14:49:48.814 INFO:tasks.workunit.client.0.smithi176.stdout:2/533: write d1/dc/d2d/d4b/d69/fb4 [295949,33604] 0 2024-03-13T14:49:48.814 INFO:tasks.workunit.client.0.smithi176.stdout:6/503: getdents d0/de/d8d 0 2024-03-13T14:49:48.817 INFO:tasks.workunit.client.0.smithi176.stdout:2/534: chown d1/dc/fa5 13 1 2024-03-13T14:49:48.818 INFO:tasks.workunit.client.1.smithi196.stdout:5/670: mknod d4/d1b/dc8/d14/d19/d26/ccb 0 2024-03-13T14:49:48.819 INFO:tasks.workunit.client.0.smithi176.stdout:1/454: rename d2/de/d10/d16/d4a/d5f/f64 to d2/de/d3f/d50/f8e 0 2024-03-13T14:49:48.819 INFO:tasks.workunit.client.0.smithi176.stdout:2/535: readlink d1/d2a/d33/l47 0 2024-03-13T14:49:48.822 INFO:tasks.workunit.client.1.smithi196.stdout:6/742: sync 2024-03-13T14:49:48.824 INFO:tasks.workunit.client.1.smithi196.stdout:6/743: chown d6/c1b 14564 1 2024-03-13T14:49:48.827 INFO:tasks.workunit.client.1.smithi196.stdout:8/549: dread d0/d16/d3d/d8b/d3f/f76 [0,4194304] 0 2024-03-13T14:49:48.830 INFO:tasks.workunit.client.1.smithi196.stdout:1/668: dwrite d1/d3f/d5b/fb8 [0,4194304] 0 2024-03-13T14:49:48.832 INFO:tasks.workunit.client.0.smithi176.stdout:3/536: mkdir d4/db8 0 2024-03-13T14:49:48.832 INFO:tasks.workunit.client.0.smithi176.stdout:9/547: mknod d0/d40/cb9 0 2024-03-13T14:49:48.833 INFO:tasks.workunit.client.0.smithi176.stdout:2/536: chown d1/dc/d1c/c87 6866957 1 2024-03-13T14:49:48.834 INFO:tasks.workunit.client.1.smithi196.stdout:2/518: creat d0/d6/d4b/d96/d9e/fa9 x:0 0 0 2024-03-13T14:49:48.835 INFO:tasks.workunit.client.0.smithi176.stdout:5/465: symlink d5/d48/l9c 0 2024-03-13T14:49:48.835 INFO:tasks.workunit.client.0.smithi176.stdout:9/548: write d0/d1e/d6f/f7b [2770261,13807] 0 2024-03-13T14:49:48.836 INFO:tasks.workunit.client.1.smithi196.stdout:3/602: truncate db/f2d 1312800 0 2024-03-13T14:49:48.837 INFO:tasks.workunit.client.0.smithi176.stdout:5/466: readlink d5/d22/l18 0 2024-03-13T14:49:48.838 INFO:tasks.workunit.client.0.smithi176.stdout:6/504: mknod d0/de/d8d/ca4 0 2024-03-13T14:49:48.840 INFO:tasks.workunit.client.0.smithi176.stdout:5/467: readlink d5/d48/l9c 0 2024-03-13T14:49:48.842 INFO:tasks.workunit.client.0.smithi176.stdout:8/443: unlink d0/l83 0 2024-03-13T14:49:48.844 INFO:tasks.workunit.client.0.smithi176.stdout:3/537: dread d4/d13/d2d/d50/f3c [0,4194304] 0 2024-03-13T14:49:48.846 INFO:tasks.workunit.client.0.smithi176.stdout:3/538: write d4/d13/d16/d3d/f54 [1271832,83442] 0 2024-03-13T14:49:48.853 INFO:tasks.workunit.client.1.smithi196.stdout:7/700: dwrite d0/d6/f2c [0,4194304] 0 2024-03-13T14:49:48.858 INFO:tasks.workunit.client.1.smithi196.stdout:9/542: creat d7/d24/d28/d6c/d7b/fb4 x:0 0 0 2024-03-13T14:49:48.858 INFO:tasks.workunit.client.1.smithi196.stdout:8/550: dwrite d0/d16/d3d/d8b/d48/f5a [4194304,4194304] 0 2024-03-13T14:49:48.862 INFO:tasks.workunit.client.1.smithi196.stdout:9/543: dread - d7/de/d4b/d1d/d22/f88 zero size 2024-03-13T14:49:48.867 INFO:tasks.workunit.client.1.smithi196.stdout:0/640: dwrite d8/da/fc [0,4194304] 0 2024-03-13T14:49:48.873 INFO:tasks.workunit.client.1.smithi196.stdout:6/744: write d6/d7/df/d27/f9d [4279621,37826] 0 2024-03-13T14:49:48.875 INFO:tasks.workunit.client.1.smithi196.stdout:3/603: mkdir db/d19/d4a/dc5 0 2024-03-13T14:49:48.882 INFO:tasks.workunit.client.1.smithi196.stdout:5/671: symlink d4/d1b/dc8/d14/d19/d26/d33/d61/lcc 0 2024-03-13T14:49:48.884 INFO:tasks.workunit.client.1.smithi196.stdout:4/566: rename d1/d1e/l3c to d1/d3/d7/d35/d76/lc0 0 2024-03-13T14:49:48.886 INFO:tasks.workunit.client.1.smithi196.stdout:4/567: readlink d1/l24 0 2024-03-13T14:49:48.887 INFO:tasks.workunit.client.1.smithi196.stdout:7/701: symlink d0/d9/d1e/d58/d72/d99/leb 0 2024-03-13T14:49:48.889 INFO:tasks.workunit.client.1.smithi196.stdout:1/669: mknod d1/d9/d4e/d55/d63/da6/dc8/ce4 0 2024-03-13T14:49:48.893 INFO:tasks.workunit.client.1.smithi196.stdout:5/672: rename d4/d1b/dc8/d14/d19/f50 to d4/d1b/dc8/d14/d19/d26/dab/fcd 0 2024-03-13T14:49:48.894 INFO:tasks.workunit.client.1.smithi196.stdout:8/551: mknod d0/d10/d12/d1e/db1/cb9 0 2024-03-13T14:49:48.896 INFO:tasks.workunit.client.1.smithi196.stdout:4/568: creat d1/d3/d4c/d72/d7d/d80/d55/fc1 x:0 0 0 2024-03-13T14:49:48.897 INFO:tasks.workunit.client.1.smithi196.stdout:8/552: write d0/d10/d12/d1e/f83 [35744,107422] 0 2024-03-13T14:49:48.898 INFO:tasks.workunit.client.1.smithi196.stdout:9/544: mkdir d7/d24/db5 0 2024-03-13T14:49:48.901 INFO:tasks.workunit.client.1.smithi196.stdout:7/702: creat d0/d9/d1e/d58/d72/fec x:0 0 0 2024-03-13T14:49:48.901 INFO:tasks.workunit.client.1.smithi196.stdout:4/569: readlink d1/d3/d7/d35/d76/d58/d69/d6c/lb3 0 2024-03-13T14:49:48.901 INFO:tasks.workunit.client.1.smithi196.stdout:6/745: symlink d6/d7/lf1 0 2024-03-13T14:49:48.903 INFO:tasks.workunit.client.1.smithi196.stdout:0/641: mknod d8/da/d25/d26/d35/d3b/d5f/cd2 0 2024-03-13T14:49:48.904 INFO:tasks.workunit.client.1.smithi196.stdout:3/604: creat db/d15/d44/d51/fc6 x:0 0 0 2024-03-13T14:49:48.909 INFO:tasks.workunit.client.1.smithi196.stdout:9/545: creat d7/de/d4b/d1d/d93/da7/fb6 x:0 0 0 2024-03-13T14:49:48.910 INFO:tasks.workunit.client.1.smithi196.stdout:2/519: truncate d0/d7/d40/f36 509248 0 2024-03-13T14:49:48.915 INFO:tasks.workunit.client.1.smithi196.stdout:7/703: fdatasync d0/d9/f24 0 2024-03-13T14:49:48.917 INFO:tasks.workunit.client.1.smithi196.stdout:4/570: symlink d1/d3/lc2 0 2024-03-13T14:49:48.917 INFO:tasks.workunit.client.1.smithi196.stdout:9/546: mknod d7/d24/d28/d6c/d7b/cb7 0 2024-03-13T14:49:48.919 INFO:tasks.workunit.client.1.smithi196.stdout:2/520: mkdir d0/d7/d9/d9a/daa 0 2024-03-13T14:49:48.922 INFO:tasks.workunit.client.1.smithi196.stdout:5/673: dread d4/d1b/dc8/d4a/fa3 [0,4194304] 0 2024-03-13T14:49:48.923 INFO:tasks.workunit.client.1.smithi196.stdout:0/642: unlink d8/cad 0 2024-03-13T14:49:48.926 INFO:tasks.workunit.client.1.smithi196.stdout:8/553: rename d0/d10/f21 to d0/d1c/d29/fba 0 2024-03-13T14:49:48.932 INFO:tasks.workunit.client.1.smithi196.stdout:4/571: dread d1/d3/f3a [0,4194304] 0 2024-03-13T14:49:48.932 INFO:tasks.workunit.client.1.smithi196.stdout:1/670: getdents d1/d9/d4e/d55 0 2024-03-13T14:49:48.936 INFO:tasks.workunit.client.1.smithi196.stdout:1/671: stat d1/d3f/d5b/d74/db5/fd5 0 2024-03-13T14:49:48.939 INFO:tasks.workunit.client.1.smithi196.stdout:6/746: write d6/f8 [301130,31565] 0 2024-03-13T14:49:48.941 INFO:tasks.workunit.client.1.smithi196.stdout:1/672: dread - d1/d3f/d5b/d74/fd3 zero size 2024-03-13T14:49:48.948 INFO:tasks.workunit.client.1.smithi196.stdout:7/704: truncate d0/d6/f2c 973581 0 2024-03-13T14:49:48.953 INFO:tasks.workunit.client.1.smithi196.stdout:7/705: readlink d0/d9/d1e/d58/d72/l83 0 2024-03-13T14:49:48.955 INFO:tasks.workunit.client.1.smithi196.stdout:1/673: rmdir d1/d3f 39 2024-03-13T14:49:48.955 INFO:tasks.workunit.client.1.smithi196.stdout:3/605: dwrite db/d19/d22/d3a/d53/f5c [0,4194304] 0 2024-03-13T14:49:48.956 INFO:tasks.workunit.client.1.smithi196.stdout:4/572: mkdir d1/d3/d7/d35/d3e/db1/dc3 0 2024-03-13T14:49:48.960 INFO:tasks.workunit.client.1.smithi196.stdout:6/747: mknod d6/d7/dd/dba/cf2 0 2024-03-13T14:49:48.961 INFO:tasks.workunit.client.1.smithi196.stdout:0/643: creat d8/da/d25/d26/d35/d3b/d50/fd3 x:0 0 0 2024-03-13T14:49:48.964 INFO:tasks.workunit.client.1.smithi196.stdout:6/748: readlink d6/d7/dd/l61 0 2024-03-13T14:49:48.965 INFO:tasks.workunit.client.1.smithi196.stdout:7/706: creat d0/d9/d31/d50/fed x:0 0 0 2024-03-13T14:49:48.966 INFO:tasks.workunit.client.1.smithi196.stdout:9/547: dwrite d7/f69 [0,4194304] 0 2024-03-13T14:49:48.966 INFO:tasks.workunit.client.1.smithi196.stdout:6/749: fdatasync d6/f8 0 2024-03-13T14:49:48.967 INFO:tasks.workunit.client.1.smithi196.stdout:4/573: mknod d1/d1e/d5c/cc4 0 2024-03-13T14:49:48.969 INFO:tasks.workunit.client.1.smithi196.stdout:8/554: dread d0/d10/f11 [0,4194304] 0 2024-03-13T14:49:48.970 INFO:tasks.workunit.client.1.smithi196.stdout:3/606: readlink db/d15/d44/d51/la6 0 2024-03-13T14:49:48.971 INFO:tasks.workunit.client.1.smithi196.stdout:1/674: rmdir d1/d9/d4e/d55/d63/da6/dc8 39 2024-03-13T14:49:48.972 INFO:tasks.workunit.client.1.smithi196.stdout:6/750: write d6/d9/d17/d19/d4e/f8a [3092879,74651] 0 2024-03-13T14:49:48.978 INFO:tasks.workunit.client.1.smithi196.stdout:1/675: dread - d1/d9/d33/f9d zero size 2024-03-13T14:49:48.979 INFO:tasks.workunit.client.1.smithi196.stdout:9/548: rename d7/de/d4b/f19 to d7/de/d74/d94/fb8 0 2024-03-13T14:49:48.981 INFO:tasks.workunit.client.1.smithi196.stdout:2/521: dread d0/d7/d9/f18 [0,4194304] 0 2024-03-13T14:49:48.981 INFO:tasks.workunit.client.1.smithi196.stdout:3/607: mknod db/cc7 0 2024-03-13T14:49:48.991 INFO:tasks.workunit.client.1.smithi196.stdout:7/707: mknod d0/d6/d35/cee 0 2024-03-13T14:49:48.991 INFO:tasks.workunit.client.1.smithi196.stdout:0/644: fsync d8/da/d25/f27 0 2024-03-13T14:49:48.991 INFO:tasks.workunit.client.1.smithi196.stdout:6/751: mknod d6/d2a/dc/d1c/da7/d9e/cf3 0 2024-03-13T14:49:48.991 INFO:tasks.workunit.client.0.smithi176.stdout:1/455: write d2/de/d3f/d3e/f61 [928708,124050] 0 2024-03-13T14:49:48.993 INFO:tasks.workunit.client.0.smithi176.stdout:3/539: dread - d4/d13/d2d/f4d zero size 2024-03-13T14:49:48.996 INFO:tasks.workunit.client.0.smithi176.stdout:0/435: rename d2/d5/dc/f15 to d2/f84 0 2024-03-13T14:49:48.996 INFO:tasks.workunit.client.1.smithi196.stdout:3/608: rmdir db/d19/d22/d3a/d53 39 2024-03-13T14:49:48.997 INFO:tasks.workunit.client.0.smithi176.stdout:3/540: dread - d4/d13/d81/fb5 zero size 2024-03-13T14:49:49.001 INFO:tasks.workunit.client.0.smithi176.stdout:6/505: mknod d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/ca5 0 2024-03-13T14:49:49.004 INFO:tasks.workunit.client.0.smithi176.stdout:6/506: readlink d0/de/l51 0 2024-03-13T14:49:49.004 INFO:tasks.workunit.client.0.smithi176.stdout:0/436: dread d2/db/d39/f47 [0,4194304] 0 2024-03-13T14:49:49.005 INFO:tasks.workunit.client.1.smithi196.stdout:6/752: read d6/d7/dd/d60/f79 [70360,90505] 0 2024-03-13T14:49:49.008 INFO:tasks.workunit.client.1.smithi196.stdout:5/674: dread d4/f11 [0,4194304] 0 2024-03-13T14:49:49.008 INFO:tasks.workunit.client.1.smithi196.stdout:4/574: dwrite d1/d3/d4c/fb8 [0,4194304] 0 2024-03-13T14:49:49.010 INFO:tasks.workunit.client.1.smithi196.stdout:1/676: mknod d1/d9/d6c/ce5 0 2024-03-13T14:49:49.010 INFO:tasks.workunit.client.0.smithi176.stdout:5/468: dwrite f0 [4194304,4194304] 0 2024-03-13T14:49:49.010 INFO:tasks.workunit.client.0.smithi176.stdout:4/531: getdents d7/d2d/da7 0 2024-03-13T14:49:49.011 INFO:tasks.workunit.client.0.smithi176.stdout:8/444: dwrite d0/d5/d3b/f3c [0,4194304] 0 2024-03-13T14:49:49.016 INFO:tasks.workunit.client.1.smithi196.stdout:3/609: fsync db/f3d 0 2024-03-13T14:49:49.019 INFO:tasks.workunit.client.1.smithi196.stdout:9/549: dwrite d7/d24/f38 [0,4194304] 0 2024-03-13T14:49:49.022 INFO:tasks.workunit.client.1.smithi196.stdout:5/675: stat d4/d1b/dc8/d14/d19/d26/d33/c86 0 2024-03-13T14:49:49.022 INFO:tasks.workunit.client.1.smithi196.stdout:4/575: dread - d1/d1e/d43/f92 zero size 2024-03-13T14:49:49.025 INFO:tasks.workunit.client.1.smithi196.stdout:2/522: dread d0/d6/f84 [4194304,4194304] 0 2024-03-13T14:49:49.031 INFO:tasks.workunit.client.0.smithi176.stdout:5/469: dwrite d5/d2b/f46 [0,4194304] 0 2024-03-13T14:49:49.032 INFO:tasks.workunit.client.1.smithi196.stdout:0/645: dwrite d8/da/d25/fd1 [0,4194304] 0 2024-03-13T14:49:49.033 INFO:tasks.workunit.client.1.smithi196.stdout:2/523: read d0/d7/d66/f76 [667201,26894] 0 2024-03-13T14:49:49.036 INFO:tasks.workunit.client.1.smithi196.stdout:6/753: symlink d6/d9/d17/da1/lf4 0 2024-03-13T14:49:49.036 INFO:tasks.workunit.client.1.smithi196.stdout:2/524: read d0/d7/d66/d67/f85 [239685,55023] 0 2024-03-13T14:49:49.037 INFO:tasks.workunit.client.0.smithi176.stdout:0/437: symlink d2/db/d31/l85 0 2024-03-13T14:49:49.037 INFO:tasks.workunit.client.0.smithi176.stdout:4/532: mknod d7/d8/d45/cbf 0 2024-03-13T14:49:49.041 INFO:tasks.workunit.client.1.smithi196.stdout:4/576: dread - d1/d3/d7/d35/d3e/d56/f8a zero size 2024-03-13T14:49:49.041 INFO:tasks.workunit.client.1.smithi196.stdout:9/550: creat d7/de/d4b/d1d/d35/fb9 x:0 0 0 2024-03-13T14:49:49.041 INFO:tasks.workunit.client.1.smithi196.stdout:3/610: truncate db/d19/d22/d3a/d6a/f89 148333 0 2024-03-13T14:49:49.051 INFO:tasks.workunit.client.1.smithi196.stdout:1/677: unlink d1/d9/c3d 0 2024-03-13T14:49:49.051 INFO:tasks.workunit.client.1.smithi196.stdout:8/555: dread d0/d10/d12/d17/d42/f4c [0,4194304] 0 2024-03-13T14:49:49.058 INFO:tasks.workunit.client.1.smithi196.stdout:5/676: rename d4/d60/lbb to d4/d1b/dc8/d14/d19/d26/d68/lce 0 2024-03-13T14:49:49.060 INFO:tasks.workunit.client.1.smithi196.stdout:6/754: creat d6/d6f/ff5 x:0 0 0 2024-03-13T14:49:49.061 INFO:tasks.workunit.client.0.smithi176.stdout:5/470: creat d5/d31/d86/f9d x:0 0 0 2024-03-13T14:49:49.064 INFO:tasks.workunit.client.1.smithi196.stdout:2/525: unlink d0/d50/l55 0 2024-03-13T14:49:49.067 INFO:tasks.workunit.client.0.smithi176.stdout:0/438: mknod d2/db/d35/c86 0 2024-03-13T14:49:49.068 INFO:tasks.workunit.client.0.smithi176.stdout:9/549: write d0/db/d89/d32/d76/d82/fb7 [559658,112228] 0 2024-03-13T14:49:49.070 INFO:tasks.workunit.client.1.smithi196.stdout:9/551: mknod d7/d9/d83/cba 0 2024-03-13T14:49:49.076 INFO:tasks.workunit.client.1.smithi196.stdout:4/577: mknod d1/d3/d4c/d72/d7d/cc5 0 2024-03-13T14:49:49.079 INFO:tasks.workunit.client.1.smithi196.stdout:8/556: dwrite d0/f51 [0,4194304] 0 2024-03-13T14:49:49.080 INFO:tasks.workunit.client.1.smithi196.stdout:7/708: write d0/d6/d33/f3c [39445,124027] 0 2024-03-13T14:49:49.082 INFO:tasks.workunit.client.0.smithi176.stdout:3/541: write d4/d28/d31/d52/f6a [443603,49849] 0 2024-03-13T14:49:49.083 INFO:tasks.workunit.client.0.smithi176.stdout:0/439: chown d2/d5/dc/f49 1893918 1 2024-03-13T14:49:49.087 INFO:tasks.workunit.client.0.smithi176.stdout:6/507: getdents d0 0 2024-03-13T14:49:49.088 INFO:tasks.workunit.client.1.smithi196.stdout:6/755: dwrite d6/d2a/d23/fb6 [0,4194304] 0 2024-03-13T14:49:49.088 INFO:tasks.workunit.client.0.smithi176.stdout:1/456: dread d2/de/d10/d16/f27 [4194304,4194304] 0 2024-03-13T14:49:49.089 INFO:tasks.workunit.client.0.smithi176.stdout:5/471: mknod d5/d2c/d74/c9e 0 2024-03-13T14:49:49.091 INFO:tasks.workunit.client.1.smithi196.stdout:8/557: dread d0/d16/d3d/d8b/d3f/f76 [0,4194304] 0 2024-03-13T14:49:49.091 INFO:tasks.workunit.client.0.smithi176.stdout:2/537: sync 2024-03-13T14:49:49.091 INFO:tasks.workunit.client.0.smithi176.stdout:7/458: sync 2024-03-13T14:49:49.092 INFO:tasks.workunit.client.0.smithi176.stdout:4/533: write d7/f24 [2463178,129970] 0 2024-03-13T14:49:49.093 INFO:tasks.workunit.client.1.smithi196.stdout:7/709: chown d0/d9/d1e/d3f/c49 8 1 2024-03-13T14:49:49.093 INFO:tasks.workunit.client.0.smithi176.stdout:8/445: truncate d0/f30 904794 0 2024-03-13T14:49:49.094 INFO:tasks.workunit.client.1.smithi196.stdout:8/558: write d0/d16/d3d/d7f/fb3 [658546,73687] 0 2024-03-13T14:49:49.096 INFO:tasks.workunit.client.0.smithi176.stdout:7/459: write d1/d1c/f77 [293845,15540] 0 2024-03-13T14:49:49.097 INFO:tasks.workunit.client.1.smithi196.stdout:5/677: rename d4/d1b/dc8/d14/d19/d26/d33/c70 to d4/d1b/dc8/d14/d19/d1c/d2d/d82/ccf 0 2024-03-13T14:49:49.098 INFO:tasks.workunit.client.0.smithi176.stdout:3/542: mknod d4/d28/d31/d45/cb9 0 2024-03-13T14:49:49.098 INFO:tasks.workunit.client.1.smithi196.stdout:8/559: chown d0/d16/d3d/d8b/l7a 51 1 2024-03-13T14:49:49.099 INFO:tasks.workunit.client.1.smithi196.stdout:2/526: mknod d0/d7/d66/d67/cab 0 2024-03-13T14:49:49.101 INFO:tasks.workunit.client.0.smithi176.stdout:1/457: stat d2/de/d10/d16/d4a/l85 0 2024-03-13T14:49:49.101 INFO:tasks.workunit.client.1.smithi196.stdout:3/611: mkdir db/d15/da1/dc8 0 2024-03-13T14:49:49.105 INFO:tasks.workunit.client.0.smithi176.stdout:9/550: dwrite d0/db/d89/d4d/d6e/f71 [0,4194304] 0 2024-03-13T14:49:49.109 INFO:tasks.workunit.client.0.smithi176.stdout:0/440: fdatasync d2/d5/d1e/f72 0 2024-03-13T14:49:49.110 INFO:tasks.workunit.client.0.smithi176.stdout:4/534: dread - d7/d8/d15/f7b zero size 2024-03-13T14:49:49.111 INFO:tasks.workunit.client.0.smithi176.stdout:2/538: rmdir d1/dc/d2d/d4b/d69 39 2024-03-13T14:49:49.114 INFO:tasks.workunit.client.0.smithi176.stdout:9/551: truncate d0/d40/d53/fad 1033942 0 2024-03-13T14:49:49.115 INFO:tasks.workunit.client.0.smithi176.stdout:7/460: chown d1/d1c/d26/d41/d2b/d40/l46 50837 1 2024-03-13T14:49:49.115 INFO:tasks.workunit.client.0.smithi176.stdout:6/508: mknod d0/de/d38/d88/d9d/ca6 0 2024-03-13T14:49:49.117 INFO:tasks.workunit.client.0.smithi176.stdout:1/458: rename d2/de/d10/d16/l4e to d2/d30/d63/l8f 0 2024-03-13T14:49:49.118 INFO:tasks.workunit.client.0.smithi176.stdout:8/446: dwrite d0/d5/d11/d47/f6a [0,4194304] 0 2024-03-13T14:49:49.119 INFO:tasks.workunit.client.1.smithi196.stdout:8/560: read d0/d10/f19 [4617442,36082] 0 2024-03-13T14:49:49.119 INFO:tasks.workunit.client.0.smithi176.stdout:6/509: read - d0/d10/d19/d9f/f98 zero size 2024-03-13T14:49:49.120 INFO:tasks.workunit.client.0.smithi176.stdout:3/543: rename d4 to d4/d13/d83/dba 22 2024-03-13T14:49:49.120 INFO:tasks.workunit.client.1.smithi196.stdout:1/678: creat d1/d9/d4e/d55/d63/da6/dc8/fe6 x:0 0 0 2024-03-13T14:49:49.122 INFO:tasks.workunit.client.0.smithi176.stdout:4/535: mknod d7/d8/d33/cc0 0 2024-03-13T14:49:49.122 INFO:tasks.workunit.client.0.smithi176.stdout:3/544: readlink d4/d28/l63 0 2024-03-13T14:49:49.122 INFO:tasks.workunit.client.0.smithi176.stdout:5/472: dread d5/d48/d4c/f92 [0,4194304] 0 2024-03-13T14:49:49.123 INFO:tasks.workunit.client.1.smithi196.stdout:0/646: getdents d8/d37 0 2024-03-13T14:49:49.125 INFO:tasks.workunit.client.0.smithi176.stdout:9/552: creat d0/d40/d53/fba x:0 0 0 2024-03-13T14:49:49.126 INFO:tasks.workunit.client.1.smithi196.stdout:8/561: truncate d0/d10/d12/d1e/f83 802582 0 2024-03-13T14:49:49.127 INFO:tasks.workunit.client.0.smithi176.stdout:9/553: write d0/d40/d53/f6b [832342,31932] 0 2024-03-13T14:49:49.129 INFO:tasks.workunit.client.1.smithi196.stdout:2/527: dread d0/d7/d9/d42/d98/f99 [0,4194304] 0 2024-03-13T14:49:49.131 INFO:tasks.workunit.client.0.smithi176.stdout:8/447: chown d0/db/dc/d40/f77 1106 1 2024-03-13T14:49:49.131 INFO:tasks.workunit.client.0.smithi176.stdout:7/461: creat d1/d28/d3a/d1b/f89 x:0 0 0 2024-03-13T14:49:49.133 INFO:tasks.workunit.client.0.smithi176.stdout:8/448: write d0/d24/f31 [2707258,10599] 0 2024-03-13T14:49:49.134 INFO:tasks.workunit.client.1.smithi196.stdout:3/612: symlink db/d19/lc9 0 2024-03-13T14:49:49.135 INFO:tasks.workunit.client.0.smithi176.stdout:1/459: dwrite d2/d30/d63/d22/d71/f87 [0,4194304] 0 2024-03-13T14:49:49.144 INFO:tasks.workunit.client.0.smithi176.stdout:3/545: mknod d4/d28/d4f/cbb 0 2024-03-13T14:49:49.147 INFO:tasks.workunit.client.0.smithi176.stdout:9/554: dwrite d0/d1e/d1f/f38 [0,4194304] 0 2024-03-13T14:49:49.148 INFO:tasks.workunit.client.0.smithi176.stdout:4/536: rename d7/d8/d15/d23/f47 to d7/dab/fc1 0 2024-03-13T14:49:49.149 INFO:tasks.workunit.client.0.smithi176.stdout:3/546: fsync d4/d28/d34/d38/f88 0 2024-03-13T14:49:49.150 INFO:tasks.workunit.client.1.smithi196.stdout:8/562: creat d0/d10/d12/d1e/fbb x:0 0 0 2024-03-13T14:49:49.151 INFO:tasks.workunit.client.0.smithi176.stdout:6/510: dwrite d0/d10/d19/d27/f4d [0,4194304] 0 2024-03-13T14:49:49.153 INFO:tasks.workunit.client.1.smithi196.stdout:0/647: dwrite d8/da/d74/f8e [0,4194304] 0 2024-03-13T14:49:49.153 INFO:tasks.workunit.client.1.smithi196.stdout:4/578: dread d1/d3/d7/d35/f37 [0,4194304] 0 2024-03-13T14:49:49.154 INFO:tasks.workunit.client.0.smithi176.stdout:8/449: dwrite d0/d5/d11/d47/f6a [0,4194304] 0 2024-03-13T14:49:49.157 INFO:tasks.workunit.client.1.smithi196.stdout:6/756: write d6/d7/dd/d60/f79 [1925805,60852] 0 2024-03-13T14:49:49.164 INFO:tasks.workunit.client.1.smithi196.stdout:2/528: mknod d0/d50/cac 0 2024-03-13T14:49:49.175 INFO:tasks.workunit.client.1.smithi196.stdout:1/679: mknod d1/d3f/d5b/d74/db5/ce7 0 2024-03-13T14:49:49.177 INFO:tasks.workunit.client.1.smithi196.stdout:7/710: getdents d0/d9/d1e/d58/d61 0 2024-03-13T14:49:49.178 INFO:tasks.workunit.client.0.smithi176.stdout:0/441: creat d2/d5/f87 x:0 0 0 2024-03-13T14:49:49.180 INFO:tasks.workunit.client.1.smithi196.stdout:8/563: mkdir d0/d16/d3d/d9c/d87/dbc 0 2024-03-13T14:49:49.182 INFO:tasks.workunit.client.1.smithi196.stdout:7/711: write d0/d9/d1e/d58/d72/f66 [1280693,5649] 0 2024-03-13T14:49:49.185 INFO:tasks.workunit.client.1.smithi196.stdout:4/579: write d1/d3/d7/d35/d76/f32 [2423055,82867] 0 2024-03-13T14:49:49.190 INFO:tasks.workunit.client.0.smithi176.stdout:4/537: unlink d7/d10/d46/l80 0 2024-03-13T14:49:49.190 INFO:tasks.workunit.client.0.smithi176.stdout:4/538: stat d7/d2d/c84 0 2024-03-13T14:49:49.191 INFO:tasks.workunit.client.0.smithi176.stdout:4/539: fdatasync d7/d8/d45/f92 0 2024-03-13T14:49:49.192 INFO:tasks.workunit.client.0.smithi176.stdout:4/540: readlink d7/d8/d33/l9c 0 2024-03-13T14:49:49.195 INFO:tasks.workunit.client.1.smithi196.stdout:8/564: mknod d0/d1c/d29/d47/cbd 0 2024-03-13T14:49:49.198 INFO:tasks.workunit.client.1.smithi196.stdout:0/648: link d8/da/d74/f8e d8/da/d25/d26/d35/d3b/d50/d55/fd4 0 2024-03-13T14:49:49.200 INFO:tasks.workunit.client.0.smithi176.stdout:2/539: creat d1/dc/d2d/d59/d80/fbb x:0 0 0 2024-03-13T14:49:49.206 INFO:tasks.workunit.client.0.smithi176.stdout:9/555: getdents d0/d40/d53/da0 0 2024-03-13T14:49:49.210 INFO:tasks.workunit.client.0.smithi176.stdout:6/511: mknod d0/de/ca7 0 2024-03-13T14:49:49.210 INFO:tasks.workunit.client.1.smithi196.stdout:1/680: truncate d1/d9/f5c 2100854 0 2024-03-13T14:49:49.212 INFO:tasks.workunit.client.0.smithi176.stdout:6/512: chown d0/de/f49 0 1 2024-03-13T14:49:49.212 INFO:tasks.workunit.client.1.smithi196.stdout:8/565: mkdir d0/d10/d12/d17/d42/dbe 0 2024-03-13T14:49:49.217 INFO:tasks.workunit.client.0.smithi176.stdout:8/450: getdents d0/d24/d29/d82 0 2024-03-13T14:49:49.219 INFO:tasks.workunit.client.0.smithi176.stdout:8/451: write d0/db/dc/f13 [2891675,121810] 0 2024-03-13T14:49:49.221 INFO:tasks.workunit.client.1.smithi196.stdout:6/757: truncate d6/d9/d11/f70 514055 0 2024-03-13T14:49:49.221 INFO:tasks.workunit.client.0.smithi176.stdout:8/452: chown d0/db/f3f 847298 1 2024-03-13T14:49:49.223 INFO:tasks.workunit.client.0.smithi176.stdout:1/460: rename d2/de/d3f/d3e/f61 to d2/de/d10/f90 0 2024-03-13T14:49:49.225 INFO:tasks.workunit.client.0.smithi176.stdout:9/556: unlink d0/db/f49 0 2024-03-13T14:49:49.225 INFO:tasks.workunit.client.0.smithi176.stdout:2/540: dread d1/dc/d2d/d59/d8c/f8f [0,4194304] 0 2024-03-13T14:49:49.226 INFO:tasks.workunit.client.1.smithi196.stdout:3/613: link db/d19/f2b db/d19/d50/fca 0 2024-03-13T14:49:49.227 INFO:tasks.workunit.client.0.smithi176.stdout:4/541: dread d7/d8/f39 [0,4194304] 0 2024-03-13T14:49:49.228 INFO:tasks.workunit.client.1.smithi196.stdout:1/681: unlink d1/d3f/fb3 0 2024-03-13T14:49:49.229 INFO:tasks.workunit.client.0.smithi176.stdout:6/513: creat d0/d10/d19/fa8 x:0 0 0 2024-03-13T14:49:49.229 INFO:tasks.workunit.client.1.smithi196.stdout:2/529: write d0/f1a [2590015,84830] 0 2024-03-13T14:49:49.233 INFO:tasks.workunit.client.1.smithi196.stdout:7/712: symlink d0/d78/lef 0 2024-03-13T14:49:49.235 INFO:tasks.workunit.client.1.smithi196.stdout:0/649: mkdir d8/da/d25/d26/d48/d82/db3/dd5 0 2024-03-13T14:49:49.239 INFO:tasks.workunit.client.0.smithi176.stdout:7/462: write d1/d3f/f4d [935343,94022] 0 2024-03-13T14:49:49.240 INFO:tasks.workunit.client.0.smithi176.stdout:8/453: fdatasync d0/d24/f57 0 2024-03-13T14:49:49.242 INFO:tasks.workunit.client.1.smithi196.stdout:4/580: write d1/f29 [3993581,33239] 0 2024-03-13T14:49:49.242 INFO:tasks.workunit.client.0.smithi176.stdout:8/454: stat d0/d5/d11/l2e 0 2024-03-13T14:49:49.245 INFO:tasks.workunit.client.1.smithi196.stdout:8/566: dwrite d0/d16/d3d/d8b/f4a [0,4194304] 0 2024-03-13T14:49:49.252 INFO:tasks.workunit.client.0.smithi176.stdout:9/557: creat d0/db/d89/d32/d58/d67/fbb x:0 0 0 2024-03-13T14:49:49.260 INFO:tasks.workunit.client.0.smithi176.stdout:9/558: chown d0/db/d89/l35 1082819758 1 2024-03-13T14:49:49.265 INFO:tasks.workunit.client.1.smithi196.stdout:1/682: mknod d1/d9/d33/d4b/d66/d7b/ce8 0 2024-03-13T14:49:49.267 INFO:tasks.workunit.client.0.smithi176.stdout:3/547: dwrite d4/d28/fa2 [0,4194304] 0 2024-03-13T14:49:49.268 INFO:tasks.workunit.client.1.smithi196.stdout:8/567: creat d0/d16/d3d/d7f/fbf x:0 0 0 2024-03-13T14:49:49.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/530: write d0/d50/f65 [1145053,113584] 0 2024-03-13T14:49:49.268 INFO:tasks.workunit.client.0.smithi176.stdout:1/461: dwrite d2/de/d3f/d50/f8e [0,4194304] 0 2024-03-13T14:49:49.272 INFO:tasks.workunit.client.0.smithi176.stdout:4/542: write d7/d10/d46/d55/f67 [359284,129526] 0 2024-03-13T14:49:49.273 INFO:tasks.workunit.client.0.smithi176.stdout:5/473: sync 2024-03-13T14:49:49.274 INFO:tasks.workunit.client.1.smithi196.stdout:0/650: write d8/da/d25/d26/d35/d3b/d6c/f9d [716107,128547] 0 2024-03-13T14:49:49.274 INFO:tasks.workunit.client.1.smithi196.stdout:6/758: write d6/d7/df/f3a [1779199,4724] 0 2024-03-13T14:49:49.274 INFO:tasks.workunit.client.1.smithi196.stdout:3/614: write db/d19/d22/d3a/f8f [221126,29818] 0 2024-03-13T14:49:49.278 INFO:tasks.workunit.client.1.smithi196.stdout:4/581: mkdir d1/d3/d7/d35/d76/d58/dc6 0 2024-03-13T14:49:49.278 INFO:tasks.workunit.client.0.smithi176.stdout:5/474: write d5/d31/d47/d80/f62 [5205045,60620] 0 2024-03-13T14:49:49.281 INFO:tasks.workunit.client.1.smithi196.stdout:3/615: truncate db/d19/d4a/d9b/fb5 437994 0 2024-03-13T14:49:49.283 INFO:tasks.workunit.client.0.smithi176.stdout:2/541: link d1/dc/d40/l92 d1/dc/d1c/lbc 0 2024-03-13T14:49:49.285 INFO:tasks.workunit.client.0.smithi176.stdout:2/542: readlink d1/dc/d40/l4a 0 2024-03-13T14:49:49.286 INFO:tasks.workunit.client.1.smithi196.stdout:1/683: creat d1/d9/d4e/d55/d63/fe9 x:0 0 0 2024-03-13T14:49:49.287 INFO:tasks.workunit.client.0.smithi176.stdout:6/514: creat d0/d10/d19/d27/fa9 x:0 0 0 2024-03-13T14:49:49.288 INFO:tasks.workunit.client.1.smithi196.stdout:7/713: creat d0/d4d/ff0 x:0 0 0 2024-03-13T14:49:49.291 INFO:tasks.workunit.client.1.smithi196.stdout:8/568: creat d0/d16/d3d/d9c/fc0 x:0 0 0 2024-03-13T14:49:49.292 INFO:tasks.workunit.client.0.smithi176.stdout:9/559: creat d0/d40/d53/d6d/d97/fbc x:0 0 0 2024-03-13T14:49:49.295 INFO:tasks.workunit.client.0.smithi176.stdout:4/543: fdatasync d7/d8/d2f/f58 0 2024-03-13T14:49:49.295 INFO:tasks.workunit.client.1.smithi196.stdout:2/531: dwrite d0/d6/d4b/d5f/f81 [0,4194304] 0 2024-03-13T14:49:49.298 INFO:tasks.workunit.client.0.smithi176.stdout:8/455: truncate d0/f42 494838 0 2024-03-13T14:49:49.300 INFO:tasks.workunit.client.1.smithi196.stdout:2/532: dread - d0/d6/d4b/f8a zero size 2024-03-13T14:49:49.301 INFO:tasks.workunit.client.0.smithi176.stdout:7/463: creat d1/d1c/d26/d41/d2b/d40/f8a x:0 0 0 2024-03-13T14:49:49.301 INFO:tasks.workunit.client.0.smithi176.stdout:8/456: read d0/db/f25 [2965420,40452] 0 2024-03-13T14:49:49.301 INFO:tasks.workunit.client.1.smithi196.stdout:8/569: dread d0/d16/d3d/d7f/fb3 [0,4194304] 0 2024-03-13T14:49:49.302 INFO:tasks.workunit.client.1.smithi196.stdout:2/533: readlink d0/d7/la3 0 2024-03-13T14:49:49.303 INFO:tasks.workunit.client.1.smithi196.stdout:8/570: fsync d0/d1c/d29/fa6 0 2024-03-13T14:49:49.303 INFO:tasks.workunit.client.1.smithi196.stdout:2/534: fsync d0/d6/d34/f62 0 2024-03-13T14:49:49.312 INFO:tasks.workunit.client.0.smithi176.stdout:5/475: rename d5/d48 to d5/d2e/d7e/d9f 0 2024-03-13T14:49:49.313 INFO:tasks.workunit.client.0.smithi176.stdout:4/544: dwrite d7/d8/f3b [0,4194304] 0 2024-03-13T14:49:49.314 INFO:tasks.workunit.client.1.smithi196.stdout:3/616: unlink db/f49 0 2024-03-13T14:49:49.315 INFO:tasks.workunit.client.0.smithi176.stdout:4/545: dread - d7/d10/d46/d4a/d5d/da1/f89 zero size 2024-03-13T14:49:49.318 INFO:tasks.workunit.client.1.smithi196.stdout:1/684: creat d1/d9/d4e/d55/d63/fea x:0 0 0 2024-03-13T14:49:49.320 INFO:tasks.workunit.client.0.smithi176.stdout:2/543: chown d1/dc/d2d/d4b/d69/f94 71231253 1 2024-03-13T14:49:49.321 INFO:tasks.workunit.client.0.smithi176.stdout:1/462: getdents d2/de/d10/d16/d40 0 2024-03-13T14:49:49.321 INFO:tasks.workunit.client.0.smithi176.stdout:6/515: rmdir d0/de/d54/d99 39 2024-03-13T14:49:49.321 INFO:tasks.workunit.client.1.smithi196.stdout:6/759: mknod d6/d2a/dc/d1c/cf6 0 2024-03-13T14:49:49.323 INFO:tasks.workunit.client.0.smithi176.stdout:0/442: sync 2024-03-13T14:49:49.323 INFO:tasks.workunit.client.0.smithi176.stdout:2/544: fsync d1/dc/fae 0 2024-03-13T14:49:49.324 INFO:tasks.workunit.client.1.smithi196.stdout:0/651: fsync d8/da/d25/d26/d54/f9a 0 2024-03-13T14:49:49.328 INFO:tasks.workunit.client.0.smithi176.stdout:1/463: write d2/de/d10/d16/d4a/d5f/d66/f84 [993774,63618] 0 2024-03-13T14:49:49.330 INFO:tasks.workunit.client.0.smithi176.stdout:0/443: write d2/d74/d61/f66 [4729584,38033] 0 2024-03-13T14:49:49.332 INFO:tasks.workunit.client.1.smithi196.stdout:4/582: mknod d1/cc7 0 2024-03-13T14:49:49.333 INFO:tasks.workunit.client.0.smithi176.stdout:1/464: fdatasync d2/de/d10/f6d 0 2024-03-13T14:49:49.333 INFO:tasks.workunit.client.0.smithi176.stdout:5/476: dwrite d5/f14 [0,4194304] 0 2024-03-13T14:49:49.334 INFO:tasks.workunit.client.0.smithi176.stdout:0/444: chown d2/db/d31 0 1 2024-03-13T14:49:49.337 INFO:tasks.workunit.client.1.smithi196.stdout:3/617: symlink db/d37/lcb 0 2024-03-13T14:49:49.339 INFO:tasks.workunit.client.1.smithi196.stdout:7/714: readlink d0/d78/d81/la6 0 2024-03-13T14:49:49.342 INFO:tasks.workunit.client.0.smithi176.stdout:3/548: dwrite d4/d13/f2a [4194304,4194304] 0 2024-03-13T14:49:49.345 INFO:tasks.workunit.client.0.smithi176.stdout:4/546: symlink d7/d2d/da7/lc2 0 2024-03-13T14:49:49.346 INFO:tasks.workunit.client.0.smithi176.stdout:6/516: mknod d0/d10/d19/caa 0 2024-03-13T14:49:49.347 INFO:tasks.workunit.client.1.smithi196.stdout:8/571: mknod d0/cc1 0 2024-03-13T14:49:49.347 INFO:tasks.workunit.client.1.smithi196.stdout:2/535: creat d0/d7/d9/d61/d83/fad x:0 0 0 2024-03-13T14:49:49.350 INFO:tasks.workunit.client.1.smithi196.stdout:4/583: mknod d1/d3/d4c/d72/d7d/d80/cc8 0 2024-03-13T14:49:49.352 INFO:tasks.workunit.client.0.smithi176.stdout:6/517: truncate d0/d10/d19/d27/f68 2050850 0 2024-03-13T14:49:49.353 INFO:tasks.workunit.client.1.smithi196.stdout:3/618: creat db/d19/d22/d3a/d6a/fcc x:0 0 0 2024-03-13T14:49:49.353 INFO:tasks.workunit.client.1.smithi196.stdout:3/619: stat db/f35 0 2024-03-13T14:49:49.358 INFO:tasks.workunit.client.0.smithi176.stdout:7/464: unlink d1/d28/d3a/l84 0 2024-03-13T14:49:49.359 INFO:tasks.workunit.client.0.smithi176.stdout:1/465: dwrite d2/de/d10/f6d [0,4194304] 0 2024-03-13T14:49:49.360 INFO:tasks.workunit.client.0.smithi176.stdout:8/457: creat d0/db/d58/d86/f89 x:0 0 0 2024-03-13T14:49:49.361 INFO:tasks.workunit.client.1.smithi196.stdout:0/652: unlink d8/da/d25/lb9 0 2024-03-13T14:49:49.363 INFO:tasks.workunit.client.1.smithi196.stdout:9/552: sync 2024-03-13T14:49:49.365 INFO:tasks.workunit.client.1.smithi196.stdout:4/584: chown d1/d3/d7/d35/d76/c5a 960 1 2024-03-13T14:49:49.365 INFO:tasks.workunit.client.1.smithi196.stdout:9/553: chown d7/l4c 0 1 2024-03-13T14:49:49.366 INFO:tasks.workunit.client.1.smithi196.stdout:4/585: readlink d1/d3/lc2 0 2024-03-13T14:49:49.368 INFO:tasks.workunit.client.0.smithi176.stdout:0/445: symlink d2/db/d31/l88 0 2024-03-13T14:49:49.368 INFO:tasks.workunit.client.0.smithi176.stdout:5/477: mknod d5/d2c/d74/ca0 0 2024-03-13T14:49:49.369 INFO:tasks.workunit.client.1.smithi196.stdout:4/586: stat d1/d3/d7/d35/d3e/f64 0 2024-03-13T14:49:49.372 INFO:tasks.workunit.client.0.smithi176.stdout:4/547: dwrite d7/f3a [0,4194304] 0 2024-03-13T14:49:49.374 INFO:tasks.workunit.client.1.smithi196.stdout:4/587: read d1/d3/d7/d35/d76/d58/f87 [4024319,55830] 0 2024-03-13T14:49:49.376 INFO:tasks.workunit.client.0.smithi176.stdout:3/549: dwrite d4/d13/d81/d89/f9c [0,4194304] 0 2024-03-13T14:49:49.376 INFO:tasks.workunit.client.1.smithi196.stdout:1/685: getdents d1/d9/d33 0 2024-03-13T14:49:49.378 INFO:tasks.workunit.client.1.smithi196.stdout:9/554: dread - d7/de/d4b/d1d/f75 zero size 2024-03-13T14:49:49.379 INFO:tasks.workunit.client.1.smithi196.stdout:7/715: link d0/d78/d81/la6 d0/d9/d1e/d58/d72/da9/da3/ddf/lf1 0 2024-03-13T14:49:49.380 INFO:tasks.workunit.client.0.smithi176.stdout:7/465: write d1/d3f/d76/f1e [9174165,88164] 0 2024-03-13T14:49:49.383 INFO:tasks.workunit.client.0.smithi176.stdout:7/466: fdatasync d1/d1c/d4a/f62 0 2024-03-13T14:49:49.386 INFO:tasks.workunit.client.0.smithi176.stdout:8/458: dread d0/db/f25 [0,4194304] 0 2024-03-13T14:49:49.387 INFO:tasks.workunit.client.0.smithi176.stdout:7/467: chown d1/cc 445 1 2024-03-13T14:49:49.388 INFO:tasks.workunit.client.0.smithi176.stdout:1/466: truncate d2/de/d3f/d50/f42 497204 0 2024-03-13T14:49:49.389 INFO:tasks.workunit.client.1.smithi196.stdout:0/653: dread d8/da/d25/f2f [0,4194304] 0 2024-03-13T14:49:49.390 INFO:tasks.workunit.client.1.smithi196.stdout:6/760: getdents d6/d2a/d23/d28 0 2024-03-13T14:49:49.391 INFO:tasks.workunit.client.0.smithi176.stdout:5/478: dwrite d5/d22/f29 [0,4194304] 0 2024-03-13T14:49:49.395 INFO:tasks.workunit.client.1.smithi196.stdout:8/572: rename d0/d16/d3d/d8b/d3f/f45 to d0/d10/fc2 0 2024-03-13T14:49:49.397 INFO:tasks.workunit.client.0.smithi176.stdout:9/560: getdents d0/db/d89/d90 0 2024-03-13T14:49:49.401 INFO:tasks.workunit.client.0.smithi176.stdout:4/548: mknod d7/d10/d46/cc3 0 2024-03-13T14:49:49.404 INFO:tasks.workunit.client.1.smithi196.stdout:5/678: sync 2024-03-13T14:49:49.409 INFO:tasks.workunit.client.0.smithi176.stdout:7/468: dwrite d1/d3f/d76/f1e [4194304,4194304] 0 2024-03-13T14:49:49.410 INFO:tasks.workunit.client.1.smithi196.stdout:7/716: rmdir d0/d78 39 2024-03-13T14:49:49.414 INFO:tasks.workunit.client.0.smithi176.stdout:5/479: creat d5/d2c/d40/d6a/d89/fa1 x:0 0 0 2024-03-13T14:49:49.415 INFO:tasks.workunit.client.1.smithi196.stdout:7/717: chown d0/f11 12240562 1 2024-03-13T14:49:49.415 INFO:tasks.workunit.client.1.smithi196.stdout:6/761: dwrite d6/d2a/d23/fb6 [0,4194304] 0 2024-03-13T14:49:49.416 INFO:tasks.workunit.client.0.smithi176.stdout:1/467: symlink d2/de/d10/d16/d4a/d5f/l91 0 2024-03-13T14:49:49.416 INFO:tasks.workunit.client.0.smithi176.stdout:5/480: chown d5/d2e/c42 1824 1 2024-03-13T14:49:49.417 INFO:tasks.workunit.client.1.smithi196.stdout:8/573: rmdir d0/d10/da5 39 2024-03-13T14:49:49.420 INFO:tasks.workunit.client.0.smithi176.stdout:4/549: mknod d7/d10/d46/cc4 0 2024-03-13T14:49:49.420 INFO:tasks.workunit.client.0.smithi176.stdout:2/545: getdents d1/dc/d61 0 2024-03-13T14:49:49.421 INFO:tasks.workunit.client.1.smithi196.stdout:6/762: chown d6/d2a/dc/d1c/d34/l4f 55 1 2024-03-13T14:49:49.423 INFO:tasks.workunit.client.1.smithi196.stdout:3/620: dwrite db/d19/d22/f82 [0,4194304] 0 2024-03-13T14:49:49.425 INFO:tasks.workunit.client.1.smithi196.stdout:9/555: write d7/d9/d17/f51 [1406253,16864] 0 2024-03-13T14:49:49.426 INFO:tasks.workunit.client.0.smithi176.stdout:3/550: link f2 d4/d28/d31/d45/fbc 0 2024-03-13T14:49:49.428 INFO:tasks.workunit.client.1.smithi196.stdout:9/556: stat d7/de/d4b/d1d/d22/d4d/f80 0 2024-03-13T14:49:49.428 INFO:tasks.workunit.client.1.smithi196.stdout:2/536: rename d0/d7/d40/f36 to d0/d6/d4b/d5f/fae 0 2024-03-13T14:49:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:49 smithi196 ceph-mon[37486]: pgmap v13: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 41 MiB/s rd, 150 MiB/s wr, 288 op/s 2024-03-13T14:49:49.429 INFO:tasks.workunit.client.0.smithi176.stdout:7/469: dwrite d1/d3f/d4b/f6b [0,4194304] 0 2024-03-13T14:49:49.432 INFO:tasks.workunit.client.0.smithi176.stdout:3/551: chown d4/d28/d5a/l85 881 1 2024-03-13T14:49:49.434 INFO:tasks.workunit.client.1.smithi196.stdout:3/621: fsync db/f3d 0 2024-03-13T14:49:49.436 INFO:tasks.workunit.client.1.smithi196.stdout:0/654: rename d8/da/d25/d26/d54/d88/c93 to d8/da/d25/d26/d35/d3b/d50/d5d/cd6 0 2024-03-13T14:49:49.440 INFO:tasks.workunit.client.0.smithi176.stdout:2/546: fdatasync d1/dc/d40/da0/fa6 0 2024-03-13T14:49:49.445 INFO:tasks.workunit.client.0.smithi176.stdout:2/547: fsync d1/dc/d1c/fa4 0 2024-03-13T14:49:49.450 INFO:tasks.workunit.client.0.smithi176.stdout:6/518: getdents d0/d10/d19/d27/d2b 0 2024-03-13T14:49:49.450 INFO:tasks.workunit.client.1.smithi196.stdout:1/686: getdents d1 0 2024-03-13T14:49:49.451 INFO:tasks.workunit.client.0.smithi176.stdout:2/548: stat d1/dc/d2d/d59/d80/d5c/d78 0 2024-03-13T14:49:49.458 INFO:tasks.workunit.client.1.smithi196.stdout:3/622: mknod db/d19/d22/d3a/d6a/ccd 0 2024-03-13T14:49:49.460 INFO:tasks.workunit.client.0.smithi176.stdout:2/549: truncate d1/dc/d2d/d59/f8a 1406647 0 2024-03-13T14:49:49.461 INFO:tasks.workunit.client.1.smithi196.stdout:4/588: truncate d1/f6 289149 0 2024-03-13T14:49:49.461 INFO:tasks.workunit.client.1.smithi196.stdout:8/574: dread d0/d1c/f7c [0,4194304] 0 2024-03-13T14:49:49.464 INFO:tasks.workunit.client.0.smithi176.stdout:3/552: dwrite d4/d28/d34/f6e [0,4194304] 0 2024-03-13T14:49:49.468 INFO:tasks.workunit.client.0.smithi176.stdout:0/446: sync 2024-03-13T14:49:49.469 INFO:tasks.workunit.client.1.smithi196.stdout:9/557: mknod d7/d9/cbb 0 2024-03-13T14:49:49.475 INFO:tasks.workunit.client.1.smithi196.stdout:1/687: dwrite d1/d3f/f80 [0,4194304] 0 2024-03-13T14:49:49.475 INFO:tasks.workunit.client.1.smithi196.stdout:5/679: dread d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/f42 [0,4194304] 0 2024-03-13T14:49:49.476 INFO:tasks.workunit.client.0.smithi176.stdout:5/481: write d5/d2c/f3d [4914912,59932] 0 2024-03-13T14:49:49.476 INFO:tasks.workunit.client.0.smithi176.stdout:8/459: getdents d0/db/dc 0 2024-03-13T14:49:49.478 INFO:tasks.workunit.client.0.smithi176.stdout:5/482: stat d5/d2c/d40/d6a/l4b 0 2024-03-13T14:49:49.481 INFO:tasks.workunit.client.1.smithi196.stdout:2/537: write d0/d7/f3f [1711318,21853] 0 2024-03-13T14:49:49.482 INFO:tasks.workunit.client.0.smithi176.stdout:6/519: creat d0/d10/d19/d9f/d76/fab x:0 0 0 2024-03-13T14:49:49.484 INFO:tasks.workunit.client.1.smithi196.stdout:7/718: dread d0/d78/d81/fc9 [0,4194304] 0 2024-03-13T14:49:49.494 INFO:tasks.workunit.client.1.smithi196.stdout:8/575: creat d0/d16/d3d/d8b/d3f/fc3 x:0 0 0 2024-03-13T14:49:49.497 INFO:tasks.workunit.client.0.smithi176.stdout:9/561: dread d0/db/d89/f16 [0,4194304] 0 2024-03-13T14:49:49.497 INFO:tasks.workunit.client.0.smithi176.stdout:3/553: mknod d4/d28/d34/da4/cbd 0 2024-03-13T14:49:49.498 INFO:tasks.workunit.client.0.smithi176.stdout:9/562: write d0/d1e/d1f/d68/faf [877213,63092] 0 2024-03-13T14:49:49.503 INFO:tasks.workunit.client.1.smithi196.stdout:6/763: rename d6/fa2 to d6/d7/ff7 0 2024-03-13T14:49:49.503 INFO:tasks.workunit.client.1.smithi196.stdout:2/538: dread d0/d7/d40/f9c [0,4194304] 0 2024-03-13T14:49:49.513 INFO:tasks.workunit.client.1.smithi196.stdout:8/576: mknod d0/d10/d12/d1e/cc4 0 2024-03-13T14:49:49.515 INFO:tasks.workunit.client.1.smithi196.stdout:5/680: mkdir d4/d1b/dc8/d3c/dd0 0 2024-03-13T14:49:49.518 INFO:tasks.workunit.client.0.smithi176.stdout:2/550: write d1/dc/d2d/d38/f5e [4536411,93975] 0 2024-03-13T14:49:49.519 INFO:tasks.workunit.client.1.smithi196.stdout:7/719: creat d0/d6/d33/d57/dae/de8/ff2 x:0 0 0 2024-03-13T14:49:49.519 INFO:tasks.workunit.client.0.smithi176.stdout:2/551: fdatasync d1/dc/d2d/d59/f8a 0 2024-03-13T14:49:49.522 INFO:tasks.workunit.client.1.smithi196.stdout:7/720: chown d0/d9/d1e/d58/d72/da9/da3/dc2 34826 1 2024-03-13T14:49:49.526 INFO:tasks.workunit.client.1.smithi196.stdout:3/623: dwrite db/f3e [0,4194304] 0 2024-03-13T14:49:49.527 INFO:tasks.workunit.client.1.smithi196.stdout:9/558: rename d7/de/d16/d3c to d7/d9/d17/dbc 0 2024-03-13T14:49:49.530 INFO:tasks.workunit.client.1.smithi196.stdout:6/764: dread d6/d2a/d23/d28/f69 [0,4194304] 0 2024-03-13T14:49:49.531 INFO:tasks.workunit.client.1.smithi196.stdout:4/589: rmdir d1/d3/d7/d35/d76/d58/d69/d9f/dbb 0 2024-03-13T14:49:49.535 INFO:tasks.workunit.client.1.smithi196.stdout:5/681: fdatasync d4/d1b/dc8/d14/f5e 0 2024-03-13T14:49:49.546 INFO:tasks.workunit.client.1.smithi196.stdout:1/688: rename d1/d9/d33/l4d to d1/d28/leb 0 2024-03-13T14:49:49.556 INFO:tasks.workunit.client.1.smithi196.stdout:9/559: write d7/f45 [2486242,43154] 0 2024-03-13T14:49:49.559 INFO:tasks.workunit.client.0.smithi176.stdout:4/550: link d7/l9 d7/d10/d46/d55/lc5 0 2024-03-13T14:49:49.561 INFO:tasks.workunit.client.1.smithi196.stdout:5/682: chown d4/d1b/dc8/d4a/l7e 34902 1 2024-03-13T14:49:49.562 INFO:tasks.workunit.client.0.smithi176.stdout:8/460: creat d0/db/dc/f8a x:0 0 0 2024-03-13T14:49:49.562 INFO:tasks.workunit.client.1.smithi196.stdout:3/624: write db/d37/d64/f90 [1582671,82785] 0 2024-03-13T14:49:49.566 INFO:tasks.workunit.client.1.smithi196.stdout:2/539: rename d0/d6/d4b/d96/d9e/fa9 to d0/d7/d9/d42/d98/faf 0 2024-03-13T14:49:49.568 INFO:tasks.workunit.client.0.smithi176.stdout:7/470: creat d1/d28/d3a/f8b x:0 0 0 2024-03-13T14:49:49.569 INFO:tasks.workunit.client.1.smithi196.stdout:4/590: mkdir d1/d3/d7/d35/d76/d58/dc9 0 2024-03-13T14:49:49.570 INFO:tasks.workunit.client.1.smithi196.stdout:4/591: chown d1/l24 0 1 2024-03-13T14:49:49.571 INFO:tasks.workunit.client.0.smithi176.stdout:1/468: dread d2/de/d3f/d50/f42 [0,4194304] 0 2024-03-13T14:49:49.571 INFO:tasks.workunit.client.1.smithi196.stdout:8/577: rmdir d0/d16/d3d/d7f/d88 0 2024-03-13T14:49:49.571 INFO:tasks.workunit.client.1.smithi196.stdout:6/765: dwrite d6/d7/df/f35 [0,4194304] 0 2024-03-13T14:49:49.572 INFO:tasks.workunit.client.0.smithi176.stdout:3/554: dread - d4/d13/d16/d3d/f4c zero size 2024-03-13T14:49:49.573 INFO:tasks.workunit.client.1.smithi196.stdout:8/578: fdatasync d0/d16/d3d/d8b/f4a 0 2024-03-13T14:49:49.574 INFO:tasks.workunit.client.1.smithi196.stdout:9/560: rmdir d7/de/d74/d94 39 2024-03-13T14:49:49.576 INFO:tasks.workunit.client.0.smithi176.stdout:2/552: rename d1/f35 to d1/dc/d2d/d59/d80/d77/fbd 0 2024-03-13T14:49:49.577 INFO:tasks.workunit.client.1.smithi196.stdout:5/683: symlink d4/d60/ld1 0 2024-03-13T14:49:49.577 INFO:tasks.workunit.client.0.smithi176.stdout:9/563: dread d0/d40/f78 [0,4194304] 0 2024-03-13T14:49:49.578 INFO:tasks.workunit.client.1.smithi196.stdout:7/721: rmdir d0/d4d/dd0 0 2024-03-13T14:49:49.579 INFO:tasks.workunit.client.0.smithi176.stdout:4/551: fsync d7/d8/f4f 0 2024-03-13T14:49:49.581 INFO:tasks.workunit.client.0.smithi176.stdout:8/461: rmdir d0/d5/d11/d47 39 2024-03-13T14:49:49.588 INFO:tasks.workunit.client.0.smithi176.stdout:1/469: mknod d2/d29/c92 0 2024-03-13T14:49:49.590 INFO:tasks.workunit.client.1.smithi196.stdout:4/592: symlink d1/d3/d7/d35/d3e/lca 0 2024-03-13T14:49:49.591 INFO:tasks.workunit.client.0.smithi176.stdout:3/555: write d4/f97 [3840710,67935] 0 2024-03-13T14:49:49.594 INFO:tasks.workunit.client.1.smithi196.stdout:6/766: symlink d6/d9/d17/da1/d50/lf8 0 2024-03-13T14:49:49.594 INFO:tasks.workunit.client.0.smithi176.stdout:7/471: dread d1/d3f/d4b/f58 [0,4194304] 0 2024-03-13T14:49:49.596 INFO:tasks.workunit.client.0.smithi176.stdout:7/472: readlink d1/d1c/d4a/l6c 0 2024-03-13T14:49:49.596 INFO:tasks.workunit.client.1.smithi196.stdout:8/579: chown d0/f56 3318950 1 2024-03-13T14:49:49.599 INFO:tasks.workunit.client.1.smithi196.stdout:9/561: creat d7/d24/d28/fbd x:0 0 0 2024-03-13T14:49:49.601 INFO:tasks.workunit.client.0.smithi176.stdout:9/564: creat d0/d1e/d1f/fbd x:0 0 0 2024-03-13T14:49:49.602 INFO:tasks.workunit.client.1.smithi196.stdout:5/684: rmdir d4/d1b/dc8/d14/d19/d1c/d2d/d82 39 2024-03-13T14:49:49.603 INFO:tasks.workunit.client.0.smithi176.stdout:9/565: write d0/db/d89/d32/d76/d82/fb7 [603792,92312] 0 2024-03-13T14:49:49.605 INFO:tasks.workunit.client.0.smithi176.stdout:9/566: chown d0/d17/c46 638779 1 2024-03-13T14:49:49.606 INFO:tasks.workunit.client.0.smithi176.stdout:2/553: truncate d1/dc/d1c/f55 754428 0 2024-03-13T14:49:49.607 INFO:tasks.workunit.client.1.smithi196.stdout:9/562: dread d7/d9/d17/f9c [0,4194304] 0 2024-03-13T14:49:49.608 INFO:tasks.workunit.client.0.smithi176.stdout:8/462: mknod d0/db/d39/c8b 0 2024-03-13T14:49:49.609 INFO:tasks.workunit.client.0.smithi176.stdout:2/554: stat d1/dc/d2d/d38/d5d/l9f 0 2024-03-13T14:49:49.609 INFO:tasks.workunit.client.1.smithi196.stdout:2/540: unlink d0/f78 0 2024-03-13T14:49:49.611 INFO:tasks.workunit.client.1.smithi196.stdout:1/689: creat d1/d9/d4e/d55/d63/da6/fec x:0 0 0 2024-03-13T14:49:49.612 INFO:tasks.workunit.client.1.smithi196.stdout:7/722: dread d0/d6/d33/f9a [0,4194304] 0 2024-03-13T14:49:49.614 INFO:tasks.workunit.client.1.smithi196.stdout:4/593: mknod d1/d1e/d5c/ccb 0 2024-03-13T14:49:49.614 INFO:tasks.workunit.client.1.smithi196.stdout:1/690: readlink d1/d9/d4e/d55/d63/da6/lac 0 2024-03-13T14:49:49.616 INFO:tasks.workunit.client.1.smithi196.stdout:8/580: dread - d0/d63/f85 zero size 2024-03-13T14:49:49.617 INFO:tasks.workunit.client.0.smithi176.stdout:0/447: getdents d2/db 0 2024-03-13T14:49:49.617 INFO:tasks.workunit.client.0.smithi176.stdout:1/470: dread d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:49:49.618 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:49 smithi176 ceph-mon[29064]: pgmap v13: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 41 MiB/s rd, 150 MiB/s wr, 288 op/s 2024-03-13T14:49:49.619 INFO:tasks.workunit.client.0.smithi176.stdout:0/448: fsync d2/d5/f87 0 2024-03-13T14:49:49.620 INFO:tasks.workunit.client.1.smithi196.stdout:8/581: write d0/d10/d12/d1e/f34 [287811,123221] 0 2024-03-13T14:49:49.620 INFO:tasks.workunit.client.0.smithi176.stdout:2/555: dread d1/dc/d2d/d4b/d69/fb4 [0,4194304] 0 2024-03-13T14:49:49.622 INFO:tasks.workunit.client.0.smithi176.stdout:4/552: write d7/d10/f2a [351031,108664] 0 2024-03-13T14:49:49.623 INFO:tasks.workunit.client.1.smithi196.stdout:3/625: write db/d19/f2f [1856333,37353] 0 2024-03-13T14:49:49.623 INFO:tasks.workunit.client.0.smithi176.stdout:7/473: mknod d1/d3f/d65/c8c 0 2024-03-13T14:49:49.626 INFO:tasks.workunit.client.0.smithi176.stdout:7/474: chown d1/d28/f2d 215745833 1 2024-03-13T14:49:49.627 INFO:tasks.workunit.client.1.smithi196.stdout:3/626: truncate db/d15/d44/fb0 785964 0 2024-03-13T14:49:49.629 INFO:tasks.workunit.client.1.smithi196.stdout:9/563: dread - d7/de/d4b/d1d/f6e zero size 2024-03-13T14:49:49.632 INFO:tasks.workunit.client.1.smithi196.stdout:5/685: dread d4/d1b/dc8/f24 [0,4194304] 0 2024-03-13T14:49:49.632 INFO:tasks.workunit.client.0.smithi176.stdout:9/567: mkdir d0/d1e/d8d/dbe 0 2024-03-13T14:49:49.633 INFO:tasks.workunit.client.1.smithi196.stdout:7/723: dwrite d0/d9/f5a [0,4194304] 0 2024-03-13T14:49:49.633 INFO:tasks.workunit.client.0.smithi176.stdout:9/568: fsync d0/d17/fa3 0 2024-03-13T14:49:49.634 INFO:tasks.workunit.client.0.smithi176.stdout:5/483: link d5/d2e/d7e/d9f/d4c/f65 d5/d31/fa2 0 2024-03-13T14:49:49.636 INFO:tasks.workunit.client.0.smithi176.stdout:9/569: chown d0/db/d89/d32/l50 19805 1 2024-03-13T14:49:49.637 INFO:tasks.workunit.client.0.smithi176.stdout:8/463: truncate d0/f5b 1014949 0 2024-03-13T14:49:49.641 INFO:tasks.workunit.client.0.smithi176.stdout:6/520: getdents d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1 0 2024-03-13T14:49:49.642 INFO:tasks.workunit.client.0.smithi176.stdout:3/556: dwrite d4/d13/d81/d89/f60 [0,4194304] 0 2024-03-13T14:49:49.643 INFO:tasks.workunit.client.0.smithi176.stdout:6/521: chown d0/d10/d19/d27/d2b 29 1 2024-03-13T14:49:49.646 INFO:tasks.workunit.client.1.smithi196.stdout:4/594: unlink d1/d3/d4c/f81 0 2024-03-13T14:49:49.649 INFO:tasks.workunit.client.0.smithi176.stdout:4/553: dread d7/d10/f83 [4194304,4194304] 0 2024-03-13T14:49:49.649 INFO:tasks.workunit.client.1.smithi196.stdout:8/582: readlink d0/d16/d71/l46 0 2024-03-13T14:49:49.649 INFO:tasks.workunit.client.1.smithi196.stdout:3/627: dwrite db/d19/f62 [0,4194304] 0 2024-03-13T14:49:49.652 INFO:tasks.workunit.client.0.smithi176.stdout:4/554: chown d7/d2d/d9b/ca3 2689342 1 2024-03-13T14:49:49.653 INFO:tasks.workunit.client.0.smithi176.stdout:0/449: rmdir d2/d5/dc/d59 39 2024-03-13T14:49:49.654 INFO:tasks.workunit.client.0.smithi176.stdout:4/555: dread - d7/d8/d45/f92 zero size 2024-03-13T14:49:49.656 INFO:tasks.workunit.client.0.smithi176.stdout:0/450: fdatasync d2/d5/d1e/f42 0 2024-03-13T14:49:49.656 INFO:tasks.workunit.client.1.smithi196.stdout:2/541: creat d0/d6/d90/fb0 x:0 0 0 2024-03-13T14:49:49.657 INFO:tasks.workunit.client.0.smithi176.stdout:1/471: write d2/f21 [2110863,44376] 0 2024-03-13T14:49:49.658 INFO:tasks.workunit.client.0.smithi176.stdout:0/451: fdatasync d2/db/d31/f7f 0 2024-03-13T14:49:49.659 INFO:tasks.workunit.client.0.smithi176.stdout:1/472: fsync d2/de/d10/d16/d59/f6e 0 2024-03-13T14:49:49.660 INFO:tasks.workunit.client.1.smithi196.stdout:9/564: rename d7/de/d4b/d1d/d35/l6d to d7/de/d4b/d1d/d22/lbe 0 2024-03-13T14:49:49.661 INFO:tasks.workunit.client.1.smithi196.stdout:2/542: write d0/d6/d34/f62 [268892,76578] 0 2024-03-13T14:49:49.661 INFO:tasks.workunit.client.0.smithi176.stdout:7/475: fdatasync d1/d1c/d26/d41/d2b/d7b/f7e 0 2024-03-13T14:49:49.664 INFO:tasks.workunit.client.1.smithi196.stdout:7/724: symlink d0/d9/d31/d77/dcf/de0/de7/lf3 0 2024-03-13T14:49:49.664 INFO:tasks.workunit.client.1.smithi196.stdout:5/686: truncate d4/d1b/dc8/d14/d19/d1c/fa2 2527987 0 2024-03-13T14:49:49.674 INFO:tasks.workunit.client.1.smithi196.stdout:8/583: dwrite d0/d1c/d29/f3c [4194304,4194304] 0 2024-03-13T14:49:49.677 INFO:tasks.workunit.client.1.smithi196.stdout:8/584: chown d0/d16/d3d/d9c/d87 694457 1 2024-03-13T14:49:49.678 INFO:tasks.workunit.client.0.smithi176.stdout:8/464: rename d0/f9 to d0/d24/d29/f8c 0 2024-03-13T14:49:49.680 INFO:tasks.workunit.client.0.smithi176.stdout:8/465: dread - d0/db/d39/d56/f65 zero size 2024-03-13T14:49:49.680 INFO:tasks.workunit.client.1.smithi196.stdout:3/628: creat db/d4c/fce x:0 0 0 2024-03-13T14:49:49.682 INFO:tasks.workunit.client.1.smithi196.stdout:3/629: fsync db/d19/d22/f98 0 2024-03-13T14:49:49.683 INFO:tasks.workunit.client.1.smithi196.stdout:2/543: creat d0/d6/d1d/fb1 x:0 0 0 2024-03-13T14:49:49.684 INFO:tasks.workunit.client.0.smithi176.stdout:3/557: write d4/d13/d83/fa1 [671282,108682] 0 2024-03-13T14:49:49.685 INFO:tasks.workunit.client.1.smithi196.stdout:5/687: creat d4/d1b/fd2 x:0 0 0 2024-03-13T14:49:49.686 INFO:tasks.workunit.client.0.smithi176.stdout:6/522: fdatasync d0/d10/d19/d27/d2b/d2e/f40 0 2024-03-13T14:49:49.688 INFO:tasks.workunit.client.0.smithi176.stdout:2/556: chown d1/dc/d1c/f55 152 1 2024-03-13T14:49:49.692 INFO:tasks.workunit.client.1.smithi196.stdout:8/585: mkdir d0/d10/d12/d1e/dc5 0 2024-03-13T14:49:49.693 INFO:tasks.workunit.client.1.smithi196.stdout:4/595: rmdir d1/d3 39 2024-03-13T14:49:49.698 INFO:tasks.workunit.client.1.smithi196.stdout:5/688: chown d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/f42 74342 1 2024-03-13T14:49:49.698 INFO:tasks.workunit.client.1.smithi196.stdout:9/565: dwrite d7/de/d16/f99 [0,4194304] 0 2024-03-13T14:49:49.702 INFO:tasks.workunit.client.0.smithi176.stdout:7/476: unlink d1/d1c/f77 0 2024-03-13T14:49:49.704 INFO:tasks.workunit.client.1.smithi196.stdout:1/691: dwrite d1/d3f/f43 [0,4194304] 0 2024-03-13T14:49:49.712 INFO:tasks.workunit.client.1.smithi196.stdout:8/586: symlink d0/d16/d3d/d9c/d87/lc6 0 2024-03-13T14:49:49.718 INFO:tasks.workunit.client.1.smithi196.stdout:5/689: creat d4/d1b/dc8/d14/d19/d26/d68/fd3 x:0 0 0 2024-03-13T14:49:49.719 INFO:tasks.workunit.client.0.smithi176.stdout:8/466: mknod d0/db/d58/c8d 0 2024-03-13T14:49:49.721 INFO:tasks.workunit.client.1.smithi196.stdout:9/566: fsync d7/de/d4b/d1d/d22/d4d/f6b 0 2024-03-13T14:49:49.722 INFO:tasks.workunit.client.0.smithi176.stdout:3/558: mknod d4/d28/d31/d52/cbe 0 2024-03-13T14:49:49.722 INFO:tasks.workunit.client.1.smithi196.stdout:2/544: rmdir d0 39 2024-03-13T14:49:49.725 INFO:tasks.workunit.client.1.smithi196.stdout:7/725: write d0/d9/d1e/d58/d72/f84 [504186,13278] 0 2024-03-13T14:49:49.727 INFO:tasks.workunit.client.1.smithi196.stdout:5/690: read d4/d1b/dc8/d14/d37/fb5 [3385097,102024] 0 2024-03-13T14:49:49.728 INFO:tasks.workunit.client.1.smithi196.stdout:7/726: chown d0/d9/d1e/d58/cab 7080603 1 2024-03-13T14:49:49.729 INFO:tasks.workunit.client.1.smithi196.stdout:5/691: chown d4/d1b/dc8/d14/c1e 46929629 1 2024-03-13T14:49:49.731 INFO:tasks.workunit.client.0.smithi176.stdout:5/484: truncate f0 7012470 0 2024-03-13T14:49:49.733 INFO:tasks.workunit.client.0.smithi176.stdout:4/556: creat d7/dab/db5/fc6 x:0 0 0 2024-03-13T14:49:49.735 INFO:tasks.workunit.client.0.smithi176.stdout:0/452: mknod d2/d5/d1e/c89 0 2024-03-13T14:49:49.736 INFO:tasks.workunit.client.0.smithi176.stdout:0/453: write d2/d5/d1e/f42 [341325,1502] 0 2024-03-13T14:49:49.737 INFO:tasks.workunit.client.0.smithi176.stdout:1/473: mknod d2/c93 0 2024-03-13T14:49:49.739 INFO:tasks.workunit.client.0.smithi176.stdout:1/474: dread - d2/de/f26 zero size 2024-03-13T14:49:49.740 INFO:tasks.workunit.client.1.smithi196.stdout:2/545: fsync d0/d6/d1d/f51 0 2024-03-13T14:49:49.741 INFO:tasks.workunit.client.0.smithi176.stdout:1/475: write d2/d29/f39 [4633551,128722] 0 2024-03-13T14:49:49.741 INFO:tasks.workunit.client.0.smithi176.stdout:7/477: creat d1/d1c/d4a/f8d x:0 0 0 2024-03-13T14:49:49.744 INFO:tasks.workunit.client.0.smithi176.stdout:2/557: write d1/dc/f52 [2347161,37404] 0 2024-03-13T14:49:49.746 INFO:tasks.workunit.client.0.smithi176.stdout:8/467: rmdir d0/db 39 2024-03-13T14:49:49.748 INFO:tasks.workunit.client.1.smithi196.stdout:4/596: creat d1/d3/d7/d35/d3e/db1/dc3/fcc x:0 0 0 2024-03-13T14:49:49.749 INFO:tasks.workunit.client.0.smithi176.stdout:3/559: mknod d4/d13/d16/cbf 0 2024-03-13T14:49:49.750 INFO:tasks.workunit.client.1.smithi196.stdout:7/727: truncate d0/f6a 2604972 0 2024-03-13T14:49:49.754 INFO:tasks.workunit.client.1.smithi196.stdout:3/630: dwrite db/d19/f2b [0,4194304] 0 2024-03-13T14:49:49.758 INFO:tasks.workunit.client.0.smithi176.stdout:5/485: fsync d5/d2e/d7e/d9f/d4c/f92 0 2024-03-13T14:49:49.760 INFO:tasks.workunit.client.1.smithi196.stdout:1/692: mkdir d1/d3f/d5b/d74/ded 0 2024-03-13T14:49:49.763 INFO:tasks.workunit.client.1.smithi196.stdout:9/567: dwrite d7/de/f29 [0,4194304] 0 2024-03-13T14:49:49.764 INFO:tasks.workunit.client.1.smithi196.stdout:6/767: sync 2024-03-13T14:49:49.764 INFO:tasks.workunit.client.1.smithi196.stdout:0/655: sync 2024-03-13T14:49:49.767 INFO:tasks.workunit.client.1.smithi196.stdout:0/656: fsync d8/da/d25/d26/d35/d3b/f7d 0 2024-03-13T14:49:49.782 INFO:tasks.workunit.client.1.smithi196.stdout:5/692: dread d4/d1b/dc8/f81 [0,4194304] 0 2024-03-13T14:49:49.791 INFO:tasks.workunit.client.0.smithi176.stdout:4/557: mknod d7/d10/d46/d4a/cc7 0 2024-03-13T14:49:49.792 INFO:tasks.workunit.client.0.smithi176.stdout:1/476: rename d2/de/f4d to d2/d29/d73/f94 0 2024-03-13T14:49:49.792 INFO:tasks.workunit.client.0.smithi176.stdout:7/478: chown d1/d28/d3a/d1b 11691401 1 2024-03-13T14:49:49.793 INFO:tasks.workunit.client.0.smithi176.stdout:9/570: getdents d0/d40/d53/d6d 0 2024-03-13T14:49:49.793 INFO:tasks.workunit.client.0.smithi176.stdout:2/558: symlink d1/dc/d2d/d59/lbe 0 2024-03-13T14:49:49.795 INFO:tasks.workunit.client.0.smithi176.stdout:6/523: link d0/d10/d19/d9f/f82 d0/d10/d19/d27/fac 0 2024-03-13T14:49:49.797 INFO:tasks.workunit.client.0.smithi176.stdout:4/558: rmdir d7/d2d 39 2024-03-13T14:49:49.798 INFO:tasks.workunit.client.0.smithi176.stdout:3/560: dwrite d4/d28/d34/f6e [0,4194304] 0 2024-03-13T14:49:49.799 INFO:tasks.workunit.client.0.smithi176.stdout:6/524: dread - d0/de/d38/d70/f8e zero size 2024-03-13T14:49:49.805 INFO:tasks.workunit.client.0.smithi176.stdout:7/479: read d1/d1c/f5f [1436511,60435] 0 2024-03-13T14:49:49.805 INFO:tasks.workunit.client.0.smithi176.stdout:3/561: chown d4/d13/d16/d80 16033199 1 2024-03-13T14:49:49.808 INFO:tasks.workunit.client.0.smithi176.stdout:8/468: dwrite d0/d5/d3b/f3c [4194304,4194304] 0 2024-03-13T14:49:49.810 INFO:tasks.workunit.client.0.smithi176.stdout:9/571: rename d0/db/d89/d32/d58/l9d to d0/db/d89/d32/d58/d5f/d84/d9a/lbf 0 2024-03-13T14:49:49.812 INFO:tasks.workunit.client.0.smithi176.stdout:1/477: write d2/de/d10/f4f [48775,23702] 0 2024-03-13T14:49:49.816 INFO:tasks.workunit.client.0.smithi176.stdout:2/559: write d1/dc/d40/da0/fa6 [4666580,57559] 0 2024-03-13T14:49:49.817 INFO:tasks.workunit.client.0.smithi176.stdout:5/486: dwrite d5/d31/f3e [0,4194304] 0 2024-03-13T14:49:49.818 INFO:tasks.workunit.client.0.smithi176.stdout:7/480: unlink d1/d7/f10 0 2024-03-13T14:49:49.827 INFO:tasks.workunit.client.1.smithi196.stdout:4/597: creat d1/d3/d4c/d72/d7d/fcd x:0 0 0 2024-03-13T14:49:49.827 INFO:tasks.workunit.client.0.smithi176.stdout:3/562: dread d4/d28/d34/f3f [0,4194304] 0 2024-03-13T14:49:49.827 INFO:tasks.workunit.client.0.smithi176.stdout:6/525: getdents d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1 0 2024-03-13T14:49:49.827 INFO:tasks.workunit.client.0.smithi176.stdout:9/572: symlink d0/db/d89/d32/d58/lc0 0 2024-03-13T14:49:49.830 INFO:tasks.workunit.client.1.smithi196.stdout:1/693: creat d1/d9/d33/d4b/d66/fee x:0 0 0 2024-03-13T14:49:49.833 INFO:tasks.workunit.client.0.smithi176.stdout:8/469: mknod d0/db/dc/d40/c8e 0 2024-03-13T14:49:49.834 INFO:tasks.workunit.client.1.smithi196.stdout:7/728: dread d0/d9/d1e/d3f/d6c/fb0 [0,4194304] 0 2024-03-13T14:49:49.835 INFO:tasks.workunit.client.1.smithi196.stdout:6/768: mknod d6/d2a/dc/d47/cf9 0 2024-03-13T14:49:49.835 INFO:tasks.workunit.client.0.smithi176.stdout:2/560: dwrite d1/dc/d2d/d59/d80/d5c/f5f [0,4194304] 0 2024-03-13T14:49:49.837 INFO:tasks.workunit.client.1.smithi196.stdout:0/657: fdatasync d8/da/f32 0 2024-03-13T14:49:49.838 INFO:tasks.workunit.client.0.smithi176.stdout:2/561: readlink d1/dc/d40/da0/la8 0 2024-03-13T14:49:49.838 INFO:tasks.workunit.client.0.smithi176.stdout:5/487: dwrite d5/d22/f29 [0,4194304] 0 2024-03-13T14:49:49.839 INFO:tasks.workunit.client.0.smithi176.stdout:1/478: symlink d2/l95 0 2024-03-13T14:49:49.841 INFO:tasks.workunit.client.0.smithi176.stdout:3/563: unlink d4/l68 0 2024-03-13T14:49:49.847 INFO:tasks.workunit.client.0.smithi176.stdout:6/526: creat d0/d10/d19/d27/d2b/d2e/fad x:0 0 0 2024-03-13T14:49:49.848 INFO:tasks.workunit.client.0.smithi176.stdout:4/559: truncate d7/f24 3438027 0 2024-03-13T14:49:49.850 INFO:tasks.workunit.client.0.smithi176.stdout:9/573: mknod d0/db/d89/d7a/d8b/cc1 0 2024-03-13T14:49:49.851 INFO:tasks.workunit.client.1.smithi196.stdout:8/587: link d0/d16/c24 d0/d10/d12/cc7 0 2024-03-13T14:49:49.853 INFO:tasks.workunit.client.0.smithi176.stdout:8/470: symlink d0/db/l8f 0 2024-03-13T14:49:49.853 INFO:tasks.workunit.client.0.smithi176.stdout:7/481: link d1/d28/f2a d1/d3f/d81/f8e 0 2024-03-13T14:49:49.855 INFO:tasks.workunit.client.0.smithi176.stdout:7/482: dread - d1/d28/d3a/f8b zero size 2024-03-13T14:49:49.856 INFO:tasks.workunit.client.0.smithi176.stdout:2/562: dread d1/dc/d1c/f4c [0,4194304] 0 2024-03-13T14:49:49.859 INFO:tasks.workunit.client.0.smithi176.stdout:5/488: symlink d5/d2b/la3 0 2024-03-13T14:49:49.863 INFO:tasks.workunit.client.0.smithi176.stdout:9/574: creat d0/db/d89/fc2 x:0 0 0 2024-03-13T14:49:49.864 INFO:tasks.workunit.client.1.smithi196.stdout:9/568: write d7/de/d4b/d1d/f75 [105759,80189] 0 2024-03-13T14:49:49.865 INFO:tasks.workunit.client.0.smithi176.stdout:4/560: creat d7/d10/d46/d4a/d5d/fc8 x:0 0 0 2024-03-13T14:49:49.870 INFO:tasks.workunit.client.0.smithi176.stdout:9/575: write d0/d1e/d1f/d68/fb4 [907550,119271] 0 2024-03-13T14:49:49.871 INFO:tasks.workunit.client.1.smithi196.stdout:9/569: read d7/d24/d28/d6c/f8f [155414,46888] 0 2024-03-13T14:49:49.872 INFO:tasks.workunit.client.1.smithi196.stdout:5/693: write d4/d1b/dc8/d14/d19/d26/d68/f77 [435760,75299] 0 2024-03-13T14:49:49.874 INFO:tasks.workunit.client.0.smithi176.stdout:7/483: dwrite d1/f61 [0,4194304] 0 2024-03-13T14:49:49.875 INFO:tasks.workunit.client.0.smithi176.stdout:2/563: creat d1/d2a/fbf x:0 0 0 2024-03-13T14:49:49.877 INFO:tasks.workunit.client.1.smithi196.stdout:1/694: fsync d1/d9/d33/d4b/f1a 0 2024-03-13T14:49:49.878 INFO:tasks.workunit.client.1.smithi196.stdout:3/631: dread db/f29 [0,4194304] 0 2024-03-13T14:49:49.879 INFO:tasks.workunit.client.0.smithi176.stdout:5/489: mknod d5/d22/d15/ca4 0 2024-03-13T14:49:49.879 INFO:tasks.workunit.client.1.smithi196.stdout:6/769: dread d6/d2a/dc/d1c/da7/fb5 [0,4194304] 0 2024-03-13T14:49:49.880 INFO:tasks.workunit.client.0.smithi176.stdout:3/564: dwrite d4/d13/f73 [0,4194304] 0 2024-03-13T14:49:49.892 INFO:tasks.workunit.client.1.smithi196.stdout:0/658: stat d8/da/d25/d26/d35/d3b/d5f/cd2 0 2024-03-13T14:49:49.893 INFO:tasks.workunit.client.0.smithi176.stdout:8/471: dread d0/d24/d29/f4e [4194304,4194304] 0 2024-03-13T14:49:49.895 INFO:tasks.workunit.client.1.smithi196.stdout:4/598: rename d1/l20 to d1/d3/lce 0 2024-03-13T14:49:49.895 INFO:tasks.workunit.client.1.smithi196.stdout:8/588: mkdir d0/d10/da5/db2/dc8 0 2024-03-13T14:49:49.897 INFO:tasks.workunit.client.0.smithi176.stdout:9/576: creat d0/db/d89/d4d/fc3 x:0 0 0 2024-03-13T14:49:49.899 INFO:tasks.workunit.client.1.smithi196.stdout:6/770: symlink d6/d7/df/lfa 0 2024-03-13T14:49:49.899 INFO:tasks.workunit.client.1.smithi196.stdout:4/599: write d1/f54 [3636645,125443] 0 2024-03-13T14:49:49.900 INFO:tasks.workunit.client.0.smithi176.stdout:2/564: mknod d1/dc/d40/da0/cc0 0 2024-03-13T14:49:49.901 INFO:tasks.workunit.client.1.smithi196.stdout:3/632: mknod db/d19/d50/db7/ccf 0 2024-03-13T14:49:49.902 INFO:tasks.workunit.client.0.smithi176.stdout:2/565: chown d1/dc/d2d/d59/d80/d5c/c7b 139710 1 2024-03-13T14:49:49.903 INFO:tasks.workunit.client.1.smithi196.stdout:4/600: write d1/d3/d4c/fa5 [356614,18579] 0 2024-03-13T14:49:49.903 INFO:tasks.workunit.client.1.smithi196.stdout:1/695: dwrite d1/d9/d33/d4b/f1d [0,4194304] 0 2024-03-13T14:49:49.904 INFO:tasks.workunit.client.1.smithi196.stdout:5/694: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/fb0 706628 0 2024-03-13T14:49:49.908 INFO:tasks.workunit.client.0.smithi176.stdout:3/565: creat d4/d28/fc0 x:0 0 0 2024-03-13T14:49:49.908 INFO:tasks.workunit.client.0.smithi176.stdout:6/527: link d0/d10/l32 d0/de/d38/d7e/lae 0 2024-03-13T14:49:49.911 INFO:tasks.workunit.client.0.smithi176.stdout:6/528: chown d0/d10/d19/d27/d2b/d2e/d3e/d9a 381 1 2024-03-13T14:49:49.917 INFO:tasks.workunit.client.0.smithi176.stdout:1/479: truncate d2/d30/d63/d22/d71/f87 1981970 0 2024-03-13T14:49:49.917 INFO:tasks.workunit.client.0.smithi176.stdout:7/484: mkdir d1/d28/d3a/d1b/d8f 0 2024-03-13T14:49:49.918 INFO:tasks.workunit.client.1.smithi196.stdout:2/546: rename d0/d50/l70 to d0/d7/d66/lb2 0 2024-03-13T14:49:49.919 INFO:tasks.workunit.client.0.smithi176.stdout:7/485: dread - d1/d1c/f79 zero size 2024-03-13T14:49:49.920 INFO:tasks.workunit.client.0.smithi176.stdout:7/486: stat d1/d3f/d4b/d63 0 2024-03-13T14:49:49.921 INFO:tasks.workunit.client.0.smithi176.stdout:5/490: mknod d5/d2e/ca5 0 2024-03-13T14:49:49.922 INFO:tasks.workunit.client.0.smithi176.stdout:2/566: write d1/dc/d2d/d38/f3f [1033896,2953] 0 2024-03-13T14:49:49.922 INFO:tasks.workunit.client.0.smithi176.stdout:0/454: sync 2024-03-13T14:49:49.923 INFO:tasks.workunit.client.1.smithi196.stdout:1/696: creat d1/d38/fef x:0 0 0 2024-03-13T14:49:49.924 INFO:tasks.workunit.client.0.smithi176.stdout:8/472: mknod d0/d5/c90 0 2024-03-13T14:49:49.925 INFO:tasks.workunit.client.1.smithi196.stdout:7/729: dwrite d0/f6a [0,4194304] 0 2024-03-13T14:49:49.926 INFO:tasks.workunit.client.0.smithi176.stdout:6/529: dwrite d0/d10/d19/d9f/f98 [0,4194304] 0 2024-03-13T14:49:49.927 INFO:tasks.workunit.client.1.smithi196.stdout:4/601: unlink d1/d3/c31 0 2024-03-13T14:49:49.927 INFO:tasks.workunit.client.0.smithi176.stdout:8/473: write d0/d5/f17 [4642813,53226] 0 2024-03-13T14:49:49.929 INFO:tasks.workunit.client.0.smithi176.stdout:3/566: rename d4/d13/d81/d89/l5f to d4/d13/d5e/lc1 0 2024-03-13T14:49:49.935 INFO:tasks.workunit.client.1.smithi196.stdout:8/589: mkdir d0/d10/d70/dc9 0 2024-03-13T14:49:49.936 INFO:tasks.workunit.client.0.smithi176.stdout:1/480: symlink d2/de/d3f/d19/l96 0 2024-03-13T14:49:49.947 INFO:tasks.workunit.client.1.smithi196.stdout:7/730: fsync d0/d9/d1e/f4b 0 2024-03-13T14:49:49.951 INFO:tasks.workunit.client.0.smithi176.stdout:2/567: truncate d1/dc/d2d/d4b/d69/fb4 737746 0 2024-03-13T14:49:49.951 INFO:tasks.workunit.client.0.smithi176.stdout:4/561: truncate d7/d8/d15/f81 815422 0 2024-03-13T14:49:49.954 INFO:tasks.workunit.client.0.smithi176.stdout:8/474: mkdir d0/db/d39/d56/d91 0 2024-03-13T14:49:49.956 INFO:tasks.workunit.client.0.smithi176.stdout:4/562: truncate d7/d10/d46/d4a/d5d/da1/f86 394763 0 2024-03-13T14:49:49.956 INFO:tasks.workunit.client.1.smithi196.stdout:8/590: mkdir d0/d16/d71/d9a/dca 0 2024-03-13T14:49:49.956 INFO:tasks.workunit.client.0.smithi176.stdout:9/577: write d0/d1e/d1f/f2c [117420,31692] 0 2024-03-13T14:49:49.957 INFO:tasks.workunit.client.1.smithi196.stdout:0/659: dread d8/da/d25/f7b [0,4194304] 0 2024-03-13T14:49:49.958 INFO:tasks.workunit.client.0.smithi176.stdout:7/487: rename d1/d1c/d26/d41/f3e to d1/d3f/f90 0 2024-03-13T14:49:49.959 INFO:tasks.workunit.client.0.smithi176.stdout:8/475: dread d0/db/fd [4194304,4194304] 0 2024-03-13T14:49:49.960 INFO:tasks.workunit.client.1.smithi196.stdout:0/660: write d8/da/d25/fcd [224496,104675] 0 2024-03-13T14:49:49.961 INFO:tasks.workunit.client.0.smithi176.stdout:7/488: truncate d1/d28/d3a/f8b 235702 0 2024-03-13T14:49:49.962 INFO:tasks.workunit.client.0.smithi176.stdout:7/489: truncate d1/d3f/d76/f6d 734609 0 2024-03-13T14:49:49.966 INFO:tasks.workunit.client.1.smithi196.stdout:1/697: symlink d1/d9/d33/d4b/lf0 0 2024-03-13T14:49:49.970 INFO:tasks.workunit.client.1.smithi196.stdout:5/695: write d4/d1b/dc8/f81 [2149085,10876] 0 2024-03-13T14:49:49.970 INFO:tasks.workunit.client.0.smithi176.stdout:2/568: creat d1/dc/d61/fc1 x:0 0 0 2024-03-13T14:49:49.973 INFO:tasks.workunit.client.1.smithi196.stdout:7/731: fsync d0/d9/d1e/d58/d72/d6e/fba 0 2024-03-13T14:49:49.974 INFO:tasks.workunit.client.0.smithi176.stdout:4/563: write d7/d8/d33/f6d [3581278,120944] 0 2024-03-13T14:49:49.975 INFO:tasks.workunit.client.0.smithi176.stdout:5/491: write d5/f55 [3832888,73948] 0 2024-03-13T14:49:49.975 INFO:tasks.workunit.client.1.smithi196.stdout:7/732: read d0/d78/d81/fc9 [622095,113773] 0 2024-03-13T14:49:49.976 INFO:tasks.workunit.client.0.smithi176.stdout:0/455: write d2/db/d31/f4c [4625489,108230] 0 2024-03-13T14:49:49.977 INFO:tasks.workunit.client.1.smithi196.stdout:8/591: mkdir d0/d63/dcb 0 2024-03-13T14:49:49.977 INFO:tasks.workunit.client.0.smithi176.stdout:4/564: stat d7/d8/d33/c9a 0 2024-03-13T14:49:49.979 INFO:tasks.workunit.client.0.smithi176.stdout:6/530: rename d0/de/d8d/f95 to d0/de/d38/d70/faf 0 2024-03-13T14:49:49.979 INFO:tasks.workunit.client.1.smithi196.stdout:7/733: chown d0/d9/d1e/d58/d72/c71 1510 1 2024-03-13T14:49:49.980 INFO:tasks.workunit.client.1.smithi196.stdout:8/592: read - d0/d10/d12/d1e/fbb zero size 2024-03-13T14:49:49.981 INFO:tasks.workunit.client.0.smithi176.stdout:6/531: write d0/d10/d19/fa8 [346361,14683] 0 2024-03-13T14:49:49.981 INFO:tasks.workunit.client.1.smithi196.stdout:9/570: rename d7/de/d4b/d1d/d22/d4d to d7/d24/dbf 0 2024-03-13T14:49:49.982 INFO:tasks.workunit.client.0.smithi176.stdout:3/567: rmdir d4/d13/d81/d89/d5b 39 2024-03-13T14:49:49.995 INFO:tasks.workunit.client.1.smithi196.stdout:4/602: dwrite d1/d3/d4c/d72/d7d/d80/fac [0,4194304] 0 2024-03-13T14:49:49.996 INFO:tasks.workunit.client.1.smithi196.stdout:5/696: creat d4/d1b/dc8/d3c/d6a/fd4 x:0 0 0 2024-03-13T14:49:49.998 INFO:tasks.workunit.client.1.smithi196.stdout:5/697: readlink d4/d60/ld1 0 2024-03-13T14:49:50.003 INFO:tasks.workunit.client.1.smithi196.stdout:6/771: rename d6/d7/dd/d60/f79 to d6/d2a/dc/d71/ffb 0 2024-03-13T14:49:50.005 INFO:tasks.workunit.client.1.smithi196.stdout:0/661: fdatasync d8/da/d25/d26/d35/d3b/d5f/d17/f70 0 2024-03-13T14:49:50.005 INFO:tasks.workunit.client.1.smithi196.stdout:6/772: chown d6/d9/d17/d19 3188532 1 2024-03-13T14:49:50.009 INFO:tasks.workunit.client.0.smithi176.stdout:8/476: creat d0/db/d58/f92 x:0 0 0 2024-03-13T14:49:50.010 INFO:tasks.workunit.client.1.smithi196.stdout:6/773: stat d6/d6f/fb2 0 2024-03-13T14:49:50.010 INFO:tasks.workunit.client.0.smithi176.stdout:9/578: mknod d0/d40/d53/da0/cc4 0 2024-03-13T14:49:50.011 INFO:tasks.workunit.client.1.smithi196.stdout:1/698: dread d1/d9/d33/d4b/f1b [4194304,4194304] 0 2024-03-13T14:49:50.011 INFO:tasks.workunit.client.0.smithi176.stdout:6/532: symlink d0/de/d38/d88/d9d/lb0 0 2024-03-13T14:49:50.012 INFO:tasks.workunit.client.0.smithi176.stdout:1/481: write d2/de/f17 [63071,96374] 0 2024-03-13T14:49:50.013 INFO:tasks.workunit.client.1.smithi196.stdout:2/547: dwrite d0/f37 [0,4194304] 0 2024-03-13T14:49:50.014 INFO:tasks.workunit.client.0.smithi176.stdout:4/565: dread - d7/d2d/fa4 zero size 2024-03-13T14:49:50.014 INFO:tasks.workunit.client.0.smithi176.stdout:3/568: creat d4/d13/d5e/fc2 x:0 0 0 2024-03-13T14:49:50.015 INFO:tasks.workunit.client.1.smithi196.stdout:6/774: dread - d6/d2a/de2/feb zero size 2024-03-13T14:49:50.015 INFO:tasks.workunit.client.0.smithi176.stdout:8/477: fdatasync d0/db/d39/d56/f5e 0 2024-03-13T14:49:50.016 INFO:tasks.workunit.client.0.smithi176.stdout:6/533: fsync d0/de/d38/d70/f8e 0 2024-03-13T14:49:50.019 INFO:tasks.workunit.client.0.smithi176.stdout:6/534: stat d0/d10/d19/d9f/d76/d7c/f9b 0 2024-03-13T14:49:50.020 INFO:tasks.workunit.client.0.smithi176.stdout:2/569: unlink d1/dc/d2d/d59/d80/d77/fbd 0 2024-03-13T14:49:50.021 INFO:tasks.workunit.client.0.smithi176.stdout:2/570: stat d1/dc/d2d/d59/d80/d5c/d78/cb3 0 2024-03-13T14:49:50.029 INFO:tasks.workunit.client.1.smithi196.stdout:0/662: dwrite d8/da/fc [0,4194304] 0 2024-03-13T14:49:50.030 INFO:tasks.workunit.client.1.smithi196.stdout:5/698: fdatasync d4/d1b/dc8/d14/d19/d26/f92 0 2024-03-13T14:49:50.035 INFO:tasks.workunit.client.0.smithi176.stdout:9/579: mkdir d0/db/d89/d32/d76/dc5 0 2024-03-13T14:49:50.037 INFO:tasks.workunit.client.0.smithi176.stdout:9/580: chown d0/d1e/d1f/f41 2783 1 2024-03-13T14:49:50.038 INFO:tasks.workunit.client.1.smithi196.stdout:8/593: dread d0/d10/d12/d1e/f5e [0,4194304] 0 2024-03-13T14:49:50.042 INFO:tasks.workunit.client.1.smithi196.stdout:0/663: read d8/f1e [3577847,23038] 0 2024-03-13T14:49:50.042 INFO:tasks.workunit.client.0.smithi176.stdout:8/478: mkdir d0/db/d39/d56/d93 0 2024-03-13T14:49:50.043 INFO:tasks.workunit.client.0.smithi176.stdout:6/535: dread d0/d10/d19/d27/d2b/f3a [0,4194304] 0 2024-03-13T14:49:50.045 INFO:tasks.workunit.client.0.smithi176.stdout:4/566: creat d7/d10/d46/d4a/d5d/fc9 x:0 0 0 2024-03-13T14:49:50.051 INFO:tasks.workunit.client.1.smithi196.stdout:7/734: rename d0/d6/f45 to d0/d6/d33/d57/dae/ff4 0 2024-03-13T14:49:50.053 INFO:tasks.workunit.client.0.smithi176.stdout:5/492: rename d5/d31/d47/d80/f90 to d5/d22/fa6 0 2024-03-13T14:49:50.055 INFO:tasks.workunit.client.0.smithi176.stdout:1/482: mkdir d2/de/d3f/d50/d83/d97 0 2024-03-13T14:49:50.056 INFO:tasks.workunit.client.0.smithi176.stdout:9/581: symlink d0/db/d89/d7a/lc6 0 2024-03-13T14:49:50.058 INFO:tasks.workunit.client.1.smithi196.stdout:5/699: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/fd5 x:0 0 0 2024-03-13T14:49:50.058 INFO:tasks.workunit.client.0.smithi176.stdout:9/582: dread - d0/db/d89/d32/d58/f83 zero size 2024-03-13T14:49:50.059 INFO:tasks.workunit.client.1.smithi196.stdout:1/699: write d1/d9/d4e/d55/d63/db9/dbb/fc2 [297166,63249] 0 2024-03-13T14:49:50.060 INFO:tasks.workunit.client.1.smithi196.stdout:5/700: fsync d4/d1b/dc8/f81 0 2024-03-13T14:49:50.060 INFO:tasks.workunit.client.0.smithi176.stdout:8/479: creat d0/db/d58/d86/f94 x:0 0 0 2024-03-13T14:49:50.062 INFO:tasks.workunit.client.1.smithi196.stdout:4/603: creat d1/d3/fcf x:0 0 0 2024-03-13T14:49:50.062 INFO:tasks.workunit.client.1.smithi196.stdout:1/700: dread - d1/d9/d4e/d55/fc3 zero size 2024-03-13T14:49:50.063 INFO:tasks.workunit.client.0.smithi176.stdout:6/536: mkdir d0/d10/d19/d27/d2b/db1 0 2024-03-13T14:49:50.064 INFO:tasks.workunit.client.0.smithi176.stdout:2/571: write d1/dc/d1c/f55 [935500,130737] 0 2024-03-13T14:49:50.066 INFO:tasks.workunit.client.1.smithi196.stdout:0/664: mkdir d8/da/d25/dd7 0 2024-03-13T14:49:50.067 INFO:tasks.workunit.client.0.smithi176.stdout:6/537: write d0/d10/d19/d27/f4d [4447135,800] 0 2024-03-13T14:49:50.068 INFO:tasks.workunit.client.1.smithi196.stdout:0/665: readlink d8/d37/lbc 0 2024-03-13T14:49:50.069 INFO:tasks.workunit.client.0.smithi176.stdout:5/493: mkdir d5/d2e/d7e/d9f/d4c/d7b/da7 0 2024-03-13T14:49:50.072 INFO:tasks.workunit.client.0.smithi176.stdout:1/483: creat d2/de/d3f/d50/d83/f98 x:0 0 0 2024-03-13T14:49:50.074 INFO:tasks.workunit.client.0.smithi176.stdout:6/538: read d0/d10/d19/d27/d2b/d2e/f2d [2667311,128200] 0 2024-03-13T14:49:50.074 INFO:tasks.workunit.client.0.smithi176.stdout:1/484: chown d2/de/d3f/d50/d83 831 1 2024-03-13T14:49:50.075 INFO:tasks.workunit.client.0.smithi176.stdout:6/539: stat d0/de/ca7 0 2024-03-13T14:49:50.076 INFO:tasks.workunit.client.1.smithi196.stdout:5/701: dread d4/d1b/dc8/d4a/fa3 [0,4194304] 0 2024-03-13T14:49:50.076 INFO:tasks.workunit.client.0.smithi176.stdout:3/569: creat d4/d28/d31/fc3 x:0 0 0 2024-03-13T14:49:50.077 INFO:tasks.workunit.client.1.smithi196.stdout:2/548: creat d0/d6/fb3 x:0 0 0 2024-03-13T14:49:50.077 INFO:tasks.workunit.client.0.smithi176.stdout:3/570: chown d4 70098 1 2024-03-13T14:49:50.078 INFO:tasks.workunit.client.0.smithi176.stdout:8/480: creat d0/db/d39/d56/f95 x:0 0 0 2024-03-13T14:49:50.081 INFO:tasks.workunit.client.0.smithi176.stdout:9/583: dwrite d0/d40/d53/fb5 [0,4194304] 0 2024-03-13T14:49:50.082 INFO:tasks.workunit.client.1.smithi196.stdout:1/701: symlink d1/d9/d4e/d55/d90/lf1 0 2024-03-13T14:49:50.086 INFO:tasks.workunit.client.1.smithi196.stdout:7/735: creat d0/d9/d1e/d58/d72/da9/da3/dc2/ff5 x:0 0 0 2024-03-13T14:49:50.086 INFO:tasks.workunit.client.1.smithi196.stdout:8/594: dwrite d0/d10/f11 [4194304,4194304] 0 2024-03-13T14:49:50.088 INFO:tasks.workunit.client.1.smithi196.stdout:6/775: dread d6/d2a/dc/d1c/fa4 [0,4194304] 0 2024-03-13T14:49:50.095 INFO:tasks.workunit.client.0.smithi176.stdout:5/494: symlink d5/d75/la8 0 2024-03-13T14:49:50.097 INFO:tasks.workunit.client.1.smithi196.stdout:6/776: read d6/d2a/dc/d1c/d34/fbd [1001774,57365] 0 2024-03-13T14:49:50.097 INFO:tasks.workunit.client.1.smithi196.stdout:5/702: symlink d4/d1b/dc8/d3c/d6a/d93/ld6 0 2024-03-13T14:49:50.100 INFO:tasks.workunit.client.1.smithi196.stdout:2/549: rmdir d0/d7/d9/d61/d83 39 2024-03-13T14:49:50.100 INFO:tasks.workunit.client.0.smithi176.stdout:4/567: dwrite d7/d8/d45/f99 [0,4194304] 0 2024-03-13T14:49:50.106 INFO:tasks.workunit.client.0.smithi176.stdout:7/490: sync 2024-03-13T14:49:50.109 INFO:tasks.workunit.client.1.smithi196.stdout:7/736: chown d0/c2 867 1 2024-03-13T14:49:50.111 INFO:tasks.workunit.client.1.smithi196.stdout:0/666: mkdir d8/da/dd8 0 2024-03-13T14:49:50.111 INFO:tasks.workunit.client.0.smithi176.stdout:2/572: symlink d1/dc/d2d/d90/d99/lc2 0 2024-03-13T14:49:50.113 INFO:tasks.workunit.client.0.smithi176.stdout:2/573: readlink d1/dc/d40/da0/lb0 0 2024-03-13T14:49:50.116 INFO:tasks.workunit.client.1.smithi196.stdout:4/604: creat d1/fd0 x:0 0 0 2024-03-13T14:49:50.117 INFO:tasks.workunit.client.0.smithi176.stdout:0/456: rename d2/db/l3a to d2/d5/l8a 0 2024-03-13T14:49:50.119 INFO:tasks.workunit.client.0.smithi176.stdout:0/457: write d2/d5/f87 [259984,59624] 0 2024-03-13T14:49:50.122 INFO:tasks.workunit.client.0.smithi176.stdout:5/495: truncate d5/d22/d15/f21 1633583 0 2024-03-13T14:49:50.122 INFO:tasks.workunit.client.0.smithi176.stdout:8/481: dread d0/d24/f32 [4194304,4194304] 0 2024-03-13T14:49:50.124 INFO:tasks.workunit.client.0.smithi176.stdout:4/568: creat d7/d10/d2c/fca x:0 0 0 2024-03-13T14:49:50.127 INFO:tasks.workunit.client.1.smithi196.stdout:6/777: mkdir d6/d9/d17/dfc 0 2024-03-13T14:49:50.128 INFO:tasks.workunit.client.0.smithi176.stdout:1/485: fsync d2/d29/d73/f94 0 2024-03-13T14:49:50.128 INFO:tasks.workunit.client.1.smithi196.stdout:5/703: creat d4/d1b/dc8/d14/d19/d1c/d25/fd7 x:0 0 0 2024-03-13T14:49:50.130 INFO:tasks.workunit.client.1.smithi196.stdout:2/550: creat d0/d7/d9/d9a/fb4 x:0 0 0 2024-03-13T14:49:50.131 INFO:tasks.workunit.client.0.smithi176.stdout:7/491: unlink d1/d1c/f5f 0 2024-03-13T14:49:50.132 INFO:tasks.workunit.client.1.smithi196.stdout:2/551: chown d0/d6/d1d/fb1 712284 1 2024-03-13T14:49:50.133 INFO:tasks.workunit.client.1.smithi196.stdout:7/737: creat d0/d9/d1e/d58/d72/da9/da3/ff6 x:0 0 0 2024-03-13T14:49:50.134 INFO:tasks.workunit.client.0.smithi176.stdout:3/571: creat d4/d13/d16/db6/fc4 x:0 0 0 2024-03-13T14:49:50.134 INFO:tasks.workunit.client.1.smithi196.stdout:1/702: dread d1/d38/f3b [0,4194304] 0 2024-03-13T14:49:50.136 INFO:tasks.workunit.client.1.smithi196.stdout:8/595: dread d0/d1c/d29/d47/f80 [0,4194304] 0 2024-03-13T14:49:50.136 INFO:tasks.workunit.client.0.smithi176.stdout:1/486: dread d2/de/d10/d16/d4a/d5f/f81 [0,4194304] 0 2024-03-13T14:49:50.137 INFO:tasks.workunit.client.0.smithi176.stdout:3/572: read - d4/d28/d31/f62 zero size 2024-03-13T14:49:50.140 INFO:tasks.workunit.client.0.smithi176.stdout:1/487: write d2/de/d10/d16/d4a/d5f/f81 [1419463,102219] 0 2024-03-13T14:49:50.142 INFO:tasks.workunit.client.0.smithi176.stdout:6/540: dwrite d0/d10/d19/d27/fac [0,4194304] 0 2024-03-13T14:49:50.143 INFO:tasks.workunit.client.0.smithi176.stdout:9/584: rename d0/d1e/d1f/d68/c86 to d0/d40/d53/d6d/d97/cc7 0 2024-03-13T14:49:50.143 INFO:tasks.workunit.client.0.smithi176.stdout:1/488: stat d2/de/d10/d16/d4a/f8a 0 2024-03-13T14:49:50.145 INFO:tasks.workunit.client.0.smithi176.stdout:6/541: fsync d0/d10/d19/d9f/f98 0 2024-03-13T14:49:50.146 INFO:tasks.workunit.client.0.smithi176.stdout:1/489: read d2/f21 [4488661,51911] 0 2024-03-13T14:49:50.146 INFO:tasks.workunit.client.0.smithi176.stdout:2/574: write d1/dc/fa5 [1870446,70496] 0 2024-03-13T14:49:50.148 INFO:tasks.workunit.client.0.smithi176.stdout:9/585: dread - d0/db/d89/d32/d58/d67/fbb zero size 2024-03-13T14:49:50.148 INFO:tasks.workunit.client.1.smithi196.stdout:2/552: rmdir d0 39 2024-03-13T14:49:50.149 INFO:tasks.workunit.client.0.smithi176.stdout:6/542: write d0/de/d38/d70/f8e [47841,24187] 0 2024-03-13T14:49:50.153 INFO:tasks.workunit.client.1.smithi196.stdout:1/703: write d1/d9/d6c/fc1 [5027037,5825] 0 2024-03-13T14:49:50.154 INFO:tasks.workunit.client.0.smithi176.stdout:7/492: readlink d1/d28/d3a/d1b/l72 0 2024-03-13T14:49:50.158 INFO:tasks.workunit.client.1.smithi196.stdout:8/596: dwrite d0/d1c/d29/fa6 [0,4194304] 0 2024-03-13T14:49:50.163 INFO:tasks.workunit.client.1.smithi196.stdout:2/553: dread - d0/d6/d34/d5b/f95 zero size 2024-03-13T14:49:50.166 INFO:tasks.workunit.client.1.smithi196.stdout:2/554: write d0/d7/d9/d5a/f7c [3771546,8626] 0 2024-03-13T14:49:50.168 INFO:tasks.workunit.client.0.smithi176.stdout:0/458: fdatasync d2/d5/d1e/f3e 0 2024-03-13T14:49:50.169 INFO:tasks.workunit.client.1.smithi196.stdout:6/778: creat d6/d7/dd/ffd x:0 0 0 2024-03-13T14:49:50.172 INFO:tasks.workunit.client.1.smithi196.stdout:3/633: sync 2024-03-13T14:49:50.178 INFO:tasks.workunit.client.0.smithi176.stdout:3/573: dwrite d4/f25 [8388608,4194304] 0 2024-03-13T14:49:50.180 INFO:tasks.workunit.client.1.smithi196.stdout:4/605: rename d1/c2a to d1/d3/cd1 0 2024-03-13T14:49:50.182 INFO:tasks.workunit.client.1.smithi196.stdout:7/738: write d0/d6/d33/f9a [1213318,21266] 0 2024-03-13T14:49:50.183 INFO:tasks.workunit.client.0.smithi176.stdout:9/586: dread d0/d40/d53/f6b [0,4194304] 0 2024-03-13T14:49:50.183 INFO:tasks.workunit.client.1.smithi196.stdout:6/779: truncate d6/d7/df/f97 199975 0 2024-03-13T14:49:50.186 INFO:tasks.workunit.client.1.smithi196.stdout:0/667: truncate d8/da/d25/d26/d35/d3b/d50/d55/fd4 1195388 0 2024-03-13T14:49:50.188 INFO:tasks.workunit.client.1.smithi196.stdout:6/780: chown d6/d9/d17/d19/d4e/d66/fea 122037039 1 2024-03-13T14:49:50.189 INFO:tasks.workunit.client.0.smithi176.stdout:3/574: read d4/d13/d16/d3d/f54 [2482487,83062] 0 2024-03-13T14:49:50.189 INFO:tasks.workunit.client.0.smithi176.stdout:3/575: stat d4/db8 0 2024-03-13T14:49:50.192 INFO:tasks.workunit.client.1.smithi196.stdout:0/668: truncate d8/da/d25/d26/d35/d3b/d6c/fc6 494561 0 2024-03-13T14:49:50.193 INFO:tasks.workunit.client.1.smithi196.stdout:5/704: link d4/d1b/dc8/d3c/f57 d4/d1b/dc8/fd8 0 2024-03-13T14:49:50.193 INFO:tasks.workunit.client.1.smithi196.stdout:3/634: creat db/d19/d22/d3a/d68/d8e/fd0 x:0 0 0 2024-03-13T14:49:50.198 INFO:tasks.workunit.client.1.smithi196.stdout:0/669: read - d8/da/d25/d26/d35/d3b/d5f/d17/f51 zero size 2024-03-13T14:49:50.198 INFO:tasks.workunit.client.1.smithi196.stdout:6/781: readlink d6/d2a/d23/l8b 0 2024-03-13T14:49:50.201 INFO:tasks.workunit.client.1.smithi196.stdout:8/597: creat d0/d10/d70/fcc x:0 0 0 2024-03-13T14:49:50.203 INFO:tasks.workunit.client.1.smithi196.stdout:1/704: dread f0 [0,4194304] 0 2024-03-13T14:49:50.203 INFO:tasks.workunit.client.1.smithi196.stdout:4/606: link d1/d3/d7/fa4 d1/d3/d7/d35/d3e/db1/fd2 0 2024-03-13T14:49:50.204 INFO:tasks.workunit.client.1.smithi196.stdout:8/598: chown d0/d1c/d29/fa6 1373482 1 2024-03-13T14:49:50.208 INFO:tasks.workunit.client.0.smithi176.stdout:6/543: dread d0/de/f57 [0,4194304] 0 2024-03-13T14:49:50.209 INFO:tasks.workunit.client.1.smithi196.stdout:3/635: rename db/d37/d64/f9a to db/d19/d4a/fd1 0 2024-03-13T14:49:50.209 INFO:tasks.workunit.client.1.smithi196.stdout:1/705: write d1/d9/d4e/d55/d63/fe9 [326566,80600] 0 2024-03-13T14:49:50.210 INFO:tasks.workunit.client.0.smithi176.stdout:2/575: creat d1/dc/d2d/d59/d8b/daf/db2/fc3 x:0 0 0 2024-03-13T14:49:50.211 INFO:tasks.workunit.client.0.smithi176.stdout:6/544: chown d0/de/c3b 60 1 2024-03-13T14:49:50.212 INFO:tasks.workunit.client.1.smithi196.stdout:6/782: creat d6/d57/ffe x:0 0 0 2024-03-13T14:49:50.213 INFO:tasks.workunit.client.0.smithi176.stdout:1/490: dwrite d2/de/d3f/d3e/f67 [0,4194304] 0 2024-03-13T14:49:50.213 INFO:tasks.workunit.client.1.smithi196.stdout:9/571: sync 2024-03-13T14:49:50.215 INFO:tasks.workunit.client.1.smithi196.stdout:6/783: fsync d6/d9/d17/d19/f2b 0 2024-03-13T14:49:50.215 INFO:tasks.workunit.client.1.smithi196.stdout:7/739: link d0/d9/d31/d77/d93/d94/cb1 d0/d6/d35/cf7 0 2024-03-13T14:49:50.221 INFO:tasks.workunit.client.1.smithi196.stdout:8/599: fdatasync d0/f7e 0 2024-03-13T14:49:50.222 INFO:tasks.workunit.client.1.smithi196.stdout:2/555: dread d0/d7/d9/d42/f80 [0,4194304] 0 2024-03-13T14:49:50.222 INFO:tasks.workunit.client.0.smithi176.stdout:9/587: creat d0/db/d89/d32/d58/d5f/fc8 x:0 0 0 2024-03-13T14:49:50.225 INFO:tasks.workunit.client.1.smithi196.stdout:4/607: rmdir d1/d3/d7/d35/d3e/db1/dc3 39 2024-03-13T14:49:50.225 INFO:tasks.workunit.client.0.smithi176.stdout:9/588: fsync d0/db/d89/d32/d58/d5f/d84/d9a/f9b 0 2024-03-13T14:49:50.228 INFO:tasks.workunit.client.0.smithi176.stdout:7/493: mkdir d1/d28/d3a/d1b/d8f/d91 0 2024-03-13T14:49:50.231 INFO:tasks.workunit.client.0.smithi176.stdout:6/545: dwrite d0/de/d54/f5e [0,4194304] 0 2024-03-13T14:49:50.236 INFO:tasks.workunit.client.1.smithi196.stdout:6/784: dwrite d6/d7/df/f35 [0,4194304] 0 2024-03-13T14:49:50.238 INFO:tasks.workunit.client.0.smithi176.stdout:2/576: dread d1/dc/d2d/d38/f3f [0,4194304] 0 2024-03-13T14:49:50.240 INFO:tasks.workunit.client.0.smithi176.stdout:5/496: getdents d5/d2e 0 2024-03-13T14:49:50.240 INFO:tasks.workunit.client.1.smithi196.stdout:5/705: read d4/d1b/dc8/d14/d19/d1c/d25/d38/f3e [2298284,1055] 0 2024-03-13T14:49:50.243 INFO:tasks.workunit.client.0.smithi176.stdout:8/482: link d0/d24/d29/d70/l84 d0/d5/l96 0 2024-03-13T14:49:50.244 INFO:tasks.workunit.client.1.smithi196.stdout:6/785: truncate d6/d9/daa/fc2 1746104 0 2024-03-13T14:49:50.244 INFO:tasks.workunit.client.1.smithi196.stdout:5/706: chown d4/d1b/dc8/d14/d19/d26/d33/d61/la8 177090201 1 2024-03-13T14:49:50.246 INFO:tasks.workunit.client.1.smithi196.stdout:0/670: symlink d8/da/d25/d26/d35/d3b/d5f/ld9 0 2024-03-13T14:49:50.247 INFO:tasks.workunit.client.0.smithi176.stdout:4/569: getdents d7/d8/d2f 0 2024-03-13T14:49:50.249 INFO:tasks.workunit.client.0.smithi176.stdout:4/570: chown d7/d8/d45/f62 157 1 2024-03-13T14:49:50.253 INFO:tasks.workunit.client.1.smithi196.stdout:9/572: creat d7/d9/d83/fc0 x:0 0 0 2024-03-13T14:49:50.255 INFO:tasks.workunit.client.0.smithi176.stdout:1/491: rename d2/de/d10/f90 to d2/d30/d63/d35/f99 0 2024-03-13T14:49:50.258 INFO:tasks.workunit.client.1.smithi196.stdout:9/573: truncate d7/d24/d28/d6c/d7b/fb4 283758 0 2024-03-13T14:49:50.258 INFO:tasks.workunit.client.1.smithi196.stdout:8/600: fsync d0/d1c/d29/d47/f52 0 2024-03-13T14:49:50.261 INFO:tasks.workunit.client.0.smithi176.stdout:8/483: dwrite d0/d24/d29/f4e [8388608,4194304] 0 2024-03-13T14:49:50.261 INFO:tasks.workunit.client.1.smithi196.stdout:9/574: write d7/d24/dbf/f90 [144486,36825] 0 2024-03-13T14:49:50.262 INFO:tasks.workunit.client.1.smithi196.stdout:2/556: rename d0/d6/d90/fa4 to d0/d6/d34/fb5 0 2024-03-13T14:49:50.263 INFO:tasks.workunit.client.1.smithi196.stdout:4/608: symlink d1/d3/d7/d35/d76/d58/d69/d6c/ld3 0 2024-03-13T14:49:50.264 INFO:tasks.workunit.client.1.smithi196.stdout:9/575: chown d7/d9/d83/f9f 293 1 2024-03-13T14:49:50.264 INFO:tasks.workunit.client.1.smithi196.stdout:2/557: readlink d0/d7/d56/l6a 0 2024-03-13T14:49:50.265 INFO:tasks.workunit.client.0.smithi176.stdout:6/546: mknod d0/d10/d19/d27/d2b/d2e/d3e/cb2 0 2024-03-13T14:49:50.267 INFO:tasks.workunit.client.1.smithi196.stdout:3/636: write db/d19/d50/db7/f69 [1056604,94197] 0 2024-03-13T14:49:50.268 INFO:tasks.workunit.client.0.smithi176.stdout:6/547: chown d0/d10/d19/d27/fac 23286782 1 2024-03-13T14:49:50.272 INFO:tasks.workunit.client.0.smithi176.stdout:9/589: dread d0/db/d89/d32/d58/d67/f7e [0,4194304] 0 2024-03-13T14:49:50.273 INFO:tasks.workunit.client.0.smithi176.stdout:2/577: dread d1/dc/f52 [0,4194304] 0 2024-03-13T14:49:50.275 INFO:tasks.workunit.client.1.smithi196.stdout:3/637: write db/d19/d50/fca [1363574,15709] 0 2024-03-13T14:49:50.275 INFO:tasks.workunit.client.1.smithi196.stdout:2/558: read d0/d6/d1d/f23 [2797431,110880] 0 2024-03-13T14:49:50.276 INFO:tasks.workunit.client.0.smithi176.stdout:9/590: truncate d0/d1e/d1f/d68/faf 1064480 0 2024-03-13T14:49:50.278 INFO:tasks.workunit.client.1.smithi196.stdout:6/786: creat d6/d2a/dc/d1c/d34/fff x:0 0 0 2024-03-13T14:49:50.278 INFO:tasks.workunit.client.0.smithi176.stdout:3/576: rename d4/d13/d83 to d4/d13/d81/d89/d5b/dc5 0 2024-03-13T14:49:50.282 INFO:tasks.workunit.client.1.smithi196.stdout:5/707: fsync d4/d1b/dc8/d14/d19/d1c/d2d/f4e 0 2024-03-13T14:49:50.286 INFO:tasks.workunit.client.1.smithi196.stdout:1/706: dwrite d1/d9/d33/d4b/f48 [0,4194304] 0 2024-03-13T14:49:50.290 INFO:tasks.workunit.client.1.smithi196.stdout:1/707: chown d1/d9/d33/l39 102 1 2024-03-13T14:49:50.291 INFO:tasks.workunit.client.0.smithi176.stdout:5/497: dread d5/f63 [0,4194304] 0 2024-03-13T14:49:50.296 INFO:tasks.workunit.client.1.smithi196.stdout:7/740: write d0/f37 [816338,111592] 0 2024-03-13T14:49:50.297 INFO:tasks.workunit.client.0.smithi176.stdout:4/571: rename d7/d10/d46/f4e to d7/d10/fcb 0 2024-03-13T14:49:50.299 INFO:tasks.workunit.client.0.smithi176.stdout:9/591: dwrite d0/d1e/d1f/fbd [0,4194304] 0 2024-03-13T14:49:50.299 INFO:tasks.workunit.client.0.smithi176.stdout:9/592: readlink d0/d1e/l24 0 2024-03-13T14:49:50.301 INFO:tasks.workunit.client.0.smithi176.stdout:3/577: unlink d4/d28/d31/c43 0 2024-03-13T14:49:50.302 INFO:tasks.workunit.client.1.smithi196.stdout:2/559: dread d0/d6/d4b/d5f/f81 [0,4194304] 0 2024-03-13T14:49:50.303 INFO:tasks.workunit.client.1.smithi196.stdout:2/560: chown d0/d7/l59 0 1 2024-03-13T14:49:50.303 INFO:tasks.workunit.client.1.smithi196.stdout:8/601: dread d0/d10/d12/d17/f57 [0,4194304] 0 2024-03-13T14:49:50.305 INFO:tasks.workunit.client.0.smithi176.stdout:0/459: link d2/d74/c6a d2/d5/d28/c8b 0 2024-03-13T14:49:50.309 INFO:tasks.workunit.client.1.smithi196.stdout:4/609: symlink d1/d3/d7/d35/d76/d58/d69/d6c/ld4 0 2024-03-13T14:49:50.309 INFO:tasks.workunit.client.0.smithi176.stdout:6/548: symlink d0/de/d38/d6b/da3/lb3 0 2024-03-13T14:49:50.310 INFO:tasks.workunit.client.0.smithi176.stdout:5/498: dread d5/d2e/d5f/f6f [0,4194304] 0 2024-03-13T14:49:50.312 INFO:tasks.workunit.client.0.smithi176.stdout:2/578: dread d1/f14 [4194304,4194304] 0 2024-03-13T14:49:50.315 INFO:tasks.workunit.client.0.smithi176.stdout:4/572: mkdir d7/d10/d46/d4a/dcc 0 2024-03-13T14:49:50.316 INFO:tasks.workunit.client.0.smithi176.stdout:7/494: link d1/d3f/d4b/d63/d7a/l7f d1/d28/d3a/l92 0 2024-03-13T14:49:50.322 INFO:tasks.workunit.client.0.smithi176.stdout:9/593: dwrite d0/f54 [0,4194304] 0 2024-03-13T14:49:50.323 INFO:tasks.workunit.client.1.smithi196.stdout:7/741: mknod d0/d6/d35/cf8 0 2024-03-13T14:49:50.324 INFO:tasks.workunit.client.0.smithi176.stdout:7/495: write d1/d28/d3a/f14 [1875772,115125] 0 2024-03-13T14:49:50.327 INFO:tasks.workunit.client.0.smithi176.stdout:3/578: dwrite d4/d13/d16/d3d/fb2 [0,4194304] 0 2024-03-13T14:49:50.328 INFO:tasks.workunit.client.1.smithi196.stdout:6/787: dread d6/d2a/f93 [0,4194304] 0 2024-03-13T14:49:50.329 INFO:tasks.workunit.client.0.smithi176.stdout:9/594: truncate d0/db/d89/d4d/fc3 1025005 0 2024-03-13T14:49:50.329 INFO:tasks.workunit.client.0.smithi176.stdout:1/492: rename d2/d30/d63/d35/f99 to d2/f9a 0 2024-03-13T14:49:50.331 INFO:tasks.workunit.client.1.smithi196.stdout:6/788: chown d6/d2a/d23/d28/f69 120636246 1 2024-03-13T14:49:50.335 INFO:tasks.workunit.client.1.smithi196.stdout:0/671: creat d8/da/d25/d26/d35/d3b/fda x:0 0 0 2024-03-13T14:49:50.337 INFO:tasks.workunit.client.0.smithi176.stdout:2/579: dwrite d1/dc/fa5 [0,4194304] 0 2024-03-13T14:49:50.340 INFO:tasks.workunit.client.1.smithi196.stdout:3/638: write db/d15/d44/d79/f80 [382921,91738] 0 2024-03-13T14:49:50.341 INFO:tasks.workunit.client.1.smithi196.stdout:0/672: write d8/da/d25/d26/d48/dba/fab [1041137,29545] 0 2024-03-13T14:49:50.345 INFO:tasks.workunit.client.0.smithi176.stdout:7/496: unlink d1/d3f/d4b/l68 0 2024-03-13T14:49:50.347 INFO:tasks.workunit.client.0.smithi176.stdout:1/493: mknod d2/d30/d63/c9b 0 2024-03-13T14:49:50.350 INFO:tasks.workunit.client.0.smithi176.stdout:0/460: truncate d2/db/d35/f3f 701444 0 2024-03-13T14:49:50.351 INFO:tasks.workunit.client.1.smithi196.stdout:1/708: creat d1/d9/d33/dd2/dde/ff2 x:0 0 0 2024-03-13T14:49:50.353 INFO:tasks.workunit.client.1.smithi196.stdout:7/742: mknod d0/d6/d33/cf9 0 2024-03-13T14:49:50.357 INFO:tasks.workunit.client.0.smithi176.stdout:4/573: link d7/d10/d2c/f77 d7/d8/d2f/fcd 0 2024-03-13T14:49:50.359 INFO:tasks.workunit.client.0.smithi176.stdout:6/549: dread d0/f8 [8388608,4194304] 0 2024-03-13T14:49:50.360 INFO:tasks.workunit.client.0.smithi176.stdout:4/574: write d7/d8/d45/fa5 [489243,8471] 0 2024-03-13T14:49:50.362 INFO:tasks.workunit.client.0.smithi176.stdout:6/550: stat d0/d10/d19/d27/d2b/d2e/f2d 0 2024-03-13T14:49:50.363 INFO:tasks.workunit.client.1.smithi196.stdout:8/602: creat d0/d16/d3d/d9c/d87/d8c/da9/db8/fcd x:0 0 0 2024-03-13T14:49:50.363 INFO:tasks.workunit.client.1.smithi196.stdout:2/561: dread d0/d6/d1d/f5e [0,4194304] 0 2024-03-13T14:49:50.363 INFO:tasks.workunit.client.1.smithi196.stdout:8/603: stat d0/d10/da5/db2 0 2024-03-13T14:49:50.363 INFO:tasks.workunit.client.1.smithi196.stdout:5/708: dread d4/d1b/dc8/d14/d37/fb5 [0,4194304] 0 2024-03-13T14:49:50.367 INFO:tasks.workunit.client.0.smithi176.stdout:2/580: dwrite d1/dc/f16 [0,4194304] 0 2024-03-13T14:49:50.369 INFO:tasks.workunit.client.1.smithi196.stdout:8/604: chown d0/d16/d3d/d8b/l7a 142 1 2024-03-13T14:49:50.370 INFO:tasks.workunit.client.0.smithi176.stdout:2/581: write d1/dc/fa5 [721778,37523] 0 2024-03-13T14:49:50.370 INFO:tasks.workunit.client.1.smithi196.stdout:9/576: getdents d7/d71 0 2024-03-13T14:49:50.372 INFO:tasks.workunit.client.1.smithi196.stdout:1/709: symlink d1/d9/d4e/d55/d63/db9/dbb/lf3 0 2024-03-13T14:49:50.376 INFO:tasks.workunit.client.1.smithi196.stdout:8/605: chown d0/d16/d3d/d8b/d48/f5a 14496 1 2024-03-13T14:49:50.378 INFO:tasks.workunit.client.0.smithi176.stdout:5/499: getdents d5/d2c/d74 0 2024-03-13T14:49:50.386 INFO:tasks.workunit.client.0.smithi176.stdout:4/575: rename d7/d8/d45/l64 to d7/d10/lce 0 2024-03-13T14:49:50.387 INFO:tasks.workunit.client.1.smithi196.stdout:6/789: write d6/d2a/dc/d1c/da7/fa6 [366000,85020] 0 2024-03-13T14:49:50.392 INFO:tasks.workunit.client.0.smithi176.stdout:2/582: dread d1/dc/d2d/d59/d80/d7e/da3/fb9 [0,4194304] 0 2024-03-13T14:49:50.392 INFO:tasks.workunit.client.0.smithi176.stdout:9/595: dread d0/d1e/d1f/d68/faf [0,4194304] 0 2024-03-13T14:49:50.392 INFO:tasks.workunit.client.0.smithi176.stdout:3/579: link d4/d13/d16/d3d/f93 d4/d28/d31/fc6 0 2024-03-13T14:49:50.392 INFO:tasks.workunit.client.0.smithi176.stdout:6/551: mkdir d0/d10/d19/d27/d2b/d2e/d3e/db4 0 2024-03-13T14:49:50.394 INFO:tasks.workunit.client.1.smithi196.stdout:4/610: dwrite d1/d3/d4c/f83 [0,4194304] 0 2024-03-13T14:49:50.397 INFO:tasks.workunit.client.1.smithi196.stdout:5/709: unlink d4/d1b/dc8/d14/d19/c53 0 2024-03-13T14:49:50.397 INFO:tasks.workunit.client.0.smithi176.stdout:0/461: mkdir d2/d5/d28/d8c 0 2024-03-13T14:49:50.400 INFO:tasks.workunit.client.0.smithi176.stdout:1/494: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:50.401 INFO:tasks.workunit.client.1.smithi196.stdout:0/673: dwrite d8/da/d25/d26/d35/d3b/d5f/d33/fb2 [0,4194304] 0 2024-03-13T14:49:50.407 INFO:tasks.workunit.client.0.smithi176.stdout:7/497: link d1/d1c/d26/l32 d1/d3f/d81/l93 0 2024-03-13T14:49:50.408 INFO:tasks.workunit.client.0.smithi176.stdout:5/500: rename d5/d31/l4f to d5/d2e/d7e/d9f/d4c/d7b/da7/la9 0 2024-03-13T14:49:50.409 INFO:tasks.workunit.client.1.smithi196.stdout:4/611: fsync d1/d3/d4c/fb8 0 2024-03-13T14:49:50.410 INFO:tasks.workunit.client.1.smithi196.stdout:6/790: stat d6/d2a/dc/d47/c59 0 2024-03-13T14:49:50.412 INFO:tasks.workunit.client.0.smithi176.stdout:9/596: dwrite d0/d1e/d6f/f7b [0,4194304] 0 2024-03-13T14:49:50.412 INFO:tasks.workunit.client.0.smithi176.stdout:3/580: dwrite d4/d13/d16/d80/faf [0,4194304] 0 2024-03-13T14:49:50.413 INFO:tasks.workunit.client.1.smithi196.stdout:3/639: rmdir db/d4c/dbb 0 2024-03-13T14:49:50.413 INFO:tasks.workunit.client.0.smithi176.stdout:5/501: chown d5/d2b/l3c 35 1 2024-03-13T14:49:50.414 INFO:tasks.workunit.client.1.smithi196.stdout:4/612: chown d1/d1e/f6a 986292 1 2024-03-13T14:49:50.414 INFO:tasks.workunit.client.0.smithi176.stdout:6/552: mkdir d0/de/d38/d6b/da3/db5 0 2024-03-13T14:49:50.415 INFO:tasks.workunit.client.0.smithi176.stdout:8/484: sync 2024-03-13T14:49:50.417 INFO:tasks.workunit.client.0.smithi176.stdout:9/597: write d0/db/fae [2718227,115346] 0 2024-03-13T14:49:50.418 INFO:tasks.workunit.client.1.smithi196.stdout:7/743: dwrite d0/d9/d1e/d3f/f7e [4194304,4194304] 0 2024-03-13T14:49:50.418 INFO:tasks.workunit.client.1.smithi196.stdout:8/606: dread d0/d10/f11 [4194304,4194304] 0 2024-03-13T14:49:50.419 INFO:tasks.workunit.client.1.smithi196.stdout:5/710: creat d4/d1b/dc8/d14/d19/d26/d33/d61/fd9 x:0 0 0 2024-03-13T14:49:50.423 INFO:tasks.workunit.client.0.smithi176.stdout:6/553: fdatasync d0/de/f2c 0 2024-03-13T14:49:50.424 INFO:tasks.workunit.client.0.smithi176.stdout:8/485: write d0/db/dc/f13 [440204,62704] 0 2024-03-13T14:49:50.424 INFO:tasks.workunit.client.0.smithi176.stdout:9/598: chown d0/db/d89/d32/d58/d5f/d84/fb1 0 1 2024-03-13T14:49:50.425 INFO:tasks.workunit.client.1.smithi196.stdout:7/744: chown d0/d9/d1e/d58/d72/d6e 142539902 1 2024-03-13T14:49:50.427 INFO:tasks.workunit.client.0.smithi176.stdout:0/462: chown d2/db/d31/f7f 133679511 1 2024-03-13T14:49:50.427 INFO:tasks.workunit.client.1.smithi196.stdout:2/562: rename d0/d7/d9/d42/d98/f99 to d0/d7/d9/d61/fb6 0 2024-03-13T14:49:50.429 INFO:tasks.workunit.client.1.smithi196.stdout:7/745: stat d0/d6/d33/d57 0 2024-03-13T14:49:50.430 INFO:tasks.workunit.client.1.smithi196.stdout:6/791: symlink d6/d7/dd/dba/l100 0 2024-03-13T14:49:50.434 INFO:tasks.workunit.client.1.smithi196.stdout:7/746: chown d0/d6/d33/f4c 153537 1 2024-03-13T14:49:50.439 INFO:tasks.workunit.client.1.smithi196.stdout:3/640: creat db/d15/d44/d79/fd2 x:0 0 0 2024-03-13T14:49:50.441 INFO:tasks.workunit.client.0.smithi176.stdout:7/498: rename d1/d1c/d26/d41/d2b/d7b to d1/d1c/d26/d41/d2b/d40/d64/d94 0 2024-03-13T14:49:50.441 INFO:tasks.workunit.client.1.smithi196.stdout:4/613: mkdir d1/d3/d7/d35/d76/d58/d69/d9f/dd5 0 2024-03-13T14:49:50.446 INFO:tasks.workunit.client.0.smithi176.stdout:7/499: read d1/d28/d3a/f8b [138030,94856] 0 2024-03-13T14:49:50.446 INFO:tasks.workunit.client.1.smithi196.stdout:4/614: chown d1/d3/d7/d35/d3e/d56/f9c 490075549 1 2024-03-13T14:49:50.447 INFO:tasks.workunit.client.0.smithi176.stdout:1/495: dwrite d2/d30/f65 [0,4194304] 0 2024-03-13T14:49:50.448 INFO:tasks.workunit.client.1.smithi196.stdout:3/641: read db/d37/d64/f74 [348669,55882] 0 2024-03-13T14:49:50.450 INFO:tasks.workunit.client.0.smithi176.stdout:1/496: readlink d2/de/d10/d16/d40/l80 0 2024-03-13T14:49:50.452 INFO:tasks.workunit.client.0.smithi176.stdout:1/497: fsync d2/de/d3f/d50/f89 0 2024-03-13T14:49:50.452 INFO:tasks.workunit.client.1.smithi196.stdout:8/607: dread d0/f51 [0,4194304] 0 2024-03-13T14:49:50.455 INFO:tasks.workunit.client.0.smithi176.stdout:1/498: read d2/de/d3f/f6f [24075,78093] 0 2024-03-13T14:49:50.456 INFO:tasks.workunit.client.1.smithi196.stdout:8/608: fdatasync d0/d16/d3d/d8b/d48/f5a 0 2024-03-13T14:49:50.459 INFO:tasks.workunit.client.0.smithi176.stdout:1/499: dread d2/de/d10/f4f [0,4194304] 0 2024-03-13T14:49:50.461 INFO:tasks.workunit.client.0.smithi176.stdout:7/500: dwrite d1/d1c/d4a/f8d [0,4194304] 0 2024-03-13T14:49:50.467 INFO:tasks.workunit.client.0.smithi176.stdout:5/502: mkdir d5/d2e/d7e/d9f/d4c/d7b/daa 0 2024-03-13T14:49:50.468 INFO:tasks.workunit.client.1.smithi196.stdout:9/577: rename d7/de/d74/c86 to d7/d71/cc1 0 2024-03-13T14:49:50.470 INFO:tasks.workunit.client.1.smithi196.stdout:3/642: dwrite db/d19/d22/d3a/d68/d8e/fd0 [0,4194304] 0 2024-03-13T14:49:50.479 INFO:tasks.workunit.client.1.smithi196.stdout:9/578: truncate d7/f45 3495277 0 2024-03-13T14:49:50.481 INFO:tasks.workunit.client.0.smithi176.stdout:3/581: truncate d4/d13/d16/d80/f8c 567498 0 2024-03-13T14:49:50.484 INFO:tasks.workunit.client.1.smithi196.stdout:6/792: mknod d6/d9/d17/da1/c101 0 2024-03-13T14:49:50.484 INFO:tasks.workunit.client.0.smithi176.stdout:3/582: fsync d4/f25 0 2024-03-13T14:49:50.485 INFO:tasks.workunit.client.0.smithi176.stdout:1/500: dwrite d2/d30/d63/d22/f2b [0,4194304] 0 2024-03-13T14:49:50.485 INFO:tasks.workunit.client.1.smithi196.stdout:2/563: creat d0/d6/d4b/d5f/fb7 x:0 0 0 2024-03-13T14:49:50.485 INFO:tasks.workunit.client.1.smithi196.stdout:1/710: write d1/d9/d4e/d55/f6e [5235857,10477] 0 2024-03-13T14:49:50.488 INFO:tasks.workunit.client.0.smithi176.stdout:1/501: write d2/de/d10/d16/d47/f5a [54968,80540] 0 2024-03-13T14:49:50.490 INFO:tasks.workunit.client.1.smithi196.stdout:5/711: mknod d4/d1b/dc8/d14/d19/d1c/cda 0 2024-03-13T14:49:50.497 INFO:tasks.workunit.client.1.smithi196.stdout:4/615: dread - d1/d3/d7/d35/d3e/db1/dc3/fcc zero size 2024-03-13T14:49:50.501 INFO:tasks.workunit.client.1.smithi196.stdout:3/643: dread db/d19/d22/f82 [0,4194304] 0 2024-03-13T14:49:50.501 INFO:tasks.workunit.client.1.smithi196.stdout:4/616: chown d1/d3/f13 40 1 2024-03-13T14:49:50.502 INFO:tasks.workunit.client.1.smithi196.stdout:3/644: readlink db/d37/d64/l67 0 2024-03-13T14:49:50.502 INFO:tasks.workunit.client.0.smithi176.stdout:4/576: link d7/d2d/c82 d7/d8/d33/d96/ccf 0 2024-03-13T14:49:50.504 INFO:tasks.workunit.client.1.smithi196.stdout:8/609: dread - d0/f7e zero size 2024-03-13T14:49:50.510 INFO:tasks.workunit.client.1.smithi196.stdout:7/747: rename d0/d9/f92 to d0/d9/d1e/d58/d72/da9/da3/ffa 0 2024-03-13T14:49:50.515 INFO:tasks.workunit.client.1.smithi196.stdout:9/579: chown d7/de/d4b/d1d/d22/lac 2356 1 2024-03-13T14:49:50.519 INFO:tasks.workunit.client.0.smithi176.stdout:7/501: write d1/d1c/d26/d41/d2b/f85 [921552,21222] 0 2024-03-13T14:49:50.520 INFO:tasks.workunit.client.1.smithi196.stdout:1/711: creat d1/d9/d4e/dcc/ff4 x:0 0 0 2024-03-13T14:49:50.524 INFO:tasks.workunit.client.1.smithi196.stdout:2/564: truncate d0/d7/f94 1278041 0 2024-03-13T14:49:50.524 INFO:tasks.workunit.client.1.smithi196.stdout:6/793: mkdir d6/d9/d17/d19/d4e/d66/d102 0 2024-03-13T14:49:50.525 INFO:tasks.workunit.client.1.smithi196.stdout:2/565: dread - d0/d6/d4b/d96/fa7 zero size 2024-03-13T14:49:50.527 INFO:tasks.workunit.client.1.smithi196.stdout:3/645: symlink db/d19/d50/ld3 0 2024-03-13T14:49:50.530 INFO:tasks.workunit.client.1.smithi196.stdout:4/617: fsync d1/d3/d4c/d72/d7d/d80/f86 0 2024-03-13T14:49:50.533 INFO:tasks.workunit.client.1.smithi196.stdout:8/610: symlink d0/d16/d3d/d8b/lce 0 2024-03-13T14:49:50.535 INFO:tasks.workunit.client.1.smithi196.stdout:8/611: dread - d0/d10/d12/d1e/fbb zero size 2024-03-13T14:49:50.535 INFO:tasks.workunit.client.1.smithi196.stdout:7/748: mkdir d0/d78/d81/dfb 0 2024-03-13T14:49:50.536 INFO:tasks.workunit.client.1.smithi196.stdout:7/749: chown d0/d78/d81 7 1 2024-03-13T14:49:50.536 INFO:tasks.workunit.client.0.smithi176.stdout:0/463: dwrite d2/d5/d1e/f3e [4194304,4194304] 0 2024-03-13T14:49:50.538 INFO:tasks.workunit.client.1.smithi196.stdout:7/750: readlink d0/d9/d1e/d58/d72/da9/d47/lcb 0 2024-03-13T14:49:50.543 INFO:tasks.workunit.client.1.smithi196.stdout:1/712: symlink d1/d3f/lf5 0 2024-03-13T14:49:50.545 INFO:tasks.workunit.client.1.smithi196.stdout:6/794: mknod d6/d7/df/d27/d39/c103 0 2024-03-13T14:49:50.547 INFO:tasks.workunit.client.1.smithi196.stdout:3/646: dwrite db/d15/d44/d79/fba [0,4194304] 0 2024-03-13T14:49:50.548 INFO:tasks.workunit.client.1.smithi196.stdout:6/795: fdatasync d6/d9/d11/f5e 0 2024-03-13T14:49:50.548 INFO:tasks.workunit.client.1.smithi196.stdout:1/713: stat d1/d3f/f80 0 2024-03-13T14:49:50.565 INFO:tasks.workunit.client.1.smithi196.stdout:1/714: dwrite d1/d9/d33/d4b/d66/fa8 [4194304,4194304] 0 2024-03-13T14:49:50.570 INFO:tasks.workunit.client.1.smithi196.stdout:9/580: write d7/de/d4b/d1d/f54 [553309,32436] 0 2024-03-13T14:49:50.572 INFO:tasks.workunit.client.1.smithi196.stdout:6/796: mkdir d6/d2a/d23/d104 0 2024-03-13T14:49:50.574 INFO:tasks.workunit.client.1.smithi196.stdout:3/647: rmdir db/d19 39 2024-03-13T14:49:50.577 INFO:tasks.workunit.client.1.smithi196.stdout:5/712: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/f3e 1926945 0 2024-03-13T14:49:50.580 INFO:tasks.workunit.client.1.smithi196.stdout:4/618: creat d1/d3/d4c/d72/daf/fd6 x:0 0 0 2024-03-13T14:49:50.587 INFO:tasks.workunit.client.1.smithi196.stdout:7/751: creat d0/d78/d81/dfb/ffc x:0 0 0 2024-03-13T14:49:50.591 INFO:tasks.workunit.client.1.smithi196.stdout:6/797: unlink d6/d7/df/l4a 0 2024-03-13T14:49:50.593 INFO:tasks.workunit.client.1.smithi196.stdout:5/713: dread d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/f64 [0,4194304] 0 2024-03-13T14:49:50.600 INFO:tasks.workunit.client.1.smithi196.stdout:8/612: creat d0/d10/d12/fcf x:0 0 0 2024-03-13T14:49:50.606 INFO:tasks.workunit.client.1.smithi196.stdout:1/715: write d1/d9/d4e/f70 [65929,76639] 0 2024-03-13T14:49:50.608 INFO:tasks.workunit.client.1.smithi196.stdout:2/566: dwrite d0/d6/d34/f41 [0,4194304] 0 2024-03-13T14:49:50.609 INFO:tasks.workunit.client.0.smithi176.stdout:8/486: creat d0/db/d53/d80/f97 x:0 0 0 2024-03-13T14:49:50.612 INFO:tasks.workunit.client.0.smithi176.stdout:8/487: fdatasync d0/db/dc/f13 0 2024-03-13T14:49:50.614 INFO:tasks.workunit.client.0.smithi176.stdout:2/583: getdents d1/d2a 0 2024-03-13T14:49:50.614 INFO:tasks.workunit.client.1.smithi196.stdout:8/613: creat d0/d16/d58/fd0 x:0 0 0 2024-03-13T14:49:50.617 INFO:tasks.workunit.client.1.smithi196.stdout:4/619: mkdir d1/d3/d7/d35/d76/d58/dc9/dd7 0 2024-03-13T14:49:50.618 INFO:tasks.workunit.client.0.smithi176.stdout:8/488: dread d0/db/dc/d40/f4f [0,4194304] 0 2024-03-13T14:49:50.618 INFO:tasks.workunit.client.1.smithi196.stdout:9/581: creat d7/d24/d28/d6c/fc2 x:0 0 0 2024-03-13T14:49:50.619 INFO:tasks.workunit.client.0.smithi176.stdout:9/599: write d0/db/f1d [1067708,80791] 0 2024-03-13T14:49:50.622 INFO:tasks.workunit.client.1.smithi196.stdout:1/716: rename d1/d38/f76 to d1/ff6 0 2024-03-13T14:49:50.624 INFO:tasks.workunit.client.1.smithi196.stdout:7/752: creat d0/d6/ddc/ffd x:0 0 0 2024-03-13T14:49:50.625 INFO:tasks.workunit.client.0.smithi176.stdout:5/503: mknod d5/d2c/d40/cab 0 2024-03-13T14:49:50.625 INFO:tasks.workunit.client.1.smithi196.stdout:2/567: stat d0/d7/d9/d42/c57 0 2024-03-13T14:49:50.627 INFO:tasks.workunit.client.1.smithi196.stdout:6/798: mkdir d6/d9/d105 0 2024-03-13T14:49:50.629 INFO:tasks.workunit.client.0.smithi176.stdout:6/554: sync 2024-03-13T14:49:50.630 INFO:tasks.workunit.client.0.smithi176.stdout:0/464: creat d2/db/d35/f8d x:0 0 0 2024-03-13T14:49:50.631 INFO:tasks.workunit.client.1.smithi196.stdout:4/620: creat d1/d3/d7/d35/d76/d58/d69/d9f/fd8 x:0 0 0 2024-03-13T14:49:50.632 INFO:tasks.workunit.client.0.smithi176.stdout:5/504: dread d5/f81 [0,4194304] 0 2024-03-13T14:49:50.633 INFO:tasks.workunit.client.1.smithi196.stdout:4/621: readlink d1/d3/d7/d35/d3e/d56/l61 0 2024-03-13T14:49:50.636 INFO:tasks.workunit.client.1.smithi196.stdout:4/622: write d1/d3/d4c/fa5 [163717,96686] 0 2024-03-13T14:49:50.639 INFO:tasks.workunit.client.0.smithi176.stdout:1/502: unlink d2/de/d10/f4f 0 2024-03-13T14:49:50.641 INFO:tasks.workunit.client.0.smithi176.stdout:1/503: fsync d2/de/d3f/d3e/f5b 0 2024-03-13T14:49:50.645 INFO:tasks.workunit.client.0.smithi176.stdout:3/583: link d4/d28/d31/d45/d82/d87/f8d d4/d13/d81/d89/d5b/dc5/fc7 0 2024-03-13T14:49:50.645 INFO:tasks.workunit.client.1.smithi196.stdout:9/582: dread d7/d9/d17/f51 [0,4194304] 0 2024-03-13T14:49:50.646 INFO:tasks.workunit.client.1.smithi196.stdout:2/568: rename d0/d50/f65 to d0/d7/d9/d9a/daa/fb8 0 2024-03-13T14:49:50.647 INFO:tasks.workunit.client.0.smithi176.stdout:1/504: write d2/de/d3f/d50/f78 [4362898,115776] 0 2024-03-13T14:49:50.650 INFO:tasks.workunit.client.1.smithi196.stdout:8/614: symlink d0/d10/d12/d17/d42/dbe/ld1 0 2024-03-13T14:49:50.651 INFO:tasks.workunit.client.0.smithi176.stdout:2/584: symlink d1/dc/d2d/d59/d8c/lc4 0 2024-03-13T14:49:50.653 INFO:tasks.workunit.client.0.smithi176.stdout:4/577: mkdir d7/d10/d46/d4a/dcc/dd0 0 2024-03-13T14:49:50.655 INFO:tasks.workunit.client.0.smithi176.stdout:9/600: symlink d0/d40/d53/lc9 0 2024-03-13T14:49:50.656 INFO:tasks.workunit.client.1.smithi196.stdout:3/648: dread db/d19/d22/f3c [0,4194304] 0 2024-03-13T14:49:50.658 INFO:tasks.workunit.client.1.smithi196.stdout:7/753: write d0/d9/f24 [865463,112650] 0 2024-03-13T14:49:50.658 INFO:tasks.workunit.client.1.smithi196.stdout:3/649: write db/d37/f76 [4462037,48105] 0 2024-03-13T14:49:50.660 INFO:tasks.workunit.client.0.smithi176.stdout:7/502: link d1/d28/d3a/f8b d1/d1c/d26/d41/d2b/d40/d64/f95 0 2024-03-13T14:49:50.662 INFO:tasks.workunit.client.1.smithi196.stdout:3/650: write db/d19/d22/f98 [766791,124196] 0 2024-03-13T14:49:50.662 INFO:tasks.workunit.client.0.smithi176.stdout:0/465: rmdir d2 39 2024-03-13T14:49:50.664 INFO:tasks.workunit.client.1.smithi196.stdout:9/583: fdatasync d7/de/d4b/f7c 0 2024-03-13T14:49:50.667 INFO:tasks.workunit.client.1.smithi196.stdout:4/623: rename d1/d3/d7/d35/d3e/cb4 to d1/d3/d7/d35/d76/d58/dc6/cd9 0 2024-03-13T14:49:50.669 INFO:tasks.workunit.client.1.smithi196.stdout:2/569: fsync d0/d7/d9/d42/f88 0 2024-03-13T14:49:50.674 INFO:tasks.workunit.client.1.smithi196.stdout:0/674: sync 2024-03-13T14:49:50.674 INFO:tasks.workunit.client.0.smithi176.stdout:4/578: readlink d7/d10/lce 0 2024-03-13T14:49:50.676 INFO:tasks.workunit.client.0.smithi176.stdout:2/585: write d1/dc/f52 [3161297,71955] 0 2024-03-13T14:49:50.678 INFO:tasks.workunit.client.0.smithi176.stdout:8/489: rmdir d0/d5/d11/d47 39 2024-03-13T14:49:50.680 INFO:tasks.workunit.client.1.smithi196.stdout:3/651: creat db/d37/fd4 x:0 0 0 2024-03-13T14:49:50.680 INFO:tasks.workunit.client.0.smithi176.stdout:7/503: dwrite d1/d3f/d76/f6d [0,4194304] 0 2024-03-13T14:49:50.681 INFO:tasks.workunit.client.1.smithi196.stdout:7/754: dwrite d0/d78/d81/dfb/ffc [0,4194304] 0 2024-03-13T14:49:50.690 INFO:tasks.workunit.client.1.smithi196.stdout:9/584: symlink d7/d9/d17/dbc/lc3 0 2024-03-13T14:49:50.691 INFO:tasks.workunit.client.1.smithi196.stdout:8/615: getdents d0/d63/dcb 0 2024-03-13T14:49:50.691 INFO:tasks.workunit.client.1.smithi196.stdout:5/714: dread d4/d1b/dc8/d14/d19/d26/dab/fcd [0,4194304] 0 2024-03-13T14:49:50.697 INFO:tasks.workunit.client.0.smithi176.stdout:7/504: dwrite d1/d3f/d4b/f6b [0,4194304] 0 2024-03-13T14:49:50.698 INFO:tasks.workunit.client.0.smithi176.stdout:1/505: mknod d2/c9c 0 2024-03-13T14:49:50.701 INFO:tasks.workunit.client.1.smithi196.stdout:4/624: mknod d1/d3/d4c/d72/d7d/d80/d55/cda 0 2024-03-13T14:49:50.709 INFO:tasks.workunit.client.1.smithi196.stdout:8/616: fdatasync d0/d10/d12/f53 0 2024-03-13T14:49:50.716 INFO:tasks.workunit.client.1.smithi196.stdout:9/585: dwrite d7/de/d4b/f7c [0,4194304] 0 2024-03-13T14:49:50.717 INFO:tasks.workunit.client.1.smithi196.stdout:2/570: unlink d0/d7/d9/d42/f5c 0 2024-03-13T14:49:50.719 INFO:tasks.workunit.client.1.smithi196.stdout:3/652: dwrite db/d19/d22/d3a/d6a/fcc [0,4194304] 0 2024-03-13T14:49:50.724 INFO:tasks.workunit.client.1.smithi196.stdout:4/625: chown d1/d3/d7/d35/l66 887676901 1 2024-03-13T14:49:50.731 INFO:tasks.workunit.client.1.smithi196.stdout:8/617: creat d0/d10/d12/d17/d42/fd2 x:0 0 0 2024-03-13T14:49:50.732 INFO:tasks.workunit.client.1.smithi196.stdout:9/586: symlink d7/de/d16/d70/lc4 0 2024-03-13T14:49:50.733 INFO:tasks.workunit.client.0.smithi176.stdout:1/506: creat d2/d30/d63/d22/d71/f9d x:0 0 0 2024-03-13T14:49:50.736 INFO:tasks.workunit.client.1.smithi196.stdout:2/571: mknod d0/d7/d9/d42/d98/cb9 0 2024-03-13T14:49:50.738 INFO:tasks.workunit.client.1.smithi196.stdout:4/626: creat d1/d3/dbc/fdb x:0 0 0 2024-03-13T14:49:50.739 INFO:tasks.workunit.client.1.smithi196.stdout:6/799: truncate d6/d2a/dc/da3/fc3 232536 0 2024-03-13T14:49:50.741 INFO:tasks.workunit.client.0.smithi176.stdout:5/505: getdents d5/d2c 0 2024-03-13T14:49:50.742 INFO:tasks.workunit.client.1.smithi196.stdout:2/572: readlink d0/d7/d9/l6d 0 2024-03-13T14:49:50.745 INFO:tasks.workunit.client.0.smithi176.stdout:6/555: truncate d0/de/d38/d70/faf 3528652 0 2024-03-13T14:49:50.745 INFO:tasks.workunit.client.1.smithi196.stdout:8/618: symlink d0/d63/dcb/ld3 0 2024-03-13T14:49:50.746 INFO:tasks.workunit.client.0.smithi176.stdout:1/507: write d2/d29/f70 [5723143,12023] 0 2024-03-13T14:49:50.747 INFO:tasks.workunit.client.0.smithi176.stdout:7/505: link d1/d1c/f5e d1/d1c/d26/d41/d2b/f96 0 2024-03-13T14:49:50.747 INFO:tasks.workunit.client.0.smithi176.stdout:6/556: chown d0/de/d38/d6b/da3 2217052 1 2024-03-13T14:49:50.747 INFO:tasks.workunit.client.0.smithi176.stdout:3/584: write d4/d13/d16/d80/f8c [849235,33225] 0 2024-03-13T14:49:50.750 INFO:tasks.workunit.client.0.smithi176.stdout:7/506: chown d1/cc 188582517 1 2024-03-13T14:49:50.750 INFO:tasks.workunit.client.0.smithi176.stdout:5/506: unlink d5/f55 0 2024-03-13T14:49:50.758 INFO:tasks.workunit.client.0.smithi176.stdout:6/557: write d0/de/f57 [5135751,43180] 0 2024-03-13T14:49:50.758 INFO:tasks.workunit.client.1.smithi196.stdout:6/800: unlink d6/d7/df/f35 0 2024-03-13T14:49:50.759 INFO:tasks.workunit.client.0.smithi176.stdout:5/507: mknod d5/d2b/cac 0 2024-03-13T14:49:50.762 INFO:tasks.workunit.client.1.smithi196.stdout:1/717: dread d1/d3f/d5b/fb8 [0,4194304] 0 2024-03-13T14:49:50.766 INFO:tasks.workunit.client.0.smithi176.stdout:3/585: mknod d4/d13/cc8 0 2024-03-13T14:49:50.766 INFO:tasks.workunit.client.1.smithi196.stdout:7/755: dread d0/d6/d33/f39 [0,4194304] 0 2024-03-13T14:49:50.766 INFO:tasks.workunit.client.1.smithi196.stdout:2/573: dread d0/d7/f21 [0,4194304] 0 2024-03-13T14:49:50.768 INFO:tasks.workunit.client.1.smithi196.stdout:3/653: dwrite db/d37/fd4 [0,4194304] 0 2024-03-13T14:49:50.768 INFO:tasks.workunit.client.0.smithi176.stdout:6/558: fdatasync d0/d9/f50 0 2024-03-13T14:49:50.771 INFO:tasks.workunit.client.1.smithi196.stdout:2/574: chown d0/d7/d9/d5a 511611968 1 2024-03-13T14:49:50.772 INFO:tasks.workunit.client.1.smithi196.stdout:7/756: fdatasync d0/d9/f5a 0 2024-03-13T14:49:50.772 INFO:tasks.workunit.client.0.smithi176.stdout:4/579: dwrite d7/f50 [0,4194304] 0 2024-03-13T14:49:50.778 INFO:tasks.workunit.client.1.smithi196.stdout:2/575: fsync d0/d6/d1d/f7f 0 2024-03-13T14:49:50.781 INFO:tasks.workunit.client.0.smithi176.stdout:4/580: symlink d7/d10/ld1 0 2024-03-13T14:49:50.781 INFO:tasks.workunit.client.0.smithi176.stdout:5/508: chown d5/d2e/d7e/d9f/d4c/d7b/da7 818031 1 2024-03-13T14:49:50.782 INFO:tasks.workunit.client.0.smithi176.stdout:6/559: fdatasync d0/de/d38/d70/f8e 0 2024-03-13T14:49:50.783 INFO:tasks.workunit.client.0.smithi176.stdout:2/586: dwrite d1/dc/d40/f44 [0,4194304] 0 2024-03-13T14:49:50.788 INFO:tasks.workunit.client.1.smithi196.stdout:4/627: dwrite d1/d3/d7/f5d [0,4194304] 0 2024-03-13T14:49:50.788 INFO:tasks.workunit.client.0.smithi176.stdout:8/490: dwrite d0/db/fd [0,4194304] 0 2024-03-13T14:49:50.792 INFO:tasks.workunit.client.0.smithi176.stdout:9/601: dwrite d0/db/f31 [0,4194304] 0 2024-03-13T14:49:50.796 INFO:tasks.workunit.client.0.smithi176.stdout:2/587: chown d1/dc/d2d/d4b/d69 2348 1 2024-03-13T14:49:50.800 INFO:tasks.workunit.client.0.smithi176.stdout:0/466: dwrite d2/d5/dc/d20/f37 [0,4194304] 0 2024-03-13T14:49:50.805 INFO:tasks.workunit.client.0.smithi176.stdout:1/508: dwrite d2/f9a [0,4194304] 0 2024-03-13T14:49:50.810 INFO:tasks.workunit.client.0.smithi176.stdout:3/586: creat d4/d28/d31/d45/fc9 x:0 0 0 2024-03-13T14:49:50.813 INFO:tasks.workunit.client.0.smithi176.stdout:1/509: stat d2/de/d10/d16/d59/l5d 0 2024-03-13T14:49:50.814 INFO:tasks.workunit.client.0.smithi176.stdout:4/581: creat d7/d8/d33/fd2 x:0 0 0 2024-03-13T14:49:50.817 INFO:tasks.workunit.client.0.smithi176.stdout:8/491: mkdir d0/d24/d98 0 2024-03-13T14:49:50.821 INFO:tasks.workunit.client.0.smithi176.stdout:4/582: chown d7/d8/d33/l63 57385 1 2024-03-13T14:49:50.821 INFO:tasks.workunit.client.0.smithi176.stdout:6/560: dwrite d0/de/f2c [4194304,4194304] 0 2024-03-13T14:49:50.824 INFO:tasks.workunit.client.1.smithi196.stdout:9/587: rmdir d7/d24/db5 0 2024-03-13T14:49:50.826 INFO:tasks.workunit.client.1.smithi196.stdout:9/588: stat d7/de/d4b/d1d/d93/da7 0 2024-03-13T14:49:50.827 INFO:tasks.workunit.client.0.smithi176.stdout:4/583: write d7/d10/d46/d4a/d5d/da1/f86 [1324185,21524] 0 2024-03-13T14:49:50.828 INFO:tasks.workunit.client.1.smithi196.stdout:9/589: stat d7/de/d4b/d1d/f66 0 2024-03-13T14:49:50.830 INFO:tasks.workunit.client.0.smithi176.stdout:0/467: rmdir d2/d5/dc 39 2024-03-13T14:49:50.830 INFO:tasks.workunit.client.0.smithi176.stdout:4/584: read d7/d10/f8a [164230,113368] 0 2024-03-13T14:49:50.833 INFO:tasks.workunit.client.0.smithi176.stdout:3/587: unlink d4/d28/d34/d38/l42 0 2024-03-13T14:49:50.834 INFO:tasks.workunit.client.0.smithi176.stdout:3/588: chown d4/d28/d31/f75 42234234 1 2024-03-13T14:49:50.839 INFO:tasks.workunit.client.0.smithi176.stdout:9/602: mknod d0/d1e/d8d/da7/cca 0 2024-03-13T14:49:50.841 INFO:tasks.workunit.client.0.smithi176.stdout:7/507: truncate d1/d3f/d4b/f6b 3836289 0 2024-03-13T14:49:50.844 INFO:tasks.workunit.client.0.smithi176.stdout:7/508: dread d1/d28/d3a/f8b [0,4194304] 0 2024-03-13T14:49:50.844 INFO:tasks.workunit.client.0.smithi176.stdout:1/510: dwrite d2/de/d10/d16/d47/f5a [0,4194304] 0 2024-03-13T14:49:50.849 INFO:tasks.workunit.client.1.smithi196.stdout:3/654: mkdir db/d19/d4a/dd5 0 2024-03-13T14:49:50.852 INFO:tasks.workunit.client.1.smithi196.stdout:2/576: write d0/d7/d66/f76 [1976285,95307] 0 2024-03-13T14:49:50.853 INFO:tasks.workunit.client.0.smithi176.stdout:3/589: read - d4/d13/f57 zero size 2024-03-13T14:49:50.855 INFO:tasks.workunit.client.1.smithi196.stdout:3/655: dread db/d19/d22/d3a/f8f [0,4194304] 0 2024-03-13T14:49:50.855 INFO:tasks.workunit.client.1.smithi196.stdout:2/577: fdatasync d0/d7/d9/d9a/fb4 0 2024-03-13T14:49:50.856 INFO:tasks.workunit.client.0.smithi176.stdout:8/492: getdents d0/db/d39/d56/d91 0 2024-03-13T14:49:50.856 INFO:tasks.workunit.client.0.smithi176.stdout:7/509: dread d1/d1c/d4a/f8d [0,4194304] 0 2024-03-13T14:49:50.858 INFO:tasks.workunit.client.1.smithi196.stdout:3/656: chown db/d19/d4a/f8c 3081 1 2024-03-13T14:49:50.862 INFO:tasks.workunit.client.1.smithi196.stdout:8/619: creat d0/d10/da5/db2/dc8/fd4 x:0 0 0 2024-03-13T14:49:50.865 INFO:tasks.workunit.client.1.smithi196.stdout:6/801: symlink d6/d7/dd/l106 0 2024-03-13T14:49:50.867 INFO:tasks.workunit.client.0.smithi176.stdout:6/561: mknod d0/d10/d19/d27/cb6 0 2024-03-13T14:49:50.869 INFO:tasks.workunit.client.1.smithi196.stdout:1/718: write d1/d9/d33/d4b/f1b [2562651,25283] 0 2024-03-13T14:49:50.870 INFO:tasks.workunit.client.1.smithi196.stdout:6/802: read d6/d2a/d23/d28/f69 [1561688,29600] 0 2024-03-13T14:49:50.871 INFO:tasks.workunit.client.1.smithi196.stdout:7/757: symlink d0/d9/d1e/lfe 0 2024-03-13T14:49:50.876 INFO:tasks.workunit.client.1.smithi196.stdout:2/578: dread d0/d6/d4b/d5f/f81 [0,4194304] 0 2024-03-13T14:49:50.876 INFO:tasks.workunit.client.1.smithi196.stdout:7/758: read - d0/d9/d1e/d58/d72/da9/da3/dc2/ff5 zero size 2024-03-13T14:49:50.878 INFO:tasks.workunit.client.0.smithi176.stdout:5/509: write d5/d2e/d7e/d9f/f5a [5146816,117855] 0 2024-03-13T14:49:50.879 INFO:tasks.workunit.client.1.smithi196.stdout:9/590: dwrite d7/d9/d17/f51 [0,4194304] 0 2024-03-13T14:49:50.885 INFO:tasks.workunit.client.1.smithi196.stdout:8/620: mknod d0/d16/d3d/d8b/d48/cd5 0 2024-03-13T14:49:50.888 INFO:tasks.workunit.client.0.smithi176.stdout:4/585: creat d7/d10/d2c/d9d/dbb/fd3 x:0 0 0 2024-03-13T14:49:50.890 INFO:tasks.workunit.client.0.smithi176.stdout:4/586: stat d7/d10/d46/d4a/d5d/f69 0 2024-03-13T14:49:50.890 INFO:tasks.workunit.client.1.smithi196.stdout:0/675: sync 2024-03-13T14:49:50.891 INFO:tasks.workunit.client.0.smithi176.stdout:2/588: dread d1/dc/d2d/d38/f5a [0,4194304] 0 2024-03-13T14:49:50.891 INFO:tasks.workunit.client.0.smithi176.stdout:4/587: chown d7/fe 106240 1 2024-03-13T14:49:50.891 INFO:tasks.workunit.client.1.smithi196.stdout:7/759: dread d0/d6/d33/f39 [0,4194304] 0 2024-03-13T14:49:50.892 INFO:tasks.workunit.client.1.smithi196.stdout:1/719: creat d1/d3f/d5b/ff7 x:0 0 0 2024-03-13T14:49:50.893 INFO:tasks.workunit.client.1.smithi196.stdout:6/803: dwrite d6/d9/daa/fc2 [0,4194304] 0 2024-03-13T14:49:50.894 INFO:tasks.workunit.client.0.smithi176.stdout:3/590: mknod d4/d28/d31/d52/cca 0 2024-03-13T14:49:50.894 INFO:tasks.workunit.client.0.smithi176.stdout:4/588: truncate d7/d8/d45/f92 894714 0 2024-03-13T14:49:50.901 INFO:tasks.workunit.client.0.smithi176.stdout:7/510: creat d1/d3f/d65/f97 x:0 0 0 2024-03-13T14:49:50.902 INFO:tasks.workunit.client.0.smithi176.stdout:4/589: read d7/d10/d46/d4a/d5d/f78 [864699,121719] 0 2024-03-13T14:49:50.904 INFO:tasks.workunit.client.0.smithi176.stdout:9/603: symlink d0/d73/db8/lcb 0 2024-03-13T14:49:50.905 INFO:tasks.workunit.client.1.smithi196.stdout:2/579: mknod d0/d6/d34/d5b/cba 0 2024-03-13T14:49:50.906 INFO:tasks.workunit.client.0.smithi176.stdout:4/590: truncate d7/d8/d33/fd2 696615 0 2024-03-13T14:49:50.907 INFO:tasks.workunit.client.0.smithi176.stdout:6/562: mkdir d0/d10/d19/d27/d2b/d2e/db7 0 2024-03-13T14:49:50.911 INFO:tasks.workunit.client.0.smithi176.stdout:6/563: read d0/f6c [3053959,90080] 0 2024-03-13T14:49:50.912 INFO:tasks.workunit.client.1.smithi196.stdout:3/657: creat db/d19/d22/d3a/d68/dbd/fd6 x:0 0 0 2024-03-13T14:49:50.913 INFO:tasks.workunit.client.0.smithi176.stdout:6/564: read - d0/d10/d19/d9f/d76/fab zero size 2024-03-13T14:49:50.913 INFO:tasks.workunit.client.1.smithi196.stdout:4/628: write d1/d1e/f7f [2501994,76084] 0 2024-03-13T14:49:50.914 INFO:tasks.workunit.client.1.smithi196.stdout:3/658: write db/d19/f60 [2801738,93124] 0 2024-03-13T14:49:50.915 INFO:tasks.workunit.client.1.smithi196.stdout:8/621: mknod d0/d16/d3d/d8b/d48/cd6 0 2024-03-13T14:49:50.922 INFO:tasks.workunit.client.0.smithi176.stdout:0/468: mkdir d2/d5/dc/d59/d8e 0 2024-03-13T14:49:50.923 INFO:tasks.workunit.client.1.smithi196.stdout:7/760: rmdir d0/d9/d1e/d58/d72/d6e/d7f 39 2024-03-13T14:49:50.924 INFO:tasks.workunit.client.0.smithi176.stdout:6/565: dread d0/d10/d19/d27/f4a [0,4194304] 0 2024-03-13T14:49:50.925 INFO:tasks.workunit.client.1.smithi196.stdout:5/715: sync 2024-03-13T14:49:50.928 INFO:tasks.workunit.client.0.smithi176.stdout:8/493: symlink d0/l99 0 2024-03-13T14:49:50.929 INFO:tasks.workunit.client.1.smithi196.stdout:5/716: chown d4/d1b/dc8/d4a/l85 1010926 1 2024-03-13T14:49:50.930 INFO:tasks.workunit.client.1.smithi196.stdout:6/804: creat d6/d7/df/d27/d39/d44/d74/f107 x:0 0 0 2024-03-13T14:49:50.931 INFO:tasks.workunit.client.0.smithi176.stdout:7/511: rmdir d1/d28/d3a/d1b 39 2024-03-13T14:49:50.932 INFO:tasks.workunit.client.1.smithi196.stdout:5/717: readlink d4/d1b/dc8/d4a/l99 0 2024-03-13T14:49:50.933 INFO:tasks.workunit.client.0.smithi176.stdout:9/604: creat d0/d40/d53/d6d/fcc x:0 0 0 2024-03-13T14:49:50.935 INFO:tasks.workunit.client.0.smithi176.stdout:4/591: symlink d7/d10/d2c/d9d/ld4 0 2024-03-13T14:49:50.937 INFO:tasks.workunit.client.1.smithi196.stdout:9/591: dread d7/f15 [0,4194304] 0 2024-03-13T14:49:50.938 INFO:tasks.workunit.client.1.smithi196.stdout:4/629: symlink d1/d3/d4c/d72/d7d/d80/d55/ldc 0 2024-03-13T14:49:50.940 INFO:tasks.workunit.client.0.smithi176.stdout:5/510: rename d5/d2e/d7e/d9f/d4c to d5/d2e/d7e/dad 0 2024-03-13T14:49:50.941 INFO:tasks.workunit.client.1.smithi196.stdout:0/676: mknod d8/da/d25/d26/d48/d82/da0/cdb 0 2024-03-13T14:49:50.943 INFO:tasks.workunit.client.1.smithi196.stdout:7/761: write d0/d9/d1e/d58/d72/da9/d47/fa7 [1275122,121230] 0 2024-03-13T14:49:50.946 INFO:tasks.workunit.client.0.smithi176.stdout:1/511: write d2/d30/d63/d22/d71/f87 [1213639,88155] 0 2024-03-13T14:49:50.948 INFO:tasks.workunit.client.1.smithi196.stdout:5/718: truncate d4/d1b/dc8/d3c/f84 610785 0 2024-03-13T14:49:50.949 INFO:tasks.workunit.client.0.smithi176.stdout:2/589: write d1/d2a/f75 [858282,60452] 0 2024-03-13T14:49:50.949 INFO:tasks.workunit.client.0.smithi176.stdout:8/494: fdatasync d0/d24/f32 0 2024-03-13T14:49:50.949 INFO:tasks.workunit.client.1.smithi196.stdout:8/622: mkdir d0/d16/dd7 0 2024-03-13T14:49:50.950 INFO:tasks.workunit.client.0.smithi176.stdout:7/512: creat d1/d1c/d4a/f98 x:0 0 0 2024-03-13T14:49:50.951 INFO:tasks.workunit.client.1.smithi196.stdout:4/630: fdatasync d1/f22 0 2024-03-13T14:49:50.952 INFO:tasks.workunit.client.0.smithi176.stdout:9/605: creat d0/db/d89/d90/fcd x:0 0 0 2024-03-13T14:49:50.954 INFO:tasks.workunit.client.0.smithi176.stdout:9/606: dread - d0/db/d89/fc2 zero size 2024-03-13T14:49:50.954 INFO:tasks.workunit.client.1.smithi196.stdout:1/720: creat d1/d3f/d5b/d74/ded/ff8 x:0 0 0 2024-03-13T14:49:50.955 INFO:tasks.workunit.client.0.smithi176.stdout:4/592: stat d7/d8/d15/f81 0 2024-03-13T14:49:50.956 INFO:tasks.workunit.client.1.smithi196.stdout:7/762: mkdir d0/d6/d35/dff 0 2024-03-13T14:49:50.957 INFO:tasks.workunit.client.1.smithi196.stdout:1/721: chown d1/d9/d33/d4b/d66/l6d 12108 1 2024-03-13T14:49:50.957 INFO:tasks.workunit.client.1.smithi196.stdout:0/677: dread d8/da/d25/d26/d54/d88/f84 [0,4194304] 0 2024-03-13T14:49:50.958 INFO:tasks.workunit.client.1.smithi196.stdout:6/805: getdents d6/d2a/d23/d28 0 2024-03-13T14:49:50.959 INFO:tasks.workunit.client.0.smithi176.stdout:3/591: rename d4/d13/d5e/c98 to d4/d28/d34/d38/ccb 0 2024-03-13T14:49:50.961 INFO:tasks.workunit.client.1.smithi196.stdout:2/580: rename d0/d7/d9/d42/d98/faf to d0/fbb 0 2024-03-13T14:49:50.962 INFO:tasks.workunit.client.0.smithi176.stdout:5/511: fsync d5/d31/d47/f5c 0 2024-03-13T14:49:50.965 INFO:tasks.workunit.client.0.smithi176.stdout:7/513: dwrite d1/d1c/d4a/f62 [0,4194304] 0 2024-03-13T14:49:50.968 INFO:tasks.workunit.client.0.smithi176.stdout:0/469: symlink d2/l8f 0 2024-03-13T14:49:50.971 INFO:tasks.workunit.client.1.smithi196.stdout:4/631: read - d1/d1e/d43/f85 zero size 2024-03-13T14:49:50.973 INFO:tasks.workunit.client.0.smithi176.stdout:1/512: chown d2/de/d10/l4b 14181289 1 2024-03-13T14:49:50.974 INFO:tasks.workunit.client.0.smithi176.stdout:8/495: fsync d0/f6c 0 2024-03-13T14:49:50.976 INFO:tasks.workunit.client.0.smithi176.stdout:2/590: symlink d1/dc/d2d/d59/d80/d7e/da3/lc5 0 2024-03-13T14:49:50.976 INFO:tasks.workunit.client.0.smithi176.stdout:1/513: stat d2/de/d10/d16/d4a/d5f/c60 0 2024-03-13T14:49:50.979 INFO:tasks.workunit.client.1.smithi196.stdout:1/722: truncate d1/d38/f25 5053932 0 2024-03-13T14:49:50.981 INFO:tasks.workunit.client.1.smithi196.stdout:1/723: readlink d1/d9/d4e/d55/l9e 0 2024-03-13T14:49:50.981 INFO:tasks.workunit.client.0.smithi176.stdout:4/593: rename d7/lad to d7/d10/d2c/d9d/dbb/ld5 0 2024-03-13T14:49:50.983 INFO:tasks.workunit.client.1.smithi196.stdout:0/678: truncate d8/da/d25/d26/d35/d3b/d5f/d17/f4d 86006 0 2024-03-13T14:49:50.985 INFO:tasks.workunit.client.1.smithi196.stdout:3/659: rename db/d19/d22/d3a/d6a/f89 to db/d19/d4a/dc5/fd7 0 2024-03-13T14:49:50.987 INFO:tasks.workunit.client.1.smithi196.stdout:2/581: mkdir d0/d7/d9/d5a/dbc 0 2024-03-13T14:49:50.988 INFO:tasks.workunit.client.1.smithi196.stdout:3/660: chown db/d19/d22/d3a/c5e 59312 1 2024-03-13T14:49:50.989 INFO:tasks.workunit.client.0.smithi176.stdout:5/512: mknod d5/d31/d86/cae 0 2024-03-13T14:49:50.991 INFO:tasks.workunit.client.1.smithi196.stdout:7/763: mknod d0/d9/d1e/c100 0 2024-03-13T14:49:50.992 INFO:tasks.workunit.client.0.smithi176.stdout:0/470: readlink d2/db/d39/l63 0 2024-03-13T14:49:50.994 INFO:tasks.workunit.client.0.smithi176.stdout:6/566: creat d0/de/d54/fb8 x:0 0 0 2024-03-13T14:49:50.994 INFO:tasks.workunit.client.1.smithi196.stdout:4/632: write d1/d3/d7/d35/d3e/d56/f9c [796855,36958] 0 2024-03-13T14:49:50.995 INFO:tasks.workunit.client.1.smithi196.stdout:1/724: fsync d1/d9/d33/f82 0 2024-03-13T14:49:50.996 INFO:tasks.workunit.client.0.smithi176.stdout:6/567: chown d0/de/d38/d70/f8e 112 1 2024-03-13T14:49:50.997 INFO:tasks.workunit.client.1.smithi196.stdout:8/623: dwrite d0/d1c/d29/f65 [0,4194304] 0 2024-03-13T14:49:50.997 INFO:tasks.workunit.client.1.smithi196.stdout:6/806: dwrite d6/d6f/fec [0,4194304] 0 2024-03-13T14:49:50.998 INFO:tasks.workunit.client.1.smithi196.stdout:0/679: creat d8/d37/fdc x:0 0 0 2024-03-13T14:49:50.999 INFO:tasks.workunit.client.0.smithi176.stdout:1/514: unlink d2/de/c11 0 2024-03-13T14:49:51.000 INFO:tasks.workunit.client.0.smithi176.stdout:2/591: mknod d1/dc/d2d/d59/d8b/daf/db2/cc6 0 2024-03-13T14:49:51.002 INFO:tasks.workunit.client.0.smithi176.stdout:4/594: dwrite d7/d8/d15/d94/fa6 [0,4194304] 0 2024-03-13T14:49:51.004 INFO:tasks.workunit.client.0.smithi176.stdout:4/595: chown d7/d10/d2c/d9d/ld4 2 1 2024-03-13T14:49:51.004 INFO:tasks.workunit.client.1.smithi196.stdout:9/592: rename d7/de/d4b/d1d/f6e to d7/d9/d17/dbc/fc5 0 2024-03-13T14:49:51.004 INFO:tasks.workunit.client.1.smithi196.stdout:0/680: readlink d8/da/d25/d26/d35/d3b/d50/d5d/l15 0 2024-03-13T14:49:51.005 INFO:tasks.workunit.client.0.smithi176.stdout:3/592: mkdir d4/db8/dcc 0 2024-03-13T14:49:51.007 INFO:tasks.workunit.client.0.smithi176.stdout:5/513: chown d5/l25 191753844 1 2024-03-13T14:49:51.009 INFO:tasks.workunit.client.0.smithi176.stdout:7/514: creat d1/d28/d3a/d1b/d8f/f99 x:0 0 0 2024-03-13T14:49:51.009 INFO:tasks.workunit.client.0.smithi176.stdout:4/596: dread d7/d10/d46/faf [0,4194304] 0 2024-03-13T14:49:51.010 INFO:tasks.workunit.client.1.smithi196.stdout:9/593: truncate d7/de/d16/f9b 307486 0 2024-03-13T14:49:51.011 INFO:tasks.workunit.client.1.smithi196.stdout:2/582: creat d0/d7/d66/d67/fbd x:0 0 0 2024-03-13T14:49:51.013 INFO:tasks.workunit.client.1.smithi196.stdout:3/661: mknod db/d19/d22/d3a/d68/cd8 0 2024-03-13T14:49:51.022 INFO:tasks.workunit.client.1.smithi196.stdout:7/764: readlink d0/d9/d31/le4 0 2024-03-13T14:49:51.025 INFO:tasks.workunit.client.0.smithi176.stdout:9/607: link d0/db/d89/d7a/d8b/f8f d0/db/d89/d4d/d5d/fce 0 2024-03-13T14:49:51.026 INFO:tasks.workunit.client.1.smithi196.stdout:7/765: chown d0/d9/d1e/d58/d72/da9/da3/dc2 514547 1 2024-03-13T14:49:51.027 INFO:tasks.workunit.client.1.smithi196.stdout:6/807: dwrite d6/d9/daa/fc2 [0,4194304] 0 2024-03-13T14:49:51.028 INFO:tasks.workunit.client.0.smithi176.stdout:9/608: dread - d0/db/d89/d32/d58/f83 zero size 2024-03-13T14:49:51.032 INFO:tasks.workunit.client.0.smithi176.stdout:2/592: symlink d1/lc7 0 2024-03-13T14:49:51.033 INFO:tasks.workunit.client.0.smithi176.stdout:2/593: stat d1/f31 0 2024-03-13T14:49:51.037 INFO:tasks.workunit.client.0.smithi176.stdout:1/515: symlink d2/de/d10/d16/d4a/d5f/l9e 0 2024-03-13T14:49:51.039 INFO:tasks.workunit.client.1.smithi196.stdout:8/624: mkdir d0/d10/d12/d1e/d2d/dd8 0 2024-03-13T14:49:51.041 INFO:tasks.workunit.client.0.smithi176.stdout:6/568: truncate d0/d10/d19/d27/d2b/f5c 163012 0 2024-03-13T14:49:51.043 INFO:tasks.workunit.client.0.smithi176.stdout:6/569: stat d0/d10/d19/d9f/d76/fab 0 2024-03-13T14:49:51.044 INFO:tasks.workunit.client.1.smithi196.stdout:4/633: rename d1/d3/d7/d35/d76/d58/d69/d6c/ld4 to d1/d3/d4c/d72/d7d/d80/d55/ldd 0 2024-03-13T14:49:51.045 INFO:tasks.workunit.client.0.smithi176.stdout:8/496: dwrite d0/d5/d11/f5f [0,4194304] 0 2024-03-13T14:49:51.047 INFO:tasks.workunit.client.0.smithi176.stdout:8/497: chown d0/db/dc/l18 20 1 2024-03-13T14:49:51.050 INFO:tasks.workunit.client.1.smithi196.stdout:3/662: mknod db/d4c/cd9 0 2024-03-13T14:49:51.051 INFO:tasks.workunit.client.0.smithi176.stdout:3/593: symlink d4/d28/lcd 0 2024-03-13T14:49:51.052 INFO:tasks.workunit.client.0.smithi176.stdout:5/514: rename d5/d2b/l3c to d5/d31/laf 0 2024-03-13T14:49:51.053 INFO:tasks.workunit.client.1.smithi196.stdout:9/594: creat d7/d9/d17/dbc/fc6 x:0 0 0 2024-03-13T14:49:51.054 INFO:tasks.workunit.client.0.smithi176.stdout:3/594: chown d4/d13/d81/d89/f9c 787412 1 2024-03-13T14:49:51.054 INFO:tasks.workunit.client.0.smithi176.stdout:5/515: dread - d5/d2e/f54 zero size 2024-03-13T14:49:51.058 INFO:tasks.workunit.client.0.smithi176.stdout:0/471: rename d2/db/d31/f51 to d2/db/d31/f90 0 2024-03-13T14:49:51.060 INFO:tasks.workunit.client.1.smithi196.stdout:8/625: dwrite d0/d16/d3d/d8b/d48/f5a [0,4194304] 0 2024-03-13T14:49:51.060 INFO:tasks.workunit.client.0.smithi176.stdout:1/516: dread d2/de/d10/d16/d47/f5c [0,4194304] 0 2024-03-13T14:49:51.065 INFO:tasks.workunit.client.0.smithi176.stdout:9/609: mknod d0/db/d89/d32/d58/d5f/ccf 0 2024-03-13T14:49:51.067 INFO:tasks.workunit.client.1.smithi196.stdout:5/719: link d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/fb0 d4/d1b/dc8/d14/d19/d26/d33/fdb 0 2024-03-13T14:49:51.067 INFO:tasks.workunit.client.0.smithi176.stdout:2/594: symlink d1/dc/d2d/d4f/lc8 0 2024-03-13T14:49:51.069 INFO:tasks.workunit.client.0.smithi176.stdout:9/610: truncate d0/d1e/d1f/d68/f85 1703583 0 2024-03-13T14:49:51.071 INFO:tasks.workunit.client.1.smithi196.stdout:5/720: chown d4/d1b/dc8/d14/d19/d26/dab/fb9 35662 1 2024-03-13T14:49:51.073 INFO:tasks.workunit.client.1.smithi196.stdout:2/583: rename d0/d7/d66/d67/f85 to d0/d6/d34/fbe 0 2024-03-13T14:49:51.081 INFO:tasks.workunit.client.1.smithi196.stdout:9/595: dwrite d7/de/d16/f60 [0,4194304] 0 2024-03-13T14:49:51.081 INFO:tasks.workunit.client.0.smithi176.stdout:6/570: creat d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/fb9 x:0 0 0 2024-03-13T14:49:51.085 INFO:tasks.workunit.client.0.smithi176.stdout:7/515: mkdir d1/d3f/d81/d9a 0 2024-03-13T14:49:51.086 INFO:tasks.workunit.client.0.smithi176.stdout:3/595: mkdir d4/d28/d31/d45/d82/dce 0 2024-03-13T14:49:51.088 INFO:tasks.workunit.client.1.smithi196.stdout:1/725: dwrite d1/d9/d33/fa5 [0,4194304] 0 2024-03-13T14:49:51.089 INFO:tasks.workunit.client.1.smithi196.stdout:0/681: write d8/da/d25/d26/d35/d3b/d50/f6d [4293012,54109] 0 2024-03-13T14:49:51.091 INFO:tasks.workunit.client.0.smithi176.stdout:3/596: read d4/d13/d81/d89/d5b/dc5/fa1 [109577,23855] 0 2024-03-13T14:49:51.095 INFO:tasks.workunit.client.0.smithi176.stdout:0/472: stat d2/d5/dc/l7c 0 2024-03-13T14:49:51.097 INFO:tasks.workunit.client.0.smithi176.stdout:0/473: fsync d2/d5/dc/d59/f5a 0 2024-03-13T14:49:51.097 INFO:tasks.workunit.client.1.smithi196.stdout:4/634: rename d1/d3/dbc/lae to d1/d3/d7/d35/d76/d58/dc6/lde 0 2024-03-13T14:49:51.098 INFO:tasks.workunit.client.0.smithi176.stdout:9/611: mknod d0/d40/d53/d6d/cd0 0 2024-03-13T14:49:51.099 INFO:tasks.workunit.client.0.smithi176.stdout:9/612: readlink d0/d1e/l27 0 2024-03-13T14:49:51.103 INFO:tasks.workunit.client.1.smithi196.stdout:7/766: rename d0/l1 to d0/l101 0 2024-03-13T14:49:51.105 INFO:tasks.workunit.client.0.smithi176.stdout:2/595: rename d1/d2a/d33/f9d to d1/dc/d2d/d90/fc9 0 2024-03-13T14:49:51.108 INFO:tasks.workunit.client.1.smithi196.stdout:6/808: dread d6/d9/d17/da1/f65 [0,4194304] 0 2024-03-13T14:49:51.110 INFO:tasks.workunit.client.1.smithi196.stdout:6/809: stat d6/d2a/dc/d71/ffb 0 2024-03-13T14:49:51.111 INFO:tasks.workunit.client.0.smithi176.stdout:0/474: dwrite d2/f6 [8388608,4194304] 0 2024-03-13T14:49:51.111 INFO:tasks.workunit.client.1.smithi196.stdout:5/721: symlink d4/d1b/dc8/d14/d19/d26/d95/ldc 0 2024-03-13T14:49:51.113 INFO:tasks.workunit.client.0.smithi176.stdout:6/571: fsync d0/de/d54/f5e 0 2024-03-13T14:49:51.119 INFO:tasks.workunit.client.1.smithi196.stdout:1/726: creat d1/d9/d4e/dcc/ff9 x:0 0 0 2024-03-13T14:49:51.121 INFO:tasks.workunit.client.1.smithi196.stdout:0/682: creat d8/da/d25/d26/d35/d3b/d59/d6a/fdd x:0 0 0 2024-03-13T14:49:51.124 INFO:tasks.workunit.client.1.smithi196.stdout:2/584: symlink d0/d7/d9/d61/lbf 0 2024-03-13T14:49:51.125 INFO:tasks.workunit.client.1.smithi196.stdout:0/683: fdatasync d8/da/d25/d26/d48/dba/fab 0 2024-03-13T14:49:51.126 INFO:tasks.workunit.client.0.smithi176.stdout:5/516: write d5/d22/d15/f16 [3990331,42911] 0 2024-03-13T14:49:51.127 INFO:tasks.workunit.client.1.smithi196.stdout:2/585: stat d0/d7/d66/f76 0 2024-03-13T14:49:51.129 INFO:tasks.workunit.client.1.smithi196.stdout:8/626: creat d0/d16/fd9 x:0 0 0 2024-03-13T14:49:51.130 INFO:tasks.workunit.client.1.smithi196.stdout:3/663: dwrite db/d37/d64/f57 [0,4194304] 0 2024-03-13T14:49:51.131 INFO:tasks.workunit.client.1.smithi196.stdout:2/586: chown d0/d6/d4b/d96/d9e 37623 1 2024-03-13T14:49:51.132 INFO:tasks.workunit.client.1.smithi196.stdout:5/722: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/fdd x:0 0 0 2024-03-13T14:49:51.135 INFO:tasks.workunit.client.1.smithi196.stdout:2/587: write d0/d7/d9/d9a/fb4 [324047,57501] 0 2024-03-13T14:49:51.143 INFO:tasks.workunit.client.1.smithi196.stdout:7/767: mknod d0/d9/d1e/d58/d72/d6e/c102 0 2024-03-13T14:49:51.144 INFO:tasks.workunit.client.1.smithi196.stdout:6/810: mknod d6/d9/d17/dfc/c108 0 2024-03-13T14:49:51.146 INFO:tasks.workunit.client.1.smithi196.stdout:3/664: dread db/d19/d22/f3c [0,4194304] 0 2024-03-13T14:49:51.147 INFO:tasks.workunit.client.1.smithi196.stdout:8/627: mkdir d0/d16/d3d/d7f/dda 0 2024-03-13T14:49:51.147 INFO:tasks.workunit.client.0.smithi176.stdout:6/572: creat d0/d10/d19/d9f/d76/fba x:0 0 0 2024-03-13T14:49:51.148 INFO:tasks.workunit.client.0.smithi176.stdout:7/516: symlink d1/d3f/d81/d9a/l9b 0 2024-03-13T14:49:51.152 INFO:tasks.workunit.client.1.smithi196.stdout:1/727: mkdir d1/dfa 0 2024-03-13T14:49:51.152 INFO:tasks.workunit.client.0.smithi176.stdout:8/498: dwrite d0/f5b [0,4194304] 0 2024-03-13T14:49:51.153 INFO:tasks.workunit.client.1.smithi196.stdout:3/665: stat db/d15/d44/d79/fd2 0 2024-03-13T14:49:51.153 INFO:tasks.workunit.client.0.smithi176.stdout:6/573: truncate d0/de/d38/d70/f8e 751912 0 2024-03-13T14:49:51.155 INFO:tasks.workunit.client.0.smithi176.stdout:3/597: symlink d4/d13/d2d/lcf 0 2024-03-13T14:49:51.156 INFO:tasks.workunit.client.1.smithi196.stdout:0/684: rename d8/da/d25/l8a to d8/da/d25/d26/d35/d3b/d50/d5d/d7e/lde 0 2024-03-13T14:49:51.159 INFO:tasks.workunit.client.1.smithi196.stdout:7/768: mkdir d0/d6/db4/dca/d103 0 2024-03-13T14:49:51.163 INFO:tasks.workunit.client.0.smithi176.stdout:9/613: unlink d0/c4 0 2024-03-13T14:49:51.164 INFO:tasks.workunit.client.1.smithi196.stdout:2/588: dread d0/d7/f17 [0,4194304] 0 2024-03-13T14:49:51.164 INFO:tasks.workunit.client.1.smithi196.stdout:6/811: write d6/d9/d11/fc7 [2783975,17870] 0 2024-03-13T14:49:51.166 INFO:tasks.workunit.client.1.smithi196.stdout:3/666: rmdir db/d19 39 2024-03-13T14:49:51.167 INFO:tasks.workunit.client.1.smithi196.stdout:2/589: chown d0/d7/d56/l6a 204698024 1 2024-03-13T14:49:51.171 INFO:tasks.workunit.client.1.smithi196.stdout:0/685: creat d8/da/d25/d26/d35/d3b/d5f/d33/d44/fdf x:0 0 0 2024-03-13T14:49:51.173 INFO:tasks.workunit.client.0.smithi176.stdout:6/574: mknod d0/d10/d19/d7d/cbb 0 2024-03-13T14:49:51.174 INFO:tasks.workunit.client.0.smithi176.stdout:5/517: unlink d5/cb 0 2024-03-13T14:49:51.174 INFO:tasks.workunit.client.1.smithi196.stdout:1/728: chown d1/d3f/d5b/d74/lb7 5596952 1 2024-03-13T14:49:51.175 INFO:tasks.workunit.client.0.smithi176.stdout:6/575: write d0/d9/f55 [2318093,104296] 0 2024-03-13T14:49:51.175 INFO:tasks.workunit.client.1.smithi196.stdout:8/628: dwrite d0/d16/d58/fd0 [0,4194304] 0 2024-03-13T14:49:51.177 INFO:tasks.workunit.client.0.smithi176.stdout:1/517: getdents d2/de/d3f/d6a 0 2024-03-13T14:49:51.178 INFO:tasks.workunit.client.1.smithi196.stdout:2/590: creat d0/d7/d9/d9a/daa/fc0 x:0 0 0 2024-03-13T14:49:51.179 INFO:tasks.workunit.client.0.smithi176.stdout:9/614: mknod d0/db/d89/d32/d58/d67/cd1 0 2024-03-13T14:49:51.180 INFO:tasks.workunit.client.0.smithi176.stdout:2/596: creat d1/dc/d2d/fca x:0 0 0 2024-03-13T14:49:51.181 INFO:tasks.workunit.client.0.smithi176.stdout:0/475: creat d2/d74/f91 x:0 0 0 2024-03-13T14:49:51.181 INFO:tasks.workunit.client.0.smithi176.stdout:2/597: readlink d1/l3e 0 2024-03-13T14:49:51.181 INFO:tasks.workunit.client.0.smithi176.stdout:8/499: mknod d0/d24/c9a 0 2024-03-13T14:49:51.184 INFO:tasks.workunit.client.1.smithi196.stdout:5/723: dwrite d4/d1b/dc8/d14/d19/d26/f3a [0,4194304] 0 2024-03-13T14:49:51.184 INFO:tasks.workunit.client.0.smithi176.stdout:9/615: chown d0/db/d89/d32/d58/d5f/d84/d9a/f9b 2 1 2024-03-13T14:49:51.187 INFO:tasks.workunit.client.1.smithi196.stdout:6/812: rmdir d6/d9/d105 0 2024-03-13T14:49:51.187 INFO:tasks.workunit.client.0.smithi176.stdout:9/616: stat d0/db/d89/d32/d58/d5f 0 2024-03-13T14:49:51.188 INFO:tasks.workunit.client.1.smithi196.stdout:5/724: readlink d4/d1b/dc8/d14/d19/l20 0 2024-03-13T14:49:51.192 INFO:tasks.workunit.client.0.smithi176.stdout:1/518: symlink d2/de/d10/d16/d4a/l9f 0 2024-03-13T14:49:51.195 INFO:tasks.workunit.client.0.smithi176.stdout:7/517: link d1/d3f/d65/c8c d1/d1c/d26/d41/d2b/d40/d64/d94/c9c 0 2024-03-13T14:49:51.196 INFO:tasks.workunit.client.0.smithi176.stdout:0/476: rmdir d2/db/d39 39 2024-03-13T14:49:51.197 INFO:tasks.workunit.client.1.smithi196.stdout:7/769: getdents d0/d9/d1e/d58/d72/d6e 0 2024-03-13T14:49:51.197 INFO:tasks.workunit.client.1.smithi196.stdout:8/629: truncate d0/d1c/d29/d47/f52 363686 0 2024-03-13T14:49:51.200 INFO:tasks.workunit.client.0.smithi176.stdout:2/598: mknod d1/dc/d2d/d90/ccb 0 2024-03-13T14:49:51.202 INFO:tasks.workunit.client.1.smithi196.stdout:6/813: creat d6/d2a/d23/d28/f109 x:0 0 0 2024-03-13T14:49:51.204 INFO:tasks.workunit.client.0.smithi176.stdout:3/598: write d4/d13/d16/d3d/f5d [4675792,82975] 0 2024-03-13T14:49:51.204 INFO:tasks.workunit.client.0.smithi176.stdout:6/576: fdatasync d0/de/d38/d70/faf 0 2024-03-13T14:49:51.205 INFO:tasks.workunit.client.1.smithi196.stdout:0/686: creat d8/da/d25/d26/d35/d3b/fe0 x:0 0 0 2024-03-13T14:49:51.205 INFO:tasks.workunit.client.1.smithi196.stdout:2/591: rmdir d0/d7/d9/d9a/daa 39 2024-03-13T14:49:51.206 INFO:tasks.workunit.client.0.smithi176.stdout:3/599: truncate d4/d13/d2d/f51 4657450 0 2024-03-13T14:49:51.207 INFO:tasks.workunit.client.0.smithi176.stdout:9/617: rename d0/db/d89/fc2 to d0/d40/fd2 0 2024-03-13T14:49:51.208 INFO:tasks.workunit.client.1.smithi196.stdout:0/687: write d8/da/d25/d26/d35/d3b/d6c/fc6 [148759,93552] 0 2024-03-13T14:49:51.209 INFO:tasks.workunit.client.0.smithi176.stdout:1/519: symlink d2/de/d3f/d3e/la0 0 2024-03-13T14:49:51.210 INFO:tasks.workunit.client.0.smithi176.stdout:1/520: fsync f1 0 2024-03-13T14:49:51.210 INFO:tasks.workunit.client.0.smithi176.stdout:9/618: fsync d0/db/d89/d4d/d6e/f71 0 2024-03-13T14:49:51.211 INFO:tasks.workunit.client.1.smithi196.stdout:0/688: dread - d8/da/d25/d26/d35/d3b/d50/fd3 zero size 2024-03-13T14:49:51.211 INFO:tasks.workunit.client.1.smithi196.stdout:3/667: creat db/d19/fda x:0 0 0 2024-03-13T14:49:51.215 INFO:tasks.workunit.client.0.smithi176.stdout:0/477: unlink d2/l8f 0 2024-03-13T14:49:51.216 INFO:tasks.workunit.client.1.smithi196.stdout:1/729: write d1/d9/d33/f42 [4497906,10223] 0 2024-03-13T14:49:51.218 INFO:tasks.workunit.client.1.smithi196.stdout:8/630: mknod d0/d10/d12/d1e/d2d/cdb 0 2024-03-13T14:49:51.218 INFO:tasks.workunit.client.1.smithi196.stdout:1/730: dread - d1/d3f/d5b/ff7 zero size 2024-03-13T14:49:51.218 INFO:tasks.workunit.client.0.smithi176.stdout:8/500: mknod d0/d5/c9b 0 2024-03-13T14:49:51.219 INFO:tasks.workunit.client.0.smithi176.stdout:5/518: dwrite d5/f6 [0,4194304] 0 2024-03-13T14:49:51.220 INFO:tasks.workunit.client.0.smithi176.stdout:0/478: dread d2/d5/d28/f54 [0,4194304] 0 2024-03-13T14:49:51.221 INFO:tasks.workunit.client.1.smithi196.stdout:6/814: rename d6/d9/d17/da1/d50/de8 to d6/d2a/d23/d28/d10a 0 2024-03-13T14:49:51.221 INFO:tasks.workunit.client.0.smithi176.stdout:5/519: fdatasync d5/d31/f61 0 2024-03-13T14:49:51.223 INFO:tasks.workunit.client.1.smithi196.stdout:6/815: chown d6/d7/df/fdd 22932770 1 2024-03-13T14:49:51.224 INFO:tasks.workunit.client.0.smithi176.stdout:2/599: mkdir d1/dc/d61/daa/dcc 0 2024-03-13T14:49:51.230 INFO:tasks.workunit.client.1.smithi196.stdout:5/725: mkdir d4/d1b/dc8/d14/d19/d26/d33/dde 0 2024-03-13T14:49:51.231 INFO:tasks.workunit.client.1.smithi196.stdout:0/689: stat d8/da/d25/d26/d35/d3b/d5f/d33/cc8 0 2024-03-13T14:49:51.232 INFO:tasks.workunit.client.1.smithi196.stdout:7/770: write d0/d9/d1e/d58/d72/d99/fb8 [4307339,96157] 0 2024-03-13T14:49:51.233 INFO:tasks.workunit.client.0.smithi176.stdout:3/600: mknod d4/d13/d5e/cd0 0 2024-03-13T14:49:51.234 INFO:tasks.workunit.client.1.smithi196.stdout:8/631: dread d0/d1c/d29/d47/f80 [0,4194304] 0 2024-03-13T14:49:51.234 INFO:tasks.workunit.client.1.smithi196.stdout:0/690: stat d8/da/d25/fd1 0 2024-03-13T14:49:51.236 INFO:tasks.workunit.client.1.smithi196.stdout:8/632: chown d0/d16/d71/d68/c90 4936 1 2024-03-13T14:49:51.237 INFO:tasks.workunit.client.1.smithi196.stdout:0/691: fsync d8/da/d25/d26/d35/d3b/d50/fd3 0 2024-03-13T14:49:51.240 INFO:tasks.workunit.client.1.smithi196.stdout:1/731: rmdir d1/d9/d4e/dcc 39 2024-03-13T14:49:51.244 INFO:tasks.workunit.client.1.smithi196.stdout:3/668: rename db/cbf to db/d19/d50/db7/cdb 0 2024-03-13T14:49:51.245 INFO:tasks.workunit.client.0.smithi176.stdout:1/521: creat d2/de/d10/d16/d59/fa1 x:0 0 0 2024-03-13T14:49:51.246 INFO:tasks.workunit.client.1.smithi196.stdout:2/592: symlink d0/d7/d9/lc1 0 2024-03-13T14:49:51.248 INFO:tasks.workunit.client.1.smithi196.stdout:6/816: mknod d6/d2a/de2/c10b 0 2024-03-13T14:49:51.249 INFO:tasks.workunit.client.1.smithi196.stdout:6/817: fsync d6/d7/df/fdd 0 2024-03-13T14:49:51.251 INFO:tasks.workunit.client.0.smithi176.stdout:4/597: sync 2024-03-13T14:49:51.253 INFO:tasks.workunit.client.0.smithi176.stdout:4/598: chown d7/d2d/fa4 9787 1 2024-03-13T14:49:51.253 INFO:tasks.workunit.client.1.smithi196.stdout:8/633: unlink d0/d10/d12/d1e/f83 0 2024-03-13T14:49:51.256 INFO:tasks.workunit.client.1.smithi196.stdout:0/692: mknod d8/da/d25/d26/d35/ce1 0 2024-03-13T14:49:51.256 INFO:tasks.workunit.client.0.smithi176.stdout:4/599: chown d7/d2d/d9b/ca3 218648086 1 2024-03-13T14:49:51.257 INFO:tasks.workunit.client.1.smithi196.stdout:0/693: write d8/f9 [679877,65118] 0 2024-03-13T14:49:51.258 INFO:tasks.workunit.client.0.smithi176.stdout:0/479: write d2/db/f25 [282367,85392] 0 2024-03-13T14:49:51.259 INFO:tasks.workunit.client.0.smithi176.stdout:2/600: unlink d1/d2a/c58 0 2024-03-13T14:49:51.261 INFO:tasks.workunit.client.0.smithi176.stdout:6/577: symlink d0/de/d38/d6b/da3/db5/lbc 0 2024-03-13T14:49:51.263 INFO:tasks.workunit.client.1.smithi196.stdout:7/771: write d0/d4d/fc4 [295204,13087] 0 2024-03-13T14:49:51.264 INFO:tasks.workunit.client.0.smithi176.stdout:9/619: dwrite d0/d40/f78 [0,4194304] 0 2024-03-13T14:49:51.264 INFO:tasks.workunit.client.1.smithi196.stdout:3/669: creat db/d15/d44/d79/fdc x:0 0 0 2024-03-13T14:49:51.269 INFO:tasks.workunit.client.1.smithi196.stdout:5/726: mknod d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd/cdf 0 2024-03-13T14:49:51.272 INFO:tasks.workunit.client.1.smithi196.stdout:1/732: write d1/d9/d4e/d55/f6f [3235215,18649] 0 2024-03-13T14:49:51.273 INFO:tasks.workunit.client.0.smithi176.stdout:1/522: unlink d2/d29/c92 0 2024-03-13T14:49:51.275 INFO:tasks.workunit.client.0.smithi176.stdout:1/523: write d2/f6 [1640993,110138] 0 2024-03-13T14:49:51.276 INFO:tasks.workunit.client.1.smithi196.stdout:6/818: fsync d6/d2a/dc/d47/f55 0 2024-03-13T14:49:51.278 INFO:tasks.workunit.client.0.smithi176.stdout:8/501: creat d0/db/d39/d56/d91/f9c x:0 0 0 2024-03-13T14:49:51.279 INFO:tasks.workunit.client.1.smithi196.stdout:8/634: fdatasync d0/d16/d71/f43 0 2024-03-13T14:49:51.280 INFO:tasks.workunit.client.0.smithi176.stdout:5/520: symlink d5/d2c/d8a/lb0 0 2024-03-13T14:49:51.284 INFO:tasks.workunit.client.0.smithi176.stdout:9/620: dwrite d0/db/d89/d32/d58/d9c/fb3 [0,4194304] 0 2024-03-13T14:49:51.287 INFO:tasks.workunit.client.0.smithi176.stdout:3/601: dwrite d4/d28/d31/d45/d82/fb3 [0,4194304] 0 2024-03-13T14:49:51.287 INFO:tasks.workunit.client.0.smithi176.stdout:4/600: rename d7/d2d/d9b to d7/d8/d15/dd6 0 2024-03-13T14:49:51.288 INFO:tasks.workunit.client.1.smithi196.stdout:3/670: fsync db/f3d 0 2024-03-13T14:49:51.289 INFO:tasks.workunit.client.0.smithi176.stdout:9/621: dread - d0/db/d89/d32/d58/d5f/fc8 zero size 2024-03-13T14:49:51.295 INFO:tasks.workunit.client.0.smithi176.stdout:7/518: getdents d1/d3f/d81 0 2024-03-13T14:49:51.299 INFO:tasks.workunit.client.0.smithi176.stdout:7/519: truncate d1/d1c/d26/d41/d2b/d40/f8a 238171 0 2024-03-13T14:49:51.303 INFO:tasks.workunit.client.0.smithi176.stdout:6/578: dread d0/d10/d19/d27/f4d [0,4194304] 0 2024-03-13T14:49:51.303 INFO:tasks.workunit.client.0.smithi176.stdout:7/520: chown d1/d1c/d26/d41/d2b/d40/l46 413 1 2024-03-13T14:49:51.304 INFO:tasks.workunit.client.1.smithi196.stdout:0/694: write d8/da/d25/d26/d54/f9a [946503,26263] 0 2024-03-13T14:49:51.305 INFO:tasks.workunit.client.0.smithi176.stdout:3/602: read d4/d13/d81/d89/f60 [1639632,109907] 0 2024-03-13T14:49:51.306 INFO:tasks.workunit.client.0.smithi176.stdout:6/579: chown d0/d10/d19/d27/l3d 1776 1 2024-03-13T14:49:51.306 INFO:tasks.workunit.client.1.smithi196.stdout:8/635: dwrite d0/d16/d58/fd0 [4194304,4194304] 0 2024-03-13T14:49:51.308 INFO:tasks.workunit.client.0.smithi176.stdout:9/622: dwrite d0/db/d89/d32/d58/d9c/fb3 [0,4194304] 0 2024-03-13T14:49:51.316 INFO:tasks.workunit.client.1.smithi196.stdout:3/671: unlink db/d37/fd4 0 2024-03-13T14:49:51.323 INFO:tasks.workunit.client.1.smithi196.stdout:7/772: truncate d0/d6/d33/d57/f95 828604 0 2024-03-13T14:49:51.323 INFO:tasks.workunit.client.0.smithi176.stdout:3/603: dwrite d4/d28/f86 [0,4194304] 0 2024-03-13T14:49:51.325 INFO:tasks.workunit.client.1.smithi196.stdout:4/635: sync 2024-03-13T14:49:51.325 INFO:tasks.workunit.client.1.smithi196.stdout:5/727: rename d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd/ca9 to d4/d1b/dc8/d14/d19/d26/d68/ce0 0 2024-03-13T14:49:51.325 INFO:tasks.workunit.client.1.smithi196.stdout:9/596: sync 2024-03-13T14:49:51.329 INFO:tasks.workunit.client.0.smithi176.stdout:7/521: creat d1/d1c/d26/d41/d2b/d40/d64/f9d x:0 0 0 2024-03-13T14:49:51.331 INFO:tasks.workunit.client.1.smithi196.stdout:9/597: chown d7/d9/c2f 144 1 2024-03-13T14:49:51.331 INFO:tasks.workunit.client.0.smithi176.stdout:9/623: mknod d0/d1e/d8d/da7/cd3 0 2024-03-13T14:49:51.333 INFO:tasks.workunit.client.1.smithi196.stdout:9/598: readlink d7/d24/d28/l37 0 2024-03-13T14:49:51.338 INFO:tasks.workunit.client.1.smithi196.stdout:2/593: write d0/d7/d9/d5a/f6f [1070072,51046] 0 2024-03-13T14:49:51.338 INFO:tasks.workunit.client.1.smithi196.stdout:3/672: dwrite db/d19/f28 [4194304,4194304] 0 2024-03-13T14:49:51.342 INFO:tasks.workunit.client.0.smithi176.stdout:0/480: rename d2/db/d31/f4c to d2/d5/d1e/f92 0 2024-03-13T14:49:51.344 INFO:tasks.workunit.client.1.smithi196.stdout:7/773: mkdir d0/d9/d31/d77/dcf/de0/d104 0 2024-03-13T14:49:51.345 INFO:tasks.workunit.client.0.smithi176.stdout:0/481: chown d2/d5/l17 27 1 2024-03-13T14:49:51.345 INFO:tasks.workunit.client.0.smithi176.stdout:1/524: write d2/de/d10/d16/f27 [3316246,26111] 0 2024-03-13T14:49:51.348 INFO:tasks.workunit.client.0.smithi176.stdout:0/482: dread - d2/db/d35/f8d zero size 2024-03-13T14:49:51.348 INFO:tasks.workunit.client.0.smithi176.stdout:6/580: unlink d0/d10/d19/d27/d2b/f5c 0 2024-03-13T14:49:51.350 INFO:tasks.workunit.client.0.smithi176.stdout:3/604: dwrite d4/d28/d4f/f90 [0,4194304] 0 2024-03-13T14:49:51.350 INFO:tasks.workunit.client.1.smithi196.stdout:8/636: symlink d0/d10/d70/ldc 0 2024-03-13T14:49:51.351 INFO:tasks.workunit.client.0.smithi176.stdout:1/525: fdatasync d2/d30/d63/d22/f82 0 2024-03-13T14:49:51.352 INFO:tasks.workunit.client.0.smithi176.stdout:2/601: getdents d1/dc/d2d/d59 0 2024-03-13T14:49:51.353 INFO:tasks.workunit.client.1.smithi196.stdout:3/673: dread db/d15/d44/d79/fba [0,4194304] 0 2024-03-13T14:49:51.356 INFO:tasks.workunit.client.0.smithi176.stdout:5/521: rename d5/d2c/d40/cab to d5/d31/d47/d80/cb1 0 2024-03-13T14:49:51.361 INFO:tasks.workunit.client.1.smithi196.stdout:7/774: mknod d0/d9/d1e/d58/d72/d99/c105 0 2024-03-13T14:49:51.361 INFO:tasks.workunit.client.0.smithi176.stdout:9/624: mknod d0/cd4 0 2024-03-13T14:49:51.366 INFO:tasks.workunit.client.0.smithi176.stdout:1/526: write d2/de/d10/d16/d59/fa1 [952780,115827] 0 2024-03-13T14:49:51.367 INFO:tasks.workunit.client.1.smithi196.stdout:6/819: truncate d6/d2a/d23/f8f 1276172 0 2024-03-13T14:49:51.368 INFO:tasks.workunit.client.1.smithi196.stdout:9/599: symlink d7/d9/lc7 0 2024-03-13T14:49:51.369 INFO:tasks.workunit.client.0.smithi176.stdout:8/502: dwrite d0/db/dc/f6b [0,4194304] 0 2024-03-13T14:49:51.371 INFO:tasks.workunit.client.1.smithi196.stdout:6/820: truncate d6/f8 1179555 0 2024-03-13T14:49:51.371 INFO:tasks.workunit.client.1.smithi196.stdout:8/637: dwrite d0/d1c/d29/fa6 [0,4194304] 0 2024-03-13T14:49:51.371 INFO:tasks.workunit.client.0.smithi176.stdout:2/602: truncate d1/f13 691488 0 2024-03-13T14:49:51.372 INFO:tasks.workunit.client.1.smithi196.stdout:4/636: rename d1/d3/d7/d35/d3e/db1 to d1/d3/d7/ddf 0 2024-03-13T14:49:51.372 INFO:tasks.workunit.client.0.smithi176.stdout:4/601: dwrite d7/d10/d2c/f66 [0,4194304] 0 2024-03-13T14:49:51.373 INFO:tasks.workunit.client.0.smithi176.stdout:3/605: getdents d4/d13/d5e/daa 0 2024-03-13T14:49:51.374 INFO:tasks.workunit.client.0.smithi176.stdout:2/603: chown d1/d2a/d33/f5b 224458 1 2024-03-13T14:49:51.375 INFO:tasks.workunit.client.1.smithi196.stdout:4/637: dread - d1/d3/d4c/d72/d7d/d80/fa7 zero size 2024-03-13T14:49:51.377 INFO:tasks.workunit.client.0.smithi176.stdout:2/604: truncate d1/dc/d61/f6d 4576347 0 2024-03-13T14:49:51.378 INFO:tasks.workunit.client.0.smithi176.stdout:7/522: rename d1/c59 to d1/d3f/d4b/d63/c9e 0 2024-03-13T14:49:51.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:51 smithi176 ceph-mon[29064]: pgmap v14: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 75 MiB/s rd, 242 MiB/s wr, 478 op/s 2024-03-13T14:49:51.379 INFO:tasks.workunit.client.1.smithi196.stdout:0/695: getdents d8/da/d25/d26 0 2024-03-13T14:49:51.380 INFO:tasks.workunit.client.0.smithi176.stdout:2/605: dread - d1/dc/d2d/d59/d8c/fad zero size 2024-03-13T14:49:51.382 INFO:tasks.workunit.client.0.smithi176.stdout:9/625: fsync d0/db/d89/f16 0 2024-03-13T14:49:51.386 INFO:tasks.workunit.client.1.smithi196.stdout:7/775: mkdir d0/d9/d1e/d3f/d6c/d106 0 2024-03-13T14:49:51.391 INFO:tasks.workunit.client.0.smithi176.stdout:8/503: creat d0/d24/d29/f9d x:0 0 0 2024-03-13T14:49:51.391 INFO:tasks.workunit.client.1.smithi196.stdout:6/821: rmdir d6/d9/d17/da1 39 2024-03-13T14:49:51.395 INFO:tasks.workunit.client.1.smithi196.stdout:2/594: creat d0/d6/d4b/fc2 x:0 0 0 2024-03-13T14:49:51.397 INFO:tasks.workunit.client.0.smithi176.stdout:1/527: dread d2/d30/f4c [0,4194304] 0 2024-03-13T14:49:51.397 INFO:tasks.workunit.client.0.smithi176.stdout:3/606: symlink d4/d13/d5e/ld1 0 2024-03-13T14:49:51.400 INFO:tasks.workunit.client.1.smithi196.stdout:5/728: rename d4/d1b/dc8 to d4/d1b/dc8/d14/d19/d1c/d2d/de1 22 2024-03-13T14:49:51.403 INFO:tasks.workunit.client.1.smithi196.stdout:8/638: dread d0/d16/d3d/d9c/f9d [0,4194304] 0 2024-03-13T14:49:51.410 INFO:tasks.workunit.client.1.smithi196.stdout:2/595: creat d0/d6/d34/d5b/fc3 x:0 0 0 2024-03-13T14:49:51.412 INFO:tasks.workunit.client.0.smithi176.stdout:1/528: fdatasync d2/f21 0 2024-03-13T14:49:51.413 INFO:tasks.workunit.client.1.smithi196.stdout:1/733: rename d1/d28/d34/f52 to d1/d3f/d5b/d74/db5/ffb 0 2024-03-13T14:49:51.416 INFO:tasks.workunit.client.1.smithi196.stdout:5/729: mknod d4/d1b/dc8/d3c/d6a/d91/ce2 0 2024-03-13T14:49:51.416 INFO:tasks.workunit.client.0.smithi176.stdout:3/607: dwrite d4/d13/d16/d3d/fb7 [0,4194304] 0 2024-03-13T14:49:51.418 INFO:tasks.workunit.client.1.smithi196.stdout:9/600: creat d7/d9/fc8 x:0 0 0 2024-03-13T14:49:51.422 INFO:tasks.workunit.client.1.smithi196.stdout:8/639: mknod d0/d16/d3d/d9c/d87/d8c/da9/db8/cdd 0 2024-03-13T14:49:51.423 INFO:tasks.workunit.client.0.smithi176.stdout:5/522: dread d5/d2c/d40/d6a/f6e [0,4194304] 0 2024-03-13T14:49:51.423 INFO:tasks.workunit.client.0.smithi176.stdout:9/626: truncate d0/f59 2280056 0 2024-03-13T14:49:51.426 INFO:tasks.workunit.client.0.smithi176.stdout:4/602: dwrite d7/d8/d2f/fb4 [0,4194304] 0 2024-03-13T14:49:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:51 smithi196 ceph-mon[37486]: pgmap v14: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 75 MiB/s rd, 242 MiB/s wr, 478 op/s 2024-03-13T14:49:51.430 INFO:tasks.workunit.client.0.smithi176.stdout:1/529: symlink d2/de/d10/d16/d4a/la2 0 2024-03-13T14:49:51.432 INFO:tasks.workunit.client.0.smithi176.stdout:1/530: write d2/d30/f3a [7350287,69291] 0 2024-03-13T14:49:51.433 INFO:tasks.workunit.client.0.smithi176.stdout:0/483: rename d2/db/d31/f41 to d2/db/f93 0 2024-03-13T14:49:51.434 INFO:tasks.workunit.client.0.smithi176.stdout:1/531: read d2/d30/f65 [485595,110264] 0 2024-03-13T14:49:51.435 INFO:tasks.workunit.client.0.smithi176.stdout:3/608: mknod d4/d13/d5e/cd2 0 2024-03-13T14:49:51.437 INFO:tasks.workunit.client.1.smithi196.stdout:4/638: dwrite d1/d3/d7/f49 [0,4194304] 0 2024-03-13T14:49:51.438 INFO:tasks.workunit.client.0.smithi176.stdout:0/484: dread - d2/d74/f91 zero size 2024-03-13T14:49:51.441 INFO:tasks.workunit.client.1.smithi196.stdout:4/639: readlink d1/d3/l1f 0 2024-03-13T14:49:51.444 INFO:tasks.workunit.client.0.smithi176.stdout:3/609: chown d4/d28/d34/da4 1852523314 1 2024-03-13T14:49:51.444 INFO:tasks.workunit.client.0.smithi176.stdout:5/523: mknod d5/d22/d15/cb2 0 2024-03-13T14:49:51.444 INFO:tasks.workunit.client.0.smithi176.stdout:4/603: mknod d7/d8/d15/cd7 0 2024-03-13T14:49:51.445 INFO:tasks.workunit.client.1.smithi196.stdout:9/601: dwrite d7/d9/d17/f51 [0,4194304] 0 2024-03-13T14:49:51.445 INFO:tasks.workunit.client.0.smithi176.stdout:9/627: dwrite d0/db/d89/d32/d58/f83 [0,4194304] 0 2024-03-13T14:49:51.458 INFO:tasks.workunit.client.0.smithi176.stdout:6/581: rename d0/d10/d19/d27/d2b/f3a to d0/de/d8d/fbd 0 2024-03-13T14:49:51.462 INFO:tasks.workunit.client.1.smithi196.stdout:2/596: creat d0/d6/fc4 x:0 0 0 2024-03-13T14:49:51.466 INFO:tasks.workunit.client.0.smithi176.stdout:5/524: dwrite d5/d2c/f3d [0,4194304] 0 2024-03-13T14:49:51.467 INFO:tasks.workunit.client.1.smithi196.stdout:8/640: symlink d0/d10/lde 0 2024-03-13T14:49:51.472 INFO:tasks.workunit.client.1.smithi196.stdout:0/696: rename d8/da/d25/d26/d35/d3b/la1 to d8/da/d25/d26/d35/d3b/d5f/le2 0 2024-03-13T14:49:51.476 INFO:tasks.workunit.client.1.smithi196.stdout:2/597: symlink d0/d6/d1d/lc5 0 2024-03-13T14:49:51.478 INFO:tasks.workunit.client.1.smithi196.stdout:0/697: dread - d8/da/d25/d26/d54/d88/fcb zero size 2024-03-13T14:49:51.478 INFO:tasks.workunit.client.1.smithi196.stdout:1/734: write d1/d38/f30 [3320845,82947] 0 2024-03-13T14:49:51.481 INFO:tasks.workunit.client.0.smithi176.stdout:6/582: dwrite d0/d10/d19/d9f/f82 [0,4194304] 0 2024-03-13T14:49:51.481 INFO:tasks.workunit.client.1.smithi196.stdout:7/776: rename d0/d9/d1e/d58/fc3 to d0/d78/d81/dfb/f107 0 2024-03-13T14:49:51.482 INFO:tasks.workunit.client.1.smithi196.stdout:5/730: getdents d4/d1b/dc8/d4a 0 2024-03-13T14:49:51.484 INFO:tasks.workunit.client.0.smithi176.stdout:0/485: mknod d2/c94 0 2024-03-13T14:49:51.487 INFO:tasks.workunit.client.1.smithi196.stdout:6/822: rename d6/d2a/dc/d71/l92 to d6/d2a/dc/de3/l10c 0 2024-03-13T14:49:51.487 INFO:tasks.workunit.client.0.smithi176.stdout:5/525: dwrite d5/d2c/d40/d76/f95 [0,4194304] 0 2024-03-13T14:49:51.492 INFO:tasks.workunit.client.0.smithi176.stdout:4/604: mkdir d7/db9/dd8 0 2024-03-13T14:49:51.495 INFO:tasks.workunit.client.0.smithi176.stdout:7/523: rename d1/d3f/f90 to d1/d3f/d81/d9a/f9f 0 2024-03-13T14:49:51.498 INFO:tasks.workunit.client.0.smithi176.stdout:7/524: write d1/d1c/d26/d41/d2b/d40/d64/d94/f7e [2940939,107755] 0 2024-03-13T14:49:51.499 INFO:tasks.workunit.client.1.smithi196.stdout:5/731: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/fe3 x:0 0 0 2024-03-13T14:49:51.499 INFO:tasks.workunit.client.1.smithi196.stdout:6/823: unlink d6/d7/df/d27/led 0 2024-03-13T14:49:51.499 INFO:tasks.workunit.client.0.smithi176.stdout:7/525: fdatasync d1/d28/d3a/d1b/f89 0 2024-03-13T14:49:51.501 INFO:tasks.workunit.client.1.smithi196.stdout:0/698: getdents d8/da/dd8 0 2024-03-13T14:49:51.508 INFO:tasks.workunit.client.1.smithi196.stdout:7/777: dwrite d0/d9/d1e/d58/d72/da9/da3/dc2/ff5 [0,4194304] 0 2024-03-13T14:49:51.511 INFO:tasks.workunit.client.1.smithi196.stdout:6/824: fsync d6/d2a/dc/d1c/d34/fbd 0 2024-03-13T14:49:51.515 INFO:tasks.workunit.client.1.smithi196.stdout:1/735: getdents d1/d3f/d5b 0 2024-03-13T14:49:51.519 INFO:tasks.workunit.client.1.smithi196.stdout:6/825: write d6/d9/d17/d19/f2b [1218873,66379] 0 2024-03-13T14:49:51.519 INFO:tasks.workunit.client.1.smithi196.stdout:1/736: stat d1/d9/d33/d4b/d66/fee 0 2024-03-13T14:49:51.521 INFO:tasks.workunit.client.0.smithi176.stdout:9/628: dread d0/d40/f47 [0,4194304] 0 2024-03-13T14:49:51.522 INFO:tasks.workunit.client.1.smithi196.stdout:1/737: write d1/d9/d33/dd2/dde/ff2 [324042,23348] 0 2024-03-13T14:49:51.524 INFO:tasks.workunit.client.1.smithi196.stdout:7/778: truncate d0/d9/d1e/d58/d72/f7c 534479 0 2024-03-13T14:49:51.528 INFO:tasks.workunit.client.1.smithi196.stdout:0/699: creat d8/da/d25/dd7/fe3 x:0 0 0 2024-03-13T14:49:51.534 INFO:tasks.workunit.client.1.smithi196.stdout:5/732: dread d4/d1b/dc8/d14/d19/f58 [0,4194304] 0 2024-03-13T14:49:51.534 INFO:tasks.workunit.client.1.smithi196.stdout:1/738: unlink d1/d9/d4e/d55/la1 0 2024-03-13T14:49:51.537 INFO:tasks.workunit.client.1.smithi196.stdout:5/733: fsync d4/d1b/dc8/d14/d19/d26/d33/d61/fd9 0 2024-03-13T14:49:51.539 INFO:tasks.workunit.client.1.smithi196.stdout:7/779: creat d0/d9/d31/d77/dcf/de0/de7/f108 x:0 0 0 2024-03-13T14:49:51.540 INFO:tasks.workunit.client.1.smithi196.stdout:5/734: dread - d4/d1b/dc8/d3c/d6a/fd4 zero size 2024-03-13T14:49:51.541 INFO:tasks.workunit.client.1.smithi196.stdout:7/780: chown d0/d6/d35/cee 72620 1 2024-03-13T14:49:51.548 INFO:tasks.workunit.client.0.smithi176.stdout:7/526: dread d1/d28/f2a [0,4194304] 0 2024-03-13T14:49:51.551 INFO:tasks.workunit.client.0.smithi176.stdout:7/527: write d1/d1c/d26/d41/d2b/d40/f8a [942532,56981] 0 2024-03-13T14:49:51.554 INFO:tasks.workunit.client.1.smithi196.stdout:5/735: link d4/d1b/dc8/d4a/faf d4/d1b/dc8/d14/d19/da7/dc7/fe4 0 2024-03-13T14:49:51.555 INFO:tasks.workunit.client.1.smithi196.stdout:0/700: write d8/da/d25/f5a [675467,78082] 0 2024-03-13T14:49:51.559 INFO:tasks.workunit.client.1.smithi196.stdout:6/826: dwrite d6/d9/d17/d19/f22 [0,4194304] 0 2024-03-13T14:49:51.564 INFO:tasks.workunit.client.1.smithi196.stdout:0/701: fdatasync d8/da/d25/f7b 0 2024-03-13T14:49:51.571 INFO:tasks.workunit.client.1.smithi196.stdout:6/827: mkdir d6/d2a/d23/d28/d10a/d10d 0 2024-03-13T14:49:51.573 INFO:tasks.workunit.client.1.smithi196.stdout:7/781: truncate d0/d9/d31/d50/f87 1975760 0 2024-03-13T14:49:51.578 INFO:tasks.workunit.client.1.smithi196.stdout:0/702: creat d8/da/d25/d26/d35/d3b/d5f/d33/fe4 x:0 0 0 2024-03-13T14:49:51.580 INFO:tasks.workunit.client.0.smithi176.stdout:1/532: link d2/de/d10/d16/d47/f5a d2/de/d3f/d50/fa3 0 2024-03-13T14:49:51.581 INFO:tasks.workunit.client.1.smithi196.stdout:6/828: stat d6/d9/d17/da1/l5b 0 2024-03-13T14:49:51.582 INFO:tasks.workunit.client.0.smithi176.stdout:1/533: fsync d2/de/d3f/d19/f32 0 2024-03-13T14:49:51.584 INFO:tasks.workunit.client.1.smithi196.stdout:1/739: link d1/d9/d4e/d55/d63/cc9 d1/d9/d33/d4b/d88/cfc 0 2024-03-13T14:49:51.587 INFO:tasks.workunit.client.0.smithi176.stdout:3/610: creat d4/d13/d2d/fd3 x:0 0 0 2024-03-13T14:49:51.589 INFO:tasks.workunit.client.0.smithi176.stdout:3/611: chown d4/c1e 0 1 2024-03-13T14:49:51.592 INFO:tasks.workunit.client.0.smithi176.stdout:2/606: rename d1/dc/d2d/d38/f5e to d1/dc/d61/d9e/fcd 0 2024-03-13T14:49:51.598 INFO:tasks.workunit.client.0.smithi176.stdout:9/629: unlink d0/db/d89/d32/d58/d67/cd1 0 2024-03-13T14:49:51.599 INFO:tasks.workunit.client.1.smithi196.stdout:7/782: link d0/d9/d1e/c8f d0/d6/d35/c109 0 2024-03-13T14:49:51.599 INFO:tasks.workunit.client.1.smithi196.stdout:5/736: dread d4/f8 [0,4194304] 0 2024-03-13T14:49:51.607 INFO:tasks.workunit.client.1.smithi196.stdout:5/737: truncate d4/d1b/dc8/d14/d19/d26/d68/fd3 292329 0 2024-03-13T14:49:51.607 INFO:tasks.workunit.client.0.smithi176.stdout:4/605: write d7/d10/d2c/f32 [3420586,107060] 0 2024-03-13T14:49:51.609 INFO:tasks.workunit.client.0.smithi176.stdout:1/534: rmdir d2/de/d10/d16/d4a/d5f 39 2024-03-13T14:49:51.610 INFO:tasks.workunit.client.0.smithi176.stdout:0/486: dwrite d2/f3 [0,4194304] 0 2024-03-13T14:49:51.612 INFO:tasks.workunit.client.1.smithi196.stdout:7/783: rmdir d0/d9/d1e/d58/d72/da9/d47 39 2024-03-13T14:49:51.616 INFO:tasks.workunit.client.1.smithi196.stdout:0/703: dwrite d8/da/d25/d26/d35/d3b/d5f/d17/f51 [0,4194304] 0 2024-03-13T14:49:51.616 INFO:tasks.workunit.client.1.smithi196.stdout:1/740: rename d1/d28/d34/d67/lad to d1/d9/d4e/lfd 0 2024-03-13T14:49:51.621 INFO:tasks.workunit.client.1.smithi196.stdout:6/829: truncate d6/d7/df/d27/f9d 722275 0 2024-03-13T14:49:51.622 INFO:tasks.workunit.client.0.smithi176.stdout:8/504: rename d0/db/dc/f35 to d0/d24/d7b/f9e 0 2024-03-13T14:49:51.623 INFO:tasks.workunit.client.0.smithi176.stdout:8/505: fsync d0/db/dc/f6b 0 2024-03-13T14:49:51.625 INFO:tasks.workunit.client.1.smithi196.stdout:1/741: creat d1/d3f/ffe x:0 0 0 2024-03-13T14:49:51.631 INFO:tasks.workunit.client.0.smithi176.stdout:9/630: fsync d0/d40/fd2 0 2024-03-13T14:49:51.631 INFO:tasks.workunit.client.1.smithi196.stdout:0/704: rename d8/d37/f5b to d8/da/d25/d26/d48/d82/db3/dd5/fe5 0 2024-03-13T14:49:51.632 INFO:tasks.workunit.client.0.smithi176.stdout:4/606: rmdir d7/d10 39 2024-03-13T14:49:51.633 INFO:tasks.workunit.client.1.smithi196.stdout:0/705: fsync d8/da/d25/d26/d35/d3b/f7d 0 2024-03-13T14:49:51.633 INFO:tasks.workunit.client.0.smithi176.stdout:2/607: truncate d1/dc/d2d/d38/d5d/f79 781634 0 2024-03-13T14:49:51.633 INFO:tasks.workunit.client.1.smithi196.stdout:6/830: dread - d6/d7/df/f8e zero size 2024-03-13T14:49:51.635 INFO:tasks.workunit.client.1.smithi196.stdout:0/706: read - d8/da/d25/dd7/fe3 zero size 2024-03-13T14:49:51.638 INFO:tasks.workunit.client.0.smithi176.stdout:4/607: dread - d7/d8/d45/fac zero size 2024-03-13T14:49:51.638 INFO:tasks.workunit.client.0.smithi176.stdout:1/535: dwrite d2/d30/d63/d22/f82 [0,4194304] 0 2024-03-13T14:49:51.640 INFO:tasks.workunit.client.0.smithi176.stdout:1/536: fdatasync d2/d29/f70 0 2024-03-13T14:49:51.642 INFO:tasks.workunit.client.1.smithi196.stdout:0/707: dread - d8/da/d25/d26/d35/d3b/d5f/fca zero size 2024-03-13T14:49:51.643 INFO:tasks.workunit.client.1.smithi196.stdout:1/742: creat d1/d9/d4e/d55/d90/fff x:0 0 0 2024-03-13T14:49:51.645 INFO:tasks.workunit.client.1.smithi196.stdout:6/831: truncate d6/d7/df/fe6 698777 0 2024-03-13T14:49:51.647 INFO:tasks.workunit.client.0.smithi176.stdout:8/506: stat d0/db/c51 0 2024-03-13T14:49:51.648 INFO:tasks.workunit.client.1.smithi196.stdout:7/784: write d0/d6/f21 [1419740,41946] 0 2024-03-13T14:49:51.649 INFO:tasks.workunit.client.1.smithi196.stdout:1/743: write d1/d9/d4e/d55/d63/fe9 [1080882,13988] 0 2024-03-13T14:49:51.653 INFO:tasks.workunit.client.1.smithi196.stdout:6/832: write d6/d9/d17/da1/d50/f5a [4329035,83373] 0 2024-03-13T14:49:51.656 INFO:tasks.workunit.client.1.smithi196.stdout:0/708: creat d8/da/d25/d26/d48/d82/db3/dd5/fe6 x:0 0 0 2024-03-13T14:49:51.659 INFO:tasks.workunit.client.0.smithi176.stdout:2/608: rmdir d1/dc/d2d/d38 39 2024-03-13T14:49:51.660 INFO:tasks.workunit.client.1.smithi196.stdout:0/709: dread - d8/da/d25/d26/d35/d3b/d5f/fca zero size 2024-03-13T14:49:51.663 INFO:tasks.workunit.client.0.smithi176.stdout:9/631: dwrite d0/d40/f78 [0,4194304] 0 2024-03-13T14:49:51.663 INFO:tasks.workunit.client.0.smithi176.stdout:2/609: chown d1/dc/d2d/d4b/l89 355893 1 2024-03-13T14:49:51.663 INFO:tasks.workunit.client.0.smithi176.stdout:5/526: getdents d5/d75 0 2024-03-13T14:49:51.666 INFO:tasks.workunit.client.1.smithi196.stdout:7/785: creat d0/d9/d1e/d58/d72/d99/f10a x:0 0 0 2024-03-13T14:49:51.667 INFO:tasks.workunit.client.0.smithi176.stdout:7/528: truncate d1/d1c/f56 1656 0 2024-03-13T14:49:51.668 INFO:tasks.workunit.client.1.smithi196.stdout:1/744: mkdir d1/d28/d34/d100 0 2024-03-13T14:49:51.672 INFO:tasks.workunit.client.0.smithi176.stdout:2/610: truncate d1/d2a/d33/fa7 27928 0 2024-03-13T14:49:51.674 INFO:tasks.workunit.client.1.smithi196.stdout:6/833: creat d6/d2a/dc/d1c/da7/d9e/f10e x:0 0 0 2024-03-13T14:49:51.678 INFO:tasks.workunit.client.1.smithi196.stdout:0/710: creat d8/da/d25/d26/d35/d3b/d5f/d33/fe7 x:0 0 0 2024-03-13T14:49:51.679 INFO:tasks.workunit.client.0.smithi176.stdout:1/537: truncate d2/de/d3f/f6f 621611 0 2024-03-13T14:49:51.683 INFO:tasks.workunit.client.1.smithi196.stdout:7/786: creat d0/d9/d1e/d58/d72/f10b x:0 0 0 2024-03-13T14:49:51.688 INFO:tasks.workunit.client.0.smithi176.stdout:9/632: rmdir d0/db/d89/d4d 39 2024-03-13T14:49:51.689 INFO:tasks.workunit.client.1.smithi196.stdout:0/711: mkdir d8/da/d25/d26/d35/d3b/d50/d5d/d7e/de8 0 2024-03-13T14:49:51.691 INFO:tasks.workunit.client.0.smithi176.stdout:0/487: creat d2/d5/f95 x:0 0 0 2024-03-13T14:49:51.692 INFO:tasks.workunit.client.0.smithi176.stdout:7/529: fsync d1/d28/f2a 0 2024-03-13T14:49:51.695 INFO:tasks.workunit.client.0.smithi176.stdout:8/507: dread d0/db/f19 [0,4194304] 0 2024-03-13T14:49:51.695 INFO:tasks.workunit.client.1.smithi196.stdout:1/745: dwrite d1/d28/f5a [0,4194304] 0 2024-03-13T14:49:51.697 INFO:tasks.workunit.client.0.smithi176.stdout:8/508: write d0/db/d39/d56/f95 [767827,19565] 0 2024-03-13T14:49:51.698 INFO:tasks.workunit.client.1.smithi196.stdout:6/834: creat d6/d9/d17/d19/d5f/dee/f10f x:0 0 0 2024-03-13T14:49:51.699 INFO:tasks.workunit.client.1.smithi196.stdout:3/674: sync 2024-03-13T14:49:51.704 INFO:tasks.workunit.client.1.smithi196.stdout:0/712: rename d8/da/d25/d26/d35/d3b/ca2 to d8/da/d25/d26/d48/d82/db3/dd5/ce9 0 2024-03-13T14:49:51.706 INFO:tasks.workunit.client.0.smithi176.stdout:6/583: rename d0/d10/d19/d9f/f82 to d0/d10/d19/d27/fbe 0 2024-03-13T14:49:51.711 INFO:tasks.workunit.client.1.smithi196.stdout:1/746: mkdir d1/d9/d6c/d101 0 2024-03-13T14:49:51.712 INFO:tasks.workunit.client.0.smithi176.stdout:5/527: fdatasync d5/d2e/d7e/dad/f92 0 2024-03-13T14:49:51.714 INFO:tasks.workunit.client.1.smithi196.stdout:1/747: write d1/d9/d33/f42 [3344481,40213] 0 2024-03-13T14:49:51.714 INFO:tasks.workunit.client.0.smithi176.stdout:5/528: chown d5/f6 1 1 2024-03-13T14:49:51.717 INFO:tasks.workunit.client.0.smithi176.stdout:2/611: dread d1/dc/d2d/f2f [0,4194304] 0 2024-03-13T14:49:51.720 INFO:tasks.workunit.client.1.smithi196.stdout:0/713: rmdir d8/da/d25/d26/d35/d3b/d5f 39 2024-03-13T14:49:51.725 INFO:tasks.workunit.client.0.smithi176.stdout:7/530: rmdir d1/d3f 39 2024-03-13T14:49:51.821 INFO:tasks.workunit.client.1.smithi196.stdout:1/748: dread - d1/d9/d4e/dcc/ff4 zero size 2024-03-13T14:49:51.822 INFO:tasks.workunit.client.0.smithi176.stdout:8/509: fdatasync d0/d5/f17 0 2024-03-13T14:49:51.824 INFO:tasks.workunit.client.0.smithi176.stdout:8/510: chown d0/db/d39/d56/d93 1 1 2024-03-13T14:49:51.827 INFO:tasks.workunit.client.0.smithi176.stdout:1/538: mkdir d2/de/d10/d16/d4a/d5f/da4 0 2024-03-13T14:49:51.831 INFO:tasks.workunit.client.0.smithi176.stdout:1/539: chown d2/de/d10/f6d 51 1 2024-03-13T14:49:51.832 INFO:tasks.workunit.client.1.smithi196.stdout:4/640: sync 2024-03-13T14:49:51.832 INFO:tasks.workunit.client.1.smithi196.stdout:8/641: sync 2024-03-13T14:49:51.832 INFO:tasks.workunit.client.1.smithi196.stdout:9/602: sync 2024-03-13T14:49:51.839 INFO:tasks.workunit.client.1.smithi196.stdout:8/642: chown d0/d16/d3d/d8b/f4b 0 1 2024-03-13T14:49:51.843 INFO:tasks.workunit.client.1.smithi196.stdout:1/749: dwrite d1/d9/d33/d4b/d66/d7b/fb4 [0,4194304] 0 2024-03-13T14:49:51.845 INFO:tasks.workunit.client.0.smithi176.stdout:4/608: truncate d7/d8/d33/f6d 1015497 0 2024-03-13T14:49:51.848 INFO:tasks.workunit.client.1.smithi196.stdout:8/643: dread d0/d16/d3d/d9c/f9d [0,4194304] 0 2024-03-13T14:49:51.852 INFO:tasks.workunit.client.0.smithi176.stdout:8/511: creat d0/db/d39/d56/f9f x:0 0 0 2024-03-13T14:49:51.853 INFO:tasks.workunit.client.0.smithi176.stdout:8/512: chown d0/db/f3f 7380 1 2024-03-13T14:49:51.857 INFO:tasks.workunit.client.0.smithi176.stdout:2/612: dwrite d1/dc/d1c/f4c [0,4194304] 0 2024-03-13T14:49:51.859 INFO:tasks.workunit.client.1.smithi196.stdout:2/598: sync 2024-03-13T14:49:51.859 INFO:tasks.workunit.client.1.smithi196.stdout:5/738: sync 2024-03-13T14:49:51.859 INFO:tasks.workunit.client.1.smithi196.stdout:0/714: sync 2024-03-13T14:49:51.864 INFO:tasks.workunit.client.1.smithi196.stdout:2/599: write d0/d7/d9/d61/fb6 [1381719,27125] 0 2024-03-13T14:49:51.864 INFO:tasks.workunit.client.1.smithi196.stdout:7/787: dwrite d0/d6/f8b [0,4194304] 0 2024-03-13T14:49:51.870 INFO:tasks.workunit.client.0.smithi176.stdout:3/612: sync 2024-03-13T14:49:51.872 INFO:tasks.workunit.client.0.smithi176.stdout:9/633: dwrite d0/d17/f48 [4194304,4194304] 0 2024-03-13T14:49:51.876 INFO:tasks.workunit.client.1.smithi196.stdout:3/675: dwrite db/d19/d22/f82 [0,4194304] 0 2024-03-13T14:49:51.878 INFO:tasks.workunit.client.1.smithi196.stdout:2/600: dread d0/d7/f17 [0,4194304] 0 2024-03-13T14:49:51.897 INFO:tasks.workunit.client.0.smithi176.stdout:4/609: rename d7/d10/c61 to d7/d10/d2c/cd9 0 2024-03-13T14:49:51.903 INFO:tasks.workunit.client.0.smithi176.stdout:8/513: dread d0/f30 [0,4194304] 0 2024-03-13T14:49:51.904 INFO:tasks.workunit.client.0.smithi176.stdout:3/613: symlink d4/d13/d16/db6/ld4 0 2024-03-13T14:49:51.906 INFO:tasks.workunit.client.0.smithi176.stdout:3/614: dread - d4/d13/d2d/fd3 zero size 2024-03-13T14:49:51.909 INFO:tasks.workunit.client.0.smithi176.stdout:3/615: write d4/d28/d34/f6e [4054229,91678] 0 2024-03-13T14:49:51.913 INFO:tasks.workunit.client.0.smithi176.stdout:7/531: getdents d1 0 2024-03-13T14:49:51.914 INFO:tasks.workunit.client.1.smithi196.stdout:1/750: dread d1/d9/d33/f62 [0,4194304] 0 2024-03-13T14:49:51.915 INFO:tasks.workunit.client.0.smithi176.stdout:4/610: read d7/d10/f8a [129470,56401] 0 2024-03-13T14:49:51.920 INFO:tasks.workunit.client.0.smithi176.stdout:9/634: mkdir d0/db/d89/d4d/dd5 0 2024-03-13T14:49:51.924 INFO:tasks.workunit.client.0.smithi176.stdout:3/616: rmdir d4/d13/d16/d80 39 2024-03-13T14:49:51.932 INFO:tasks.workunit.client.0.smithi176.stdout:7/532: creat d1/d3f/d4b/d63/d7a/fa0 x:0 0 0 2024-03-13T14:49:51.932 INFO:tasks.workunit.client.0.smithi176.stdout:4/611: dwrite d7/dab/db5/fc6 [0,4194304] 0 2024-03-13T14:49:51.934 INFO:tasks.workunit.client.0.smithi176.stdout:9/635: chown d0/l15 115171 1 2024-03-13T14:49:51.938 INFO:tasks.workunit.client.0.smithi176.stdout:9/636: chown d0/db/d89/d32/d58/lc0 14485284 1 2024-03-13T14:49:51.941 INFO:tasks.workunit.client.0.smithi176.stdout:5/529: creat d5/d22/fb3 x:0 0 0 2024-03-13T14:49:51.942 INFO:tasks.workunit.client.1.smithi196.stdout:6/835: link d6/d2a/c1a d6/d9/d17/da1/d50/c110 0 2024-03-13T14:49:51.952 INFO:tasks.workunit.client.0.smithi176.stdout:4/612: dread - d7/d8/d45/f8d zero size 2024-03-13T14:49:51.954 INFO:tasks.workunit.client.1.smithi196.stdout:4/641: creat d1/d3/d7/d35/d76/d58/d69/d9f/fe0 x:0 0 0 2024-03-13T14:49:51.958 INFO:tasks.workunit.client.0.smithi176.stdout:7/533: dread d1/d7/f3d [0,4194304] 0 2024-03-13T14:49:51.958 INFO:tasks.workunit.client.1.smithi196.stdout:8/644: mknod d0/d16/d3d/d9c/d87/d8c/da9/db8/cdf 0 2024-03-13T14:49:51.963 INFO:tasks.workunit.client.1.smithi196.stdout:0/715: unlink c4 0 2024-03-13T14:49:51.967 INFO:tasks.workunit.client.0.smithi176.stdout:9/637: dread d0/d40/d53/fb5 [0,4194304] 0 2024-03-13T14:49:51.967 INFO:tasks.workunit.client.0.smithi176.stdout:5/530: dread d5/d2b/f70 [0,4194304] 0 2024-03-13T14:49:51.968 INFO:tasks.workunit.client.0.smithi176.stdout:5/531: chown d5/d2c/l3b 338399120 1 2024-03-13T14:49:51.976 INFO:tasks.workunit.client.1.smithi196.stdout:1/751: fsync d1/d9/d33/d4b/d66/d7b/f8d 0 2024-03-13T14:49:51.977 INFO:tasks.workunit.client.0.smithi176.stdout:7/534: dread d1/d28/d3a/f14 [0,4194304] 0 2024-03-13T14:49:51.978 INFO:tasks.workunit.client.0.smithi176.stdout:9/638: creat d0/d1e/d1f/fd6 x:0 0 0 2024-03-13T14:49:51.980 INFO:tasks.workunit.client.1.smithi196.stdout:6/836: mkdir d6/d2a/d23/d28/d111 0 2024-03-13T14:49:51.980 INFO:tasks.workunit.client.0.smithi176.stdout:7/535: dread d1/d1c/d26/d41/d2b/d40/d64/f95 [0,4194304] 0 2024-03-13T14:49:51.988 INFO:tasks.workunit.client.0.smithi176.stdout:9/639: symlink d0/d1e/d6f/ld7 0 2024-03-13T14:49:51.991 INFO:tasks.workunit.client.1.smithi196.stdout:2/601: dwrite d0/f1a [0,4194304] 0 2024-03-13T14:49:51.992 INFO:tasks.workunit.client.1.smithi196.stdout:0/716: dread d8/da/d74/fae [0,4194304] 0 2024-03-13T14:49:52.001 INFO:tasks.workunit.client.0.smithi176.stdout:7/536: symlink d1/d28/d3a/d1b/la1 0 2024-03-13T14:49:52.004 INFO:tasks.workunit.client.1.smithi196.stdout:3/676: symlink db/d19/d50/db7/da9/dc0/ldd 0 2024-03-13T14:49:52.006 INFO:tasks.workunit.client.1.smithi196.stdout:3/677: write db/d19/d22/f82 [949246,36120] 0 2024-03-13T14:49:52.007 INFO:tasks.workunit.client.0.smithi176.stdout:6/584: write d0/f6c [4083726,117285] 0 2024-03-13T14:49:52.010 INFO:tasks.workunit.client.0.smithi176.stdout:9/640: rename d0/db/d89/d32/d76 to d0/dd8 0 2024-03-13T14:49:52.012 INFO:tasks.workunit.client.0.smithi176.stdout:9/641: fdatasync d0/d1e/d1f/d68/fb4 0 2024-03-13T14:49:52.014 INFO:tasks.workunit.client.0.smithi176.stdout:9/642: chown d0/d17/f48 103592834 1 2024-03-13T14:49:52.015 INFO:tasks.workunit.client.1.smithi196.stdout:6/837: symlink d6/d57/l112 0 2024-03-13T14:49:52.015 INFO:tasks.workunit.client.0.smithi176.stdout:7/537: truncate d1/d1c/d26/d41/d2b/f96 4576963 0 2024-03-13T14:49:52.017 INFO:tasks.workunit.client.1.smithi196.stdout:4/642: mknod d1/d3/d7/d35/d76/d58/dc9/dd7/ce1 0 2024-03-13T14:49:52.021 INFO:tasks.workunit.client.0.smithi176.stdout:0/488: creat d2/d5/f96 x:0 0 0 2024-03-13T14:49:52.024 INFO:tasks.workunit.client.1.smithi196.stdout:5/739: link d4/d1b/dc8/d14/d19/d26/d33/d61/l7c d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/le5 0 2024-03-13T14:49:52.026 INFO:tasks.workunit.client.0.smithi176.stdout:7/538: mknod d1/d1c/d26/d41/d2b/d40/d64/ca2 0 2024-03-13T14:49:52.027 INFO:tasks.workunit.client.0.smithi176.stdout:9/643: dread d0/d40/d53/fb5 [0,4194304] 0 2024-03-13T14:49:52.030 INFO:tasks.workunit.client.1.smithi196.stdout:0/717: creat d8/da/d25/d26/d35/d3b/d59/fea x:0 0 0 2024-03-13T14:49:52.035 INFO:tasks.workunit.client.1.smithi196.stdout:3/678: mknod db/d19/d22/d3a/d6a/cde 0 2024-03-13T14:49:52.040 INFO:tasks.workunit.client.0.smithi176.stdout:7/539: symlink d1/d3f/d81/d9a/la3 0 2024-03-13T14:49:52.041 INFO:tasks.workunit.client.0.smithi176.stdout:7/540: dread - d1/d28/d3a/d1b/f89 zero size 2024-03-13T14:49:52.046 INFO:tasks.workunit.client.1.smithi196.stdout:5/740: creat d4/d1b/dc8/d14/d19/d26/fe6 x:0 0 0 2024-03-13T14:49:52.049 INFO:tasks.workunit.client.0.smithi176.stdout:7/541: symlink d1/d28/d3a/d1b/d8f/la4 0 2024-03-13T14:49:52.049 INFO:tasks.workunit.client.1.smithi196.stdout:5/741: stat d4/d1b/dc8/d3c/d6a/d91/fa6 0 2024-03-13T14:49:52.049 INFO:tasks.workunit.client.1.smithi196.stdout:2/602: mknod d0/d7/d9/d5a/dbc/cc6 0 2024-03-13T14:49:52.050 INFO:tasks.workunit.client.1.smithi196.stdout:0/718: symlink d8/da/d25/d26/d35/d3b/d6c/leb 0 2024-03-13T14:49:52.051 INFO:tasks.workunit.client.0.smithi176.stdout:7/542: fdatasync d1/d3f/d4b/d63/d7a/fa0 0 2024-03-13T14:49:52.052 INFO:tasks.workunit.client.1.smithi196.stdout:0/719: chown d8/c61 3863 1 2024-03-13T14:49:52.053 INFO:tasks.workunit.client.0.smithi176.stdout:7/543: dread - d1/d28/d3a/d1b/f89 zero size 2024-03-13T14:49:52.055 INFO:tasks.workunit.client.1.smithi196.stdout:5/742: fdatasync d4/d1b/dc8/d14/d19/f96 0 2024-03-13T14:49:52.059 INFO:tasks.workunit.client.0.smithi176.stdout:7/544: mkdir d1/d3f/d76/da5 0 2024-03-13T14:49:52.061 INFO:tasks.workunit.client.1.smithi196.stdout:3/679: chown db/d19/d22/d3a/d53/c9f 108 1 2024-03-13T14:49:52.062 INFO:tasks.workunit.client.1.smithi196.stdout:3/680: dread - db/d4c/fce zero size 2024-03-13T14:49:52.062 INFO:tasks.workunit.client.1.smithi196.stdout:4/643: creat d1/d3/d7/ddf/fe2 x:0 0 0 2024-03-13T14:49:52.063 INFO:tasks.workunit.client.1.smithi196.stdout:0/720: readlink d8/da/d25/d26/d35/d3b/d50/l56 0 2024-03-13T14:49:52.065 INFO:tasks.workunit.client.1.smithi196.stdout:4/644: write d1/d3/d7/d35/d76/f94 [1719608,61352] 0 2024-03-13T14:49:52.066 INFO:tasks.workunit.client.1.smithi196.stdout:3/681: truncate db/d19/d4a/fb2 593978 0 2024-03-13T14:49:52.066 INFO:tasks.workunit.client.0.smithi176.stdout:2/613: write d1/dc/d2d/d59/d8c/f8f [982474,100566] 0 2024-03-13T14:49:52.068 INFO:tasks.workunit.client.1.smithi196.stdout:5/743: unlink d4/d1b/dc8/d14/d37/f4d 0 2024-03-13T14:49:52.070 INFO:tasks.workunit.client.1.smithi196.stdout:0/721: dread d8/da/d25/fcd [0,4194304] 0 2024-03-13T14:49:52.070 INFO:tasks.workunit.client.1.smithi196.stdout:9/603: rename d7/d24/d28/fbd to d7/de/fc9 0 2024-03-13T14:49:52.072 INFO:tasks.workunit.client.0.smithi176.stdout:7/545: creat d1/d28/d3a/d1b/fa6 x:0 0 0 2024-03-13T14:49:52.078 INFO:tasks.workunit.client.0.smithi176.stdout:2/614: mknod d1/dc/d2d/d59/d80/cce 0 2024-03-13T14:49:52.083 INFO:tasks.workunit.client.1.smithi196.stdout:0/722: fsync d8/da/d25/d26/d35/d3b/d5f/d33/fe4 0 2024-03-13T14:49:52.087 INFO:tasks.workunit.client.1.smithi196.stdout:4/645: dwrite d1/d3/d7/d35/d3e/d56/f9c [0,4194304] 0 2024-03-13T14:49:52.099 INFO:tasks.workunit.client.0.smithi176.stdout:3/617: write d4/f1d [1191170,21987] 0 2024-03-13T14:49:52.101 INFO:tasks.workunit.client.1.smithi196.stdout:9/604: dread d7/de/d4b/d1d/f23 [0,4194304] 0 2024-03-13T14:49:52.101 INFO:tasks.workunit.client.0.smithi176.stdout:3/618: chown d4/d13/d16/db6/fc4 114121525 1 2024-03-13T14:49:52.102 INFO:tasks.workunit.client.0.smithi176.stdout:8/514: dwrite d0/d5/d11/f68 [0,4194304] 0 2024-03-13T14:49:52.104 INFO:tasks.workunit.client.0.smithi176.stdout:3/619: chown d4/d28/d31/f75 98233 1 2024-03-13T14:49:52.105 INFO:tasks.workunit.client.0.smithi176.stdout:3/620: fsync d4/d13/f73 0 2024-03-13T14:49:52.109 INFO:tasks.workunit.client.1.smithi196.stdout:4/646: dwrite d1/d3/d7/ddf/fe2 [0,4194304] 0 2024-03-13T14:49:52.111 INFO:tasks.workunit.client.1.smithi196.stdout:0/723: link d8/da/d25/d26/d35/d3b/d50/d5d/da5/lc4 d8/da/d25/d26/d48/d82/db3/lec 0 2024-03-13T14:49:52.115 INFO:tasks.workunit.client.1.smithi196.stdout:4/647: mkdir d1/d3/d7/d35/d76/d58/d69/de3 0 2024-03-13T14:49:52.117 INFO:tasks.workunit.client.0.smithi176.stdout:8/515: mknod d0/d5/d11/d47/ca0 0 2024-03-13T14:49:52.120 INFO:tasks.workunit.client.1.smithi196.stdout:0/724: dread d8/da/d25/d26/d35/d3b/d6c/fc6 [0,4194304] 0 2024-03-13T14:49:52.127 INFO:tasks.workunit.client.1.smithi196.stdout:0/725: chown d8/da/d25/d26/d35/d3b/d5f/le2 616207 1 2024-03-13T14:49:52.128 INFO:tasks.workunit.client.0.smithi176.stdout:4/613: dwrite d7/d10/f44 [0,4194304] 0 2024-03-13T14:49:52.128 INFO:tasks.workunit.client.1.smithi196.stdout:8/645: write d0/d16/d3d/d8b/f38 [3808,98139] 0 2024-03-13T14:49:52.129 INFO:tasks.workunit.client.0.smithi176.stdout:4/614: dread - d7/d8/fbd zero size 2024-03-13T14:49:52.131 INFO:tasks.workunit.client.0.smithi176.stdout:3/621: link d4/c33 d4/d28/d31/cd5 0 2024-03-13T14:49:52.131 INFO:tasks.workunit.client.0.smithi176.stdout:9/644: creat d0/db/fd9 x:0 0 0 2024-03-13T14:49:52.134 INFO:tasks.workunit.client.0.smithi176.stdout:9/645: dread - d0/d1e/d6f/fac zero size 2024-03-13T14:49:52.134 INFO:tasks.workunit.client.1.smithi196.stdout:8/646: fsync d0/d10/d12/d1e/f34 0 2024-03-13T14:49:52.137 INFO:tasks.workunit.client.1.smithi196.stdout:9/605: dread d7/f45 [0,4194304] 0 2024-03-13T14:49:52.138 INFO:tasks.workunit.client.0.smithi176.stdout:3/622: chown d4/d28/d4f/l74 1084 1 2024-03-13T14:49:52.139 INFO:tasks.workunit.client.1.smithi196.stdout:7/788: truncate d0/d6/f21 3732058 0 2024-03-13T14:49:52.141 INFO:tasks.workunit.client.0.smithi176.stdout:8/516: dread d0/d5/d3b/f41 [0,4194304] 0 2024-03-13T14:49:52.142 INFO:tasks.workunit.client.0.smithi176.stdout:3/623: fsync d4/d28/d31/f62 0 2024-03-13T14:49:52.143 INFO:tasks.workunit.client.1.smithi196.stdout:4/648: link d1/d3/fa d1/d3/d7/d35/d3e/fe4 0 2024-03-13T14:49:52.143 INFO:tasks.workunit.client.0.smithi176.stdout:9/646: chown d0/d40/d53/caa 1796 1 2024-03-13T14:49:52.145 INFO:tasks.workunit.client.0.smithi176.stdout:5/532: dwrite d5/d2b/f5b [0,4194304] 0 2024-03-13T14:49:52.153 INFO:tasks.workunit.client.0.smithi176.stdout:8/517: creat d0/db/d53/fa1 x:0 0 0 2024-03-13T14:49:52.156 INFO:tasks.workunit.client.1.smithi196.stdout:1/752: dwrite d1/d9/d33/d4b/f1a [0,4194304] 0 2024-03-13T14:49:52.158 INFO:tasks.workunit.client.0.smithi176.stdout:6/585: dwrite d0/d10/d19/d9f/f74 [0,4194304] 0 2024-03-13T14:49:52.163 INFO:tasks.workunit.client.0.smithi176.stdout:4/615: unlink d7/d8/d33/l41 0 2024-03-13T14:49:52.166 INFO:tasks.workunit.client.1.smithi196.stdout:0/726: link d8/da/d25/d26/d48/d82/da0/cdb d8/da/d25/d26/d35/d3b/d50/d55/ced 0 2024-03-13T14:49:52.169 INFO:tasks.workunit.client.1.smithi196.stdout:0/727: write d8/da/d25/d26/d35/d3b/d5f/fca [396030,111290] 0 2024-03-13T14:49:52.171 INFO:tasks.workunit.client.0.smithi176.stdout:6/586: fdatasync d0/d9/f6f 0 2024-03-13T14:49:52.172 INFO:tasks.workunit.client.1.smithi196.stdout:4/649: mknod d1/d3/d7/d35/d76/ce5 0 2024-03-13T14:49:52.176 INFO:tasks.workunit.client.1.smithi196.stdout:9/606: creat d7/de/d4b/d1d/fca x:0 0 0 2024-03-13T14:49:52.180 INFO:tasks.workunit.client.0.smithi176.stdout:0/489: dwrite d2/d5/d28/f5e [0,4194304] 0 2024-03-13T14:49:52.183 INFO:tasks.workunit.client.1.smithi196.stdout:1/753: mknod d1/d94/c102 0 2024-03-13T14:49:52.184 INFO:tasks.workunit.client.1.smithi196.stdout:6/838: write d6/d2a/dc/d1c/f25 [827069,79416] 0 2024-03-13T14:49:52.184 INFO:tasks.workunit.client.1.smithi196.stdout:4/650: rmdir d1/d3/d4c/d72/d7d/d80 39 2024-03-13T14:49:52.188 INFO:tasks.workunit.client.0.smithi176.stdout:4/616: creat d7/d10/d46/d4a/d5d/fda x:0 0 0 2024-03-13T14:49:52.192 INFO:tasks.workunit.client.0.smithi176.stdout:4/617: dread d7/d8/d33/fd2 [0,4194304] 0 2024-03-13T14:49:52.193 INFO:tasks.workunit.client.0.smithi176.stdout:4/618: readlink d7/d8/l51 0 2024-03-13T14:49:52.193 INFO:tasks.workunit.client.0.smithi176.stdout:3/624: getdents d4/d28/d34 0 2024-03-13T14:49:52.194 INFO:tasks.workunit.client.1.smithi196.stdout:7/789: link d0/d9/d1e/c100 d0/d4d/c10c 0 2024-03-13T14:49:52.194 INFO:tasks.workunit.client.0.smithi176.stdout:4/619: fsync d7/d8/f28 0 2024-03-13T14:49:52.195 INFO:tasks.workunit.client.1.smithi196.stdout:2/603: write d0/d7/d9/d42/f88 [512528,17314] 0 2024-03-13T14:49:52.195 INFO:tasks.workunit.client.0.smithi176.stdout:3/625: chown d4/d28/c58 7169182 1 2024-03-13T14:49:52.196 INFO:tasks.workunit.client.1.smithi196.stdout:6/839: mkdir d6/d57/d113 0 2024-03-13T14:49:52.197 INFO:tasks.workunit.client.1.smithi196.stdout:7/790: stat d0/d9/d1e/lb3 0 2024-03-13T14:49:52.198 INFO:tasks.workunit.client.0.smithi176.stdout:9/647: creat d0/db/d89/fda x:0 0 0 2024-03-13T14:49:52.206 INFO:tasks.workunit.client.1.smithi196.stdout:4/651: dwrite d1/d3/d4c/fa5 [0,4194304] 0 2024-03-13T14:49:52.207 INFO:tasks.workunit.client.0.smithi176.stdout:9/648: dread d0/d17/f48 [4194304,4194304] 0 2024-03-13T14:49:52.208 INFO:tasks.workunit.client.0.smithi176.stdout:9/649: write d0/db/fd9 [130692,62104] 0 2024-03-13T14:49:52.209 INFO:tasks.workunit.client.1.smithi196.stdout:3/682: write db/f13 [2480609,126777] 0 2024-03-13T14:49:52.219 INFO:tasks.workunit.client.1.smithi196.stdout:6/840: mkdir d6/d2a/dc/de3/d114 0 2024-03-13T14:49:52.219 INFO:tasks.workunit.client.0.smithi176.stdout:6/587: truncate d0/f8 3857503 0 2024-03-13T14:49:52.222 INFO:tasks.workunit.client.1.smithi196.stdout:4/652: creat d1/d3/dbc/fe6 x:0 0 0 2024-03-13T14:49:52.224 INFO:tasks.workunit.client.0.smithi176.stdout:0/490: mkdir d2/d5/d1e/d97 0 2024-03-13T14:49:52.228 INFO:tasks.workunit.client.1.smithi196.stdout:5/744: dwrite d4/d1b/dc8/d14/d19/d26/dab/fb7 [0,4194304] 0 2024-03-13T14:49:52.229 INFO:tasks.workunit.client.1.smithi196.stdout:2/604: dwrite d0/f77 [0,4194304] 0 2024-03-13T14:49:52.229 INFO:tasks.workunit.client.0.smithi176.stdout:2/615: dwrite d1/f7f [0,4194304] 0 2024-03-13T14:49:52.230 INFO:tasks.workunit.client.1.smithi196.stdout:0/728: link d8/da/d74/cce d8/da/d25/d26/d35/d3b/d50/d5d/d7e/de8/cee 0 2024-03-13T14:49:52.240 INFO:tasks.workunit.client.1.smithi196.stdout:7/791: mknod d0/d9/d31/d77/dcf/de0/d104/c10d 0 2024-03-13T14:49:52.241 INFO:tasks.workunit.client.0.smithi176.stdout:6/588: dwrite d0/d10/d19/d27/fa9 [0,4194304] 0 2024-03-13T14:49:52.244 INFO:tasks.workunit.client.0.smithi176.stdout:6/589: chown d0/de/d38/d88 1922 1 2024-03-13T14:49:52.244 INFO:tasks.workunit.client.0.smithi176.stdout:4/620: dwrite d7/d10/d46/d4a/d5d/f78 [0,4194304] 0 2024-03-13T14:49:52.251 INFO:tasks.workunit.client.0.smithi176.stdout:4/621: write d7/d8/d45/f76 [176689,73665] 0 2024-03-13T14:49:52.251 INFO:tasks.workunit.client.0.smithi176.stdout:1/540: write d2/de/d3f/f6f [148580,91498] 0 2024-03-13T14:49:52.251 INFO:tasks.workunit.client.0.smithi176.stdout:6/590: write d0/de/d38/f80 [173383,99284] 0 2024-03-13T14:49:52.256 INFO:tasks.workunit.client.1.smithi196.stdout:8/647: truncate d0/d1c/d29/d47/f80 1261141 0 2024-03-13T14:49:52.265 INFO:tasks.workunit.client.0.smithi176.stdout:8/518: truncate d0/d5/d11/f1b 3243564 0 2024-03-13T14:49:52.266 INFO:tasks.workunit.client.1.smithi196.stdout:2/605: dwrite d0/d7/d9/d42/f88 [0,4194304] 0 2024-03-13T14:49:52.267 INFO:tasks.workunit.client.0.smithi176.stdout:7/546: sync 2024-03-13T14:49:52.268 INFO:tasks.workunit.client.0.smithi176.stdout:9/650: getdents d0/d73 0 2024-03-13T14:49:52.268 INFO:tasks.workunit.client.0.smithi176.stdout:5/533: dwrite d5/d22/d15/f2d [0,4194304] 0 2024-03-13T14:49:52.269 INFO:tasks.workunit.client.1.smithi196.stdout:2/606: dread - d0/d6/d4b/f8a zero size 2024-03-13T14:49:52.272 INFO:tasks.workunit.client.1.smithi196.stdout:2/607: chown d0/d6/fb3 114083573 1 2024-03-13T14:49:52.273 INFO:tasks.workunit.client.0.smithi176.stdout:1/541: creat d2/d30/d63/d35/fa5 x:0 0 0 2024-03-13T14:49:52.274 INFO:tasks.workunit.client.0.smithi176.stdout:0/491: rename d2/d74/c6a to d2/db/c98 0 2024-03-13T14:49:52.275 INFO:tasks.workunit.client.1.smithi196.stdout:9/607: write d7/de/d4b/f14 [1717089,54684] 0 2024-03-13T14:49:52.277 INFO:tasks.workunit.client.1.smithi196.stdout:9/608: chown d7/d9/d17 1288186 1 2024-03-13T14:49:52.280 INFO:tasks.workunit.client.1.smithi196.stdout:9/609: chown d7/de/d4b/f97 813786 1 2024-03-13T14:49:52.285 INFO:tasks.workunit.client.0.smithi176.stdout:9/651: stat d0/d17/l18 0 2024-03-13T14:49:52.289 INFO:tasks.workunit.client.0.smithi176.stdout:5/534: dwrite d5/d2c/f3d [0,4194304] 0 2024-03-13T14:49:52.289 INFO:tasks.workunit.client.0.smithi176.stdout:2/616: rename d1/dc/d40 to d1/dc/d2d/d90/dcf 0 2024-03-13T14:49:52.293 INFO:tasks.workunit.client.1.smithi196.stdout:1/754: dwrite d1/d9/d33/d4b/d66/d7b/f7f [0,4194304] 0 2024-03-13T14:49:52.303 INFO:tasks.workunit.client.0.smithi176.stdout:9/652: unlink d0/db/d89/l35 0 2024-03-13T14:49:52.304 INFO:tasks.workunit.client.0.smithi176.stdout:5/535: creat d5/d31/d86/fb4 x:0 0 0 2024-03-13T14:49:52.307 INFO:tasks.workunit.client.1.smithi196.stdout:2/608: dwrite d0/d6/d4b/d5f/fb7 [0,4194304] 0 2024-03-13T14:49:52.313 INFO:tasks.workunit.client.1.smithi196.stdout:2/609: stat d0/d50/fa2 0 2024-03-13T14:49:52.313 INFO:tasks.workunit.client.1.smithi196.stdout:6/841: write d6/d2a/dc/d47/f55 [822494,44184] 0 2024-03-13T14:49:52.315 INFO:tasks.workunit.client.1.smithi196.stdout:6/842: fsync d6/d6f/fec 0 2024-03-13T14:49:52.319 INFO:tasks.workunit.client.1.smithi196.stdout:9/610: dwrite d7/de/d4b/d1d/d22/f88 [0,4194304] 0 2024-03-13T14:49:52.321 INFO:tasks.workunit.client.1.smithi196.stdout:3/683: write db/d19/d22/d3a/d53/f59 [1150310,62116] 0 2024-03-13T14:49:52.326 INFO:tasks.workunit.client.1.smithi196.stdout:3/684: write db/d19/d50/db7/f69 [1963661,24502] 0 2024-03-13T14:49:52.326 INFO:tasks.workunit.client.1.smithi196.stdout:4/653: write d1/d3/fa [468307,11962] 0 2024-03-13T14:49:52.328 INFO:tasks.workunit.client.0.smithi176.stdout:0/492: truncate d2/d5/d1e/f52 1356935 0 2024-03-13T14:49:52.335 INFO:tasks.workunit.client.1.smithi196.stdout:5/745: truncate d4/d1b/dc8/d14/d19/d26/d68/f77 966557 0 2024-03-13T14:49:52.340 INFO:tasks.workunit.client.0.smithi176.stdout:4/622: getdents d7/dab 0 2024-03-13T14:49:52.342 INFO:tasks.workunit.client.0.smithi176.stdout:7/547: getdents d1/d3f/d81 0 2024-03-13T14:49:52.343 INFO:tasks.workunit.client.0.smithi176.stdout:7/548: chown d1/d1c/d26/d41/d2b/d40 1417822 1 2024-03-13T14:49:52.346 INFO:tasks.workunit.client.1.smithi196.stdout:1/755: unlink d1/d3f/d5b/d74/l59 0 2024-03-13T14:49:52.347 INFO:tasks.workunit.client.0.smithi176.stdout:9/653: creat d0/d17/fdb x:0 0 0 2024-03-13T14:49:52.348 INFO:tasks.workunit.client.1.smithi196.stdout:9/611: creat d7/d9/d83/fcb x:0 0 0 2024-03-13T14:49:52.349 INFO:tasks.workunit.client.0.smithi176.stdout:5/536: mknod d5/d31/d47/d80/cb5 0 2024-03-13T14:49:52.356 INFO:tasks.workunit.client.1.smithi196.stdout:3/685: dread - db/fbc zero size 2024-03-13T14:49:52.359 INFO:tasks.workunit.client.1.smithi196.stdout:2/610: mkdir d0/d7/d9/d61/dc7 0 2024-03-13T14:49:52.361 INFO:tasks.workunit.client.0.smithi176.stdout:3/626: sync 2024-03-13T14:49:52.362 INFO:tasks.workunit.client.0.smithi176.stdout:5/537: dread d5/d2c/d40/d6a/f6e [0,4194304] 0 2024-03-13T14:49:52.364 INFO:tasks.workunit.client.1.smithi196.stdout:5/746: mknod d4/d1b/dc8/d14/d19/d26/d95/ce7 0 2024-03-13T14:49:52.364 INFO:tasks.workunit.client.0.smithi176.stdout:9/654: dwrite d0/db/d89/d32/d58/d5f/d84/d9a/f9b [0,4194304] 0 2024-03-13T14:49:52.365 INFO:tasks.workunit.client.0.smithi176.stdout:4/623: creat d7/d8/d45/fdb x:0 0 0 2024-03-13T14:49:52.365 INFO:tasks.workunit.client.0.smithi176.stdout:1/542: truncate d2/de/d10/f6d 249119 0 2024-03-13T14:49:52.366 INFO:tasks.workunit.client.1.smithi196.stdout:1/756: dwrite d1/d9/d4e/f70 [0,4194304] 0 2024-03-13T14:49:52.370 INFO:tasks.workunit.client.0.smithi176.stdout:4/624: dread - d7/d10/d46/d4a/d5d/fc9 zero size 2024-03-13T14:49:52.370 INFO:tasks.workunit.client.1.smithi196.stdout:9/612: mkdir d7/de/d4b/d1d/d35/dcc 0 2024-03-13T14:49:52.376 INFO:tasks.workunit.client.0.smithi176.stdout:8/519: dwrite d0/d5/d11/f1b [0,4194304] 0 2024-03-13T14:49:52.376 INFO:tasks.workunit.client.1.smithi196.stdout:4/654: mknod d1/d3/d7/d35/d76/d58/d69/de3/ce7 0 2024-03-13T14:49:52.384 INFO:tasks.workunit.client.1.smithi196.stdout:3/686: symlink db/d15/da1/ldf 0 2024-03-13T14:49:52.384 INFO:tasks.workunit.client.0.smithi176.stdout:3/627: creat d4/d28/d34/d38/fd6 x:0 0 0 2024-03-13T14:49:52.386 INFO:tasks.workunit.client.1.smithi196.stdout:2/611: mkdir d0/d7/d56/dc8 0 2024-03-13T14:49:52.387 INFO:tasks.workunit.client.0.smithi176.stdout:2/617: write d1/dc/d2d/d38/f5a [3273641,96897] 0 2024-03-13T14:49:52.389 INFO:tasks.workunit.client.0.smithi176.stdout:5/538: creat d5/d31/fb6 x:0 0 0 2024-03-13T14:49:52.389 INFO:tasks.workunit.client.0.smithi176.stdout:4/625: dread d7/d10/fcb [0,4194304] 0 2024-03-13T14:49:52.390 INFO:tasks.workunit.client.1.smithi196.stdout:6/843: dwrite d6/d57/fb0 [0,4194304] 0 2024-03-13T14:49:52.390 INFO:tasks.workunit.client.0.smithi176.stdout:4/626: stat d7/db9 0 2024-03-13T14:49:52.393 INFO:tasks.workunit.client.1.smithi196.stdout:5/747: creat d4/d1b/dc8/d3c/d6a/d93/fe8 x:0 0 0 2024-03-13T14:49:52.394 INFO:tasks.workunit.client.1.smithi196.stdout:6/844: dread - d6/d7/dd/ffd zero size 2024-03-13T14:49:52.394 INFO:tasks.workunit.client.0.smithi176.stdout:4/627: truncate d7/d10/d46/d4a/d5d/fc9 838035 0 2024-03-13T14:49:52.397 INFO:tasks.workunit.client.1.smithi196.stdout:1/757: symlink d1/d28/da3/l103 0 2024-03-13T14:49:52.399 INFO:tasks.workunit.client.0.smithi176.stdout:9/655: mknod d0/db/d89/d32/d58/d5f/d84/cdc 0 2024-03-13T14:49:52.400 INFO:tasks.workunit.client.1.smithi196.stdout:9/613: mkdir d7/de/d4b/d1d/d22/dcd 0 2024-03-13T14:49:52.404 INFO:tasks.workunit.client.1.smithi196.stdout:4/655: fsync d1/d3/d7/f1d 0 2024-03-13T14:49:52.404 INFO:tasks.workunit.client.1.smithi196.stdout:9/614: fdatasync d7/de/d4b/d1d/fca 0 2024-03-13T14:49:52.404 INFO:tasks.workunit.client.1.smithi196.stdout:3/687: rmdir db/d37/d64 39 2024-03-13T14:49:52.406 INFO:tasks.workunit.client.0.smithi176.stdout:0/493: creat d2/f99 x:0 0 0 2024-03-13T14:49:52.408 INFO:tasks.workunit.client.1.smithi196.stdout:9/615: read d7/de/d4b/d1d/f23 [128712,100743] 0 2024-03-13T14:49:52.409 INFO:tasks.workunit.client.1.smithi196.stdout:2/612: unlink d0/d7/d66/f76 0 2024-03-13T14:49:52.412 INFO:tasks.workunit.client.1.smithi196.stdout:2/613: write d0/d6/d1d/f7f [2829510,82594] 0 2024-03-13T14:49:52.413 INFO:tasks.workunit.client.0.smithi176.stdout:5/539: mknod d5/d2e/cb7 0 2024-03-13T14:49:52.416 INFO:tasks.workunit.client.0.smithi176.stdout:4/628: dwrite d7/d8/f28 [0,4194304] 0 2024-03-13T14:49:52.417 INFO:tasks.workunit.client.1.smithi196.stdout:2/614: truncate d0/d7/d9/d42/d98/fa8 995787 0 2024-03-13T14:49:52.420 INFO:tasks.workunit.client.1.smithi196.stdout:5/748: rename d4/d1b/dc8/d14/d19/d26/d95/ce7 to d4/d1b/dc8/d14/d19/d26/d33/dde/ce9 0 2024-03-13T14:49:52.422 INFO:tasks.workunit.client.1.smithi196.stdout:5/749: chown d4/d1b/dc8/d14/d19/d1c/f3f 6762411 1 2024-03-13T14:49:52.430 INFO:tasks.workunit.client.0.smithi176.stdout:1/543: rename d2/de/d3f/c6c to d2/de/d10/d16/d4a/d5f/da4/ca6 0 2024-03-13T14:49:52.431 INFO:tasks.workunit.client.1.smithi196.stdout:4/656: mknod d1/d1e/d43/ce8 0 2024-03-13T14:49:52.433 INFO:tasks.workunit.client.0.smithi176.stdout:1/544: chown d2/de/d10/d16/d47/c8b 22711947 1 2024-03-13T14:49:52.436 INFO:tasks.workunit.client.1.smithi196.stdout:3/688: truncate db/d19/d22/f3c 3167790 0 2024-03-13T14:49:52.440 INFO:tasks.workunit.client.0.smithi176.stdout:7/549: getdents d1/d7 0 2024-03-13T14:49:52.440 INFO:tasks.workunit.client.0.smithi176.stdout:9/656: dread d0/db/d89/d7a/d8b/f8f [0,4194304] 0 2024-03-13T14:49:52.441 INFO:tasks.workunit.client.1.smithi196.stdout:6/845: fsync d6/d57/f63 0 2024-03-13T14:49:52.446 INFO:tasks.workunit.client.0.smithi176.stdout:5/540: dread d5/d31/f3e [0,4194304] 0 2024-03-13T14:49:52.447 INFO:tasks.workunit.client.0.smithi176.stdout:8/520: link d0/d5/c90 d0/db/d39/d56/d91/ca2 0 2024-03-13T14:49:52.450 INFO:tasks.workunit.client.0.smithi176.stdout:1/545: dwrite d2/d29/f39 [8388608,4194304] 0 2024-03-13T14:49:52.452 INFO:tasks.workunit.client.1.smithi196.stdout:4/657: creat d1/d3/d7/d35/d76/d58/d69/d9f/fe9 x:0 0 0 2024-03-13T14:49:52.453 INFO:tasks.workunit.client.0.smithi176.stdout:1/546: fsync d2/de/d10/d16/d40/f77 0 2024-03-13T14:49:52.454 INFO:tasks.workunit.client.0.smithi176.stdout:1/547: chown d2/de/d3f/d19 18395 1 2024-03-13T14:49:52.455 INFO:tasks.workunit.client.0.smithi176.stdout:1/548: fdatasync d2/de/d10/d16/d47/f54 0 2024-03-13T14:49:52.459 INFO:tasks.workunit.client.1.smithi196.stdout:9/616: symlink d7/d24/lce 0 2024-03-13T14:49:52.461 INFO:tasks.workunit.client.1.smithi196.stdout:6/846: symlink d6/d7/df/d27/d39/d44/d74/l115 0 2024-03-13T14:49:52.462 INFO:tasks.workunit.client.0.smithi176.stdout:7/550: truncate d1/d3f/d81/f8e 3286105 0 2024-03-13T14:49:52.465 INFO:tasks.workunit.client.1.smithi196.stdout:6/847: dread - d6/d2a/dc/d1c/da7/d9e/f10e zero size 2024-03-13T14:49:52.466 INFO:tasks.workunit.client.0.smithi176.stdout:4/629: dread d7/d8/d33/f7f [0,4194304] 0 2024-03-13T14:49:52.469 INFO:tasks.workunit.client.0.smithi176.stdout:6/591: sync 2024-03-13T14:49:52.470 INFO:tasks.workunit.client.0.smithi176.stdout:4/630: write d7/d8/d15/d23/fa8 [903061,95419] 0 2024-03-13T14:49:52.470 INFO:tasks.workunit.client.1.smithi196.stdout:3/689: fdatasync db/d37/d64/f74 0 2024-03-13T14:49:52.472 INFO:tasks.workunit.client.0.smithi176.stdout:5/541: dread d5/f9a [0,4194304] 0 2024-03-13T14:49:52.474 INFO:tasks.workunit.client.1.smithi196.stdout:1/758: getdents d1/d3f/d5b/d74/db5 0 2024-03-13T14:49:52.475 INFO:tasks.workunit.client.0.smithi176.stdout:9/657: chown d0/d1e/d6f 85511335 1 2024-03-13T14:49:52.476 INFO:tasks.workunit.client.0.smithi176.stdout:7/551: creat d1/d1c/d4a/fa7 x:0 0 0 2024-03-13T14:49:52.477 INFO:tasks.workunit.client.0.smithi176.stdout:0/494: dwrite d2/db/f2c [0,4194304] 0 2024-03-13T14:49:52.478 INFO:tasks.workunit.client.1.smithi196.stdout:9/617: read d7/de/d4b/f1a [1024013,9534] 0 2024-03-13T14:49:52.478 INFO:tasks.workunit.client.0.smithi176.stdout:7/552: stat d1/d1c/d26/d41/d2b/d40/f8a 0 2024-03-13T14:49:52.481 INFO:tasks.workunit.client.1.smithi196.stdout:5/750: dwrite d4/f8 [8388608,4194304] 0 2024-03-13T14:49:52.482 INFO:tasks.workunit.client.0.smithi176.stdout:3/628: rename d4/d28/d31/d45/d82/d87/f8d to d4/d13/d2d/fd7 0 2024-03-13T14:49:52.482 INFO:tasks.workunit.client.0.smithi176.stdout:8/521: symlink d0/d5/d11/la3 0 2024-03-13T14:49:52.483 INFO:tasks.workunit.client.0.smithi176.stdout:3/629: dread - d4/d13/d81/fb5 zero size 2024-03-13T14:49:52.487 INFO:tasks.workunit.client.0.smithi176.stdout:8/522: dread - d0/db/d39/d56/f9f zero size 2024-03-13T14:49:52.489 INFO:tasks.workunit.client.1.smithi196.stdout:2/615: write d0/d7/d40/f60 [285772,60262] 0 2024-03-13T14:49:52.490 INFO:tasks.workunit.client.0.smithi176.stdout:3/630: readlink d4/d28/d34/da4/la8 0 2024-03-13T14:49:52.493 INFO:tasks.workunit.client.0.smithi176.stdout:4/631: creat d7/d8/d15/dd6/fdc x:0 0 0 2024-03-13T14:49:52.494 INFO:tasks.workunit.client.1.smithi196.stdout:2/616: chown d0/d7/d8e 90 1 2024-03-13T14:49:52.495 INFO:tasks.workunit.client.0.smithi176.stdout:4/632: dread - d7/d8/d15/fbc zero size 2024-03-13T14:49:52.495 INFO:tasks.workunit.client.1.smithi196.stdout:6/848: dwrite d6/d6f/ff5 [0,4194304] 0 2024-03-13T14:49:52.496 INFO:tasks.workunit.client.1.smithi196.stdout:2/617: write d0/d7/f89 [371434,88803] 0 2024-03-13T14:49:52.499 INFO:tasks.workunit.client.0.smithi176.stdout:4/633: dread - d7/d8/d15/f7b zero size 2024-03-13T14:49:52.503 INFO:tasks.workunit.client.1.smithi196.stdout:6/849: readlink d6/d2a/dc/l1e 0 2024-03-13T14:49:52.505 INFO:tasks.workunit.client.1.smithi196.stdout:2/618: truncate d0/d6/d4b/f8a 640731 0 2024-03-13T14:49:52.506 INFO:tasks.workunit.client.0.smithi176.stdout:6/592: rename d0/d9/f50 to d0/d10/d19/fbf 0 2024-03-13T14:49:52.508 INFO:tasks.workunit.client.0.smithi176.stdout:6/593: chown d0/d10/d19/d27/f4a 87 1 2024-03-13T14:49:52.508 INFO:tasks.workunit.client.1.smithi196.stdout:3/690: dwrite db/d19/d22/d3a/d53/f59 [0,4194304] 0 2024-03-13T14:49:52.510 INFO:tasks.workunit.client.1.smithi196.stdout:1/759: rename d1/d28/ca0 to d1/d9/c104 0 2024-03-13T14:49:52.512 INFO:tasks.workunit.client.1.smithi196.stdout:1/760: chown d1/d9/d33/d4b/d66/d7b 33205448 1 2024-03-13T14:49:52.514 INFO:tasks.workunit.client.0.smithi176.stdout:9/658: dread d0/d1e/d1f/d68/f85 [0,4194304] 0 2024-03-13T14:49:52.515 INFO:tasks.workunit.client.1.smithi196.stdout:1/761: chown d1/d28/d34/f41 17112 1 2024-03-13T14:49:52.518 INFO:tasks.workunit.client.0.smithi176.stdout:1/549: truncate d2/de/d3f/f52 4863824 0 2024-03-13T14:49:52.528 INFO:tasks.workunit.client.1.smithi196.stdout:1/762: creat d1/d9/d4e/d55/d63/db9/f105 x:0 0 0 2024-03-13T14:49:52.529 INFO:tasks.workunit.client.0.smithi176.stdout:9/659: dwrite d0/d1e/d1f/f38 [0,4194304] 0 2024-03-13T14:49:52.530 INFO:tasks.workunit.client.0.smithi176.stdout:9/660: readlink d0/l21 0 2024-03-13T14:49:52.535 INFO:tasks.workunit.client.1.smithi196.stdout:9/618: dwrite d7/de/d4b/d1d/f66 [4194304,4194304] 0 2024-03-13T14:49:52.537 INFO:tasks.workunit.client.1.smithi196.stdout:3/691: dwrite db/d15/d44/d51/fc6 [0,4194304] 0 2024-03-13T14:49:52.539 INFO:tasks.workunit.client.0.smithi176.stdout:2/618: sync 2024-03-13T14:49:52.542 INFO:tasks.workunit.client.1.smithi196.stdout:5/751: truncate d4/d1b/dc8/d14/d19/d26/dab/fb7 2625855 0 2024-03-13T14:49:52.550 INFO:tasks.workunit.client.1.smithi196.stdout:2/619: rename d0/d7/f94 to d0/d7/d9/fc9 0 2024-03-13T14:49:52.555 INFO:tasks.workunit.client.1.smithi196.stdout:6/850: write f4 [2628747,27240] 0 2024-03-13T14:49:52.555 INFO:tasks.workunit.client.1.smithi196.stdout:9/619: mknod d7/d24/d28/ccf 0 2024-03-13T14:49:52.559 INFO:tasks.workunit.client.1.smithi196.stdout:9/620: write d7/d71/f9e [635746,103715] 0 2024-03-13T14:49:52.560 INFO:tasks.workunit.client.1.smithi196.stdout:5/752: creat d4/d1b/dc8/d14/d19/d26/d33/d61/fea x:0 0 0 2024-03-13T14:49:52.562 INFO:tasks.workunit.client.1.smithi196.stdout:3/692: read db/d19/d22/f39 [1913697,649] 0 2024-03-13T14:49:52.564 INFO:tasks.workunit.client.1.smithi196.stdout:6/851: truncate d6/d57/f63 1492527 0 2024-03-13T14:49:52.565 INFO:tasks.workunit.client.1.smithi196.stdout:9/621: creat d7/de/d4b/d1d/d93/da7/fd0 x:0 0 0 2024-03-13T14:49:52.566 INFO:tasks.workunit.client.1.smithi196.stdout:1/763: rename d1/d9/c12 to d1/d3f/c106 0 2024-03-13T14:49:52.572 INFO:tasks.workunit.client.1.smithi196.stdout:2/620: creat d0/d7/d9/d9a/fca x:0 0 0 2024-03-13T14:49:52.573 INFO:tasks.workunit.client.1.smithi196.stdout:6/852: write d6/d2a/dc/d1c/d34/fbd [1469032,105574] 0 2024-03-13T14:49:52.578 INFO:tasks.workunit.client.1.smithi196.stdout:9/622: mkdir d7/de/d4b/dd1 0 2024-03-13T14:49:52.579 INFO:tasks.workunit.client.1.smithi196.stdout:9/623: readlink d7/d24/l48 0 2024-03-13T14:49:52.582 INFO:tasks.workunit.client.0.smithi176.stdout:3/631: rmdir d4/d28/d31/d45/d82 39 2024-03-13T14:49:52.584 INFO:tasks.workunit.client.0.smithi176.stdout:3/632: chown d4/d13/d81/d89/l70 1126853 1 2024-03-13T14:49:52.585 INFO:tasks.workunit.client.0.smithi176.stdout:8/523: truncate d0/d5/d11/f76 223452 0 2024-03-13T14:49:52.586 INFO:tasks.workunit.client.1.smithi196.stdout:5/753: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/f42 1225810 0 2024-03-13T14:49:52.586 INFO:tasks.workunit.client.0.smithi176.stdout:0/495: sync 2024-03-13T14:49:52.587 INFO:tasks.workunit.client.0.smithi176.stdout:8/524: chown d0/db/d58/d86/f89 3114468 1 2024-03-13T14:49:52.592 INFO:tasks.workunit.client.1.smithi196.stdout:1/764: rename d1/d9/d33/d4b/d88/cfc to d1/d28/d34/c107 0 2024-03-13T14:49:52.593 INFO:tasks.workunit.client.1.smithi196.stdout:3/693: getdents db/d19/d50/db7/da9 0 2024-03-13T14:49:52.597 INFO:tasks.workunit.client.0.smithi176.stdout:1/550: mknod d2/de/ca7 0 2024-03-13T14:49:52.598 INFO:tasks.workunit.client.1.smithi196.stdout:6/853: dread d6/d2a/d23/d28/f69 [0,4194304] 0 2024-03-13T14:49:52.601 INFO:tasks.workunit.client.1.smithi196.stdout:9/624: rename d7/de/d4b/d1d/l8b to d7/d24/d28/ld2 0 2024-03-13T14:49:52.602 INFO:tasks.workunit.client.0.smithi176.stdout:9/661: fdatasync d0/db/d89/d32/d58/d67/f7e 0 2024-03-13T14:49:52.605 INFO:tasks.workunit.client.0.smithi176.stdout:2/619: rename d1/dc/cd to d1/dc/d2d/d59/d80/d77/cd0 0 2024-03-13T14:49:52.607 INFO:tasks.workunit.client.1.smithi196.stdout:5/754: fsync d4/d1b/dc8/d14/d19/d1c/fa2 0 2024-03-13T14:49:52.609 INFO:tasks.workunit.client.1.smithi196.stdout:5/755: chown d4/d1b/dc8/d14/d19/d26/dab 1874004733 1 2024-03-13T14:49:52.611 INFO:tasks.workunit.client.0.smithi176.stdout:3/633: mkdir d4/d28/dd8 0 2024-03-13T14:49:52.613 INFO:tasks.workunit.client.1.smithi196.stdout:2/621: getdents d0/d6/d4b/d96/d9e 0 2024-03-13T14:49:52.621 INFO:tasks.workunit.client.0.smithi176.stdout:4/634: creat d7/db9/dd8/fdd x:0 0 0 2024-03-13T14:49:52.622 INFO:tasks.workunit.client.0.smithi176.stdout:4/635: stat d7/d10/d2c/d9d 0 2024-03-13T14:49:52.623 INFO:tasks.workunit.client.1.smithi196.stdout:1/765: mkdir d1/d3f/d5b/d74/dd4/d108 0 2024-03-13T14:49:52.628 INFO:tasks.workunit.client.0.smithi176.stdout:1/551: mkdir d2/de/d10/d16/d4a/d5f/da8 0 2024-03-13T14:49:52.630 INFO:tasks.workunit.client.0.smithi176.stdout:1/552: write f1 [3986192,53833] 0 2024-03-13T14:49:52.632 INFO:tasks.workunit.client.1.smithi196.stdout:6/854: creat d6/d2a/d23/d28/d10a/d10d/f116 x:0 0 0 2024-03-13T14:49:52.633 INFO:tasks.workunit.client.0.smithi176.stdout:9/662: mknod d0/dd8/d82/cdd 0 2024-03-13T14:49:52.633 INFO:tasks.workunit.client.1.smithi196.stdout:3/694: dread db/d19/d22/d3a/d53/f48 [0,4194304] 0 2024-03-13T14:49:52.634 INFO:tasks.workunit.client.0.smithi176.stdout:9/663: read d0/d40/f47 [828599,73113] 0 2024-03-13T14:49:52.637 INFO:tasks.workunit.client.1.smithi196.stdout:2/622: dread d0/f37 [0,4194304] 0 2024-03-13T14:49:52.642 INFO:tasks.workunit.client.0.smithi176.stdout:5/542: readlink d5/d2e/d7e/dad/d7b/da7/la9 0 2024-03-13T14:49:52.644 INFO:tasks.workunit.client.0.smithi176.stdout:5/543: readlink d5/d2e/d7e/d9f/l9c 0 2024-03-13T14:49:52.644 INFO:tasks.workunit.client.1.smithi196.stdout:9/625: creat d7/de/d74/fd3 x:0 0 0 2024-03-13T14:49:52.648 INFO:tasks.workunit.client.1.smithi196.stdout:6/855: rmdir d6/d2a/dc/d1c/da7/d9f 39 2024-03-13T14:49:52.648 INFO:tasks.workunit.client.0.smithi176.stdout:4/636: readlink d7/l3c 0 2024-03-13T14:49:52.651 INFO:tasks.workunit.client.0.smithi176.stdout:7/553: getdents d1/d3f/d4b 0 2024-03-13T14:49:52.653 INFO:tasks.workunit.client.0.smithi176.stdout:6/594: link d0/f6c d0/d10/fc0 0 2024-03-13T14:49:52.657 INFO:tasks.workunit.client.0.smithi176.stdout:9/664: readlink d0/d1e/la9 0 2024-03-13T14:49:52.658 INFO:tasks.workunit.client.0.smithi176.stdout:9/665: fsync d0/db/d89/d32/d58/f83 0 2024-03-13T14:49:52.660 INFO:tasks.workunit.client.0.smithi176.stdout:8/525: rename d0/d24/c7c to d0/d5/d11/d47/ca4 0 2024-03-13T14:49:52.662 INFO:tasks.workunit.client.1.smithi196.stdout:6/856: dwrite d6/d9/d17/d19/f2b [0,4194304] 0 2024-03-13T14:49:52.667 INFO:tasks.workunit.client.1.smithi196.stdout:1/766: creat d1/d3f/d5b/d74/f109 x:0 0 0 2024-03-13T14:49:52.668 INFO:tasks.workunit.client.0.smithi176.stdout:3/634: mknod d4/d13/d81/d89/cd9 0 2024-03-13T14:49:52.669 INFO:tasks.workunit.client.0.smithi176.stdout:5/544: chown d5/c36 31 1 2024-03-13T14:49:52.670 INFO:tasks.workunit.client.1.smithi196.stdout:2/623: dwrite d0/d6/d34/fbe [0,4194304] 0 2024-03-13T14:49:52.670 INFO:tasks.workunit.client.1.smithi196.stdout:1/767: dread - d1/d38/fef zero size 2024-03-13T14:49:52.670 INFO:tasks.workunit.client.0.smithi176.stdout:5/545: chown d5/d2e/d7e/d96 32573144 1 2024-03-13T14:49:52.673 INFO:tasks.workunit.client.0.smithi176.stdout:0/496: write d2/d5/d1e/f3d [611813,16209] 0 2024-03-13T14:49:52.675 INFO:tasks.workunit.client.0.smithi176.stdout:4/637: write d7/d2d/f6b [1428655,130702] 0 2024-03-13T14:49:52.677 INFO:tasks.workunit.client.1.smithi196.stdout:9/626: rename d7/d24/dbf/f80 to d7/de/d74/fd4 0 2024-03-13T14:49:52.679 INFO:tasks.workunit.client.1.smithi196.stdout:5/756: dwrite d4/d1b/dc8/d14/d19/d1c/d25/f49 [0,4194304] 0 2024-03-13T14:49:52.681 INFO:tasks.workunit.client.0.smithi176.stdout:7/554: read d1/d3f/d81/d9a/f9f [164944,18884] 0 2024-03-13T14:49:52.684 INFO:tasks.workunit.client.1.smithi196.stdout:3/695: write db/d15/f41 [228285,89415] 0 2024-03-13T14:49:52.685 INFO:tasks.workunit.client.1.smithi196.stdout:9/627: write d7/de/d4b/d1d/f75 [259162,56856] 0 2024-03-13T14:49:52.685 INFO:tasks.workunit.client.0.smithi176.stdout:2/620: truncate d1/d2a/d33/fa2 1769451 0 2024-03-13T14:49:52.685 INFO:tasks.workunit.client.1.smithi196.stdout:2/624: write d0/d6/d34/fb5 [1189233,111894] 0 2024-03-13T14:49:52.686 INFO:tasks.workunit.client.1.smithi196.stdout:6/857: mknod d6/d2a/c117 0 2024-03-13T14:49:52.692 INFO:tasks.workunit.client.1.smithi196.stdout:3/696: fdatasync db/d37/d64/f90 0 2024-03-13T14:49:52.692 INFO:tasks.workunit.client.1.smithi196.stdout:5/757: stat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0 0 2024-03-13T14:49:52.694 INFO:tasks.workunit.client.1.smithi196.stdout:3/697: chown db/d19/f2b 29 1 2024-03-13T14:49:52.695 INFO:tasks.workunit.client.0.smithi176.stdout:6/595: rename d0/de/d38/d6b/da3/db5 to d0/dc1 0 2024-03-13T14:49:52.696 INFO:tasks.workunit.client.1.smithi196.stdout:3/698: chown db/d19/d22/d3a/fad 2 1 2024-03-13T14:49:52.702 INFO:tasks.workunit.client.0.smithi176.stdout:4/638: symlink d7/d8/d45/lde 0 2024-03-13T14:49:52.705 INFO:tasks.workunit.client.1.smithi196.stdout:1/768: creat d1/f10a x:0 0 0 2024-03-13T14:49:52.706 INFO:tasks.workunit.client.1.smithi196.stdout:2/625: symlink d0/d7/d9/d61/dc7/lcb 0 2024-03-13T14:49:52.707 INFO:tasks.workunit.client.0.smithi176.stdout:1/553: creat d2/de/d10/fa9 x:0 0 0 2024-03-13T14:49:52.708 INFO:tasks.workunit.client.0.smithi176.stdout:7/555: symlink d1/d3f/d4b/d63/d7a/la8 0 2024-03-13T14:49:52.709 INFO:tasks.workunit.client.1.smithi196.stdout:1/769: write d1/d9/d33/d4b/d66/f7c [6946586,47660] 0 2024-03-13T14:49:52.712 INFO:tasks.workunit.client.0.smithi176.stdout:3/635: mknod d4/d13/d81/d89/cda 0 2024-03-13T14:49:52.713 INFO:tasks.workunit.client.1.smithi196.stdout:9/628: creat d7/de/d74/d94/fd5 x:0 0 0 2024-03-13T14:49:52.716 INFO:tasks.workunit.client.0.smithi176.stdout:5/546: creat d5/d2e/d7e/dad/d7b/daa/fb8 x:0 0 0 2024-03-13T14:49:52.717 INFO:tasks.workunit.client.0.smithi176.stdout:5/547: chown d5/f9a 0 1 2024-03-13T14:49:52.718 INFO:tasks.workunit.client.1.smithi196.stdout:3/699: creat db/d19/d22/d3a/d68/d8e/fe0 x:0 0 0 2024-03-13T14:49:52.720 INFO:tasks.workunit.client.0.smithi176.stdout:2/621: rename d1/l3e to d1/dc/d2d/d38/d5d/ld1 0 2024-03-13T14:49:52.722 INFO:tasks.workunit.client.1.smithi196.stdout:2/626: creat d0/d7/d66/d67/fcc x:0 0 0 2024-03-13T14:49:52.723 INFO:tasks.workunit.client.1.smithi196.stdout:2/627: dread - d0/fbb zero size 2024-03-13T14:49:52.724 INFO:tasks.workunit.client.1.smithi196.stdout:5/758: mknod d4/d1b/dc8/d14/d19/d1c/d2d/ceb 0 2024-03-13T14:49:52.727 INFO:tasks.workunit.client.0.smithi176.stdout:6/596: unlink d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/ca5 0 2024-03-13T14:49:52.727 INFO:tasks.workunit.client.0.smithi176.stdout:9/666: dwrite d0/db/d89/d32/f62 [0,4194304] 0 2024-03-13T14:49:52.728 INFO:tasks.workunit.client.1.smithi196.stdout:9/629: symlink d7/d24/d28/ld6 0 2024-03-13T14:49:52.729 INFO:tasks.workunit.client.0.smithi176.stdout:0/497: fdatasync d2/db/d31/f90 0 2024-03-13T14:49:52.730 INFO:tasks.workunit.client.1.smithi196.stdout:9/630: chown d7/d9/d17/l4a 63145297 1 2024-03-13T14:49:52.730 INFO:tasks.workunit.client.1.smithi196.stdout:3/700: creat db/d19/d22/fe1 x:0 0 0 2024-03-13T14:49:52.733 INFO:tasks.workunit.client.0.smithi176.stdout:4/639: creat d7/db9/dd8/fdf x:0 0 0 2024-03-13T14:49:52.734 INFO:tasks.workunit.client.1.smithi196.stdout:6/858: dread d6/d2a/f93 [4194304,4194304] 0 2024-03-13T14:49:52.735 INFO:tasks.workunit.client.1.smithi196.stdout:1/770: creat d1/dfa/f10b x:0 0 0 2024-03-13T14:49:52.736 INFO:tasks.workunit.client.1.smithi196.stdout:3/701: truncate db/d19/d22/fc2 582680 0 2024-03-13T14:49:52.737 INFO:tasks.workunit.client.0.smithi176.stdout:7/556: chown d1/d28/d3a/l92 4 1 2024-03-13T14:49:52.740 INFO:tasks.workunit.client.0.smithi176.stdout:3/636: truncate d4/d13/d16/f84 491356 0 2024-03-13T14:49:52.741 INFO:tasks.workunit.client.1.smithi196.stdout:5/759: dread d4/d1b/dc8/fd8 [0,4194304] 0 2024-03-13T14:49:52.742 INFO:tasks.workunit.client.1.smithi196.stdout:9/631: mkdir d7/de/d4b/d1d/d93/dd7 0 2024-03-13T14:49:52.745 INFO:tasks.workunit.client.0.smithi176.stdout:5/548: rmdir d5/d2c/d40/d76 39 2024-03-13T14:49:52.749 INFO:tasks.workunit.client.0.smithi176.stdout:8/526: creat d0/d5/fa5 x:0 0 0 2024-03-13T14:49:52.750 INFO:tasks.workunit.client.1.smithi196.stdout:8/648: sync 2024-03-13T14:49:52.750 INFO:tasks.workunit.client.1.smithi196.stdout:0/729: sync 2024-03-13T14:49:52.751 INFO:tasks.workunit.client.1.smithi196.stdout:3/702: unlink db/lc 0 2024-03-13T14:49:52.752 INFO:tasks.workunit.client.1.smithi196.stdout:8/649: stat d0/d10/d12/d1e/d2d/cdb 0 2024-03-13T14:49:52.758 INFO:tasks.workunit.client.0.smithi176.stdout:6/597: creat d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/fc2 x:0 0 0 2024-03-13T14:49:52.759 INFO:tasks.workunit.client.0.smithi176.stdout:7/557: dwrite d1/d1c/d26/d41/d2b/f85 [0,4194304] 0 2024-03-13T14:49:52.759 INFO:tasks.workunit.client.1.smithi196.stdout:2/628: link d0/d6/d4b/d5f/f81 d0/d6/fcd 0 2024-03-13T14:49:52.759 INFO:tasks.workunit.client.1.smithi196.stdout:5/760: creat d4/d1b/dc8/d14/d19/d26/fec x:0 0 0 2024-03-13T14:49:52.760 INFO:tasks.workunit.client.0.smithi176.stdout:6/598: chown d0/d9/f55 4849 1 2024-03-13T14:49:52.761 INFO:tasks.workunit.client.1.smithi196.stdout:6/859: mknod d6/d9/c118 0 2024-03-13T14:49:52.762 INFO:tasks.workunit.client.0.smithi176.stdout:4/640: symlink d7/d8/d15/d23/le0 0 2024-03-13T14:49:52.763 INFO:tasks.workunit.client.1.smithi196.stdout:6/860: readlink d6/d2a/dc/d1c/l7d 0 2024-03-13T14:49:52.771 INFO:tasks.workunit.client.1.smithi196.stdout:3/703: mknod db/d19/d22/d3a/d53/ce2 0 2024-03-13T14:49:52.771 INFO:tasks.workunit.client.1.smithi196.stdout:8/650: symlink d0/d16/d3d/d8b/le0 0 2024-03-13T14:49:52.771 INFO:tasks.workunit.client.1.smithi196.stdout:0/730: creat d8/fef x:0 0 0 2024-03-13T14:49:52.771 INFO:tasks.workunit.client.0.smithi176.stdout:3/637: rename d4/d13/d5e/lae to d4/d13/d81/d89/d5b/ldb 0 2024-03-13T14:49:52.774 INFO:tasks.workunit.client.0.smithi176.stdout:5/549: creat d5/d2c/d40/d6a/fb9 x:0 0 0 2024-03-13T14:49:52.775 INFO:tasks.workunit.client.1.smithi196.stdout:2/629: chown d0/d6/d34/d5b/c82 478306 1 2024-03-13T14:49:52.776 INFO:tasks.workunit.client.1.smithi196.stdout:5/761: fdatasync d4/d1b/dc8/d3c/fc3 0 2024-03-13T14:49:52.779 INFO:tasks.workunit.client.0.smithi176.stdout:2/622: symlink d1/dc/d2d/d4b/ld2 0 2024-03-13T14:49:52.781 INFO:tasks.workunit.client.0.smithi176.stdout:8/527: rmdir d0/db/d53/d80 39 2024-03-13T14:49:52.782 INFO:tasks.workunit.client.1.smithi196.stdout:1/771: write d1/d9/f20 [4293226,96175] 0 2024-03-13T14:49:52.782 INFO:tasks.workunit.client.1.smithi196.stdout:9/632: dwrite d7/fd [0,4194304] 0 2024-03-13T14:49:52.785 INFO:tasks.workunit.client.0.smithi176.stdout:7/558: creat d1/d1c/d26/fa9 x:0 0 0 2024-03-13T14:49:52.789 INFO:tasks.workunit.client.0.smithi176.stdout:3/638: dwrite d4/d28/d31/d52/f6a [0,4194304] 0 2024-03-13T14:49:52.793 INFO:tasks.workunit.client.1.smithi196.stdout:6/861: dread d6/f8 [0,4194304] 0 2024-03-13T14:49:52.793 INFO:tasks.workunit.client.1.smithi196.stdout:7/792: sync 2024-03-13T14:49:52.793 INFO:tasks.workunit.client.0.smithi176.stdout:6/599: creat d0/de/d38/d88/fc3 x:0 0 0 2024-03-13T14:49:52.793 INFO:tasks.workunit.client.1.smithi196.stdout:4/658: sync 2024-03-13T14:49:52.797 INFO:tasks.workunit.client.0.smithi176.stdout:1/554: creat d2/faa x:0 0 0 2024-03-13T14:49:52.799 INFO:tasks.workunit.client.1.smithi196.stdout:3/704: dwrite db/f3e [0,4194304] 0 2024-03-13T14:49:52.801 INFO:tasks.workunit.client.1.smithi196.stdout:4/659: read d1/d3/d7/d35/f52 [118107,5689] 0 2024-03-13T14:49:52.802 INFO:tasks.workunit.client.0.smithi176.stdout:9/667: rename d0/d1e/d1f/fbd to d0/d1e/d8d/dbe/fde 0 2024-03-13T14:49:52.804 INFO:tasks.workunit.client.1.smithi196.stdout:8/651: dwrite d0/d10/d12/d1e/fbb [0,4194304] 0 2024-03-13T14:49:52.805 INFO:tasks.workunit.client.1.smithi196.stdout:4/660: dread - d1/d1e/d43/f85 zero size 2024-03-13T14:49:52.813 INFO:tasks.workunit.client.0.smithi176.stdout:6/600: dread - d0/de/d54/f75 zero size 2024-03-13T14:49:52.814 INFO:tasks.workunit.client.1.smithi196.stdout:7/793: dwrite d0/f37 [0,4194304] 0 2024-03-13T14:49:52.820 INFO:tasks.workunit.client.0.smithi176.stdout:0/498: dwrite d2/db/f3b [0,4194304] 0 2024-03-13T14:49:52.823 INFO:tasks.workunit.client.0.smithi176.stdout:0/499: readlink d2/d5/dc/l5f 0 2024-03-13T14:49:52.825 INFO:tasks.workunit.client.0.smithi176.stdout:8/528: rename d0/f5b to d0/d24/d29/fa6 0 2024-03-13T14:49:52.826 INFO:tasks.workunit.client.0.smithi176.stdout:9/668: truncate d0/d1e/d1f/f41 1896864 0 2024-03-13T14:49:52.827 INFO:tasks.workunit.client.0.smithi176.stdout:7/559: mknod d1/d28/d3a/d1b/d8f/d91/caa 0 2024-03-13T14:49:52.828 INFO:tasks.workunit.client.0.smithi176.stdout:2/623: mknod d1/dc/d2d/d59/d80/cd3 0 2024-03-13T14:49:52.833 INFO:tasks.workunit.client.0.smithi176.stdout:0/500: chown d2/d74/f91 122406 1 2024-03-13T14:49:52.835 INFO:tasks.workunit.client.0.smithi176.stdout:8/529: stat d0/db/dc/f13 0 2024-03-13T14:49:52.836 INFO:tasks.workunit.client.0.smithi176.stdout:5/550: dwrite d5/d2e/f54 [0,4194304] 0 2024-03-13T14:49:52.837 INFO:tasks.workunit.client.0.smithi176.stdout:7/560: chown d1/d1c/c21 464 1 2024-03-13T14:49:52.845 INFO:tasks.workunit.client.0.smithi176.stdout:9/669: write d0/db/d89/d32/d58/d5f/fc8 [569215,77268] 0 2024-03-13T14:49:52.846 INFO:tasks.workunit.client.0.smithi176.stdout:1/555: rename d2/ca to d2/de/d10/d16/d4a/d5f/d66/cab 0 2024-03-13T14:49:52.849 INFO:tasks.workunit.client.0.smithi176.stdout:9/670: chown d0/d40/d53/da0 1383 1 2024-03-13T14:49:52.851 INFO:tasks.workunit.client.0.smithi176.stdout:2/624: creat d1/dc/d2d/d59/d8c/fd4 x:0 0 0 2024-03-13T14:49:52.856 INFO:tasks.workunit.client.0.smithi176.stdout:1/556: mknod d2/de/d3f/d19/cac 0 2024-03-13T14:49:52.856 INFO:tasks.workunit.client.0.smithi176.stdout:6/601: dwrite d0/de/d38/d7e/f84 [4194304,4194304] 0 2024-03-13T14:49:52.859 INFO:tasks.workunit.client.0.smithi176.stdout:0/501: mknod d2/d5/dc/d59/d79/c9a 0 2024-03-13T14:49:52.862 INFO:tasks.workunit.client.0.smithi176.stdout:1/557: chown d2/de/d3f/d50/d83/d97 1408 1 2024-03-13T14:49:52.864 INFO:tasks.workunit.client.0.smithi176.stdout:0/502: fdatasync d2/d74/f91 0 2024-03-13T14:49:52.865 INFO:tasks.workunit.client.1.smithi196.stdout:2/630: rmdir d0/d50 39 2024-03-13T14:49:52.868 INFO:tasks.workunit.client.0.smithi176.stdout:6/602: mkdir d0/d10/d19/d27/d2b/d2e/dc4 0 2024-03-13T14:49:52.872 INFO:tasks.workunit.client.0.smithi176.stdout:7/561: dwrite d1/d3f/d65/f97 [0,4194304] 0 2024-03-13T14:49:52.872 INFO:tasks.workunit.client.0.smithi176.stdout:4/641: dwrite d7/d8/d33/f6d [0,4194304] 0 2024-03-13T14:49:52.874 INFO:tasks.workunit.client.0.smithi176.stdout:3/639: truncate d4/d13/d16/d3d/f5d 3925919 0 2024-03-13T14:49:52.880 INFO:tasks.workunit.client.0.smithi176.stdout:4/642: write d7/d8/d15/fbc [15882,118705] 0 2024-03-13T14:49:52.882 INFO:tasks.workunit.client.0.smithi176.stdout:1/558: creat d2/de/d3f/d50/d83/fad x:0 0 0 2024-03-13T14:49:52.889 INFO:tasks.workunit.client.1.smithi196.stdout:3/705: creat db/d19/d22/d3a/d6a/fe3 x:0 0 0 2024-03-13T14:49:52.892 INFO:tasks.workunit.client.1.smithi196.stdout:4/661: creat d1/d3/d4c/d72/fea x:0 0 0 2024-03-13T14:49:52.894 INFO:tasks.workunit.client.0.smithi176.stdout:4/643: rmdir d7/d8/d15/d23 39 2024-03-13T14:49:52.896 INFO:tasks.workunit.client.1.smithi196.stdout:7/794: truncate d0/d9/d31/d77/d93/fbd 433289 0 2024-03-13T14:49:52.896 INFO:tasks.workunit.client.0.smithi176.stdout:4/644: stat d7/d8/d15/dd6/dba 0 2024-03-13T14:49:52.897 INFO:tasks.workunit.client.0.smithi176.stdout:6/603: mkdir d0/d10/d19/d27/d2b/d2e/dc4/dc5 0 2024-03-13T14:49:52.899 INFO:tasks.workunit.client.0.smithi176.stdout:1/559: rename d2/d29/f39 to d2/de/d3f/d50/d83/fae 0 2024-03-13T14:49:52.902 INFO:tasks.workunit.client.0.smithi176.stdout:8/530: dwrite d0/db/f1a [0,4194304] 0 2024-03-13T14:49:52.903 INFO:tasks.workunit.client.1.smithi196.stdout:2/631: mkdir d0/d6/d4b/d5f/dce 0 2024-03-13T14:49:52.904 INFO:tasks.workunit.client.0.smithi176.stdout:2/625: truncate d1/dc/fa5 3650491 0 2024-03-13T14:49:52.905 INFO:tasks.workunit.client.0.smithi176.stdout:0/503: creat d2/f9b x:0 0 0 2024-03-13T14:49:52.905 INFO:tasks.workunit.client.0.smithi176.stdout:8/531: chown d0/db/d58/d86 24470 1 2024-03-13T14:49:52.909 INFO:tasks.workunit.client.1.smithi196.stdout:5/762: rename d4/d1b/dc8/d14/d19/d26/d33/d61/l7c to d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/led 0 2024-03-13T14:49:52.909 INFO:tasks.workunit.client.1.smithi196.stdout:0/731: dread d8/da/f4e [0,4194304] 0 2024-03-13T14:49:52.918 INFO:tasks.workunit.client.0.smithi176.stdout:6/604: fsync d0/d10/d19/d27/d2b/d2e/f52 0 2024-03-13T14:49:52.919 INFO:tasks.workunit.client.1.smithi196.stdout:8/652: creat d0/d10/d12/d1e/d2d/dd8/fe1 x:0 0 0 2024-03-13T14:49:52.922 INFO:tasks.workunit.client.0.smithi176.stdout:3/640: dwrite d4/d28/f67 [0,4194304] 0 2024-03-13T14:49:52.925 INFO:tasks.workunit.client.0.smithi176.stdout:2/626: fsync d1/dc/f2e 0 2024-03-13T14:49:52.927 INFO:tasks.workunit.client.1.smithi196.stdout:4/662: mkdir d1/d3/d7/d35/d76/d58/dc9/dd7/deb 0 2024-03-13T14:49:52.928 INFO:tasks.workunit.client.0.smithi176.stdout:2/627: write d1/dc/d1c/f4c [4820344,89243] 0 2024-03-13T14:49:52.930 INFO:tasks.workunit.client.0.smithi176.stdout:2/628: chown d1/dc/d2d/d4b/d69 7 1 2024-03-13T14:49:52.930 INFO:tasks.workunit.client.0.smithi176.stdout:7/562: dread d1/d1c/d26/d41/d2b/d40/d64/d94/f7e [0,4194304] 0 2024-03-13T14:49:52.930 INFO:tasks.workunit.client.1.smithi196.stdout:7/795: chown d0/d9/d1e/d58/d72/da9/d47/fa7 171123901 1 2024-03-13T14:49:52.935 INFO:tasks.workunit.client.0.smithi176.stdout:4/645: mkdir d7/de1 0 2024-03-13T14:49:52.936 INFO:tasks.workunit.client.1.smithi196.stdout:4/663: write d1/d3/d7/f5d [520595,67252] 0 2024-03-13T14:49:52.937 INFO:tasks.workunit.client.1.smithi196.stdout:1/772: creat d1/d9/d4e/f10c x:0 0 0 2024-03-13T14:49:52.940 INFO:tasks.workunit.client.1.smithi196.stdout:9/633: rename d7/d9/c2f to d7/d9/cd8 0 2024-03-13T14:49:52.944 INFO:tasks.workunit.client.1.smithi196.stdout:0/732: rename d8/da/d25/d26/d54/d88/fcb to d8/da/d25/ff0 0 2024-03-13T14:49:52.948 INFO:tasks.workunit.client.1.smithi196.stdout:3/706: dwrite db/d19/d22/f31 [0,4194304] 0 2024-03-13T14:49:52.949 INFO:tasks.workunit.client.1.smithi196.stdout:6/862: link d6/d9/d17/d19/d4e/f6e d6/d2a/d23/d28/f119 0 2024-03-13T14:49:52.950 INFO:tasks.workunit.client.1.smithi196.stdout:2/632: dwrite d0/d6/f84 [4194304,4194304] 0 2024-03-13T14:49:52.952 INFO:tasks.workunit.client.1.smithi196.stdout:3/707: chown db/d71/fa5 6387752 1 2024-03-13T14:49:52.952 INFO:tasks.workunit.client.1.smithi196.stdout:2/633: fsync d0/f1a 0 2024-03-13T14:49:52.954 INFO:tasks.workunit.client.0.smithi176.stdout:3/641: rmdir d4/d28/d34/d38 39 2024-03-13T14:49:52.960 INFO:tasks.workunit.client.0.smithi176.stdout:4/646: dread d7/f50 [0,4194304] 0 2024-03-13T14:49:52.961 INFO:tasks.workunit.client.0.smithi176.stdout:0/504: fdatasync d2/d5/d1e/f4e 0 2024-03-13T14:49:52.963 INFO:tasks.workunit.client.1.smithi196.stdout:8/653: fsync d0/d10/d12/f53 0 2024-03-13T14:49:52.963 INFO:tasks.workunit.client.1.smithi196.stdout:4/664: dread - d1/d1e/fb7 zero size 2024-03-13T14:49:52.964 INFO:tasks.workunit.client.0.smithi176.stdout:4/647: truncate d7/db9/dd8/fdd 342523 0 2024-03-13T14:49:52.965 INFO:tasks.workunit.client.0.smithi176.stdout:8/532: mkdir d0/db/d39/d56/d93/da7 0 2024-03-13T14:49:52.969 INFO:tasks.workunit.client.1.smithi196.stdout:7/796: creat d0/d9/d1e/d58/d72/da9/d47/f10e x:0 0 0 2024-03-13T14:49:52.969 INFO:tasks.workunit.client.0.smithi176.stdout:2/629: creat d1/dc/d2d/d59/d80/d7e/fd5 x:0 0 0 2024-03-13T14:49:52.971 INFO:tasks.workunit.client.1.smithi196.stdout:1/773: write d1/d9/d4e/d55/f6f [1012235,111303] 0 2024-03-13T14:49:52.972 INFO:tasks.workunit.client.0.smithi176.stdout:2/630: dread - d1/dc/d2d/fca zero size 2024-03-13T14:49:52.972 INFO:tasks.workunit.client.0.smithi176.stdout:7/563: rename d1/d1c/d26/d41/d2b/f37 to d1/d3f/fab 0 2024-03-13T14:49:52.974 INFO:tasks.workunit.client.0.smithi176.stdout:2/631: fdatasync d1/dc/d2d/d59/d80/fbb 0 2024-03-13T14:49:52.977 INFO:tasks.workunit.client.0.smithi176.stdout:2/632: chown d1/d2a/d33/fa7 115 1 2024-03-13T14:49:52.977 INFO:tasks.workunit.client.1.smithi196.stdout:1/774: dread - d1/d9/d4e/d55/d63/da6/dc8/fe6 zero size 2024-03-13T14:49:52.981 INFO:tasks.workunit.client.1.smithi196.stdout:5/763: link d4/d1b/dc8/d14/d19/d1c/d25/d38/f56 d4/d1b/dc8/d14/d37/fee 0 2024-03-13T14:49:52.982 INFO:tasks.workunit.client.0.smithi176.stdout:3/642: dread d4/d13/f73 [0,4194304] 0 2024-03-13T14:49:52.984 INFO:tasks.workunit.client.0.smithi176.stdout:1/560: link d2/d30/d63/c9b d2/d30/d63/d35/caf 0 2024-03-13T14:49:52.984 INFO:tasks.workunit.client.1.smithi196.stdout:0/733: chown d8/da/d25/d26/d35/d3b/d50/d5d/cd6 7288 1 2024-03-13T14:49:52.987 INFO:tasks.workunit.client.0.smithi176.stdout:1/561: chown d2/de/d10/d16/d4a/l88 1 1 2024-03-13T14:49:52.988 INFO:tasks.workunit.client.1.smithi196.stdout:3/708: read f7 [2917407,22306] 0 2024-03-13T14:49:52.989 INFO:tasks.workunit.client.0.smithi176.stdout:2/633: dread d1/d2a/f82 [0,4194304] 0 2024-03-13T14:49:52.989 INFO:tasks.workunit.client.0.smithi176.stdout:7/564: dwrite d1/d1c/f79 [0,4194304] 0 2024-03-13T14:49:52.989 INFO:tasks.workunit.client.1.smithi196.stdout:6/863: creat d6/d2a/dc/de3/f11a x:0 0 0 2024-03-13T14:49:52.993 INFO:tasks.workunit.client.0.smithi176.stdout:6/605: write d0/de/d38/d70/f9e [152470,10799] 0 2024-03-13T14:49:52.993 INFO:tasks.workunit.client.1.smithi196.stdout:8/654: unlink d0/d16/d3d/d8b/f60 0 2024-03-13T14:49:52.996 INFO:tasks.workunit.client.0.smithi176.stdout:0/505: dread d2/db/d39/f47 [0,4194304] 0 2024-03-13T14:49:52.997 INFO:tasks.workunit.client.0.smithi176.stdout:0/506: stat d2/f75 0 2024-03-13T14:49:52.999 INFO:tasks.workunit.client.1.smithi196.stdout:7/797: truncate d0/d4d/fc8 76484 0 2024-03-13T14:49:52.999 INFO:tasks.workunit.client.0.smithi176.stdout:0/507: fsync d2/f6 0 2024-03-13T14:49:53.001 INFO:tasks.workunit.client.1.smithi196.stdout:1/775: symlink d1/d9/d33/d4b/d66/l10d 0 2024-03-13T14:49:53.002 INFO:tasks.workunit.client.0.smithi176.stdout:1/562: truncate d2/de/d10/d16/d47/f5c 293601 0 2024-03-13T14:49:53.003 INFO:tasks.workunit.client.1.smithi196.stdout:0/734: creat d8/da/d74/ff1 x:0 0 0 2024-03-13T14:49:53.006 INFO:tasks.workunit.client.0.smithi176.stdout:4/648: link d7/d8/d2f/fb4 d7/d10/d2c/d9d/fe2 0 2024-03-13T14:49:53.008 INFO:tasks.workunit.client.1.smithi196.stdout:3/709: rmdir db/d19/d4a 39 2024-03-13T14:49:53.009 INFO:tasks.workunit.client.0.smithi176.stdout:8/533: dwrite d0/f52 [0,4194304] 0 2024-03-13T14:49:53.012 INFO:tasks.workunit.client.0.smithi176.stdout:2/634: rename d1/dc/d2d/d90/dcf/da0/cc0 to d1/dc/d2d/d59/d80/d5c/cd6 0 2024-03-13T14:49:53.013 INFO:tasks.workunit.client.1.smithi196.stdout:3/710: dread db/d15/f1a [0,4194304] 0 2024-03-13T14:49:53.017 INFO:tasks.workunit.client.0.smithi176.stdout:6/606: symlink d0/de/d38/d7e/lc6 0 2024-03-13T14:49:53.021 INFO:tasks.workunit.client.1.smithi196.stdout:8/655: unlink d0/c74 0 2024-03-13T14:49:53.023 INFO:tasks.workunit.client.1.smithi196.stdout:9/634: creat d7/d9/fd9 x:0 0 0 2024-03-13T14:49:53.024 INFO:tasks.workunit.client.0.smithi176.stdout:7/565: rename d1/d1c/d26/d41/d2b/d40/f6e to d1/d28/d3a/d1b/d8f/fac 0 2024-03-13T14:49:53.025 INFO:tasks.workunit.client.1.smithi196.stdout:1/776: mkdir d1/d94/d10e 0 2024-03-13T14:49:53.025 INFO:tasks.workunit.client.1.smithi196.stdout:4/665: dread d1/d3/d7/f1a [4194304,4194304] 0 2024-03-13T14:49:53.028 INFO:tasks.workunit.client.1.smithi196.stdout:0/735: mkdir d8/da/d25/d26/d48/d82/da0/df2 0 2024-03-13T14:49:53.029 INFO:tasks.workunit.client.1.smithi196.stdout:4/666: read - d1/d3/d7/d35/d76/d58/d69/d9f/fe0 zero size 2024-03-13T14:49:53.030 INFO:tasks.workunit.client.0.smithi176.stdout:2/635: creat d1/dc/d2d/d4b/d69/fd7 x:0 0 0 2024-03-13T14:49:53.034 INFO:tasks.workunit.client.0.smithi176.stdout:8/534: dwrite d0/db/dc/f8a [0,4194304] 0 2024-03-13T14:49:53.035 INFO:tasks.workunit.client.0.smithi176.stdout:2/636: chown d1/d2a/d33/l47 344706 1 2024-03-13T14:49:53.038 INFO:tasks.workunit.client.0.smithi176.stdout:1/563: dwrite d2/d30/f65 [4194304,4194304] 0 2024-03-13T14:49:53.044 INFO:tasks.workunit.client.0.smithi176.stdout:7/566: rmdir d1/d1c/d26/d41/d2b/d40/d64/d94 39 2024-03-13T14:49:53.045 INFO:tasks.workunit.client.0.smithi176.stdout:5/551: sync 2024-03-13T14:49:53.045 INFO:tasks.workunit.client.0.smithi176.stdout:9/671: sync 2024-03-13T14:49:53.046 INFO:tasks.workunit.client.0.smithi176.stdout:0/508: dread d2/d5/d1e/f72 [0,4194304] 0 2024-03-13T14:49:53.049 INFO:tasks.workunit.client.1.smithi196.stdout:6/864: dwrite d6/d7/df/d27/d39/d44/d74/fd4 [0,4194304] 0 2024-03-13T14:49:53.049 INFO:tasks.workunit.client.1.smithi196.stdout:9/635: mkdir d7/de/d74/d94/dda 0 2024-03-13T14:49:53.050 INFO:tasks.workunit.client.1.smithi196.stdout:5/764: dread d4/d1b/dc8/d3c/d6a/d93/f9e [0,4194304] 0 2024-03-13T14:49:53.051 INFO:tasks.workunit.client.0.smithi176.stdout:2/637: creat d1/dc/d2d/d90/dcf/da0/fd8 x:0 0 0 2024-03-13T14:49:53.055 INFO:tasks.workunit.client.0.smithi176.stdout:8/535: dwrite d0/d24/d29/f8c [0,4194304] 0 2024-03-13T14:49:53.056 INFO:tasks.workunit.client.0.smithi176.stdout:8/536: chown d0/d5/f4d 30 1 2024-03-13T14:49:53.059 INFO:tasks.workunit.client.1.smithi196.stdout:1/777: fdatasync d1/f47 0 2024-03-13T14:49:53.059 INFO:tasks.workunit.client.0.smithi176.stdout:8/537: write d0/d5/d11/f7f [460456,128701] 0 2024-03-13T14:49:53.060 INFO:tasks.workunit.client.0.smithi176.stdout:4/649: write d7/f24 [3784597,128869] 0 2024-03-13T14:49:53.061 INFO:tasks.workunit.client.0.smithi176.stdout:5/552: rmdir d5/d2c/d8a 39 2024-03-13T14:49:53.064 INFO:tasks.workunit.client.0.smithi176.stdout:7/567: chown d1/d3f/d81/f8e 147983 1 2024-03-13T14:49:53.068 INFO:tasks.workunit.client.1.smithi196.stdout:5/765: chown d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/fb0 3 1 2024-03-13T14:49:53.069 INFO:tasks.workunit.client.0.smithi176.stdout:4/650: write d7/d10/d2c/d9d/dbb/fd3 [664232,6229] 0 2024-03-13T14:49:53.070 INFO:tasks.workunit.client.1.smithi196.stdout:4/667: symlink d1/d3/d4c/d72/d7d/d80/lec 0 2024-03-13T14:49:53.077 INFO:tasks.workunit.client.1.smithi196.stdout:7/798: dwrite d0/d6/d33/d57/dae/ff4 [0,4194304] 0 2024-03-13T14:49:53.077 INFO:tasks.workunit.client.0.smithi176.stdout:2/638: creat d1/dc/d2d/d59/d80/d76/fd9 x:0 0 0 2024-03-13T14:49:53.077 INFO:tasks.workunit.client.0.smithi176.stdout:5/553: mknod d5/d2e/d7e/dad/d7b/cba 0 2024-03-13T14:49:53.077 INFO:tasks.workunit.client.0.smithi176.stdout:6/607: dwrite d0/f4e [0,4194304] 0 2024-03-13T14:49:53.080 INFO:tasks.workunit.client.0.smithi176.stdout:8/538: dread d0/d24/f32 [0,4194304] 0 2024-03-13T14:49:53.082 INFO:tasks.workunit.client.1.smithi196.stdout:0/736: write d8/da/f13 [948555,119898] 0 2024-03-13T14:49:53.082 INFO:tasks.workunit.client.0.smithi176.stdout:6/608: chown d0/d10/d19/d27/d2b/d2e/db7 1953 1 2024-03-13T14:49:53.083 INFO:tasks.workunit.client.1.smithi196.stdout:9/636: dwrite d7/d9/d83/fcb [0,4194304] 0 2024-03-13T14:49:53.083 INFO:tasks.workunit.client.0.smithi176.stdout:1/564: dread d2/de/d10/d16/d4a/d5f/f81 [0,4194304] 0 2024-03-13T14:49:53.084 INFO:tasks.workunit.client.0.smithi176.stdout:9/672: truncate d0/f59 1394584 0 2024-03-13T14:49:53.085 INFO:tasks.workunit.client.1.smithi196.stdout:3/711: getdents db/d19/d50 0 2024-03-13T14:49:53.086 INFO:tasks.workunit.client.1.smithi196.stdout:3/712: stat db/d15/d44/d79/f80 0 2024-03-13T14:49:53.087 INFO:tasks.workunit.client.1.smithi196.stdout:9/637: dread - d7/de/d4b/d1d/d93/da7/fd0 zero size 2024-03-13T14:49:53.091 INFO:tasks.workunit.client.0.smithi176.stdout:4/651: mknod d7/d10/d2c/d9d/ce3 0 2024-03-13T14:49:53.098 INFO:tasks.workunit.client.0.smithi176.stdout:8/539: rename d0/db/d58/d86/f89 to d0/db/d58/d86/fa8 0 2024-03-13T14:49:53.099 INFO:tasks.workunit.client.1.smithi196.stdout:7/799: creat d0/d6/d33/d57/f10f x:0 0 0 2024-03-13T14:49:53.100 INFO:tasks.workunit.client.0.smithi176.stdout:6/609: symlink d0/de/d54/d99/lc7 0 2024-03-13T14:49:53.101 INFO:tasks.workunit.client.1.smithi196.stdout:3/713: creat db/d15/d44/d79/fe4 x:0 0 0 2024-03-13T14:49:53.104 INFO:tasks.workunit.client.0.smithi176.stdout:9/673: stat d0/db/d89/d32/d58/d5f/d84/d9a/lbf 0 2024-03-13T14:49:53.106 INFO:tasks.workunit.client.0.smithi176.stdout:7/568: rmdir d1/d1c/d26/d41/d2b/d40/d64/d94 39 2024-03-13T14:49:53.106 INFO:tasks.workunit.client.0.smithi176.stdout:2/639: mknod d1/dc/d2d/d4b/cda 0 2024-03-13T14:49:53.107 INFO:tasks.workunit.client.1.smithi196.stdout:7/800: mkdir d0/d6/d33/d110 0 2024-03-13T14:49:53.108 INFO:tasks.workunit.client.0.smithi176.stdout:1/565: dwrite d2/d30/f3a [8388608,4194304] 0 2024-03-13T14:49:53.109 INFO:tasks.workunit.client.0.smithi176.stdout:9/674: truncate d0/d40/d53/f6b 971376 0 2024-03-13T14:49:53.111 INFO:tasks.workunit.client.1.smithi196.stdout:4/668: creat d1/d3/d7/d35/d76/fed x:0 0 0 2024-03-13T14:49:53.111 INFO:tasks.workunit.client.0.smithi176.stdout:8/540: mkdir d0/db/d58/d86/da9 0 2024-03-13T14:49:53.112 INFO:tasks.workunit.client.0.smithi176.stdout:6/610: creat d0/de/d38/d6b/fc8 x:0 0 0 2024-03-13T14:49:53.116 INFO:tasks.workunit.client.1.smithi196.stdout:3/714: creat db/d19/d4a/dc5/fe5 x:0 0 0 2024-03-13T14:49:53.118 INFO:tasks.workunit.client.1.smithi196.stdout:7/801: mknod d0/d6/d33/d57/dae/de8/c111 0 2024-03-13T14:49:53.118 INFO:tasks.workunit.client.1.smithi196.stdout:3/715: chown db/f29 62607 1 2024-03-13T14:49:53.119 INFO:tasks.workunit.client.1.smithi196.stdout:4/669: rmdir d1 39 2024-03-13T14:49:53.124 INFO:tasks.workunit.client.0.smithi176.stdout:1/566: mkdir d2/de/d10/d16/d47/db0 0 2024-03-13T14:49:53.130 INFO:tasks.workunit.client.0.smithi176.stdout:7/569: dwrite d1/d28/d3a/f14 [0,4194304] 0 2024-03-13T14:49:53.132 INFO:tasks.workunit.client.1.smithi196.stdout:1/778: dread d1/d9/f18 [0,4194304] 0 2024-03-13T14:49:53.135 INFO:tasks.workunit.client.1.smithi196.stdout:7/802: creat d0/d6/db4/f112 x:0 0 0 2024-03-13T14:49:53.137 INFO:tasks.workunit.client.0.smithi176.stdout:3/643: sync 2024-03-13T14:49:53.138 INFO:tasks.workunit.client.1.smithi196.stdout:3/716: creat db/d15/d44/d79/fe6 x:0 0 0 2024-03-13T14:49:53.138 INFO:tasks.workunit.client.0.smithi176.stdout:3/644: fsync d4/f97 0 2024-03-13T14:49:53.140 INFO:tasks.workunit.client.0.smithi176.stdout:3/645: chown d4/d28/d34/da4 0 1 2024-03-13T14:49:53.141 INFO:tasks.workunit.client.1.smithi196.stdout:9/638: write d7/de/d4b/f1a [3831523,88281] 0 2024-03-13T14:49:53.142 INFO:tasks.workunit.client.0.smithi176.stdout:4/652: link d7/d8/d15/c21 d7/d10/d46/ce4 0 2024-03-13T14:49:53.145 INFO:tasks.workunit.client.1.smithi196.stdout:5/766: dwrite d4/d1b/dc8/d14/d19/da7/dc7/fe4 [0,4194304] 0 2024-03-13T14:49:53.146 INFO:tasks.workunit.client.1.smithi196.stdout:4/670: mkdir d1/d3/d7/d35/d76/d58/d69/d9f/dee 0 2024-03-13T14:49:53.147 INFO:tasks.workunit.client.0.smithi176.stdout:2/640: mkdir d1/dc/d2d/d4b/ddb 0 2024-03-13T14:49:53.148 INFO:tasks.workunit.client.1.smithi196.stdout:0/737: dwrite d8/da/f32 [0,4194304] 0 2024-03-13T14:49:53.149 INFO:tasks.workunit.client.0.smithi176.stdout:1/567: stat d2/de/d3f/d3e/l7b 0 2024-03-13T14:49:53.150 INFO:tasks.workunit.client.1.smithi196.stdout:1/779: creat d1/d28/d34/f10f x:0 0 0 2024-03-13T14:49:53.154 INFO:tasks.workunit.client.1.smithi196.stdout:7/803: fsync d0/d9/d1e/d3f/f82 0 2024-03-13T14:49:53.156 INFO:tasks.workunit.client.0.smithi176.stdout:5/554: rename d5/d22/d15/c23 to d5/d2c/cbb 0 2024-03-13T14:49:53.158 INFO:tasks.workunit.client.1.smithi196.stdout:7/804: read - d0/d9/d31/d77/dcf/de0/de7/f108 zero size 2024-03-13T14:49:53.159 INFO:tasks.workunit.client.1.smithi196.stdout:9/639: mknod d7/cdb 0 2024-03-13T14:49:53.165 INFO:tasks.workunit.client.1.smithi196.stdout:1/780: chown d1/d9/ca4 96 1 2024-03-13T14:49:53.165 INFO:tasks.workunit.client.1.smithi196.stdout:9/640: truncate d7/d9/d83/fc0 845565 0 2024-03-13T14:49:53.177 INFO:tasks.workunit.client.1.smithi196.stdout:5/767: unlink d4/cb1 0 2024-03-13T14:49:53.181 INFO:tasks.workunit.client.0.smithi176.stdout:5/555: mknod d5/d31/cbc 0 2024-03-13T14:49:53.183 INFO:tasks.workunit.client.0.smithi176.stdout:9/675: rename d0/db/d89/c30 to d0/dd8/cdf 0 2024-03-13T14:49:53.185 INFO:tasks.workunit.client.1.smithi196.stdout:1/781: mknod d1/d38/c110 0 2024-03-13T14:49:53.186 INFO:tasks.workunit.client.0.smithi176.stdout:6/611: dread d0/de/f57 [0,4194304] 0 2024-03-13T14:49:53.186 INFO:tasks.workunit.client.0.smithi176.stdout:7/570: write d1/d1c/d26/d41/d2b/d40/d64/d94/f7e [4138608,84433] 0 2024-03-13T14:49:53.187 INFO:tasks.workunit.client.1.smithi196.stdout:9/641: creat d7/de/d16/fdc x:0 0 0 2024-03-13T14:49:53.188 INFO:tasks.workunit.client.0.smithi176.stdout:0/509: sync 2024-03-13T14:49:53.189 INFO:tasks.workunit.client.0.smithi176.stdout:3/646: creat d4/d13/d16/dad/fdc x:0 0 0 2024-03-13T14:49:53.191 INFO:tasks.workunit.client.0.smithi176.stdout:8/541: dwrite d0/db/dc/f69 [0,4194304] 0 2024-03-13T14:49:53.191 INFO:tasks.workunit.client.1.smithi196.stdout:4/671: truncate d1/d3/d7/d35/d76/d58/d69/d6c/f7e 706702 0 2024-03-13T14:49:53.195 INFO:tasks.workunit.client.1.smithi196.stdout:3/717: dread db/d71/fa5 [0,4194304] 0 2024-03-13T14:49:53.197 INFO:tasks.workunit.client.1.smithi196.stdout:7/805: creat d0/d6/d35/dff/f113 x:0 0 0 2024-03-13T14:49:53.199 INFO:tasks.workunit.client.1.smithi196.stdout:5/768: symlink d4/d1b/dc8/d3c/d6a/lef 0 2024-03-13T14:49:53.201 INFO:tasks.workunit.client.1.smithi196.stdout:1/782: readlink d1/l6 0 2024-03-13T14:49:53.202 INFO:tasks.workunit.client.0.smithi176.stdout:1/568: creat d2/de/d10/d16/d4a/d5f/da8/fb1 x:0 0 0 2024-03-13T14:49:53.202 INFO:tasks.workunit.client.0.smithi176.stdout:4/653: write d7/d8/d2f/f58 [3079743,93821] 0 2024-03-13T14:49:53.202 INFO:tasks.workunit.client.1.smithi196.stdout:0/738: write d8/da/d25/d26/d35/d3b/d6c/f6e [3040702,53599] 0 2024-03-13T14:49:53.205 INFO:tasks.workunit.client.0.smithi176.stdout:2/641: write d1/dc/d2d/d4b/f63 [868297,118487] 0 2024-03-13T14:49:53.206 INFO:tasks.workunit.client.1.smithi196.stdout:9/642: rename d7/d24/d28/l76 to d7/de/d4b/d1d/d22/dcd/ldd 0 2024-03-13T14:49:53.206 INFO:tasks.workunit.client.0.smithi176.stdout:8/542: dread d0/db/f19 [0,4194304] 0 2024-03-13T14:49:53.208 INFO:tasks.workunit.client.1.smithi196.stdout:0/739: write d8/da/d25/d26/d35/d3b/d59/fea [822906,44056] 0 2024-03-13T14:49:53.210 INFO:tasks.workunit.client.0.smithi176.stdout:9/676: dwrite d0/db/d89/d32/d58/d5f/d84/fb1 [0,4194304] 0 2024-03-13T14:49:53.211 INFO:tasks.workunit.client.0.smithi176.stdout:1/569: write d2/de/d3f/d50/d83/f98 [511900,58868] 0 2024-03-13T14:49:53.213 INFO:tasks.workunit.client.0.smithi176.stdout:8/543: chown d0/db/dc/l1d 551 1 2024-03-13T14:49:53.213 INFO:tasks.workunit.client.0.smithi176.stdout:9/677: readlink d0/d73/db8/lcb 0 2024-03-13T14:49:53.213 INFO:tasks.workunit.client.0.smithi176.stdout:6/612: dwrite d0/d10/d19/d27/fa9 [4194304,4194304] 0 2024-03-13T14:49:53.215 INFO:tasks.workunit.client.1.smithi196.stdout:5/769: creat d4/d1b/dc8/d4a/ff0 x:0 0 0 2024-03-13T14:49:53.216 INFO:tasks.workunit.client.0.smithi176.stdout:9/678: chown d0/d40/d53/caa 12 1 2024-03-13T14:49:53.219 INFO:tasks.workunit.client.1.smithi196.stdout:3/718: mknod db/d19/d22/d3a/ce7 0 2024-03-13T14:49:53.220 INFO:tasks.workunit.client.0.smithi176.stdout:0/510: mkdir d2/db/d39/d9c 0 2024-03-13T14:49:53.222 INFO:tasks.workunit.client.1.smithi196.stdout:3/719: read db/f36 [3798484,107075] 0 2024-03-13T14:49:53.224 INFO:tasks.workunit.client.0.smithi176.stdout:9/679: write d0/db/d89/d32/d58/d9c/fb3 [2873903,52818] 0 2024-03-13T14:49:53.227 INFO:tasks.workunit.client.0.smithi176.stdout:9/680: dread - d0/d40/d53/fba zero size 2024-03-13T14:49:53.229 INFO:tasks.workunit.client.1.smithi196.stdout:9/643: creat d7/fde x:0 0 0 2024-03-13T14:49:53.230 INFO:tasks.workunit.client.0.smithi176.stdout:6/613: dread d0/d10/d19/d27/f68 [0,4194304] 0 2024-03-13T14:49:53.233 INFO:tasks.workunit.client.0.smithi176.stdout:7/571: dread d1/d3f/d4b/f6b [0,4194304] 0 2024-03-13T14:49:53.234 INFO:tasks.workunit.client.1.smithi196.stdout:9/644: stat d7/de/d74/d94/ca2 0 2024-03-13T14:49:53.236 INFO:tasks.workunit.client.1.smithi196.stdout:9/645: fsync d7/d9/d17/f51 0 2024-03-13T14:49:53.237 INFO:tasks.workunit.client.1.smithi196.stdout:5/770: stat d4/d1b/dc8/d14/d19/d26/d33/dde/ce9 0 2024-03-13T14:49:53.239 INFO:tasks.workunit.client.1.smithi196.stdout:7/806: write d0/d4d/f5b [1495663,51945] 0 2024-03-13T14:49:53.240 INFO:tasks.workunit.client.0.smithi176.stdout:5/556: rename d5/d75/la8 to d5/d2e/lbd 0 2024-03-13T14:49:53.241 INFO:tasks.workunit.client.1.smithi196.stdout:7/807: fsync d0/d6/ddc/ffd 0 2024-03-13T14:49:53.243 INFO:tasks.workunit.client.0.smithi176.stdout:3/647: mkdir d4/d13/d16/ddd 0 2024-03-13T14:49:53.243 INFO:tasks.workunit.client.0.smithi176.stdout:6/614: dwrite d0/f4e [0,4194304] 0 2024-03-13T14:49:53.246 INFO:tasks.workunit.client.0.smithi176.stdout:4/654: link d7/d8/d45/fa5 d7/db9/dd8/fe5 0 2024-03-13T14:49:53.253 INFO:tasks.workunit.client.0.smithi176.stdout:2/642: creat d1/dc/d2d/d4b/ddb/fdc x:0 0 0 2024-03-13T14:49:53.254 INFO:tasks.workunit.client.0.smithi176.stdout:8/544: creat d0/db/d39/d56/d93/da7/faa x:0 0 0 2024-03-13T14:49:53.254 INFO:tasks.workunit.client.1.smithi196.stdout:1/783: dwrite d1/d28/f35 [0,4194304] 0 2024-03-13T14:49:53.255 INFO:tasks.workunit.client.0.smithi176.stdout:9/681: mkdir d0/db/d89/d4d/d5d/de0 0 2024-03-13T14:49:53.261 INFO:tasks.workunit.client.0.smithi176.stdout:4/655: dwrite d7/d2d/da7/fae [0,4194304] 0 2024-03-13T14:49:53.262 INFO:tasks.workunit.client.1.smithi196.stdout:5/771: mkdir d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/df1 0 2024-03-13T14:49:53.263 INFO:tasks.workunit.client.1.smithi196.stdout:4/672: rename d1/d3/d7/d35/d76/d58/f70 to d1/d3/d7/ddf/fef 0 2024-03-13T14:49:53.265 INFO:tasks.workunit.client.1.smithi196.stdout:4/673: chown d1/d1e/fb7 288655 1 2024-03-13T14:49:53.268 INFO:tasks.workunit.client.1.smithi196.stdout:4/674: write d1/d3/d7/ddf/fe2 [2946401,83686] 0 2024-03-13T14:49:53.271 INFO:tasks.workunit.client.0.smithi176.stdout:7/572: creat d1/d1c/d26/d41/fad x:0 0 0 2024-03-13T14:49:53.274 INFO:tasks.workunit.client.0.smithi176.stdout:8/545: dwrite d0/db/dc/f6b [0,4194304] 0 2024-03-13T14:49:53.274 INFO:tasks.workunit.client.0.smithi176.stdout:5/557: write d5/d2e/d43/f50 [371431,4561] 0 2024-03-13T14:49:53.277 INFO:tasks.workunit.client.0.smithi176.stdout:5/558: rename d5/d2b to d5/d2b/dbe 22 2024-03-13T14:49:53.281 INFO:tasks.workunit.client.1.smithi196.stdout:0/740: write d8/da/d25/d26/d35/d3b/d5f/f7a [280708,72303] 0 2024-03-13T14:49:53.283 INFO:tasks.workunit.client.0.smithi176.stdout:0/511: mkdir d2/d5/d28/d8c/d9d 0 2024-03-13T14:49:53.285 INFO:tasks.workunit.client.0.smithi176.stdout:1/570: truncate d2/de/d3f/d50/f42 2536993 0 2024-03-13T14:49:53.289 INFO:tasks.workunit.client.0.smithi176.stdout:2/643: unlink d1/dc/d1c/l66 0 2024-03-13T14:49:53.289 INFO:tasks.workunit.client.1.smithi196.stdout:7/808: write d0/d6/d33/f96 [461953,56210] 0 2024-03-13T14:49:53.291 INFO:tasks.workunit.client.0.smithi176.stdout:9/682: mkdir d0/d40/d53/d6d/de1 0 2024-03-13T14:49:53.292 INFO:tasks.workunit.client.1.smithi196.stdout:2/634: sync 2024-03-13T14:49:53.295 INFO:tasks.workunit.client.0.smithi176.stdout:4/656: creat d7/d10/d46/d4a/fe6 x:0 0 0 2024-03-13T14:49:53.295 INFO:tasks.workunit.client.0.smithi176.stdout:9/683: chown d0/d1e/d6f/f7b 31 1 2024-03-13T14:49:53.296 INFO:tasks.workunit.client.1.smithi196.stdout:9/646: dwrite d7/de/d16/d70/f89 [0,4194304] 0 2024-03-13T14:49:53.299 INFO:tasks.workunit.client.1.smithi196.stdout:5/772: write d4/d1b/dc8/d14/d19/d1c/fa2 [2640593,107358] 0 2024-03-13T14:49:53.300 INFO:tasks.workunit.client.1.smithi196.stdout:7/809: truncate d0/d9/d31/d50/fed 25602 0 2024-03-13T14:49:53.301 INFO:tasks.workunit.client.1.smithi196.stdout:7/810: stat d0/d4d 0 2024-03-13T14:49:53.304 INFO:tasks.workunit.client.1.smithi196.stdout:0/741: dread d8/da/f32 [0,4194304] 0 2024-03-13T14:49:53.304 INFO:tasks.workunit.client.0.smithi176.stdout:3/648: dwrite d4/d13/d16/d3d/f93 [0,4194304] 0 2024-03-13T14:49:53.305 INFO:tasks.workunit.client.0.smithi176.stdout:5/559: creat d5/d31/d47/d80/fbf x:0 0 0 2024-03-13T14:49:53.310 INFO:tasks.workunit.client.1.smithi196.stdout:0/742: stat d8/da/d25/d26/d35/d3b/d59/d6a/f75 0 2024-03-13T14:49:53.310 INFO:tasks.workunit.client.0.smithi176.stdout:5/560: truncate d5/d31/d47/f85 721905 0 2024-03-13T14:49:53.314 INFO:tasks.workunit.client.0.smithi176.stdout:8/546: rename d0/db/d58 to d0/d24/d7b/dab 0 2024-03-13T14:49:53.314 INFO:tasks.workunit.client.1.smithi196.stdout:0/743: stat d8/da/d25/d26/d35/d3b/d59/fea 0 2024-03-13T14:49:53.317 INFO:tasks.workunit.client.1.smithi196.stdout:3/720: getdents db/d71 0 2024-03-13T14:49:53.317 INFO:tasks.workunit.client.0.smithi176.stdout:6/615: dread d0/d10/f3f [0,4194304] 0 2024-03-13T14:49:53.320 INFO:tasks.workunit.client.0.smithi176.stdout:1/571: mknod d2/de/d3f/d19/cb2 0 2024-03-13T14:49:53.321 INFO:tasks.workunit.client.1.smithi196.stdout:3/721: read db/f13 [3906333,18290] 0 2024-03-13T14:49:53.329 INFO:tasks.workunit.client.0.smithi176.stdout:7/573: symlink d1/d28/d7c/lae 0 2024-03-13T14:49:53.331 INFO:tasks.workunit.client.1.smithi196.stdout:4/675: dwrite d1/d3/d7/d35/f52 [0,4194304] 0 2024-03-13T14:49:53.336 INFO:tasks.workunit.client.0.smithi176.stdout:2/644: rename d1/f14 to d1/dc/d2d/d59/d8b/fdd 0 2024-03-13T14:49:53.337 INFO:tasks.workunit.client.0.smithi176.stdout:8/547: mknod d0/d24/d29/d70/cac 0 2024-03-13T14:49:53.338 INFO:tasks.workunit.client.1.smithi196.stdout:2/635: symlink d0/d7/d9/lcf 0 2024-03-13T14:49:53.339 INFO:tasks.workunit.client.0.smithi176.stdout:9/684: dread d0/d40/d53/fb5 [0,4194304] 0 2024-03-13T14:49:53.339 INFO:tasks.workunit.client.0.smithi176.stdout:0/512: creat d2/db/d39/d9c/f9e x:0 0 0 2024-03-13T14:49:53.340 INFO:tasks.workunit.client.0.smithi176.stdout:0/513: readlink d2/db/d35/l4f 0 2024-03-13T14:49:53.341 INFO:tasks.workunit.client.1.smithi196.stdout:9/647: mknod d7/d24/cdf 0 2024-03-13T14:49:53.341 INFO:tasks.workunit.client.0.smithi176.stdout:0/514: fdatasync d2/db/f3b 0 2024-03-13T14:49:53.342 INFO:tasks.workunit.client.0.smithi176.stdout:9/685: dread - d0/d40/d53/d6d/d97/fbc zero size 2024-03-13T14:49:53.342 INFO:tasks.workunit.client.1.smithi196.stdout:3/722: fsync db/d71/fa5 0 2024-03-13T14:49:53.342 INFO:tasks.workunit.client.0.smithi176.stdout:0/515: stat d2/db/d35 0 2024-03-13T14:49:53.343 INFO:tasks.workunit.client.1.smithi196.stdout:1/784: getdents d1/d9/d4e/d55/d63/db9 0 2024-03-13T14:49:53.343 INFO:tasks.workunit.client.1.smithi196.stdout:8/656: sync 2024-03-13T14:49:53.343 INFO:tasks.workunit.client.1.smithi196.stdout:6/865: sync 2024-03-13T14:49:53.344 INFO:tasks.workunit.client.0.smithi176.stdout:0/516: chown d2/d5/d28/d8c/d9d 0 1 2024-03-13T14:49:53.346 INFO:tasks.workunit.client.0.smithi176.stdout:9/686: write d0/dd8/d82/fb7 [217038,121911] 0 2024-03-13T14:49:53.348 INFO:tasks.workunit.client.0.smithi176.stdout:4/657: fsync d7/d8/d15/d23/f35 0 2024-03-13T14:49:53.348 INFO:tasks.workunit.client.0.smithi176.stdout:7/574: fdatasync d1/d3f/d81/d9a/f9f 0 2024-03-13T14:49:53.348 INFO:tasks.workunit.client.1.smithi196.stdout:6/866: write d6/d2a/d23/d28/f109 [871409,56715] 0 2024-03-13T14:49:53.349 INFO:tasks.workunit.client.0.smithi176.stdout:9/687: fdatasync d0/db/d89/d32/d58/d5f/d84/d9a/f9b 0 2024-03-13T14:49:53.350 INFO:tasks.workunit.client.1.smithi196.stdout:6/867: readlink d6/d7/dd/dba/l100 0 2024-03-13T14:49:53.354 INFO:tasks.workunit.client.0.smithi176.stdout:3/649: mkdir d4/d28/d34/dde 0 2024-03-13T14:49:53.356 INFO:tasks.workunit.client.1.smithi196.stdout:4/676: creat d1/d3/d7/d35/d76/d58/d69/d6c/ff0 x:0 0 0 2024-03-13T14:49:53.356 INFO:tasks.workunit.client.0.smithi176.stdout:8/548: dwrite d0/d24/d29/f4e [8388608,4194304] 0 2024-03-13T14:49:53.356 INFO:tasks.workunit.client.0.smithi176.stdout:3/650: chown d4/d13/d81/d89/f32 3159 1 2024-03-13T14:49:53.360 INFO:tasks.workunit.client.0.smithi176.stdout:2/645: creat d1/d2a/fde x:0 0 0 2024-03-13T14:49:53.360 INFO:tasks.workunit.client.1.smithi196.stdout:2/636: symlink d0/d6/d1d/ld0 0 2024-03-13T14:49:53.366 INFO:tasks.workunit.client.0.smithi176.stdout:6/616: symlink d0/d10/d19/d27/d2b/d2e/db7/lc9 0 2024-03-13T14:49:53.368 INFO:tasks.workunit.client.1.smithi196.stdout:8/657: symlink d0/d10/d12/d17/d42/le2 0 2024-03-13T14:49:53.369 INFO:tasks.workunit.client.0.smithi176.stdout:1/572: mkdir d2/de/d3f/d3e/d86/db3 0 2024-03-13T14:49:53.371 INFO:tasks.workunit.client.1.smithi196.stdout:0/744: creat d8/da/d25/d26/d54/ff3 x:0 0 0 2024-03-13T14:49:53.372 INFO:tasks.workunit.client.1.smithi196.stdout:1/785: truncate d1/d9/d33/d4b/d66/d7b/f8d 346695 0 2024-03-13T14:49:53.372 INFO:tasks.workunit.client.0.smithi176.stdout:3/651: dwrite d4/d28/d31/fc6 [0,4194304] 0 2024-03-13T14:49:53.374 INFO:tasks.workunit.client.0.smithi176.stdout:3/652: stat d4/d28/c58 0 2024-03-13T14:49:53.375 INFO:tasks.workunit.client.0.smithi176.stdout:3/653: chown d4/d28/d31/d45/c77 910 1 2024-03-13T14:49:53.378 INFO:tasks.workunit.client.1.smithi196.stdout:4/677: rename d1/d3/d7/d35/d76/f32 to d1/d1e/d5c/ff1 0 2024-03-13T14:49:53.379 INFO:tasks.workunit.client.1.smithi196.stdout:0/745: write d8/da/d25/d26/d35/d3b/d5f/fca [978841,76157] 0 2024-03-13T14:49:53.381 INFO:tasks.workunit.client.1.smithi196.stdout:7/811: dwrite d0/d6/f21 [0,4194304] 0 2024-03-13T14:49:53.382 INFO:tasks.workunit.client.1.smithi196.stdout:2/637: write d0/d50/fa2 [892945,41974] 0 2024-03-13T14:49:53.382 INFO:tasks.workunit.client.0.smithi176.stdout:9/688: write d0/d1e/d1f/d68/faf [180168,26970] 0 2024-03-13T14:49:53.384 INFO:tasks.workunit.client.0.smithi176.stdout:9/689: readlink d0/db/d89/d32/d58/d5f/l79 0 2024-03-13T14:49:53.385 INFO:tasks.workunit.client.1.smithi196.stdout:7/812: chown d0/d6/d33 3682174 1 2024-03-13T14:49:53.385 INFO:tasks.workunit.client.0.smithi176.stdout:8/549: mknod d0/db/d39/d56/d91/cad 0 2024-03-13T14:49:53.388 INFO:tasks.workunit.client.1.smithi196.stdout:2/638: truncate d0/d6/fb3 308437 0 2024-03-13T14:49:53.390 INFO:tasks.workunit.client.0.smithi176.stdout:5/561: truncate d5/d2c/f66 667459 0 2024-03-13T14:49:53.394 INFO:tasks.workunit.client.1.smithi196.stdout:4/678: dread - d1/d3/d7/ddf/fef zero size 2024-03-13T14:49:53.399 INFO:tasks.workunit.client.0.smithi176.stdout:1/573: mkdir d2/de/d10/d16/d47/db4 0 2024-03-13T14:49:53.402 INFO:tasks.workunit.client.1.smithi196.stdout:4/679: read - d1/d3/d7/ddf/fef zero size 2024-03-13T14:49:53.403 INFO:tasks.workunit.client.0.smithi176.stdout:0/517: mkdir d2/d74/d9f 0 2024-03-13T14:49:53.406 INFO:tasks.workunit.client.1.smithi196.stdout:9/648: write d7/d24/dbf/f6b [186605,79405] 0 2024-03-13T14:49:53.407 INFO:tasks.workunit.client.1.smithi196.stdout:3/723: write db/d15/d44/d51/fb4 [428745,37013] 0 2024-03-13T14:49:53.412 INFO:tasks.workunit.client.0.smithi176.stdout:3/654: unlink d4/d28/c3e 0 2024-03-13T14:49:53.414 INFO:tasks.workunit.client.1.smithi196.stdout:7/813: creat d0/d9/d1e/d3f/d6c/d106/f114 x:0 0 0 2024-03-13T14:49:53.415 INFO:tasks.workunit.client.1.smithi196.stdout:8/658: dread d0/f51 [0,4194304] 0 2024-03-13T14:49:53.416 INFO:tasks.workunit.client.1.smithi196.stdout:0/746: rename d8/da/d25/d26/d35/d3b/d5f/le2 to d8/da/d25/d26/d35/d3b/lf4 0 2024-03-13T14:49:53.421 INFO:tasks.workunit.client.1.smithi196.stdout:6/868: write d6/d7/df/d27/d39/d44/d74/f87 [2004184,5041] 0 2024-03-13T14:49:53.422 INFO:tasks.workunit.client.1.smithi196.stdout:3/724: write db/d19/d4a/dc5/fd7 [142426,36072] 0 2024-03-13T14:49:53.424 INFO:tasks.workunit.client.0.smithi176.stdout:4/658: write d7/d8/d33/fd2 [575491,87458] 0 2024-03-13T14:49:53.424 INFO:tasks.workunit.client.1.smithi196.stdout:9/649: mkdir d7/de/d4b/d1d/d93/de0 0 2024-03-13T14:49:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:53 smithi196 ceph-mon[37486]: pgmap v15: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 57 MiB/s rd, 163 MiB/s wr, 334 op/s 2024-03-13T14:49:53.430 INFO:tasks.workunit.client.1.smithi196.stdout:7/814: fsync d0/d9/d31/d77/d93/d94/f9e 0 2024-03-13T14:49:53.430 INFO:tasks.workunit.client.0.smithi176.stdout:8/550: dread d0/d5/d11/f76 [0,4194304] 0 2024-03-13T14:49:53.433 INFO:tasks.workunit.client.1.smithi196.stdout:8/659: symlink d0/d10/d12/d1e/d2d/dd8/le3 0 2024-03-13T14:49:53.433 INFO:tasks.workunit.client.1.smithi196.stdout:7/815: chown d0/d9/d31/d77/d93/fc6 7080620 1 2024-03-13T14:49:53.437 INFO:tasks.workunit.client.1.smithi196.stdout:0/747: fdatasync d8/da/d25/d26/d54/d88/f84 0 2024-03-13T14:49:53.438 INFO:tasks.workunit.client.1.smithi196.stdout:4/680: dwrite d1/d3/d7/d35/d76/d58/d69/d9f/fe9 [0,4194304] 0 2024-03-13T14:49:53.438 INFO:tasks.workunit.client.0.smithi176.stdout:6/617: mkdir d0/d10/dca 0 2024-03-13T14:49:53.439 INFO:tasks.workunit.client.1.smithi196.stdout:7/816: fsync d0/d6/d35/dff/f113 0 2024-03-13T14:49:53.441 INFO:tasks.workunit.client.1.smithi196.stdout:3/725: symlink db/d19/d4a/dc5/le8 0 2024-03-13T14:49:53.442 INFO:tasks.workunit.client.1.smithi196.stdout:2/639: dread d0/d6/d4b/f7d [0,4194304] 0 2024-03-13T14:49:53.444 INFO:tasks.workunit.client.1.smithi196.stdout:4/681: stat d1/d3/d7/d35/d76/d58/d69/d9f/fe9 0 2024-03-13T14:49:53.444 INFO:tasks.workunit.client.0.smithi176.stdout:0/518: symlink d2/db/d39/d9c/la0 0 2024-03-13T14:49:53.445 INFO:tasks.workunit.client.0.smithi176.stdout:5/562: dwrite d5/d31/d47/d80/f4d [0,4194304] 0 2024-03-13T14:49:53.446 INFO:tasks.workunit.client.0.smithi176.stdout:1/574: dread d2/de/d10/d16/d4a/f8a [0,4194304] 0 2024-03-13T14:49:53.455 INFO:tasks.workunit.client.1.smithi196.stdout:1/786: dwrite d1/f2a [0,4194304] 0 2024-03-13T14:49:53.456 INFO:tasks.workunit.client.0.smithi176.stdout:9/690: creat d0/d40/d53/d6d/de1/fe2 x:0 0 0 2024-03-13T14:49:53.457 INFO:tasks.workunit.client.1.smithi196.stdout:8/660: mkdir d0/d10/da5/de4 0 2024-03-13T14:49:53.458 INFO:tasks.workunit.client.0.smithi176.stdout:4/659: creat d7/db9/fe7 x:0 0 0 2024-03-13T14:49:53.459 INFO:tasks.workunit.client.1.smithi196.stdout:9/650: getdents d7/de/d4b/dd1 0 2024-03-13T14:49:53.462 INFO:tasks.workunit.client.0.smithi176.stdout:8/551: unlink d0/db/dc/l2d 0 2024-03-13T14:49:53.464 INFO:tasks.workunit.client.0.smithi176.stdout:8/552: chown d0/d24/d29/c55 51 1 2024-03-13T14:49:53.465 INFO:tasks.workunit.client.0.smithi176.stdout:5/563: dwrite d5/d31/d47/d80/f4d [0,4194304] 0 2024-03-13T14:49:53.467 INFO:tasks.workunit.client.1.smithi196.stdout:7/817: mkdir d0/d9/d1e/d58/d72/da9/da3/d115 0 2024-03-13T14:49:53.469 INFO:tasks.workunit.client.1.smithi196.stdout:4/682: dwrite d1/d3/d4c/d72/d7d/d80/d55/fc1 [0,4194304] 0 2024-03-13T14:49:53.471 INFO:tasks.workunit.client.1.smithi196.stdout:9/651: creat d7/de/d16/fe1 x:0 0 0 2024-03-13T14:49:53.473 INFO:tasks.workunit.client.1.smithi196.stdout:2/640: creat d0/d7/d9/d9a/daa/fd1 x:0 0 0 2024-03-13T14:49:53.473 INFO:tasks.workunit.client.0.smithi176.stdout:7/575: getdents d1/d3f/d65 0 2024-03-13T14:49:53.476 INFO:tasks.workunit.client.1.smithi196.stdout:8/661: link d0/d10/d70/fcc d0/d10/d12/d1e/fe5 0 2024-03-13T14:49:53.480 INFO:tasks.workunit.client.1.smithi196.stdout:8/662: symlink d0/d16/d3d/d9c/le6 0 2024-03-13T14:49:53.481 INFO:tasks.workunit.client.0.smithi176.stdout:9/691: mkdir d0/d1e/d8d/d9f/de3 0 2024-03-13T14:49:53.483 INFO:tasks.workunit.client.0.smithi176.stdout:7/576: creat d1/faf x:0 0 0 2024-03-13T14:49:53.485 INFO:tasks.workunit.client.0.smithi176.stdout:7/577: stat d1/d3f/d4b/d63/d7a 0 2024-03-13T14:49:53.486 INFO:tasks.workunit.client.1.smithi196.stdout:8/663: mknod d0/d10/d12/d17/ce7 0 2024-03-13T14:49:53.487 INFO:tasks.workunit.client.1.smithi196.stdout:2/641: getdents d0/d7/d9/d61/dc7 0 2024-03-13T14:49:53.491 INFO:tasks.workunit.client.1.smithi196.stdout:9/652: dwrite d7/de/d16/f60 [4194304,4194304] 0 2024-03-13T14:49:53.493 INFO:tasks.workunit.client.0.smithi176.stdout:3/655: getdents d4/d13/d2d/d50 0 2024-03-13T14:49:53.493 INFO:tasks.workunit.client.1.smithi196.stdout:2/642: read d0/d6/d4b/d5f/f81 [4113755,74865] 0 2024-03-13T14:49:53.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:53 smithi176 ceph-mon[29064]: pgmap v15: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 57 MiB/s rd, 163 MiB/s wr, 334 op/s 2024-03-13T14:49:53.495 INFO:tasks.workunit.client.0.smithi176.stdout:1/575: getdents d2/de/d3f/d3e/d86 0 2024-03-13T14:49:53.495 INFO:tasks.workunit.client.0.smithi176.stdout:3/656: dread - d4/d28/d31/d45/fc9 zero size 2024-03-13T14:49:53.496 INFO:tasks.workunit.client.0.smithi176.stdout:8/553: dwrite d0/d5/d11/f7f [0,4194304] 0 2024-03-13T14:49:53.497 INFO:tasks.workunit.client.0.smithi176.stdout:1/576: readlink d2/de/d3f/d19/l96 0 2024-03-13T14:49:53.497 INFO:tasks.workunit.client.1.smithi196.stdout:9/653: symlink d7/le2 0 2024-03-13T14:49:53.497 INFO:tasks.workunit.client.0.smithi176.stdout:8/554: readlink d0/d24/l85 0 2024-03-13T14:49:53.499 INFO:tasks.workunit.client.0.smithi176.stdout:8/555: write d0/d24/f57 [1488672,129810] 0 2024-03-13T14:49:53.499 INFO:tasks.workunit.client.1.smithi196.stdout:8/664: getdents d0/d10/d12/d1e/db1 0 2024-03-13T14:49:53.500 INFO:tasks.workunit.client.1.smithi196.stdout:9/654: chown d7/lb 1 1 2024-03-13T14:49:53.501 INFO:tasks.workunit.client.1.smithi196.stdout:2/643: creat d0/d7/d9/d61/dc7/fd2 x:0 0 0 2024-03-13T14:49:53.512 INFO:tasks.workunit.client.1.smithi196.stdout:9/655: mknod d7/de/ce3 0 2024-03-13T14:49:53.519 INFO:tasks.workunit.client.1.smithi196.stdout:9/656: mkdir d7/d24/d28/de4 0 2024-03-13T14:49:53.520 INFO:tasks.workunit.client.1.smithi196.stdout:8/665: getdents d0/d10/d70 0 2024-03-13T14:49:53.523 INFO:tasks.workunit.client.0.smithi176.stdout:7/578: creat d1/d1c/d26/fb0 x:0 0 0 2024-03-13T14:49:53.524 INFO:tasks.workunit.client.1.smithi196.stdout:8/666: dread - d0/d16/d3d/d9c/d87/d8c/da9/db8/fcd zero size 2024-03-13T14:49:53.525 INFO:tasks.workunit.client.0.smithi176.stdout:7/579: stat d1/d1c/d26/d41/d2b/d40/l49 0 2024-03-13T14:49:53.525 INFO:tasks.workunit.client.0.smithi176.stdout:9/692: rename d0/dd8/cdf to d0/d40/d53/d6d/ce4 0 2024-03-13T14:49:53.526 INFO:tasks.workunit.client.0.smithi176.stdout:9/693: fdatasync d0/d1e/d1f/d68/fb4 0 2024-03-13T14:49:53.527 INFO:tasks.workunit.client.0.smithi176.stdout:1/577: creat d2/de/d3f/d6a/fb5 x:0 0 0 2024-03-13T14:49:53.528 INFO:tasks.workunit.client.0.smithi176.stdout:8/556: creat d0/db/d39/fae x:0 0 0 2024-03-13T14:49:53.528 INFO:tasks.workunit.client.0.smithi176.stdout:1/578: fsync d2/faa 0 2024-03-13T14:49:53.529 INFO:tasks.workunit.client.1.smithi196.stdout:7/818: dread d0/d9/f30 [0,4194304] 0 2024-03-13T14:49:53.529 INFO:tasks.workunit.client.0.smithi176.stdout:8/557: stat d0/db/dc 0 2024-03-13T14:49:53.530 INFO:tasks.workunit.client.1.smithi196.stdout:7/819: dread - d0/d6/db4/f112 zero size 2024-03-13T14:49:53.530 INFO:tasks.workunit.client.0.smithi176.stdout:8/558: stat d0/d5/l5d 0 2024-03-13T14:49:53.533 INFO:tasks.workunit.client.1.smithi196.stdout:9/657: rename d7/de/d4b/d1d/d22/dcd to d7/de/de5 0 2024-03-13T14:49:53.534 INFO:tasks.workunit.client.0.smithi176.stdout:7/580: symlink d1/d28/d3a/d1b/lb1 0 2024-03-13T14:49:53.536 INFO:tasks.workunit.client.0.smithi176.stdout:7/581: chown d1/d1c/d26/d41/d2b/f85 0 1 2024-03-13T14:49:53.536 INFO:tasks.workunit.client.0.smithi176.stdout:1/579: mknod d2/de/d3f/d3e/d86/cb6 0 2024-03-13T14:49:53.538 INFO:tasks.workunit.client.0.smithi176.stdout:1/580: stat d2/de/d10/d16/d59 0 2024-03-13T14:49:53.538 INFO:tasks.workunit.client.1.smithi196.stdout:9/658: unlink d7/d9/d17/dbc/l91 0 2024-03-13T14:49:53.540 INFO:tasks.workunit.client.0.smithi176.stdout:7/582: mkdir d1/d1c/d26/d41/db2 0 2024-03-13T14:49:53.542 INFO:tasks.workunit.client.0.smithi176.stdout:1/581: chown d2/de/d10/f6d 748 1 2024-03-13T14:49:53.544 INFO:tasks.workunit.client.0.smithi176.stdout:7/583: creat d1/d3f/d4b/d63/fb3 x:0 0 0 2024-03-13T14:49:53.547 INFO:tasks.workunit.client.0.smithi176.stdout:7/584: creat d1/d3f/d4b/d63/fb4 x:0 0 0 2024-03-13T14:49:53.548 INFO:tasks.workunit.client.1.smithi196.stdout:7/820: dread d0/d78/d81/dfb/ffc [0,4194304] 0 2024-03-13T14:49:53.548 INFO:tasks.workunit.client.0.smithi176.stdout:9/694: dwrite d0/d1e/d8d/dbe/fde [0,4194304] 0 2024-03-13T14:49:53.551 INFO:tasks.workunit.client.1.smithi196.stdout:7/821: fsync d0/d9/d1e/d58/d61/fd4 0 2024-03-13T14:49:53.551 INFO:tasks.workunit.client.0.smithi176.stdout:9/695: write d0/db/d89/d32/f62 [2548990,94049] 0 2024-03-13T14:49:53.555 INFO:tasks.workunit.client.1.smithi196.stdout:7/822: getdents d0/d9/d1e/d58/d72/da9/da3/d115 0 2024-03-13T14:49:53.557 INFO:tasks.workunit.client.0.smithi176.stdout:7/585: link d1/c2c d1/d3f/d76/cb5 0 2024-03-13T14:49:53.559 INFO:tasks.workunit.client.0.smithi176.stdout:9/696: link d0/d40/d53/fba d0/db/d89/d90/fe5 0 2024-03-13T14:49:53.562 INFO:tasks.workunit.client.1.smithi196.stdout:7/823: link d0/d6/d33/d57/f85 d0/d9/d1e/d58/d72/d99/f116 0 2024-03-13T14:49:53.564 INFO:tasks.workunit.client.0.smithi176.stdout:9/697: truncate d0/db/d89/d32/d58/d67/fbb 419466 0 2024-03-13T14:49:53.567 INFO:tasks.workunit.client.0.smithi176.stdout:9/698: mkdir d0/db/d89/d90/de6 0 2024-03-13T14:49:53.569 INFO:tasks.workunit.client.0.smithi176.stdout:9/699: creat d0/d1e/d8d/fe7 x:0 0 0 2024-03-13T14:49:53.575 INFO:tasks.workunit.client.1.smithi196.stdout:7/824: dwrite d0/d9/d1e/d58/d72/da9/d47/f10e [0,4194304] 0 2024-03-13T14:49:53.580 INFO:tasks.workunit.client.1.smithi196.stdout:7/825: link d0/d4d/dd1/dd6/fe5 d0/d6/d33/d57/d8a/d9f/f117 0 2024-03-13T14:49:53.582 INFO:tasks.workunit.client.1.smithi196.stdout:7/826: unlink d0/d9/d31/d50/fed 0 2024-03-13T14:49:53.585 INFO:tasks.workunit.client.1.smithi196.stdout:7/827: mkdir d0/d9/d31/d77/dcf/de0/de7/d118 0 2024-03-13T14:49:53.613 INFO:tasks.workunit.client.1.smithi196.stdout:7/828: dread d0/d9/d1e/d3f/f7e [0,4194304] 0 2024-03-13T14:49:53.620 INFO:tasks.workunit.client.1.smithi196.stdout:7/829: link d0/d9/d1e/d3f/cc5 d0/d9/d1e/d58/d72/d6e/c119 0 2024-03-13T14:49:53.625 INFO:tasks.workunit.client.1.smithi196.stdout:7/830: creat d0/d9/d31/d77/f11a x:0 0 0 2024-03-13T14:49:53.641 INFO:tasks.workunit.client.0.smithi176.stdout:8/559: read d0/d5/d11/f26 [3098626,86003] 0 2024-03-13T14:49:53.650 INFO:tasks.workunit.client.0.smithi176.stdout:8/560: symlink d0/db/d39/d56/d91/laf 0 2024-03-13T14:49:53.651 INFO:tasks.workunit.client.0.smithi176.stdout:2/646: truncate d1/dc/d2d/d59/d80/d7e/da3/fb9 3586063 0 2024-03-13T14:49:53.653 INFO:tasks.workunit.client.0.smithi176.stdout:0/519: rmdir d2 39 2024-03-13T14:49:53.655 INFO:tasks.workunit.client.0.smithi176.stdout:8/561: mknod d0/db/d39/d56/d91/cb0 0 2024-03-13T14:49:53.659 INFO:tasks.workunit.client.0.smithi176.stdout:2/647: readlink d1/dc/d2d/d90/dcf/l92 0 2024-03-13T14:49:53.662 INFO:tasks.workunit.client.1.smithi196.stdout:0/748: write d8/da/d25/d26/f60 [50871,105163] 0 2024-03-13T14:49:53.671 INFO:tasks.workunit.client.1.smithi196.stdout:0/749: symlink d8/da/d25/d26/d35/d3b/d50/d5d/da5/lf5 0 2024-03-13T14:49:53.674 INFO:tasks.workunit.client.0.smithi176.stdout:5/564: write d5/d2c/f66 [1331548,9918] 0 2024-03-13T14:49:53.680 INFO:tasks.workunit.client.1.smithi196.stdout:0/750: link d8/da/d25/d26/d35/d3b/d59/d6a/f8d d8/da/d25/d26/d48/dba/ff6 0 2024-03-13T14:49:53.685 INFO:tasks.workunit.client.1.smithi196.stdout:2/644: truncate d0/f77 3974012 0 2024-03-13T14:49:53.685 INFO:tasks.workunit.client.0.smithi176.stdout:5/565: mkdir d5/d31/d86/dc0 0 2024-03-13T14:49:53.689 INFO:tasks.workunit.client.0.smithi176.stdout:4/660: dwrite d7/db9/dd8/fe5 [0,4194304] 0 2024-03-13T14:49:53.692 INFO:tasks.workunit.client.0.smithi176.stdout:1/582: dread d2/de/d10/f72 [0,4194304] 0 2024-03-13T14:49:53.692 INFO:tasks.workunit.client.1.smithi196.stdout:2/645: mkdir d0/d7/d9/d61/dd3 0 2024-03-13T14:49:53.693 INFO:tasks.workunit.client.0.smithi176.stdout:3/657: write d4/d13/d81/d89/d5b/dc5/fc7 [1070073,111248] 0 2024-03-13T14:49:53.696 INFO:tasks.workunit.client.0.smithi176.stdout:1/583: chown d2/de/d10/d16/d59/fa1 14945385 1 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:9/659: write d7/de/d16/d70/f7a [1176324,90444] 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:8/667: dwrite d0/d16/f86 [0,4194304] 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:8/668: mknod d0/d16/d3d/d8b/ce8 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:8/669: truncate d0/d10/d12/d17/f57 4941905 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:8/670: dread - d0/d10/da5/db2/dc8/fd4 zero size 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:2/646: dread d0/d6/d1d/f7f [4194304,4194304] 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:8/671: truncate d0/d16/d3d/d8b/d48/f5f 4422748 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:7/831: getdents d0/d9/d31/d77/dcf/de0/de7/d118 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:0/751: getdents d8/da/d25/d26/d35/d3b/d50/d5d/da5 0 2024-03-13T14:49:53.783 INFO:tasks.workunit.client.1.smithi196.stdout:5/773: sync 2024-03-13T14:49:53.784 INFO:tasks.workunit.client.1.smithi196.stdout:0/752: mknod d8/da/d25/d26/d35/d3b/d6c/cf7 0 2024-03-13T14:49:53.784 INFO:tasks.workunit.client.1.smithi196.stdout:0/753: truncate d8/da/d25/d26/d54/ff3 539345 0 2024-03-13T14:49:53.784 INFO:tasks.workunit.client.1.smithi196.stdout:0/754: chown d8/da/d25 7 1 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:5/566: symlink d5/d22/lc1 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:1/584: fdatasync d2/de/f17 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:2/648: dread d1/dc/d2d/d4b/d69/f94 [0,4194304] 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:1/585: truncate d2/de/d3f/d50/f78 5150266 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:1/586: readlink d2/de/d10/d16/d59/l5d 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:1/587: stat d2/c20 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:5/567: link d5/d31/d47/f85 d5/d31/d86/dc0/fc2 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:2/649: fdatasync d1/dc/fa5 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:3/658: dwrite d4/d13/d16/d3d/fb7 [0,4194304] 0 2024-03-13T14:49:53.786 INFO:tasks.workunit.client.0.smithi176.stdout:1/588: creat d2/d30/d63/d22/d71/fb7 x:0 0 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:5/568: chown d5/d31/d47/d80/cb1 4016771 1 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:2/650: symlink d1/dc/d2d/d59/d8b/daf/db2/ldf 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:7/586: write d1/d7/f3d [1250115,81845] 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:5/569: symlink d5/d2b/lc3 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:3/659: dwrite d4/d28/d31/fc3 [0,4194304] 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:5/570: creat d5/d2c/d40/d6a/d89/fc4 x:0 0 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:9/700: dwrite d0/d17/f48 [0,4194304] 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:5/571: chown d5/d2e/d7e/dad/d7b/cba 1396310 1 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:7/587: dread d1/d3f/d4b/f6b [0,4194304] 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:3/660: mkdir d4/d13/d16/ddf 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:3/661: readlink d4/d28/d5a/l85 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:5/572: chown d5/d2e/c42 15399 1 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:9/701: rename d0/db/d89/d4d/d6e/d81 to d0/d1e/d8d/dbe/de8 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:3/662: write d4/d13/d16/d3d/fb7 [2734541,81259] 0 2024-03-13T14:49:53.787 INFO:tasks.workunit.client.0.smithi176.stdout:3/663: write d4/d28/f67 [2810358,99334] 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:9/702: symlink d0/d1e/d8d/d9f/le9 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:3/664: symlink d4/d13/d81/le0 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:3/665: dread d4/d13/d81/d89/f27 [0,4194304] 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:0/520: dwrite d2/d5/d28/f54 [0,4194304] 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:9/703: unlink d0/d40/d53/fad 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:4/661: write d7/d8/f4f [977754,77241] 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:6/618: sync 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:4/662: dread - d7/d10/d46/d4a/d5d/fc8 zero size 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:3/666: creat d4/d13/d5e/fe1 x:0 0 0 2024-03-13T14:49:53.788 INFO:tasks.workunit.client.0.smithi176.stdout:2/651: dread d1/dc/d1c/f4c [0,4194304] 0 2024-03-13T14:49:53.789 INFO:tasks.workunit.client.1.smithi196.stdout:9/660: write d7/de/f61 [121804,53018] 0 2024-03-13T14:49:53.789 INFO:tasks.workunit.client.0.smithi176.stdout:0/521: mknod d2/db/d39/d9c/ca1 0 2024-03-13T14:49:53.790 INFO:tasks.workunit.client.0.smithi176.stdout:9/704: creat d0/db/d89/d4d/d6e/fea x:0 0 0 2024-03-13T14:49:53.794 INFO:tasks.workunit.client.0.smithi176.stdout:6/619: fdatasync d0/d10/d19/d27/f4a 0 2024-03-13T14:49:53.795 INFO:tasks.workunit.client.0.smithi176.stdout:3/667: stat d4/d13/d81/d89/d5b/c6f 0 2024-03-13T14:49:53.797 INFO:tasks.workunit.client.0.smithi176.stdout:1/589: dwrite d2/d30/f4c [0,4194304] 0 2024-03-13T14:49:53.797 INFO:tasks.workunit.client.0.smithi176.stdout:2/652: truncate d1/dc/fa5 118622 0 2024-03-13T14:49:53.801 INFO:tasks.workunit.client.0.smithi176.stdout:5/573: dread d5/d2e/d7e/d9f/f4e [0,4194304] 0 2024-03-13T14:49:53.804 INFO:tasks.workunit.client.1.smithi196.stdout:2/647: dwrite d0/d7/f17 [0,4194304] 0 2024-03-13T14:49:53.804 INFO:tasks.workunit.client.0.smithi176.stdout:4/663: mkdir d7/d10/d46/d4a/d5d/da1/de8 0 2024-03-13T14:49:53.805 INFO:tasks.workunit.client.0.smithi176.stdout:5/574: truncate d5/d31/d86/fb4 81519 0 2024-03-13T14:49:53.807 INFO:tasks.workunit.client.0.smithi176.stdout:4/664: write d7/d8/d33/f6d [4869017,25560] 0 2024-03-13T14:49:53.807 INFO:tasks.workunit.client.1.smithi196.stdout:2/648: chown d0/d6/d1d/f27 7997469 1 2024-03-13T14:49:53.807 INFO:tasks.workunit.client.0.smithi176.stdout:9/705: creat d0/d1e/d1f/feb x:0 0 0 2024-03-13T14:49:53.812 INFO:tasks.workunit.client.1.smithi196.stdout:8/672: dwrite d0/f56 [0,4194304] 0 2024-03-13T14:49:53.814 INFO:tasks.workunit.client.0.smithi176.stdout:7/588: dwrite d1/f1f [0,4194304] 0 2024-03-13T14:49:53.821 INFO:tasks.workunit.client.1.smithi196.stdout:3/726: sync 2024-03-13T14:49:53.828 INFO:tasks.workunit.client.0.smithi176.stdout:2/653: mkdir d1/dc/de0 0 2024-03-13T14:49:53.832 INFO:tasks.workunit.client.1.smithi196.stdout:8/673: dwrite d0/d1c/d29/fa6 [0,4194304] 0 2024-03-13T14:49:53.838 INFO:tasks.workunit.client.1.smithi196.stdout:8/674: chown d0/d16/c96 2148 1 2024-03-13T14:49:53.840 INFO:tasks.workunit.client.1.smithi196.stdout:8/675: chown d0/d1c/d29/c6d 11086603 1 2024-03-13T14:49:53.852 INFO:tasks.workunit.client.0.smithi176.stdout:9/706: symlink d0/d1e/d6f/lec 0 2024-03-13T14:49:53.853 INFO:tasks.workunit.client.0.smithi176.stdout:9/707: dread - d0/db/d89/fda zero size 2024-03-13T14:49:53.856 INFO:tasks.workunit.client.0.smithi176.stdout:7/589: mkdir d1/d1c/d26/d41/db6 0 2024-03-13T14:49:53.856 INFO:tasks.workunit.client.0.smithi176.stdout:6/620: write d0/d10/d19/d27/d2b/d2e/f7b [970595,99516] 0 2024-03-13T14:49:53.858 INFO:tasks.workunit.client.0.smithi176.stdout:7/590: truncate d1/f61 4728566 0 2024-03-13T14:49:53.859 INFO:tasks.workunit.client.0.smithi176.stdout:2/654: rename d1/d2a/fde to d1/dc/de0/fe1 0 2024-03-13T14:49:53.861 INFO:tasks.workunit.client.0.smithi176.stdout:5/575: symlink d5/lc5 0 2024-03-13T14:49:53.862 INFO:tasks.workunit.client.0.smithi176.stdout:5/576: stat d5/d75/f8c 0 2024-03-13T14:49:53.865 INFO:tasks.workunit.client.0.smithi176.stdout:3/668: write d4/d13/d16/d3d/f4c [108504,128455] 0 2024-03-13T14:49:53.867 INFO:tasks.workunit.client.0.smithi176.stdout:7/591: mkdir d1/d28/d7c/db7 0 2024-03-13T14:49:53.867 INFO:tasks.workunit.client.1.smithi196.stdout:1/787: sync 2024-03-13T14:49:53.867 INFO:tasks.workunit.client.1.smithi196.stdout:6/869: sync 2024-03-13T14:49:53.867 INFO:tasks.workunit.client.1.smithi196.stdout:4/683: sync 2024-03-13T14:49:53.869 INFO:tasks.workunit.client.0.smithi176.stdout:7/592: chown d1/d3f/d76/f1e 3313 1 2024-03-13T14:49:53.871 INFO:tasks.workunit.client.0.smithi176.stdout:7/593: chown d1/d3f/l47 182748310 1 2024-03-13T14:49:53.874 INFO:tasks.workunit.client.0.smithi176.stdout:0/522: truncate d2/db/f25 4054182 0 2024-03-13T14:49:53.880 INFO:tasks.workunit.client.0.smithi176.stdout:1/590: dwrite d2/de/d3f/d50/f7a [0,4194304] 0 2024-03-13T14:49:53.880 INFO:tasks.workunit.client.0.smithi176.stdout:6/621: dread d0/d10/d19/d9f/f98 [0,4194304] 0 2024-03-13T14:49:53.881 INFO:tasks.workunit.client.0.smithi176.stdout:1/591: fdatasync d2/de/d3f/f6f 0 2024-03-13T14:49:53.883 INFO:tasks.workunit.client.0.smithi176.stdout:1/592: dread - d2/de/d3f/d3e/f5b zero size 2024-03-13T14:49:53.887 INFO:tasks.workunit.client.1.smithi196.stdout:4/684: dwrite d1/d3/dbc/fe6 [0,4194304] 0 2024-03-13T14:49:53.888 INFO:tasks.workunit.client.1.smithi196.stdout:0/755: symlink d8/da/d25/d26/d35/d3b/d5f/d17/lf8 0 2024-03-13T14:49:53.894 INFO:tasks.workunit.client.0.smithi176.stdout:0/523: creat d2/db/d39/fa2 x:0 0 0 2024-03-13T14:49:53.896 INFO:tasks.workunit.client.1.smithi196.stdout:2/649: rmdir d0/d6/d34 39 2024-03-13T14:49:53.900 INFO:tasks.workunit.client.1.smithi196.stdout:2/650: write d0/d6/f84 [6048819,75094] 0 2024-03-13T14:49:53.902 INFO:tasks.workunit.client.0.smithi176.stdout:9/708: link d0/d1e/l24 d0/db/d89/d7a/led 0 2024-03-13T14:49:53.903 INFO:tasks.workunit.client.0.smithi176.stdout:3/669: dread d4/d28/fa2 [0,4194304] 0 2024-03-13T14:49:53.904 INFO:tasks.workunit.client.0.smithi176.stdout:7/594: creat d1/d3f/d76/da5/fb8 x:0 0 0 2024-03-13T14:49:53.905 INFO:tasks.workunit.client.0.smithi176.stdout:6/622: dread d0/de/d38/d70/faf [0,4194304] 0 2024-03-13T14:49:53.907 INFO:tasks.workunit.client.0.smithi176.stdout:8/562: sync 2024-03-13T14:49:53.910 INFO:tasks.workunit.client.1.smithi196.stdout:1/788: symlink d1/d3f/l111 0 2024-03-13T14:49:53.910 INFO:tasks.workunit.client.0.smithi176.stdout:5/577: link d5/ld d5/d2e/d7e/dad/d7b/lc6 0 2024-03-13T14:49:53.910 INFO:tasks.workunit.client.0.smithi176.stdout:2/655: write d1/f31 [4666954,2376] 0 2024-03-13T14:49:53.912 INFO:tasks.workunit.client.1.smithi196.stdout:1/789: readlink d1/d9/d33/d4b/lf0 0 2024-03-13T14:49:53.914 INFO:tasks.workunit.client.1.smithi196.stdout:6/870: creat d6/d9/d17/dfc/f11b x:0 0 0 2024-03-13T14:49:53.915 INFO:tasks.workunit.client.0.smithi176.stdout:3/670: write d4/d13/f2a [4256328,125450] 0 2024-03-13T14:49:53.915 INFO:tasks.workunit.client.1.smithi196.stdout:6/871: chown d6/d57 1 1 2024-03-13T14:49:53.918 INFO:tasks.workunit.client.0.smithi176.stdout:0/524: symlink d2/d5/dc/d59/d8e/la3 0 2024-03-13T14:49:53.919 INFO:tasks.workunit.client.1.smithi196.stdout:6/872: fsync d6/d7/df/d27/d39/d44/d74/f87 0 2024-03-13T14:49:53.921 INFO:tasks.workunit.client.1.smithi196.stdout:7/832: getdents d0/d6/d35/dff 0 2024-03-13T14:49:53.921 INFO:tasks.workunit.client.0.smithi176.stdout:2/656: dread - d1/dc/d2d/d59/d80/fbb zero size 2024-03-13T14:49:53.922 INFO:tasks.workunit.client.0.smithi176.stdout:8/563: unlink d0/d5/d11/f26 0 2024-03-13T14:49:53.923 INFO:tasks.workunit.client.0.smithi176.stdout:0/525: write d2/db/f3b [2138386,130495] 0 2024-03-13T14:49:53.924 INFO:tasks.workunit.client.1.smithi196.stdout:0/756: creat d8/ff9 x:0 0 0 2024-03-13T14:49:53.926 INFO:tasks.workunit.client.0.smithi176.stdout:9/709: dread d0/db/d89/d7a/d8b/f8f [0,4194304] 0 2024-03-13T14:49:53.927 INFO:tasks.workunit.client.0.smithi176.stdout:0/526: read d2/db/d39/f47 [242603,2974] 0 2024-03-13T14:49:53.928 INFO:tasks.workunit.client.0.smithi176.stdout:9/710: readlink d0/db/d89/d32/d58/d5f/l8c 0 2024-03-13T14:49:53.934 INFO:tasks.workunit.client.1.smithi196.stdout:9/661: dread d7/de/d4b/d1d/d93/da7/fae [0,4194304] 0 2024-03-13T14:49:53.936 INFO:tasks.workunit.client.1.smithi196.stdout:5/774: dwrite d4/d1b/dc8/d14/f5e [4194304,4194304] 0 2024-03-13T14:49:53.936 INFO:tasks.workunit.client.0.smithi176.stdout:4/665: sync 2024-03-13T14:49:53.938 INFO:tasks.workunit.client.1.smithi196.stdout:2/651: stat d0/c1f 0 2024-03-13T14:49:53.939 INFO:tasks.workunit.client.0.smithi176.stdout:8/564: fdatasync d0/f5c 0 2024-03-13T14:49:53.943 INFO:tasks.workunit.client.0.smithi176.stdout:8/565: read d0/d24/d29/f4e [10508935,25692] 0 2024-03-13T14:49:53.943 INFO:tasks.workunit.client.0.smithi176.stdout:6/623: write d0/d10/d19/d27/d2b/f65 [1145492,25133] 0 2024-03-13T14:49:53.945 INFO:tasks.workunit.client.0.smithi176.stdout:2/657: symlink d1/dc/d61/d9e/le2 0 2024-03-13T14:49:53.945 INFO:tasks.workunit.client.1.smithi196.stdout:8/676: symlink d0/d1c/le9 0 2024-03-13T14:49:53.946 INFO:tasks.workunit.client.0.smithi176.stdout:1/593: dread d2/f21 [0,4194304] 0 2024-03-13T14:49:53.946 INFO:tasks.workunit.client.0.smithi176.stdout:7/595: write d1/d1c/d26/d41/d2b/d40/d64/f60 [622546,44752] 0 2024-03-13T14:49:53.947 INFO:tasks.workunit.client.1.smithi196.stdout:1/790: mknod d1/d3f/d5b/d74/db5/c112 0 2024-03-13T14:49:53.947 INFO:tasks.workunit.client.1.smithi196.stdout:3/727: truncate db/d19/d22/d3a/d6a/fcc 1674595 0 2024-03-13T14:49:53.949 INFO:tasks.workunit.client.0.smithi176.stdout:0/527: dwrite d2/db/f3b [0,4194304] 0 2024-03-13T14:49:53.952 INFO:tasks.workunit.client.0.smithi176.stdout:1/594: chown d2/de/d3f/d50/f89 1002486939 1 2024-03-13T14:49:53.954 INFO:tasks.workunit.client.1.smithi196.stdout:7/833: mknod d0/d6/d33/d57/d8a/d9f/c11b 0 2024-03-13T14:49:53.959 INFO:tasks.workunit.client.1.smithi196.stdout:4/685: truncate d1/d3/d4c/fb8 2495272 0 2024-03-13T14:49:53.960 INFO:tasks.workunit.client.1.smithi196.stdout:9/662: fsync d7/f15 0 2024-03-13T14:49:53.960 INFO:tasks.workunit.client.0.smithi176.stdout:0/528: rmdir d2/d74 39 2024-03-13T14:49:53.961 INFO:tasks.workunit.client.1.smithi196.stdout:5/775: read d4/d1b/dc8/d14/d37/f78 [28418,10976] 0 2024-03-13T14:49:53.961 INFO:tasks.workunit.client.0.smithi176.stdout:0/529: fdatasync d2/d5/d28/f54 0 2024-03-13T14:49:53.961 INFO:tasks.workunit.client.1.smithi196.stdout:4/686: write d1/d3/d7/ddf/dc3/fcc [223255,130830] 0 2024-03-13T14:49:53.967 INFO:tasks.workunit.client.1.smithi196.stdout:8/677: readlink d0/d10/d79/l9e 0 2024-03-13T14:49:53.967 INFO:tasks.workunit.client.0.smithi176.stdout:9/711: write d0/db/d89/d32/d58/d67/f7e [1470820,77584] 0 2024-03-13T14:49:53.968 INFO:tasks.workunit.client.0.smithi176.stdout:9/712: fsync d0/db/fae 0 2024-03-13T14:49:53.971 INFO:tasks.workunit.client.1.smithi196.stdout:1/791: rmdir d1/d9 39 2024-03-13T14:49:53.975 INFO:tasks.workunit.client.1.smithi196.stdout:3/728: mkdir db/d19/d4a/d9b/de9 0 2024-03-13T14:49:53.975 INFO:tasks.workunit.client.0.smithi176.stdout:4/666: write d7/d2d/f2e [5111135,66100] 0 2024-03-13T14:49:53.977 INFO:tasks.workunit.client.0.smithi176.stdout:4/667: chown d7/d8/d15/d94 212 1 2024-03-13T14:49:53.978 INFO:tasks.workunit.client.1.smithi196.stdout:6/873: dwrite d6/d9/d17/d19/d4e/f6e [0,4194304] 0 2024-03-13T14:49:53.983 INFO:tasks.workunit.client.0.smithi176.stdout:5/578: rename d5/d2c/d40/d6a to d5/d2c/d40/dc7 0 2024-03-13T14:49:53.984 INFO:tasks.workunit.client.0.smithi176.stdout:5/579: fdatasync d5/d2e/d43/f50 0 2024-03-13T14:49:53.986 INFO:tasks.workunit.client.0.smithi176.stdout:6/624: unlink d0/d10/d19/d27/d2b/d2e/d3e/d9a/l9c 0 2024-03-13T14:49:53.988 INFO:tasks.workunit.client.1.smithi196.stdout:2/652: write d0/d6/fcd [3200944,26381] 0 2024-03-13T14:49:53.993 INFO:tasks.workunit.client.1.smithi196.stdout:0/757: truncate d8/da/d74/fae 3563046 0 2024-03-13T14:49:53.995 INFO:tasks.workunit.client.1.smithi196.stdout:4/687: symlink d1/d3/d7/d35/d76/d58/dc6/lf2 0 2024-03-13T14:49:53.996 INFO:tasks.workunit.client.1.smithi196.stdout:8/678: fsync d0/d16/d71/f89 0 2024-03-13T14:49:53.998 INFO:tasks.workunit.client.0.smithi176.stdout:7/596: dwrite d1/d1c/d26/d41/d2b/d40/d64/f95 [0,4194304] 0 2024-03-13T14:49:53.999 INFO:tasks.workunit.client.0.smithi176.stdout:0/530: write d2/f84 [2328845,90893] 0 2024-03-13T14:49:54.001 INFO:tasks.workunit.client.0.smithi176.stdout:9/713: mkdir d0/d40/d53/d6d/dee 0 2024-03-13T14:49:54.002 INFO:tasks.workunit.client.0.smithi176.stdout:7/597: chown d1/d1c/d26/d41/d2b/c31 1 1 2024-03-13T14:49:54.006 INFO:tasks.workunit.client.1.smithi196.stdout:6/874: mknod d6/d7/df/d27/d39/d44/d74/c11c 0 2024-03-13T14:49:54.007 INFO:tasks.workunit.client.0.smithi176.stdout:6/625: symlink d0/de/d38/d6b/lcb 0 2024-03-13T14:49:54.007 INFO:tasks.workunit.client.1.smithi196.stdout:7/834: mknod d0/d9/d1e/d58/c11c 0 2024-03-13T14:49:54.011 INFO:tasks.workunit.client.0.smithi176.stdout:5/580: read d5/d2c/f3d [1168555,117972] 0 2024-03-13T14:49:54.013 INFO:tasks.workunit.client.0.smithi176.stdout:6/626: dread d0/de/d38/d70/f8e [0,4194304] 0 2024-03-13T14:49:54.014 INFO:tasks.workunit.client.0.smithi176.stdout:8/566: rmdir d0/d24/d7b/dab/d86/da9 0 2024-03-13T14:49:54.015 INFO:tasks.workunit.client.0.smithi176.stdout:8/567: chown d0/d5/d11/d47/f6a 1 1 2024-03-13T14:49:54.015 INFO:tasks.workunit.client.0.smithi176.stdout:2/658: truncate d1/dc/d61/d9e/fcd 3493055 0 2024-03-13T14:49:54.016 INFO:tasks.workunit.client.0.smithi176.stdout:8/568: stat d0/d24/d29/f8c 0 2024-03-13T14:49:54.021 INFO:tasks.workunit.client.1.smithi196.stdout:9/663: write d7/d24/f31 [3072690,75961] 0 2024-03-13T14:49:54.024 INFO:tasks.workunit.client.1.smithi196.stdout:0/758: symlink d8/da/d25/d26/d54/d88/lfa 0 2024-03-13T14:49:54.028 INFO:tasks.workunit.client.1.smithi196.stdout:0/759: read d8/da/d25/f7b [2651279,72321] 0 2024-03-13T14:49:54.029 INFO:tasks.workunit.client.1.smithi196.stdout:8/679: mknod d0/d10/d70/cea 0 2024-03-13T14:49:54.030 INFO:tasks.workunit.client.1.smithi196.stdout:3/729: write db/d15/d44/d79/fa7 [2714273,20850] 0 2024-03-13T14:49:54.031 INFO:tasks.workunit.client.1.smithi196.stdout:8/680: chown d0/d10/fc2 20829051 1 2024-03-13T14:49:54.032 INFO:tasks.workunit.client.0.smithi176.stdout:5/581: creat d5/d2e/d7e/dad/d7b/fc8 x:0 0 0 2024-03-13T14:49:54.038 INFO:tasks.workunit.client.0.smithi176.stdout:0/531: write d2/d5/dc/d20/f48 [2951305,89666] 0 2024-03-13T14:49:54.039 INFO:tasks.workunit.client.1.smithi196.stdout:7/835: chown d0/d9/d31/d77/d93/lcd 42 1 2024-03-13T14:49:54.039 INFO:tasks.workunit.client.0.smithi176.stdout:8/569: creat d0/db/dc/fb1 x:0 0 0 2024-03-13T14:49:54.039 INFO:tasks.workunit.client.0.smithi176.stdout:2/659: dread d1/dc/d1c/f37 [0,4194304] 0 2024-03-13T14:49:54.040 INFO:tasks.workunit.client.0.smithi176.stdout:2/660: stat d1/l3 0 2024-03-13T14:49:54.041 INFO:tasks.workunit.client.0.smithi176.stdout:8/570: chown d0/d24/d29/d82 228571215 1 2024-03-13T14:49:54.042 INFO:tasks.workunit.client.0.smithi176.stdout:1/595: getdents d2 0 2024-03-13T14:49:54.043 INFO:tasks.workunit.client.0.smithi176.stdout:2/661: dread - d1/dc/d2d/d59/d8c/fd4 zero size 2024-03-13T14:49:54.045 INFO:tasks.workunit.client.0.smithi176.stdout:7/598: link d1/d3f/d76/f1e d1/d3f/d76/fb9 0 2024-03-13T14:49:54.046 INFO:tasks.workunit.client.1.smithi196.stdout:5/776: link d4/d1b/dc8/d14/d19/d26/d33/fdb d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/ff2 0 2024-03-13T14:49:54.046 INFO:tasks.workunit.client.0.smithi176.stdout:7/599: write d1/f1f [3640651,87385] 0 2024-03-13T14:49:54.048 INFO:tasks.workunit.client.1.smithi196.stdout:0/760: read - d8/da/d25/d26/d35/d3b/d59/d6a/f8d zero size 2024-03-13T14:49:54.050 INFO:tasks.workunit.client.1.smithi196.stdout:3/730: rmdir db/d15 39 2024-03-13T14:49:54.053 INFO:tasks.workunit.client.1.smithi196.stdout:6/875: creat d6/d7/df/dcb/f11d x:0 0 0 2024-03-13T14:49:54.053 INFO:tasks.workunit.client.0.smithi176.stdout:0/532: rmdir d2/d5/dc 39 2024-03-13T14:49:54.056 INFO:tasks.workunit.client.0.smithi176.stdout:9/714: rename d0/d73 to d0/db/def 0 2024-03-13T14:49:54.057 INFO:tasks.workunit.client.0.smithi176.stdout:8/571: symlink d0/d5/d11/d47/lb2 0 2024-03-13T14:49:54.058 INFO:tasks.workunit.client.0.smithi176.stdout:8/572: readlink d0/l78 0 2024-03-13T14:49:54.058 INFO:tasks.workunit.client.1.smithi196.stdout:2/653: creat d0/d7/fd4 x:0 0 0 2024-03-13T14:49:54.060 INFO:tasks.workunit.client.0.smithi176.stdout:7/600: creat d1/d3f/d76/da5/fba x:0 0 0 2024-03-13T14:49:54.062 INFO:tasks.workunit.client.0.smithi176.stdout:5/582: symlink d5/lc9 0 2024-03-13T14:49:54.062 INFO:tasks.workunit.client.1.smithi196.stdout:5/777: dwrite d4/d1b/dc8/d4a/faf [0,4194304] 0 2024-03-13T14:49:54.062 INFO:tasks.workunit.client.0.smithi176.stdout:8/573: truncate d0/d24/d7b/dab/d86/f94 100876 0 2024-03-13T14:49:54.064 INFO:tasks.workunit.client.0.smithi176.stdout:6/627: creat d0/de/fcc x:0 0 0 2024-03-13T14:49:54.066 INFO:tasks.workunit.client.1.smithi196.stdout:1/792: creat d1/d9/d4e/d55/f113 x:0 0 0 2024-03-13T14:49:54.067 INFO:tasks.workunit.client.1.smithi196.stdout:7/836: mknod d0/d9/c11d 0 2024-03-13T14:49:54.068 INFO:tasks.workunit.client.1.smithi196.stdout:5/778: chown d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd 28213685 1 2024-03-13T14:49:54.069 INFO:tasks.workunit.client.0.smithi176.stdout:2/662: dread d1/dc/d2d/f97 [0,4194304] 0 2024-03-13T14:49:54.071 INFO:tasks.workunit.client.0.smithi176.stdout:2/663: fdatasync d1/dc/d1c/fa4 0 2024-03-13T14:49:54.072 INFO:tasks.workunit.client.0.smithi176.stdout:7/601: rename d1/d3f/d81/d9a to d1/d28/d7c/dbb 0 2024-03-13T14:49:54.074 INFO:tasks.workunit.client.1.smithi196.stdout:6/876: creat d6/d2a/dc/d1c/d34/f11e x:0 0 0 2024-03-13T14:49:54.076 INFO:tasks.workunit.client.1.smithi196.stdout:9/664: link d7/d24/lb2 d7/de/de5/le6 0 2024-03-13T14:49:54.077 INFO:tasks.workunit.client.0.smithi176.stdout:5/583: dwrite d5/f9a [0,4194304] 0 2024-03-13T14:49:54.079 INFO:tasks.workunit.client.1.smithi196.stdout:2/654: write d0/d7/d9/d9a/daa/fb8 [1282417,125223] 0 2024-03-13T14:49:54.083 INFO:tasks.workunit.client.0.smithi176.stdout:0/533: mknod d2/d74/d61/ca4 0 2024-03-13T14:49:54.084 INFO:tasks.workunit.client.1.smithi196.stdout:1/793: write d1/d9/d4e/d55/d90/fff [256246,12089] 0 2024-03-13T14:49:54.091 INFO:tasks.workunit.client.0.smithi176.stdout:7/602: dwrite d1/d1c/d4a/fa7 [0,4194304] 0 2024-03-13T14:49:54.091 INFO:tasks.workunit.client.0.smithi176.stdout:0/534: mkdir d2/db/d39/da5 0 2024-03-13T14:49:54.102 INFO:tasks.workunit.client.1.smithi196.stdout:9/665: mkdir d7/d9/d17/dbc/de7 0 2024-03-13T14:49:54.103 INFO:tasks.workunit.client.0.smithi176.stdout:5/584: mkdir d5/d2e/d7e/d96/dca 0 2024-03-13T14:49:54.103 INFO:tasks.workunit.client.1.smithi196.stdout:3/731: dwrite db/d37/f97 [0,4194304] 0 2024-03-13T14:49:54.107 INFO:tasks.workunit.client.0.smithi176.stdout:6/628: write d0/f6c [683094,88879] 0 2024-03-13T14:49:54.108 INFO:tasks.workunit.client.0.smithi176.stdout:7/603: dwrite d1/d3f/d4b/d63/d7a/fa0 [0,4194304] 0 2024-03-13T14:49:54.110 INFO:tasks.workunit.client.1.smithi196.stdout:1/794: mknod d1/d9/d6c/c114 0 2024-03-13T14:49:54.111 INFO:tasks.workunit.client.1.smithi196.stdout:0/761: truncate d8/da/d25/d26/d35/d3b/d5f/d17/f51 361544 0 2024-03-13T14:49:54.112 INFO:tasks.workunit.client.1.smithi196.stdout:1/795: chown d1/d9/d33/d4b/fd6 266953076 1 2024-03-13T14:49:54.117 INFO:tasks.workunit.client.0.smithi176.stdout:0/535: dread d2/db/d31/f90 [0,4194304] 0 2024-03-13T14:49:54.117 INFO:tasks.workunit.client.0.smithi176.stdout:8/574: dwrite d0/db/dc/d40/f46 [0,4194304] 0 2024-03-13T14:49:54.118 INFO:tasks.workunit.client.1.smithi196.stdout:5/779: write d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/fbf [4559890,19204] 0 2024-03-13T14:49:54.119 INFO:tasks.workunit.client.1.smithi196.stdout:7/837: truncate d0/d9/f5a 2291229 0 2024-03-13T14:49:54.122 INFO:tasks.workunit.client.0.smithi176.stdout:5/585: creat d5/d2e/d5f/fcb x:0 0 0 2024-03-13T14:49:54.123 INFO:tasks.workunit.client.0.smithi176.stdout:0/536: rmdir d2/d74/d61 39 2024-03-13T14:49:54.128 INFO:tasks.workunit.client.1.smithi196.stdout:9/666: unlink d7/de/d4b/d1d/d22/f88 0 2024-03-13T14:49:54.131 INFO:tasks.workunit.client.1.smithi196.stdout:3/732: creat db/d19/d50/db7/da9/fea x:0 0 0 2024-03-13T14:49:54.133 INFO:tasks.workunit.client.1.smithi196.stdout:2/655: fdatasync d0/d6/d4b/f79 0 2024-03-13T14:49:54.135 INFO:tasks.workunit.client.0.smithi176.stdout:5/586: dread - d5/d2c/d40/f8f zero size 2024-03-13T14:49:54.136 INFO:tasks.workunit.client.1.smithi196.stdout:2/656: fsync d0/d6/fcd 0 2024-03-13T14:49:54.136 INFO:tasks.workunit.client.1.smithi196.stdout:1/796: creat d1/d9/d33/dd2/f115 x:0 0 0 2024-03-13T14:49:54.137 INFO:tasks.workunit.client.1.smithi196.stdout:0/762: mknod d8/da/d25/cfb 0 2024-03-13T14:49:54.139 INFO:tasks.workunit.client.1.smithi196.stdout:7/838: symlink d0/d4d/dd1/dd6/l11e 0 2024-03-13T14:49:54.140 INFO:tasks.workunit.client.1.smithi196.stdout:7/839: stat d0/d6/d33 0 2024-03-13T14:49:54.142 INFO:tasks.workunit.client.1.smithi196.stdout:6/877: link d6/d57/c85 d6/d7/df/d27/d39/d44/dc8/c11f 0 2024-03-13T14:49:54.143 INFO:tasks.workunit.client.0.smithi176.stdout:5/587: fsync d5/d22/d15/f7d 0 2024-03-13T14:49:54.148 INFO:tasks.workunit.client.0.smithi176.stdout:7/604: getdents d1/d3f/d76/da5 0 2024-03-13T14:49:54.154 INFO:tasks.workunit.client.0.smithi176.stdout:5/588: symlink d5/d31/lcc 0 2024-03-13T14:49:54.162 INFO:tasks.workunit.client.1.smithi196.stdout:9/667: dwrite d7/de/d74/fd4 [0,4194304] 0 2024-03-13T14:49:54.162 INFO:tasks.workunit.client.1.smithi196.stdout:2/657: read d0/d6/d4b/d5f/fae [486813,59389] 0 2024-03-13T14:49:54.164 INFO:tasks.workunit.client.1.smithi196.stdout:7/840: creat d0/d78/d81/dfb/f11f x:0 0 0 2024-03-13T14:49:54.165 INFO:tasks.workunit.client.1.smithi196.stdout:9/668: chown d7/de/f40 770503 1 2024-03-13T14:49:54.168 INFO:tasks.workunit.client.1.smithi196.stdout:9/669: write d7/de/d4b/f1a [5729692,84288] 0 2024-03-13T14:49:54.169 INFO:tasks.workunit.client.1.smithi196.stdout:6/878: creat d6/d2a/dc/d47/f120 x:0 0 0 2024-03-13T14:49:54.172 INFO:tasks.workunit.client.1.smithi196.stdout:2/658: creat d0/d50/fd5 x:0 0 0 2024-03-13T14:49:54.173 INFO:tasks.workunit.client.1.smithi196.stdout:7/841: unlink d0/d6/f21 0 2024-03-13T14:49:54.174 INFO:tasks.workunit.client.1.smithi196.stdout:6/879: fdatasync d6/fa 0 2024-03-13T14:49:54.177 INFO:tasks.workunit.client.1.smithi196.stdout:9/670: rename d7/d9/d17/dbc/lc3 to d7/de/d74/d94/le8 0 2024-03-13T14:49:54.181 INFO:tasks.workunit.client.1.smithi196.stdout:3/733: link db/f2c db/d4c/feb 0 2024-03-13T14:49:54.181 INFO:tasks.workunit.client.0.smithi176.stdout:0/537: creat d2/fa6 x:0 0 0 2024-03-13T14:49:54.182 INFO:tasks.workunit.client.0.smithi176.stdout:8/575: write d0/db/dc/d40/f62 [2630716,65698] 0 2024-03-13T14:49:54.182 INFO:tasks.workunit.client.0.smithi176.stdout:6/629: dwrite d0/d10/d19/f63 [0,4194304] 0 2024-03-13T14:49:54.190 INFO:tasks.workunit.client.0.smithi176.stdout:5/589: fdatasync f0 0 2024-03-13T14:49:54.191 INFO:tasks.workunit.client.1.smithi196.stdout:0/763: dwrite d8/da/d25/d26/d35/d3b/d50/d5d/f95 [0,4194304] 0 2024-03-13T14:49:54.198 INFO:tasks.workunit.client.0.smithi176.stdout:7/605: creat d1/d1c/d26/d41/d2b/fbc x:0 0 0 2024-03-13T14:49:54.202 INFO:tasks.workunit.client.1.smithi196.stdout:5/780: getdents d4/d1b/dc8/d14/d19/d26 0 2024-03-13T14:49:54.202 INFO:tasks.workunit.client.0.smithi176.stdout:8/576: mknod d0/db/d39/d56/d93/cb3 0 2024-03-13T14:49:54.204 INFO:tasks.workunit.client.1.smithi196.stdout:9/671: creat d7/de/d4b/fe9 x:0 0 0 2024-03-13T14:49:54.205 INFO:tasks.workunit.client.0.smithi176.stdout:8/577: write d0/db/d39/d56/f9f [575654,103615] 0 2024-03-13T14:49:54.206 INFO:tasks.workunit.client.0.smithi176.stdout:6/630: creat d0/de/d54/d99/fcd x:0 0 0 2024-03-13T14:49:54.206 INFO:tasks.workunit.client.1.smithi196.stdout:5/781: chown d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/ca1 1915 1 2024-03-13T14:49:54.208 INFO:tasks.workunit.client.1.smithi196.stdout:9/672: read d7/de/d4b/f14 [191608,31101] 0 2024-03-13T14:49:54.208 INFO:tasks.workunit.client.1.smithi196.stdout:1/797: getdents d1/d9/d33/d4b/d66/d7b 0 2024-03-13T14:49:54.209 INFO:tasks.workunit.client.1.smithi196.stdout:3/734: rmdir db/d19/d22/d3a/d68 39 2024-03-13T14:49:54.212 INFO:tasks.workunit.client.1.smithi196.stdout:2/659: mknod d0/d7/d56/dc8/cd6 0 2024-03-13T14:49:54.212 INFO:tasks.workunit.client.0.smithi176.stdout:5/590: creat d5/d2b/fcd x:0 0 0 2024-03-13T14:49:54.212 INFO:tasks.workunit.client.1.smithi196.stdout:9/673: chown d7/de/d4b/f6a 7583030 1 2024-03-13T14:49:54.214 INFO:tasks.workunit.client.1.smithi196.stdout:7/842: symlink d0/d6/d33/d110/l120 0 2024-03-13T14:49:54.214 INFO:tasks.workunit.client.0.smithi176.stdout:7/606: creat d1/d3f/d4b/d63/fbd x:0 0 0 2024-03-13T14:49:54.214 INFO:tasks.workunit.client.0.smithi176.stdout:6/631: truncate d0/d10/d19/d27/d2b/d2e/fad 329379 0 2024-03-13T14:49:54.216 INFO:tasks.workunit.client.1.smithi196.stdout:3/735: fsync db/d19/d22/f98 0 2024-03-13T14:49:54.216 INFO:tasks.workunit.client.0.smithi176.stdout:0/538: dread d2/d5/dc/d59/f80 [0,4194304] 0 2024-03-13T14:49:54.218 INFO:tasks.workunit.client.1.smithi196.stdout:2/660: dread - d0/d7/d66/d67/fcc zero size 2024-03-13T14:49:54.219 INFO:tasks.workunit.client.1.smithi196.stdout:0/764: creat d8/da/d25/d26/ffc x:0 0 0 2024-03-13T14:49:54.220 INFO:tasks.workunit.client.0.smithi176.stdout:6/632: mknod d0/cce 0 2024-03-13T14:49:54.223 INFO:tasks.workunit.client.1.smithi196.stdout:1/798: unlink d1/d9/d33/f82 0 2024-03-13T14:49:54.223 INFO:tasks.workunit.client.1.smithi196.stdout:6/880: dwrite d6/d7/df/f8e [0,4194304] 0 2024-03-13T14:49:54.224 INFO:tasks.workunit.client.0.smithi176.stdout:0/539: mknod d2/d5/dc/d59/ca7 0 2024-03-13T14:49:54.227 INFO:tasks.workunit.client.1.smithi196.stdout:5/782: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/ff3 x:0 0 0 2024-03-13T14:49:54.227 INFO:tasks.workunit.client.0.smithi176.stdout:3/671: sync 2024-03-13T14:49:54.228 INFO:tasks.workunit.client.1.smithi196.stdout:6/881: truncate d6/d2a/dc/d1c/d34/fff 543362 0 2024-03-13T14:49:54.231 INFO:tasks.workunit.client.0.smithi176.stdout:3/672: write d4/d13/d5e/fc2 [142065,65785] 0 2024-03-13T14:49:54.232 INFO:tasks.workunit.client.1.smithi196.stdout:6/882: read d6/d2a/d23/d28/f69 [1955852,22172] 0 2024-03-13T14:49:54.235 INFO:tasks.workunit.client.0.smithi176.stdout:0/540: write d2/d5/d1e/f6d [4969078,3278] 0 2024-03-13T14:49:54.235 INFO:tasks.workunit.client.1.smithi196.stdout:6/883: write d6/d2a/d23/d28/f119 [1735890,19807] 0 2024-03-13T14:49:54.242 INFO:tasks.workunit.client.0.smithi176.stdout:8/578: dread d0/db/dc/f13 [0,4194304] 0 2024-03-13T14:49:54.245 INFO:tasks.workunit.client.0.smithi176.stdout:5/591: dread d5/d31/d47/f5c [0,4194304] 0 2024-03-13T14:49:54.252 INFO:tasks.workunit.client.1.smithi196.stdout:3/736: dread db/d19/d22/d3a/d53/f59 [0,4194304] 0 2024-03-13T14:49:54.254 INFO:tasks.workunit.client.1.smithi196.stdout:9/674: mknod d7/d24/d28/d6c/cea 0 2024-03-13T14:49:54.258 INFO:tasks.workunit.client.1.smithi196.stdout:1/799: mknod d1/d9/d4e/d55/d63/da6/dc8/c116 0 2024-03-13T14:49:54.261 INFO:tasks.workunit.client.1.smithi196.stdout:2/661: rename d0/d50/c5d to d0/d7/d9/d9a/cd7 0 2024-03-13T14:49:54.262 INFO:tasks.workunit.client.1.smithi196.stdout:9/675: unlink d7/d9/d17/dbc/fc6 0 2024-03-13T14:49:54.264 INFO:tasks.workunit.client.1.smithi196.stdout:2/662: read d0/d6/fcd [2090628,23966] 0 2024-03-13T14:49:54.264 INFO:tasks.workunit.client.1.smithi196.stdout:1/800: unlink d1/d28/f32 0 2024-03-13T14:49:54.266 INFO:tasks.workunit.client.1.smithi196.stdout:9/676: creat d7/de/d16/d70/feb x:0 0 0 2024-03-13T14:49:54.267 INFO:tasks.workunit.client.1.smithi196.stdout:2/663: chown d0/d7/d9/d9a/fca 8157479 1 2024-03-13T14:49:54.268 INFO:tasks.workunit.client.1.smithi196.stdout:9/677: stat d7/de/d4b/f97 0 2024-03-13T14:49:54.269 INFO:tasks.workunit.client.1.smithi196.stdout:3/737: dwrite db/d19/d22/f31 [0,4194304] 0 2024-03-13T14:49:54.269 INFO:tasks.workunit.client.1.smithi196.stdout:1/801: creat d1/d9/d33/d4b/d66/d7b/f117 x:0 0 0 2024-03-13T14:49:54.271 INFO:tasks.workunit.client.1.smithi196.stdout:9/678: fdatasync d7/de/d74/d94/fb8 0 2024-03-13T14:49:54.277 INFO:tasks.workunit.client.1.smithi196.stdout:3/738: chown db/d19/d50/db7/da9/dc0/ldd 0 1 2024-03-13T14:49:54.278 INFO:tasks.workunit.client.1.smithi196.stdout:3/739: readlink db/d15/d44/d51/la6 0 2024-03-13T14:49:54.283 INFO:tasks.workunit.client.1.smithi196.stdout:3/740: truncate db/ff 1654402 0 2024-03-13T14:49:54.292 INFO:tasks.workunit.client.1.smithi196.stdout:3/741: truncate db/d19/d4a/f92 350750 0 2024-03-13T14:49:54.294 INFO:tasks.workunit.client.1.smithi196.stdout:3/742: rmdir db/d15/da1 39 2024-03-13T14:49:54.296 INFO:tasks.workunit.client.1.smithi196.stdout:9/679: dwrite d7/de/d16/f99 [0,4194304] 0 2024-03-13T14:49:54.304 INFO:tasks.workunit.client.1.smithi196.stdout:3/743: creat db/d19/d50/db7/da9/dc0/fec x:0 0 0 2024-03-13T14:49:54.311 INFO:tasks.workunit.client.1.smithi196.stdout:3/744: mknod db/d4c/ced 0 2024-03-13T14:49:54.314 INFO:tasks.workunit.client.1.smithi196.stdout:5/783: dwrite d4/d1b/dc8/fd8 [0,4194304] 0 2024-03-13T14:49:54.318 INFO:tasks.workunit.client.1.smithi196.stdout:2/664: dread d0/d7/d40/f46 [0,4194304] 0 2024-03-13T14:49:54.318 INFO:tasks.workunit.client.1.smithi196.stdout:3/745: mknod db/d19/d22/cee 0 2024-03-13T14:49:54.325 INFO:tasks.workunit.client.1.smithi196.stdout:6/884: dwrite d6/f8 [0,4194304] 0 2024-03-13T14:49:54.328 INFO:tasks.workunit.client.1.smithi196.stdout:1/802: dwrite d1/d9/d33/d4b/f75 [0,4194304] 0 2024-03-13T14:49:54.329 INFO:tasks.workunit.client.1.smithi196.stdout:2/665: unlink d0/d7/f17 0 2024-03-13T14:49:54.330 INFO:tasks.workunit.client.1.smithi196.stdout:3/746: symlink db/d19/d50/db7/lef 0 2024-03-13T14:49:54.334 INFO:tasks.workunit.client.1.smithi196.stdout:3/747: chown db/d19/d4a/fb2 200 1 2024-03-13T14:49:54.337 INFO:tasks.workunit.client.1.smithi196.stdout:3/748: chown db/d19/fda 280497480 1 2024-03-13T14:49:54.341 INFO:tasks.workunit.client.1.smithi196.stdout:3/749: write db/d19/f2f [568763,75400] 0 2024-03-13T14:49:54.341 INFO:tasks.workunit.client.0.smithi176.stdout:5/592: mkdir d5/d2b/dce 0 2024-03-13T14:49:54.342 INFO:tasks.workunit.client.0.smithi176.stdout:3/673: mkdir d4/d13/d81/d89/de2 0 2024-03-13T14:49:54.342 INFO:tasks.workunit.client.1.smithi196.stdout:5/784: rename d4/d1b/dc8/d14/d19/d26/d68/f77 to d4/d1b/dc8/d3c/dd0/ff4 0 2024-03-13T14:49:54.343 INFO:tasks.workunit.client.1.smithi196.stdout:6/885: symlink d6/d7/df/d27/d39/d44/dc8/l121 0 2024-03-13T14:49:54.345 INFO:tasks.workunit.client.1.smithi196.stdout:3/750: fsync db/d19/f62 0 2024-03-13T14:49:54.346 INFO:tasks.workunit.client.1.smithi196.stdout:1/803: mknod d1/d94/d10e/c118 0 2024-03-13T14:49:54.346 INFO:tasks.workunit.client.1.smithi196.stdout:9/680: dwrite d7/f15 [0,4194304] 0 2024-03-13T14:49:54.348 INFO:tasks.workunit.client.1.smithi196.stdout:2/666: creat d0/d7/d9/d9a/fd8 x:0 0 0 2024-03-13T14:49:54.349 INFO:tasks.workunit.client.0.smithi176.stdout:3/674: chown d4/d13/d81/d89/f94 29596 1 2024-03-13T14:49:54.350 INFO:tasks.workunit.client.1.smithi196.stdout:9/681: readlink d7/d24/l25 0 2024-03-13T14:49:54.351 INFO:tasks.workunit.client.1.smithi196.stdout:1/804: write d1/d38/f16 [1542267,102555] 0 2024-03-13T14:49:54.353 INFO:tasks.workunit.client.1.smithi196.stdout:3/751: fdatasync db/d19/d50/db7/f42 0 2024-03-13T14:49:54.359 INFO:tasks.workunit.client.0.smithi176.stdout:6/633: dwrite d0/d10/d19/d27/d2b/d2e/f2d [4194304,4194304] 0 2024-03-13T14:49:54.360 INFO:tasks.workunit.client.1.smithi196.stdout:9/682: mknod d7/d9/d83/cec 0 2024-03-13T14:49:54.360 INFO:tasks.workunit.client.0.smithi176.stdout:5/593: mknod d5/d2c/d74/ccf 0 2024-03-13T14:49:54.362 INFO:tasks.workunit.client.1.smithi196.stdout:6/886: truncate d6/d7/f95 1110660 0 2024-03-13T14:49:54.362 INFO:tasks.workunit.client.0.smithi176.stdout:6/634: fsync d0/de/f2c 0 2024-03-13T14:49:54.368 INFO:tasks.workunit.client.1.smithi196.stdout:1/805: creat d1/d9/d4e/d55/d63/db9/dbb/f119 x:0 0 0 2024-03-13T14:49:54.370 INFO:tasks.workunit.client.1.smithi196.stdout:2/667: dwrite d0/d7/d9/d5a/f7c [0,4194304] 0 2024-03-13T14:49:54.371 INFO:tasks.workunit.client.0.smithi176.stdout:0/541: dwrite d2/d5/d1e/f71 [0,4194304] 0 2024-03-13T14:49:54.372 INFO:tasks.workunit.client.0.smithi176.stdout:8/579: dread d0/db/f1a [0,4194304] 0 2024-03-13T14:49:54.373 INFO:tasks.workunit.client.0.smithi176.stdout:1/596: sync 2024-03-13T14:49:54.375 INFO:tasks.workunit.client.0.smithi176.stdout:4/668: sync 2024-03-13T14:49:54.376 INFO:tasks.workunit.client.1.smithi196.stdout:3/752: mkdir db/d19/d4a/d9b/de9/df0 0 2024-03-13T14:49:54.379 INFO:tasks.workunit.client.0.smithi176.stdout:6/635: read d0/f8 [3372074,59110] 0 2024-03-13T14:49:54.381 INFO:tasks.workunit.client.0.smithi176.stdout:5/594: truncate d5/d22/f1e 4736822 0 2024-03-13T14:49:54.382 INFO:tasks.workunit.client.0.smithi176.stdout:5/595: dread - d5/d22/fb3 zero size 2024-03-13T14:49:54.384 INFO:tasks.workunit.client.1.smithi196.stdout:2/668: write d0/d7/d9/d9a/daa/fb8 [2068397,19562] 0 2024-03-13T14:49:54.385 INFO:tasks.workunit.client.0.smithi176.stdout:1/597: unlink d2/de/d10/d16/d47/f5a 0 2024-03-13T14:49:54.385 INFO:tasks.workunit.client.0.smithi176.stdout:5/596: write d5/d2e/d5f/fcb [403209,52280] 0 2024-03-13T14:49:54.386 INFO:tasks.workunit.client.1.smithi196.stdout:2/669: chown d0/fbb 348029 1 2024-03-13T14:49:54.387 INFO:tasks.workunit.client.0.smithi176.stdout:1/598: dread - d2/de/d10/d16/d59/f6e zero size 2024-03-13T14:49:54.388 INFO:tasks.workunit.client.0.smithi176.stdout:3/675: getdents d4/db8 0 2024-03-13T14:49:54.389 INFO:tasks.workunit.client.1.smithi196.stdout:2/670: write d0/d50/fd5 [304844,70141] 0 2024-03-13T14:49:54.391 INFO:tasks.workunit.client.1.smithi196.stdout:2/671: chown d0/d7/d9/f48 85 1 2024-03-13T14:49:54.391 INFO:tasks.workunit.client.0.smithi176.stdout:0/542: rename d2/d5/dc/d59/d8e to d2/d5/dc/d20/da8 0 2024-03-13T14:49:54.393 INFO:tasks.workunit.client.1.smithi196.stdout:4/688: sync 2024-03-13T14:49:54.393 INFO:tasks.workunit.client.1.smithi196.stdout:8/681: sync 2024-03-13T14:49:54.395 INFO:tasks.workunit.client.0.smithi176.stdout:6/636: rmdir d0/de/d38/d6b/da3 39 2024-03-13T14:49:54.401 INFO:tasks.workunit.client.1.smithi196.stdout:6/887: rename d6/d2a/d23/d28/f109 to d6/d9/d17/d19/d4e/f122 0 2024-03-13T14:49:54.404 INFO:tasks.workunit.client.1.smithi196.stdout:8/682: chown d0/d10/d12/d1e/f6a 1047 1 2024-03-13T14:49:54.405 INFO:tasks.workunit.client.0.smithi176.stdout:1/599: creat d2/fb8 x:0 0 0 2024-03-13T14:49:54.407 INFO:tasks.workunit.client.1.smithi196.stdout:8/683: dread - d0/d16/d3d/d7f/fbf zero size 2024-03-13T14:49:54.409 INFO:tasks.workunit.client.1.smithi196.stdout:8/684: readlink d0/d1c/le9 0 2024-03-13T14:49:54.409 INFO:tasks.workunit.client.0.smithi176.stdout:2/664: sync 2024-03-13T14:49:54.409 INFO:tasks.workunit.client.0.smithi176.stdout:9/715: sync 2024-03-13T14:49:54.410 INFO:tasks.workunit.client.0.smithi176.stdout:2/665: chown d1/dc 14947163 1 2024-03-13T14:49:54.410 INFO:tasks.workunit.client.1.smithi196.stdout:4/689: rename d1/d3/d7/d35/f37 to d1/d3/d4c/d72/ff3 0 2024-03-13T14:49:54.415 INFO:tasks.workunit.client.1.smithi196.stdout:6/888: dread d6/d7/df/f97 [0,4194304] 0 2024-03-13T14:49:54.417 INFO:tasks.workunit.client.0.smithi176.stdout:8/580: creat d0/d24/fb4 x:0 0 0 2024-03-13T14:49:54.421 INFO:tasks.workunit.client.1.smithi196.stdout:4/690: fsync d1/d3/d4c/d72/fa0 0 2024-03-13T14:49:54.425 INFO:tasks.workunit.client.0.smithi176.stdout:9/716: creat d0/dd8/ff0 x:0 0 0 2024-03-13T14:49:54.426 INFO:tasks.workunit.client.1.smithi196.stdout:8/685: unlink d0/lf 0 2024-03-13T14:49:54.430 INFO:tasks.workunit.client.0.smithi176.stdout:2/666: rename d1/dc/f17 to d1/dc/d2d/d59/d80/d7e/fe3 0 2024-03-13T14:49:54.435 INFO:tasks.workunit.client.1.smithi196.stdout:4/691: fsync d1/d3/f3a 0 2024-03-13T14:49:54.438 INFO:tasks.workunit.client.1.smithi196.stdout:4/692: fdatasync d1/d3/d7/f49 0 2024-03-13T14:49:54.439 INFO:tasks.workunit.client.1.smithi196.stdout:6/889: dwrite d6/d57/ffe [0,4194304] 0 2024-03-13T14:49:54.444 INFO:tasks.workunit.client.1.smithi196.stdout:1/806: dread d1/d9/d33/d4b/f1d [0,4194304] 0 2024-03-13T14:49:54.444 INFO:tasks.workunit.client.1.smithi196.stdout:8/686: truncate d0/d10/f19 727831 0 2024-03-13T14:49:54.445 INFO:tasks.workunit.client.1.smithi196.stdout:2/672: dread d0/d6/d4b/d5f/fb7 [0,4194304] 0 2024-03-13T14:49:54.452 INFO:tasks.workunit.client.0.smithi176.stdout:2/667: readlink d1/d2a/l72 0 2024-03-13T14:49:54.454 INFO:tasks.workunit.client.1.smithi196.stdout:3/753: dwrite db/d19/fa2 [4194304,4194304] 0 2024-03-13T14:49:54.458 INFO:tasks.workunit.client.0.smithi176.stdout:4/669: truncate d7/d10/f2a 968500 0 2024-03-13T14:49:54.458 INFO:tasks.workunit.client.1.smithi196.stdout:9/683: dread d7/de/d4b/f43 [0,4194304] 0 2024-03-13T14:49:54.461 INFO:tasks.workunit.client.0.smithi176.stdout:5/597: truncate d5/d31/d47/d80/f4d 2538806 0 2024-03-13T14:49:54.464 INFO:tasks.workunit.client.0.smithi176.stdout:0/543: dread d2/d5/d1e/f4e [0,4194304] 0 2024-03-13T14:49:54.474 INFO:tasks.workunit.client.0.smithi176.stdout:3/676: truncate d4/d28/d4f/f90 1455475 0 2024-03-13T14:49:54.474 INFO:tasks.workunit.client.1.smithi196.stdout:4/693: rename d1/d3/d7/ddf/dc3 to d1/d3/d7/d35/d76/d58/d69/de3/df4 0 2024-03-13T14:49:54.475 INFO:tasks.workunit.client.0.smithi176.stdout:9/717: symlink d0/d1e/d8d/d9f/de3/lf1 0 2024-03-13T14:49:54.483 INFO:tasks.workunit.client.0.smithi176.stdout:2/668: creat d1/dc/d61/daa/fe4 x:0 0 0 2024-03-13T14:49:54.484 INFO:tasks.workunit.client.1.smithi196.stdout:8/687: dread d0/d10/f77 [0,4194304] 0 2024-03-13T14:49:54.489 INFO:tasks.workunit.client.0.smithi176.stdout:6/637: getdents d0/d10/d19/d27/d2b/d2e 0 2024-03-13T14:49:54.490 INFO:tasks.workunit.client.1.smithi196.stdout:9/684: chown d7/de/lab 41551 1 2024-03-13T14:49:54.498 INFO:tasks.workunit.client.1.smithi196.stdout:6/890: dwrite d6/d2a/d23/d28/f69 [0,4194304] 0 2024-03-13T14:49:54.503 INFO:tasks.workunit.client.0.smithi176.stdout:0/544: rename d2/f75 to d2/db/d39/fa9 0 2024-03-13T14:49:54.503 INFO:tasks.workunit.client.0.smithi176.stdout:6/638: fsync d0/d10/d19/d27/d2b/d2e/f7b 0 2024-03-13T14:49:54.504 INFO:tasks.workunit.client.1.smithi196.stdout:2/673: mkdir d0/d7/d9/d61/d83/dd9 0 2024-03-13T14:49:54.506 INFO:tasks.workunit.client.0.smithi176.stdout:1/600: mknod d2/de/d10/d16/d4a/d5f/cb9 0 2024-03-13T14:49:54.507 INFO:tasks.workunit.client.1.smithi196.stdout:8/688: symlink d0/d16/d3d/d9c/d87/leb 0 2024-03-13T14:49:54.510 INFO:tasks.workunit.client.1.smithi196.stdout:4/694: write d1/d1e/fb7 [336061,95426] 0 2024-03-13T14:49:54.512 INFO:tasks.workunit.client.0.smithi176.stdout:7/607: sync 2024-03-13T14:49:54.515 INFO:tasks.workunit.client.0.smithi176.stdout:2/669: dwrite d1/dc/d2d/d59/f8a [0,4194304] 0 2024-03-13T14:49:54.517 INFO:tasks.workunit.client.0.smithi176.stdout:2/670: chown d1/f31 402349907 1 2024-03-13T14:49:54.520 INFO:tasks.workunit.client.1.smithi196.stdout:6/891: stat d6/d2a/d23/f8f 0 2024-03-13T14:49:54.521 INFO:tasks.workunit.client.1.smithi196.stdout:6/892: stat d6/d9 0 2024-03-13T14:49:54.521 INFO:tasks.workunit.client.1.smithi196.stdout:9/685: unlink d7/d9/d17/dbc/c5e 0 2024-03-13T14:49:54.524 INFO:tasks.workunit.client.1.smithi196.stdout:1/807: getdents d1/d28/d34 0 2024-03-13T14:49:54.525 INFO:tasks.workunit.client.1.smithi196.stdout:1/808: dread - d1/d38/fef zero size 2024-03-13T14:49:54.526 INFO:tasks.workunit.client.1.smithi196.stdout:4/695: unlink d1/d3/d4c/d72/d7d/c95 0 2024-03-13T14:49:54.532 INFO:tasks.workunit.client.1.smithi196.stdout:8/689: write d0/f7e [313041,46694] 0 2024-03-13T14:49:54.536 INFO:tasks.workunit.client.1.smithi196.stdout:1/809: symlink d1/d9/d4e/dcc/l11a 0 2024-03-13T14:49:54.541 INFO:tasks.workunit.client.1.smithi196.stdout:4/696: rename d1/d3/d7/d35/d76/d58/d69/d9f/fe9 to d1/d3/d7/d35/d3e/d56/ff5 0 2024-03-13T14:49:54.544 INFO:tasks.workunit.client.1.smithi196.stdout:9/686: write f5 [5993669,65654] 0 2024-03-13T14:49:54.546 INFO:tasks.workunit.client.1.smithi196.stdout:2/674: getdents d0/d7/d66/d67 0 2024-03-13T14:49:54.551 INFO:tasks.workunit.client.1.smithi196.stdout:8/690: dwrite d0/d16/d3d/d9c/d87/d8c/da9/db8/fcd [0,4194304] 0 2024-03-13T14:49:54.565 INFO:tasks.workunit.client.1.smithi196.stdout:2/675: creat d0/d7/d66/d67/fda x:0 0 0 2024-03-13T14:49:54.567 INFO:tasks.workunit.client.1.smithi196.stdout:1/810: write d1/d9/d33/d4b/d66/d7b/fa9 [1479637,49731] 0 2024-03-13T14:49:54.568 INFO:tasks.workunit.client.1.smithi196.stdout:6/893: getdents d6/d9/d17/da1 0 2024-03-13T14:49:54.571 INFO:tasks.workunit.client.1.smithi196.stdout:6/894: chown d6/d57/d91/fe0 1638339897 1 2024-03-13T14:49:54.572 INFO:tasks.workunit.client.1.smithi196.stdout:6/895: chown d6/d2a 242293702 1 2024-03-13T14:49:54.572 INFO:tasks.workunit.client.0.smithi176.stdout:3/677: creat d4/d13/d5e/daa/fe3 x:0 0 0 2024-03-13T14:49:54.575 INFO:tasks.workunit.client.0.smithi176.stdout:4/670: creat d7/fe9 x:0 0 0 2024-03-13T14:49:54.576 INFO:tasks.workunit.client.1.smithi196.stdout:9/687: write d7/de/f40 [1070645,1175] 0 2024-03-13T14:49:54.577 INFO:tasks.workunit.client.0.smithi176.stdout:8/581: sync 2024-03-13T14:49:54.578 INFO:tasks.workunit.client.1.smithi196.stdout:9/688: readlink d7/d9/lc7 0 2024-03-13T14:49:54.578 INFO:tasks.workunit.client.1.smithi196.stdout:4/697: dwrite d1/d3/d7/d35/f40 [0,4194304] 0 2024-03-13T14:49:54.579 INFO:tasks.workunit.client.0.smithi176.stdout:5/598: creat d5/d2c/fd0 x:0 0 0 2024-03-13T14:49:54.581 INFO:tasks.workunit.client.0.smithi176.stdout:9/718: write d0/f59 [248962,70081] 0 2024-03-13T14:49:54.584 INFO:tasks.workunit.client.1.smithi196.stdout:2/676: symlink d0/d7/d9/d61/dd3/ldb 0 2024-03-13T14:49:54.586 INFO:tasks.workunit.client.1.smithi196.stdout:8/691: rename d0/d16/d3d/d8b/d3f/f64 to d0/d16/fec 0 2024-03-13T14:49:54.587 INFO:tasks.workunit.client.0.smithi176.stdout:8/582: write d0/db/d39/d56/f65 [331304,76358] 0 2024-03-13T14:49:54.588 INFO:tasks.workunit.client.1.smithi196.stdout:9/689: rmdir d7/d9/d17 39 2024-03-13T14:49:54.589 INFO:tasks.workunit.client.0.smithi176.stdout:9/719: write d0/db/d89/d32/d58/f83 [4120403,29268] 0 2024-03-13T14:49:54.590 INFO:tasks.workunit.client.0.smithi176.stdout:8/583: chown d0/db/d39/d56 1820442426 1 2024-03-13T14:49:54.591 INFO:tasks.workunit.client.1.smithi196.stdout:4/698: rmdir d1/d3/d7/d35/d76/d58/d69/d9f 39 2024-03-13T14:49:54.591 INFO:tasks.workunit.client.1.smithi196.stdout:9/690: read - d7/d24/dbf/f5c zero size 2024-03-13T14:49:54.596 INFO:tasks.workunit.client.1.smithi196.stdout:1/811: rename d1/d9/d4e/d55/d63/da6/lcd to d1/d9/d33/dd2/l11b 0 2024-03-13T14:49:54.598 INFO:tasks.workunit.client.1.smithi196.stdout:6/896: creat d6/d2a/dc/d1c/f123 x:0 0 0 2024-03-13T14:49:54.599 INFO:tasks.workunit.client.1.smithi196.stdout:9/691: symlink d7/de/d4b/d1d/d22/led 0 2024-03-13T14:49:54.600 INFO:tasks.workunit.client.0.smithi176.stdout:2/671: fdatasync d1/dc/d2d/f7a 0 2024-03-13T14:49:54.600 INFO:tasks.workunit.client.1.smithi196.stdout:4/699: symlink d1/d3/d7/d35/d76/d58/d69/d6c/lf6 0 2024-03-13T14:49:54.601 INFO:tasks.workunit.client.1.smithi196.stdout:6/897: readlink d6/d7/df/d27/d39/d44/dc8/l121 0 2024-03-13T14:49:54.601 INFO:tasks.workunit.client.1.smithi196.stdout:2/677: dread d0/d6/d1d/f7f [4194304,4194304] 0 2024-03-13T14:49:54.603 INFO:tasks.workunit.client.1.smithi196.stdout:6/898: chown d6/d6f/fec 352342480 1 2024-03-13T14:49:54.604 INFO:tasks.workunit.client.1.smithi196.stdout:8/692: rename d0/d16/d3d/d8b/d3f/l55 to d0/d16/d3d/d9c/d87/d8c/da9/db8/led 0 2024-03-13T14:49:54.614 INFO:tasks.workunit.client.0.smithi176.stdout:7/608: symlink d1/d1c/d26/d41/db6/lbe 0 2024-03-13T14:49:54.614 INFO:tasks.workunit.client.0.smithi176.stdout:8/584: truncate d0/db/dc/f2a 877205 0 2024-03-13T14:49:54.615 INFO:tasks.workunit.client.0.smithi176.stdout:8/585: fdatasync d0/d24/fb4 0 2024-03-13T14:49:54.617 INFO:tasks.workunit.client.0.smithi176.stdout:7/609: fdatasync d1/d3f/d76/da5/fb8 0 2024-03-13T14:49:54.617 INFO:tasks.workunit.client.0.smithi176.stdout:0/545: truncate d2/db/f2c 3000130 0 2024-03-13T14:49:54.618 INFO:tasks.workunit.client.1.smithi196.stdout:7/843: sync 2024-03-13T14:49:54.619 INFO:tasks.workunit.client.0.smithi176.stdout:3/678: mknod d4/d28/d31/ce4 0 2024-03-13T14:49:54.619 INFO:tasks.workunit.client.0.smithi176.stdout:7/610: chown d1/d28/d7c/lae 22 1 2024-03-13T14:49:54.620 INFO:tasks.workunit.client.1.smithi196.stdout:2/678: write d0/d7/f87 [1511933,43220] 0 2024-03-13T14:49:54.626 INFO:tasks.workunit.client.0.smithi176.stdout:9/720: symlink d0/d1e/d8d/dbe/de8/d8a/lf2 0 2024-03-13T14:49:54.626 INFO:tasks.workunit.client.1.smithi196.stdout:2/679: stat d0/d6/l29 0 2024-03-13T14:49:54.626 INFO:tasks.workunit.client.1.smithi196.stdout:8/693: dwrite d0/d16/d3d/d8b/d48/f5a [4194304,4194304] 0 2024-03-13T14:49:54.629 INFO:tasks.workunit.client.1.smithi196.stdout:6/899: mknod d6/d2a/d23/d104/c124 0 2024-03-13T14:49:54.633 INFO:tasks.workunit.client.0.smithi176.stdout:1/601: getdents d2/de/d3f 0 2024-03-13T14:49:54.635 INFO:tasks.workunit.client.1.smithi196.stdout:1/812: dwrite d1/d28/d34/d67/fdf [0,4194304] 0 2024-03-13T14:49:54.645 INFO:tasks.workunit.client.1.smithi196.stdout:9/692: creat d7/d24/d28/d6c/fee x:0 0 0 2024-03-13T14:49:54.647 INFO:tasks.workunit.client.1.smithi196.stdout:4/700: truncate d1/d3/d4c/d72/d7d/d80/d55/fc1 1478503 0 2024-03-13T14:49:54.647 INFO:tasks.workunit.client.1.smithi196.stdout:7/844: read d0/d6/d33/f4c [190450,113592] 0 2024-03-13T14:49:54.647 INFO:tasks.workunit.client.1.smithi196.stdout:2/680: fdatasync d0/d6/d4b/d5f/fae 0 2024-03-13T14:49:54.651 INFO:tasks.workunit.client.0.smithi176.stdout:9/721: creat d0/db/d89/d7a/ff3 x:0 0 0 2024-03-13T14:49:54.654 INFO:tasks.workunit.client.0.smithi176.stdout:8/586: mknod d0/d5/cb5 0 2024-03-13T14:49:54.655 INFO:tasks.workunit.client.0.smithi176.stdout:8/587: chown d0/d5/d11/la3 0 1 2024-03-13T14:49:54.656 INFO:tasks.workunit.client.0.smithi176.stdout:1/602: mkdir d2/de/d10/d16/d40/dba 0 2024-03-13T14:49:54.656 INFO:tasks.workunit.client.1.smithi196.stdout:5/785: sync 2024-03-13T14:49:54.656 INFO:tasks.workunit.client.1.smithi196.stdout:0/765: sync 2024-03-13T14:49:54.656 INFO:tasks.workunit.client.1.smithi196.stdout:3/754: sync 2024-03-13T14:49:54.656 INFO:tasks.workunit.client.1.smithi196.stdout:8/694: creat d0/d16/d3d/d9c/d87/d8c/da9/fee x:0 0 0 2024-03-13T14:49:54.657 INFO:tasks.workunit.client.1.smithi196.stdout:7/845: truncate d0/d9/d31/d77/dcf/de0/de7/f108 625006 0 2024-03-13T14:49:54.658 INFO:tasks.workunit.client.0.smithi176.stdout:6/639: sync 2024-03-13T14:49:54.658 INFO:tasks.workunit.client.1.smithi196.stdout:0/766: readlink d8/da/d25/l2e 0 2024-03-13T14:49:54.662 INFO:tasks.workunit.client.1.smithi196.stdout:4/701: creat d1/d1e/d5c/ff7 x:0 0 0 2024-03-13T14:49:54.662 INFO:tasks.workunit.client.0.smithi176.stdout:3/679: mknod d4/db8/dcc/ce5 0 2024-03-13T14:49:54.663 INFO:tasks.workunit.client.0.smithi176.stdout:9/722: fsync d0/d40/d53/fba 0 2024-03-13T14:49:54.663 INFO:tasks.workunit.client.0.smithi176.stdout:0/546: dwrite d2/d5/d28/f56 [0,4194304] 0 2024-03-13T14:49:54.665 INFO:tasks.workunit.client.0.smithi176.stdout:9/723: dread - d0/dd8/ff0 zero size 2024-03-13T14:49:54.666 INFO:tasks.workunit.client.1.smithi196.stdout:4/702: chown d1/d3/d7/d35/d76/fed 15 1 2024-03-13T14:49:54.666 INFO:tasks.workunit.client.1.smithi196.stdout:2/681: mknod d0/d7/d40/cdc 0 2024-03-13T14:49:54.668 INFO:tasks.workunit.client.0.smithi176.stdout:2/672: getdents d1/dc/d61/daa 0 2024-03-13T14:49:54.668 INFO:tasks.workunit.client.0.smithi176.stdout:9/724: chown d0/db/d89/d90 0 1 2024-03-13T14:49:54.671 INFO:tasks.workunit.client.0.smithi176.stdout:1/603: creat d2/de/d10/d16/d47/fbb x:0 0 0 2024-03-13T14:49:54.671 INFO:tasks.workunit.client.0.smithi176.stdout:9/725: readlink d0/db/d89/d32/d58/d5f/l8c 0 2024-03-13T14:49:54.671 INFO:tasks.workunit.client.1.smithi196.stdout:6/900: symlink d6/d7/l125 0 2024-03-13T14:49:54.673 INFO:tasks.workunit.client.1.smithi196.stdout:2/682: write d0/d7/d66/d67/fda [202259,124225] 0 2024-03-13T14:49:54.673 INFO:tasks.workunit.client.0.smithi176.stdout:1/604: chown d2/de/d3f/d19/f32 463903971 1 2024-03-13T14:49:54.675 INFO:tasks.workunit.client.1.smithi196.stdout:6/901: stat d6/d2a/lb3 0 2024-03-13T14:49:54.675 INFO:tasks.workunit.client.1.smithi196.stdout:3/755: mknod db/d19/d4a/d9b/cf1 0 2024-03-13T14:49:54.677 INFO:tasks.workunit.client.0.smithi176.stdout:6/640: write d0/d10/d19/d9f/d76/d7c/f9b [1239107,34624] 0 2024-03-13T14:49:54.678 INFO:tasks.workunit.client.1.smithi196.stdout:8/695: mkdir d0/d10/d12/d17/def 0 2024-03-13T14:49:54.682 INFO:tasks.workunit.client.1.smithi196.stdout:3/756: chown db/d15/d44/l66 6121 1 2024-03-13T14:49:54.683 INFO:tasks.workunit.client.1.smithi196.stdout:3/757: chown l9 471 1 2024-03-13T14:49:54.685 INFO:tasks.workunit.client.0.smithi176.stdout:2/673: dwrite d1/f7f [0,4194304] 0 2024-03-13T14:49:54.687 INFO:tasks.workunit.client.0.smithi176.stdout:3/680: unlink d4/d28/d31/fc6 0 2024-03-13T14:49:54.687 INFO:tasks.workunit.client.0.smithi176.stdout:0/547: mkdir d2/db/d35/daa 0 2024-03-13T14:49:54.689 INFO:tasks.workunit.client.0.smithi176.stdout:7/611: dread d1/d1c/d26/d41/d2b/f96 [0,4194304] 0 2024-03-13T14:49:54.689 INFO:tasks.workunit.client.1.smithi196.stdout:0/767: mkdir d8/da/d25/d26/d35/d3b/d5f/d33/dfd 0 2024-03-13T14:49:54.690 INFO:tasks.workunit.client.0.smithi176.stdout:7/612: chown d1/d1c/d26/d41/db6 216 1 2024-03-13T14:49:54.692 INFO:tasks.workunit.client.0.smithi176.stdout:7/613: chown d1/d1c/d26/d41/l30 250611 1 2024-03-13T14:49:54.695 INFO:tasks.workunit.client.0.smithi176.stdout:9/726: creat d0/d1e/d8d/ff4 x:0 0 0 2024-03-13T14:49:54.702 INFO:tasks.workunit.client.0.smithi176.stdout:1/605: creat d2/de/d3f/d3e/d86/fbc x:0 0 0 2024-03-13T14:49:54.704 INFO:tasks.workunit.client.1.smithi196.stdout:4/703: dread d1/d3/d7/d35/d76/d58/d69/d6c/f7e [0,4194304] 0 2024-03-13T14:49:54.704 INFO:tasks.workunit.client.1.smithi196.stdout:6/902: dwrite d6/d2a/d23/d28/f69 [0,4194304] 0 2024-03-13T14:49:54.705 INFO:tasks.workunit.client.0.smithi176.stdout:6/641: mknod d0/d10/d19/d9f/d76/ccf 0 2024-03-13T14:49:54.708 INFO:tasks.workunit.client.1.smithi196.stdout:2/683: rename d0/d7/d9/d61/dc7/fd2 to d0/d7/d8e/fdd 0 2024-03-13T14:49:54.710 INFO:tasks.workunit.client.1.smithi196.stdout:8/696: symlink d0/d16/d3d/d8b/lf0 0 2024-03-13T14:49:54.716 INFO:tasks.workunit.client.1.smithi196.stdout:5/786: mkdir d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5 0 2024-03-13T14:49:54.716 INFO:tasks.workunit.client.1.smithi196.stdout:7/846: truncate d0/d9/d31/d50/f87 2253615 0 2024-03-13T14:49:54.717 INFO:tasks.workunit.client.1.smithi196.stdout:9/693: link d7/de/d4b/d1d/d22/f67 d7/de/de5/fef 0 2024-03-13T14:49:54.720 INFO:tasks.workunit.client.1.smithi196.stdout:9/694: readlink d7/d24/l25 0 2024-03-13T14:49:54.721 INFO:tasks.workunit.client.0.smithi176.stdout:8/588: rename d0/db/d39/d56/l64 to d0/db/d53/lb6 0 2024-03-13T14:49:54.721 INFO:tasks.workunit.client.1.smithi196.stdout:0/768: creat d8/da/d25/d26/d48/d82/db3/ffe x:0 0 0 2024-03-13T14:49:54.722 INFO:tasks.workunit.client.0.smithi176.stdout:1/606: symlink d2/de/d10/d16/lbd 0 2024-03-13T14:49:54.724 INFO:tasks.workunit.client.1.smithi196.stdout:1/813: getdents d1/d28/d34 0 2024-03-13T14:49:54.725 INFO:tasks.workunit.client.0.smithi176.stdout:0/548: mknod d2/db/cab 0 2024-03-13T14:49:54.727 INFO:tasks.workunit.client.0.smithi176.stdout:7/614: dread d1/d28/f2d [0,4194304] 0 2024-03-13T14:49:54.728 INFO:tasks.workunit.client.1.smithi196.stdout:4/704: mknod d1/d3/d4c/cf8 0 2024-03-13T14:49:54.728 INFO:tasks.workunit.client.0.smithi176.stdout:4/671: sync 2024-03-13T14:49:54.729 INFO:tasks.workunit.client.0.smithi176.stdout:8/589: unlink d0/d24/d29/f4e 0 2024-03-13T14:49:54.732 INFO:tasks.workunit.client.1.smithi196.stdout:4/705: write d1/d3/d7/d35/d76/fed [21174,59759] 0 2024-03-13T14:49:54.734 INFO:tasks.workunit.client.1.smithi196.stdout:2/684: creat d0/d7/d9/d42/d98/fde x:0 0 0 2024-03-13T14:49:54.735 INFO:tasks.workunit.client.1.smithi196.stdout:4/706: read - d1/d3/d7/fa4 zero size 2024-03-13T14:49:54.735 INFO:tasks.workunit.client.0.smithi176.stdout:6/642: symlink d0/d10/d19/d27/ld0 0 2024-03-13T14:49:54.739 INFO:tasks.workunit.client.1.smithi196.stdout:3/758: mknod db/cf2 0 2024-03-13T14:49:54.740 INFO:tasks.workunit.client.1.smithi196.stdout:8/697: dread d0/d16/d58/fd0 [0,4194304] 0 2024-03-13T14:49:54.742 INFO:tasks.workunit.client.0.smithi176.stdout:7/615: dwrite d1/f61 [0,4194304] 0 2024-03-13T14:49:54.743 INFO:tasks.workunit.client.1.smithi196.stdout:7/847: unlink d0/d9/f30 0 2024-03-13T14:49:54.748 INFO:tasks.workunit.client.0.smithi176.stdout:4/672: symlink d7/d8/d15/d23/lea 0 2024-03-13T14:49:54.751 INFO:tasks.workunit.client.1.smithi196.stdout:6/903: symlink d6/dd2/l126 0 2024-03-13T14:49:54.751 INFO:tasks.workunit.client.0.smithi176.stdout:8/590: mkdir d0/d5/d11/d47/db7 0 2024-03-13T14:49:54.754 INFO:tasks.workunit.client.0.smithi176.stdout:3/681: dread d4/d28/d31/d45/d82/fb3 [0,4194304] 0 2024-03-13T14:49:54.755 INFO:tasks.workunit.client.0.smithi176.stdout:1/607: symlink d2/d29/d73/lbe 0 2024-03-13T14:49:54.756 INFO:tasks.workunit.client.0.smithi176.stdout:3/682: chown d4/l15 1758 1 2024-03-13T14:49:54.758 INFO:tasks.workunit.client.1.smithi196.stdout:8/698: dread d0/f51 [0,4194304] 0 2024-03-13T14:49:54.758 INFO:tasks.workunit.client.0.smithi176.stdout:9/727: write d0/d1e/f2e [2290603,127189] 0 2024-03-13T14:49:54.758 INFO:tasks.workunit.client.0.smithi176.stdout:6/643: creat d0/de/d54/d99/fd1 x:0 0 0 2024-03-13T14:49:54.762 INFO:tasks.workunit.client.1.smithi196.stdout:5/787: dread d4/d1b/dc8/d14/d19/d26/f43 [0,4194304] 0 2024-03-13T14:49:54.765 INFO:tasks.workunit.client.1.smithi196.stdout:4/707: mkdir d1/d3/d7/d35/d3e/df9 0 2024-03-13T14:49:54.766 INFO:tasks.workunit.client.0.smithi176.stdout:8/591: creat d0/db/dc/fb8 x:0 0 0 2024-03-13T14:49:54.767 INFO:tasks.workunit.client.1.smithi196.stdout:4/708: dread - d1/d3/d7/ddf/fef zero size 2024-03-13T14:49:54.772 INFO:tasks.workunit.client.0.smithi176.stdout:3/683: creat d4/d13/d16/dad/fe6 x:0 0 0 2024-03-13T14:49:54.773 INFO:tasks.workunit.client.1.smithi196.stdout:0/769: dwrite d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 [0,4194304] 0 2024-03-13T14:49:54.781 INFO:tasks.workunit.client.0.smithi176.stdout:8/592: mkdir d0/db/dc/db9 0 2024-03-13T14:49:54.782 INFO:tasks.workunit.client.0.smithi176.stdout:1/608: rename d2/d30/d63/d35 to d2/de/dbf 0 2024-03-13T14:49:54.785 INFO:tasks.workunit.client.0.smithi176.stdout:3/684: mknod d4/d28/d31/d52/ce7 0 2024-03-13T14:49:54.786 INFO:tasks.workunit.client.0.smithi176.stdout:0/549: getdents d2/db 0 2024-03-13T14:49:54.787 INFO:tasks.workunit.client.0.smithi176.stdout:0/550: fsync d2/f11 0 2024-03-13T14:49:54.788 INFO:tasks.workunit.client.1.smithi196.stdout:5/788: symlink d4/d1b/dc8/d14/lf6 0 2024-03-13T14:49:54.793 INFO:tasks.workunit.client.0.smithi176.stdout:6/644: mkdir d0/d10/dca/dd2 0 2024-03-13T14:49:54.794 INFO:tasks.workunit.client.1.smithi196.stdout:7/848: dwrite d0/fa [0,4194304] 0 2024-03-13T14:49:54.798 INFO:tasks.workunit.client.0.smithi176.stdout:7/616: link d1/d28/d3a/d1b/d8f/d91/caa d1/d1c/d26/d41/d2b/cbf 0 2024-03-13T14:49:54.799 INFO:tasks.workunit.client.1.smithi196.stdout:7/849: chown d0/d6/d33/d57/dae/de8/ff2 52708 1 2024-03-13T14:49:54.801 INFO:tasks.workunit.client.0.smithi176.stdout:7/617: write d1/d1c/d4a/f98 [329276,65600] 0 2024-03-13T14:49:54.801 INFO:tasks.workunit.client.0.smithi176.stdout:8/593: mknod d0/d5/d11/cba 0 2024-03-13T14:49:54.810 INFO:tasks.workunit.client.0.smithi176.stdout:1/609: link d2/de/d10/f72 d2/de/dbf/fc0 0 2024-03-13T14:49:54.810 INFO:tasks.workunit.client.0.smithi176.stdout:4/673: dread d7/fe [0,4194304] 0 2024-03-13T14:49:54.811 INFO:tasks.workunit.client.1.smithi196.stdout:3/759: dread db/d15/d44/fb6 [0,4194304] 0 2024-03-13T14:49:54.813 INFO:tasks.workunit.client.0.smithi176.stdout:6/645: creat d0/d10/d19/d27/d2b/db1/fd3 x:0 0 0 2024-03-13T14:49:54.813 INFO:tasks.workunit.client.0.smithi176.stdout:1/610: dread - d2/de/f26 zero size 2024-03-13T14:49:54.814 INFO:tasks.workunit.client.0.smithi176.stdout:0/551: dwrite d2/d5/d1e/f3e [4194304,4194304] 0 2024-03-13T14:49:54.821 INFO:tasks.workunit.client.0.smithi176.stdout:9/728: dwrite d0/d1e/f4a [0,4194304] 0 2024-03-13T14:49:54.826 INFO:tasks.workunit.client.0.smithi176.stdout:9/729: stat d0/d1e/d1f/c4f 0 2024-03-13T14:49:54.826 INFO:tasks.workunit.client.0.smithi176.stdout:0/552: readlink d2/d5/l8a 0 2024-03-13T14:49:54.829 INFO:tasks.workunit.client.0.smithi176.stdout:5/599: sync 2024-03-13T14:49:54.829 INFO:tasks.workunit.client.0.smithi176.stdout:9/730: chown d0/d40/d53/d6d/c96 5 1 2024-03-13T14:49:54.831 INFO:tasks.workunit.client.0.smithi176.stdout:4/674: rename d7/ca to d7/d10/d2c/d9d/ceb 0 2024-03-13T14:49:54.832 INFO:tasks.workunit.client.0.smithi176.stdout:0/553: dread d2/db/d39/f47 [0,4194304] 0 2024-03-13T14:49:54.834 INFO:tasks.workunit.client.0.smithi176.stdout:8/594: dwrite d0/db/d39/d56/d93/da7/faa [0,4194304] 0 2024-03-13T14:49:54.837 INFO:tasks.workunit.client.1.smithi196.stdout:6/904: mknod d6/d2a/dc/c127 0 2024-03-13T14:49:54.839 INFO:tasks.workunit.client.0.smithi176.stdout:5/600: write d5/d22/fa6 [497499,37249] 0 2024-03-13T14:49:54.839 INFO:tasks.workunit.client.1.smithi196.stdout:6/905: chown d6/d7/dd/dba/l100 1247479825 1 2024-03-13T14:49:54.840 INFO:tasks.workunit.client.1.smithi196.stdout:2/685: creat d0/d7/d9/fdf x:0 0 0 2024-03-13T14:49:54.841 INFO:tasks.workunit.client.0.smithi176.stdout:5/601: fsync d5/d2e/f54 0 2024-03-13T14:49:54.842 INFO:tasks.workunit.client.0.smithi176.stdout:1/611: dwrite d2/f6 [4194304,4194304] 0 2024-03-13T14:49:54.843 INFO:tasks.workunit.client.1.smithi196.stdout:2/686: write d0/d6/fc4 [1012097,60] 0 2024-03-13T14:49:54.846 INFO:tasks.workunit.client.0.smithi176.stdout:4/675: unlink d7/d10/d46/d4a/d5d/da1/f86 0 2024-03-13T14:49:54.849 INFO:tasks.workunit.client.1.smithi196.stdout:5/789: symlink d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/lf7 0 2024-03-13T14:49:54.851 INFO:tasks.workunit.client.0.smithi176.stdout:3/685: dread d4/d13/d2d/f30 [4194304,4194304] 0 2024-03-13T14:49:54.856 INFO:tasks.workunit.client.1.smithi196.stdout:1/814: rename d1/d94/laa to d1/d9/d33/d4b/l11c 0 2024-03-13T14:49:54.861 INFO:tasks.workunit.client.1.smithi196.stdout:0/770: dwrite d8/da/d25/d26/d35/d3b/d6c/fc6 [0,4194304] 0 2024-03-13T14:49:54.861 INFO:tasks.workunit.client.1.smithi196.stdout:8/699: rmdir d0/d16/d3d/d9c/d87/dbc 0 2024-03-13T14:49:54.862 INFO:tasks.workunit.client.0.smithi176.stdout:5/602: rmdir d5/d2c 39 2024-03-13T14:49:54.862 INFO:tasks.workunit.client.0.smithi176.stdout:7/618: dread d1/d3f/d76/fb9 [0,4194304] 0 2024-03-13T14:49:54.863 INFO:tasks.workunit.client.0.smithi176.stdout:5/603: chown d5/d31/d47/f7f 16175 1 2024-03-13T14:49:54.865 INFO:tasks.workunit.client.1.smithi196.stdout:6/906: dwrite d6/d2a/dc/d1c/d34/fff [0,4194304] 0 2024-03-13T14:49:54.867 INFO:tasks.workunit.client.0.smithi176.stdout:6/646: write d0/d10/d19/d7d/f86 [941705,30598] 0 2024-03-13T14:49:54.869 INFO:tasks.workunit.client.0.smithi176.stdout:2/674: sync 2024-03-13T14:49:54.871 INFO:tasks.workunit.client.1.smithi196.stdout:0/771: write d8/da/d25/d26/d35/d3b/d50/f6d [1861989,35352] 0 2024-03-13T14:49:54.874 INFO:tasks.workunit.client.0.smithi176.stdout:4/676: unlink d7/d10/d46/d4a/cc7 0 2024-03-13T14:49:54.884 INFO:tasks.workunit.client.1.smithi196.stdout:6/907: fsync d6/d9/d11/f13 0 2024-03-13T14:49:54.886 INFO:tasks.workunit.client.0.smithi176.stdout:8/595: rename d0/db/d53/lb6 to d0/db/d53/d80/lbb 0 2024-03-13T14:49:54.890 INFO:tasks.workunit.client.0.smithi176.stdout:5/604: read - d5/d2c/d40/dc7/f79 zero size 2024-03-13T14:49:54.892 INFO:tasks.workunit.client.0.smithi176.stdout:9/731: getdents d0/d1e/d8d/d9f/de3 0 2024-03-13T14:49:54.895 INFO:tasks.workunit.client.1.smithi196.stdout:8/700: truncate d0/d16/fec 258994 0 2024-03-13T14:49:54.896 INFO:tasks.workunit.client.0.smithi176.stdout:4/677: dwrite d7/d10/d2c/f66 [0,4194304] 0 2024-03-13T14:49:54.898 INFO:tasks.workunit.client.1.smithi196.stdout:6/908: mknod d6/d7/df/dcb/c128 0 2024-03-13T14:49:54.900 INFO:tasks.workunit.client.0.smithi176.stdout:8/596: rename d0/d24/fb4 to d0/db/fbc 0 2024-03-13T14:49:54.901 INFO:tasks.workunit.client.1.smithi196.stdout:3/760: getdents db/d19/d4a 0 2024-03-13T14:49:54.902 INFO:tasks.workunit.client.0.smithi176.stdout:5/605: chown d5/d2c/d8a/lb0 95093386 1 2024-03-13T14:49:54.904 INFO:tasks.workunit.client.1.smithi196.stdout:4/709: rename d1/d1e/l77 to d1/d3/d7/ddf/lfa 0 2024-03-13T14:49:54.906 INFO:tasks.workunit.client.1.smithi196.stdout:1/815: getdents d1/d9/d4e/d55/d63/db9 0 2024-03-13T14:49:54.910 INFO:tasks.workunit.client.0.smithi176.stdout:2/675: creat d1/dc/d2d/d59/d80/fe5 x:0 0 0 2024-03-13T14:49:54.916 INFO:tasks.workunit.client.0.smithi176.stdout:4/678: symlink d7/d8/d15/dd6/lec 0 2024-03-13T14:49:54.922 INFO:tasks.workunit.client.0.smithi176.stdout:8/597: fsync d0/d24/d29/fa6 0 2024-03-13T14:49:54.923 INFO:tasks.workunit.client.1.smithi196.stdout:5/790: write d4/d1b/dc8/d3c/d6a/f9f [471103,81170] 0 2024-03-13T14:49:54.923 INFO:tasks.workunit.client.0.smithi176.stdout:0/554: dread d2/db/d31/f62 [0,4194304] 0 2024-03-13T14:49:54.925 INFO:tasks.workunit.client.1.smithi196.stdout:7/850: rename d0/d9/d1e/d58/d72/da9/da3/dc2 to d0/d9/d31/d77/dcf/de0/d121 0 2024-03-13T14:49:54.925 INFO:tasks.workunit.client.0.smithi176.stdout:8/598: chown d0/d5/f87 12 1 2024-03-13T14:49:54.926 INFO:tasks.workunit.client.0.smithi176.stdout:6/647: getdents d0/d10/dca 0 2024-03-13T14:49:54.928 INFO:tasks.workunit.client.1.smithi196.stdout:1/816: truncate d1/d9/d33/d4b/fe0 452466 0 2024-03-13T14:49:54.929 INFO:tasks.workunit.client.0.smithi176.stdout:7/619: write d1/d28/f75 [930029,32196] 0 2024-03-13T14:49:54.930 INFO:tasks.workunit.client.1.smithi196.stdout:8/701: symlink d0/d16/d71/d9a/dca/lf1 0 2024-03-13T14:49:54.930 INFO:tasks.workunit.client.0.smithi176.stdout:1/612: dread d2/de/d3f/d50/d83/fae [4194304,4194304] 0 2024-03-13T14:49:54.934 INFO:tasks.workunit.client.1.smithi196.stdout:4/710: mkdir d1/dfb 0 2024-03-13T14:49:54.935 INFO:tasks.workunit.client.0.smithi176.stdout:2/676: dwrite d1/dc/d61/daa/fe4 [0,4194304] 0 2024-03-13T14:49:54.939 INFO:tasks.workunit.client.0.smithi176.stdout:2/677: dread - d1/dc/d61/daa/fb8 zero size 2024-03-13T14:49:54.939 INFO:tasks.workunit.client.1.smithi196.stdout:5/791: creat d4/d1b/dc8/d14/d19/d26/d33/dde/ff8 x:0 0 0 2024-03-13T14:49:54.945 INFO:tasks.workunit.client.0.smithi176.stdout:8/599: creat d0/d24/d29/fbd x:0 0 0 2024-03-13T14:49:54.946 INFO:tasks.workunit.client.1.smithi196.stdout:7/851: fdatasync d0/d9/d1e/d58/d72/f66 0 2024-03-13T14:49:54.948 INFO:tasks.workunit.client.1.smithi196.stdout:0/772: dwrite d8/da/d25/d26/d54/d88/f45 [0,4194304] 0 2024-03-13T14:49:54.950 INFO:tasks.workunit.client.0.smithi176.stdout:6/648: unlink d0/de/d38/d6b/fc8 0 2024-03-13T14:49:54.951 INFO:tasks.workunit.client.0.smithi176.stdout:5/606: write d5/d2b/f30 [1070212,99066] 0 2024-03-13T14:49:54.952 INFO:tasks.workunit.client.0.smithi176.stdout:7/620: stat d1/d28/d3a/f5c 0 2024-03-13T14:49:54.960 INFO:tasks.workunit.client.0.smithi176.stdout:2/678: creat d1/dc/d2d/d38/d5d/fe6 x:0 0 0 2024-03-13T14:49:54.960 INFO:tasks.workunit.client.1.smithi196.stdout:9/695: sync 2024-03-13T14:49:54.960 INFO:tasks.workunit.client.1.smithi196.stdout:6/909: dwrite d6/d7/dd/faf [0,4194304] 0 2024-03-13T14:49:54.962 INFO:tasks.workunit.client.1.smithi196.stdout:5/792: truncate d4/d1b/dc8/d14/d19/f58 192419 0 2024-03-13T14:49:54.962 INFO:tasks.workunit.client.0.smithi176.stdout:0/555: mkdir d2/d5/d28/dac 0 2024-03-13T14:49:54.962 INFO:tasks.workunit.client.1.smithi196.stdout:3/761: dwrite db/d19/d50/db7/f42 [0,4194304] 0 2024-03-13T14:49:54.963 INFO:tasks.workunit.client.1.smithi196.stdout:5/793: readlink d4/d1b/dc8/d4a/l99 0 2024-03-13T14:49:54.964 INFO:tasks.workunit.client.0.smithi176.stdout:9/732: getdents d0/db/d89/d32/d58/d5f/d84 0 2024-03-13T14:49:54.964 INFO:tasks.workunit.client.1.smithi196.stdout:9/696: stat d7/d24/d28/d6c/d7b/fb4 0 2024-03-13T14:49:54.965 INFO:tasks.workunit.client.0.smithi176.stdout:0/556: chown d2/d5/l7e 1591 1 2024-03-13T14:49:54.969 INFO:tasks.workunit.client.0.smithi176.stdout:9/733: chown d0/db/d89/d32/d58/d5f/c87 6 1 2024-03-13T14:49:54.971 INFO:tasks.workunit.client.1.smithi196.stdout:8/702: creat d0/d10/d12/d1e/dc5/ff2 x:0 0 0 2024-03-13T14:49:54.971 INFO:tasks.workunit.client.1.smithi196.stdout:1/817: write d1/d9/d4e/d55/d63/f73 [186159,117750] 0 2024-03-13T14:49:54.971 INFO:tasks.workunit.client.0.smithi176.stdout:4/679: dwrite d7/d8/d15/f81 [0,4194304] 0 2024-03-13T14:49:54.975 INFO:tasks.workunit.client.0.smithi176.stdout:6/649: creat d0/de/d38/d88/d9d/fd4 x:0 0 0 2024-03-13T14:49:54.977 INFO:tasks.workunit.client.0.smithi176.stdout:5/607: creat d5/d2c/d40/dc7/d89/fd1 x:0 0 0 2024-03-13T14:49:54.981 INFO:tasks.workunit.client.0.smithi176.stdout:7/621: rename d1/d3f/d4b/f58 to d1/d28/d3a/d1b/d8f/d91/fc0 0 2024-03-13T14:49:54.982 INFO:tasks.workunit.client.0.smithi176.stdout:1/613: creat d2/de/d10/d16/d47/db4/fc1 x:0 0 0 2024-03-13T14:49:54.983 INFO:tasks.workunit.client.1.smithi196.stdout:4/711: unlink d1/d3/d7/d35/d76/lc0 0 2024-03-13T14:49:54.987 INFO:tasks.workunit.client.1.smithi196.stdout:9/697: creat d7/de/d4b/d1d/d93/ff0 x:0 0 0 2024-03-13T14:49:54.987 INFO:tasks.workunit.client.0.smithi176.stdout:1/614: write d2/de/d3f/d3e/f5b [715783,117369] 0 2024-03-13T14:49:54.988 INFO:tasks.workunit.client.1.smithi196.stdout:2/687: sync 2024-03-13T14:49:54.988 INFO:tasks.workunit.client.1.smithi196.stdout:9/698: readlink d7/le2 0 2024-03-13T14:49:54.991 INFO:tasks.workunit.client.1.smithi196.stdout:6/910: fdatasync d6/fcc 0 2024-03-13T14:49:54.991 INFO:tasks.workunit.client.0.smithi176.stdout:0/557: unlink d2/db/d35/c86 0 2024-03-13T14:49:54.994 INFO:tasks.workunit.client.0.smithi176.stdout:0/558: fdatasync d2/f3 0 2024-03-13T14:49:54.994 INFO:tasks.workunit.client.0.smithi176.stdout:9/734: chown d0/d40/d53/c88 0 1 2024-03-13T14:49:54.995 INFO:tasks.workunit.client.1.smithi196.stdout:3/762: mknod db/d19/d22/d3a/d53/cf3 0 2024-03-13T14:49:54.995 INFO:tasks.workunit.client.0.smithi176.stdout:5/608: truncate d5/d22/d15/f7d 1637111 0 2024-03-13T14:49:54.997 INFO:tasks.workunit.client.1.smithi196.stdout:1/818: mkdir d1/d9/d4e/d55/d63/da6/dc8/d11d 0 2024-03-13T14:49:54.998 INFO:tasks.workunit.client.0.smithi176.stdout:1/615: symlink d2/de/d3f/d50/d83/lc2 0 2024-03-13T14:49:54.999 INFO:tasks.workunit.client.1.smithi196.stdout:8/703: dwrite d0/d16/d3d/d9c/d87/d8c/da9/fee [0,4194304] 0 2024-03-13T14:49:55.001 INFO:tasks.workunit.client.1.smithi196.stdout:6/911: fsync d6/d2a/de2/fd5 0 2024-03-13T14:49:55.002 INFO:tasks.workunit.client.1.smithi196.stdout:2/688: read d0/d7/d9/d9a/fb4 [284443,129178] 0 2024-03-13T14:49:55.006 INFO:tasks.workunit.client.0.smithi176.stdout:6/650: dwrite d0/d9/f55 [0,4194304] 0 2024-03-13T14:49:55.006 INFO:tasks.workunit.client.0.smithi176.stdout:0/559: mknod d2/d5/d28/d8c/cad 0 2024-03-13T14:49:55.008 INFO:tasks.workunit.client.1.smithi196.stdout:8/704: write d0/d10/d12/d1e/d2d/dd8/fe1 [798695,73446] 0 2024-03-13T14:49:55.012 INFO:tasks.workunit.client.1.smithi196.stdout:8/705: chown d0/d10/d12/d1e/dc5/ff2 56278 1 2024-03-13T14:49:55.012 INFO:tasks.workunit.client.1.smithi196.stdout:5/794: rename d4/d1b/dc8/d14/d19/d26/d33/c86 to d4/d1b/dc8/d14/d19/d1c/d2d/cf9 0 2024-03-13T14:49:55.015 INFO:tasks.workunit.client.1.smithi196.stdout:9/699: dwrite d7/de/d16/fdc [0,4194304] 0 2024-03-13T14:49:55.016 INFO:tasks.workunit.client.1.smithi196.stdout:6/912: mkdir d6/d2a/d23/d129 0 2024-03-13T14:49:55.019 INFO:tasks.workunit.client.1.smithi196.stdout:9/700: stat d7/le2 0 2024-03-13T14:49:55.025 INFO:tasks.workunit.client.1.smithi196.stdout:2/689: mknod d0/d7/d66/ce0 0 2024-03-13T14:49:55.031 INFO:tasks.workunit.client.1.smithi196.stdout:8/706: creat d0/d16/d3d/d8b/ff3 x:0 0 0 2024-03-13T14:49:55.042 INFO:tasks.workunit.client.1.smithi196.stdout:7/852: dwrite d0/d4d/fc8 [0,4194304] 0 2024-03-13T14:49:55.044 INFO:tasks.workunit.client.1.smithi196.stdout:2/690: symlink d0/d7/d56/le1 0 2024-03-13T14:49:55.045 INFO:tasks.workunit.client.0.smithi176.stdout:5/609: link d5/d31/d47/f5c d5/d2e/fd2 0 2024-03-13T14:49:55.045 INFO:tasks.workunit.client.1.smithi196.stdout:3/763: getdents db/d19/d4a/d9b 0 2024-03-13T14:49:55.047 INFO:tasks.workunit.client.1.smithi196.stdout:5/795: getdents d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/df1 0 2024-03-13T14:49:55.047 INFO:tasks.workunit.client.1.smithi196.stdout:8/707: chown d0/d1c/d29/d47/f52 5 1 2024-03-13T14:49:55.048 INFO:tasks.workunit.client.1.smithi196.stdout:0/773: dwrite d8/da/d74/f8e [0,4194304] 0 2024-03-13T14:49:55.048 INFO:tasks.workunit.client.1.smithi196.stdout:3/764: chown db/d19/d50/db7/c6d 31414 1 2024-03-13T14:49:55.051 INFO:tasks.workunit.client.1.smithi196.stdout:9/701: mkdir d7/de/d4b/d1d/d93/dd7/df1 0 2024-03-13T14:49:55.052 INFO:tasks.workunit.client.0.smithi176.stdout:4/680: dread d7/d10/d2c/f30 [0,4194304] 0 2024-03-13T14:49:55.052 INFO:tasks.workunit.client.1.smithi196.stdout:7/853: mkdir d0/d78/d81/d122 0 2024-03-13T14:49:55.053 INFO:tasks.workunit.client.1.smithi196.stdout:7/854: stat d0/d6/faa 0 2024-03-13T14:49:55.054 INFO:tasks.workunit.client.0.smithi176.stdout:1/616: dread d2/f6 [0,4194304] 0 2024-03-13T14:49:55.054 INFO:tasks.workunit.client.1.smithi196.stdout:4/712: rename d1/l25 to d1/d3/d7/d35/lfc 0 2024-03-13T14:49:55.056 INFO:tasks.workunit.client.1.smithi196.stdout:3/765: mknod db/d19/d4a/d9b/cf4 0 2024-03-13T14:49:55.057 INFO:tasks.workunit.client.0.smithi176.stdout:2/679: dread d1/dc/d2d/d38/f3f [0,4194304] 0 2024-03-13T14:49:55.058 INFO:tasks.workunit.client.0.smithi176.stdout:1/617: write d2/de/d3f/d3e/d86/fbc [931644,78874] 0 2024-03-13T14:49:55.060 INFO:tasks.workunit.client.1.smithi196.stdout:0/774: unlink d8/da/d25/d26/d35/d3b/d5f/fca 0 2024-03-13T14:49:55.069 INFO:tasks.workunit.client.0.smithi176.stdout:1/618: unlink d2/f9a 0 2024-03-13T14:49:55.070 INFO:tasks.workunit.client.1.smithi196.stdout:3/766: truncate db/d37/d64/f74 4186652 0 2024-03-13T14:49:55.071 INFO:tasks.workunit.client.1.smithi196.stdout:9/702: getdents d7/d24/d28/de4 0 2024-03-13T14:49:55.072 INFO:tasks.workunit.client.1.smithi196.stdout:9/703: fsync d7/d9/fd9 0 2024-03-13T14:49:55.075 INFO:tasks.workunit.client.0.smithi176.stdout:3/686: sync 2024-03-13T14:49:55.076 INFO:tasks.workunit.client.0.smithi176.stdout:1/619: stat d2/de/d10/fa9 0 2024-03-13T14:49:55.076 INFO:tasks.workunit.client.1.smithi196.stdout:5/796: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/f42 650244 0 2024-03-13T14:49:55.077 INFO:tasks.workunit.client.1.smithi196.stdout:0/775: unlink d8/da/d25/d26/d35/d3b/d6c/leb 0 2024-03-13T14:49:55.078 INFO:tasks.workunit.client.0.smithi176.stdout:1/620: readlink d2/de/d10/d16/d4a/l88 0 2024-03-13T14:49:55.081 INFO:tasks.workunit.client.0.smithi176.stdout:9/735: truncate d0/d40/f47 2205368 0 2024-03-13T14:49:55.083 INFO:tasks.workunit.client.0.smithi176.stdout:9/736: chown d0/d1e/f2e 86221396 1 2024-03-13T14:49:55.084 INFO:tasks.workunit.client.1.smithi196.stdout:9/704: creat d7/d71/ff2 x:0 0 0 2024-03-13T14:49:55.084 INFO:tasks.workunit.client.1.smithi196.stdout:9/705: dread - d7/d71/ff2 zero size 2024-03-13T14:49:55.086 INFO:tasks.workunit.client.0.smithi176.stdout:1/621: read d2/de/d10/d16/d4a/d5f/d66/f84 [586158,94826] 0 2024-03-13T14:49:55.087 INFO:tasks.workunit.client.1.smithi196.stdout:1/819: truncate d1/d28/d34/d67/fdf 767043 0 2024-03-13T14:49:55.088 INFO:tasks.workunit.client.0.smithi176.stdout:9/737: read - d0/dd8/d82/fa5 zero size 2024-03-13T14:49:55.089 INFO:tasks.workunit.client.0.smithi176.stdout:0/560: dwrite d2/d5/dc/f49 [0,4194304] 0 2024-03-13T14:49:55.090 INFO:tasks.workunit.client.1.smithi196.stdout:5/797: mknod d4/d1b/dc8/d14/cfa 0 2024-03-13T14:49:55.092 INFO:tasks.workunit.client.1.smithi196.stdout:8/708: getdents d0/d10/d12/d1e 0 2024-03-13T14:49:55.092 INFO:tasks.workunit.client.1.smithi196.stdout:5/798: chown d4/d1b/dc8/d14/d19/l88 12291356 1 2024-03-13T14:49:55.093 INFO:tasks.workunit.client.0.smithi176.stdout:3/687: dwrite d4/d13/d16/dad/fdc [0,4194304] 0 2024-03-13T14:49:55.094 INFO:tasks.workunit.client.0.smithi176.stdout:5/610: write d5/d2e/d7e/dad/f92 [2466461,27614] 0 2024-03-13T14:49:55.099 INFO:tasks.workunit.client.0.smithi176.stdout:4/681: write d7/f3a [993531,79107] 0 2024-03-13T14:49:55.102 INFO:tasks.workunit.client.0.smithi176.stdout:2/680: write d1/f96 [37130,123032] 0 2024-03-13T14:49:55.104 INFO:tasks.workunit.client.0.smithi176.stdout:1/622: mknod d2/d30/d63/cc3 0 2024-03-13T14:49:55.107 INFO:tasks.workunit.client.1.smithi196.stdout:4/713: write d1/f22 [133701,285] 0 2024-03-13T14:49:55.107 INFO:tasks.workunit.client.0.smithi176.stdout:3/688: mkdir d4/d13/d16/db6/de8 0 2024-03-13T14:49:55.109 INFO:tasks.workunit.client.0.smithi176.stdout:3/689: write d4/d28/d31/fc3 [229616,74017] 0 2024-03-13T14:49:55.112 INFO:tasks.workunit.client.1.smithi196.stdout:2/691: dwrite d0/f77 [0,4194304] 0 2024-03-13T14:49:55.113 INFO:tasks.workunit.client.0.smithi176.stdout:5/611: creat d5/d2e/d5f/fd3 x:0 0 0 2024-03-13T14:49:55.115 INFO:tasks.workunit.client.0.smithi176.stdout:0/561: rmdir d2/d74/d61 39 2024-03-13T14:49:55.118 INFO:tasks.workunit.client.1.smithi196.stdout:5/799: mknod d4/d1b/dc8/d14/d19/d1c/d94/cfb 0 2024-03-13T14:49:55.123 INFO:tasks.workunit.client.1.smithi196.stdout:3/767: write db/f29 [4547354,70746] 0 2024-03-13T14:49:55.128 INFO:tasks.workunit.client.1.smithi196.stdout:3/768: write db/d19/d50/db7/da9/dc0/fec [20135,54591] 0 2024-03-13T14:49:55.130 INFO:tasks.workunit.client.0.smithi176.stdout:9/738: dread d0/db/d89/d32/d58/d9c/fb3 [0,4194304] 0 2024-03-13T14:49:55.131 INFO:tasks.workunit.client.1.smithi196.stdout:3/769: write db/d37/f97 [1163498,115073] 0 2024-03-13T14:49:55.134 INFO:tasks.workunit.client.0.smithi176.stdout:9/739: write d0/d1e/d1f/d68/faf [654462,122057] 0 2024-03-13T14:49:55.134 INFO:tasks.workunit.client.1.smithi196.stdout:3/770: chown db/d15/d44/d51/l7a 3 1 2024-03-13T14:49:55.136 INFO:tasks.workunit.client.0.smithi176.stdout:9/740: truncate d0/db/d89/fda 183125 0 2024-03-13T14:49:55.140 INFO:tasks.workunit.client.1.smithi196.stdout:5/800: rename d4/d1b/dc8/d3c/d6a/d91/fa6 to d4/d1b/dc8/d14/d19/d26/d68/ffc 0 2024-03-13T14:49:55.144 INFO:tasks.workunit.client.0.smithi176.stdout:1/623: fdatasync d2/f21 0 2024-03-13T14:49:55.144 INFO:tasks.workunit.client.1.smithi196.stdout:8/709: dwrite d0/d16/d3d/d8b/f4b [0,4194304] 0 2024-03-13T14:49:55.147 INFO:tasks.workunit.client.1.smithi196.stdout:7/855: link d0/d9/d1e/d58/d72/d6e/c102 d0/d6/db4/dca/c123 0 2024-03-13T14:49:55.153 INFO:tasks.workunit.client.1.smithi196.stdout:1/820: dread - d1/d3f/d5b/d74/db5/fd5 zero size 2024-03-13T14:49:55.154 INFO:tasks.workunit.client.1.smithi196.stdout:1/821: stat d1/d28/f5a 0 2024-03-13T14:49:55.154 INFO:tasks.workunit.client.0.smithi176.stdout:0/562: dread d2/d5/d1e/f6d [0,4194304] 0 2024-03-13T14:49:55.156 INFO:tasks.workunit.client.1.smithi196.stdout:0/776: getdents d8/da/d25/d26/d35/d3b/d59 0 2024-03-13T14:49:55.159 INFO:tasks.workunit.client.0.smithi176.stdout:4/682: rename d7/d8/c1d to d7/d8/d15/ced 0 2024-03-13T14:49:55.160 INFO:tasks.workunit.client.1.smithi196.stdout:1/822: readlink d1/d28/d34/d67/l6b 0 2024-03-13T14:49:55.161 INFO:tasks.workunit.client.0.smithi176.stdout:9/741: mkdir d0/db/d89/d90/df5 0 2024-03-13T14:49:55.165 INFO:tasks.workunit.client.0.smithi176.stdout:1/624: symlink d2/de/dbf/lc4 0 2024-03-13T14:49:55.166 INFO:tasks.workunit.client.1.smithi196.stdout:8/710: dwrite d0/d10/d12/d1e/fbb [0,4194304] 0 2024-03-13T14:49:55.166 INFO:tasks.workunit.client.1.smithi196.stdout:5/801: dwrite d4/d1b/dc8/d14/d19/d26/f3a [0,4194304] 0 2024-03-13T14:49:55.167 INFO:tasks.workunit.client.1.smithi196.stdout:4/714: dread d1/d3/d4c/f59 [0,4194304] 0 2024-03-13T14:49:55.168 INFO:tasks.workunit.client.1.smithi196.stdout:9/706: getdents d7/de/d4b/d1d/d93/da7 0 2024-03-13T14:49:55.169 INFO:tasks.workunit.client.0.smithi176.stdout:3/690: rmdir d4/d28/dd8 0 2024-03-13T14:49:55.170 INFO:tasks.workunit.client.0.smithi176.stdout:3/691: readlink d4/d28/lcd 0 2024-03-13T14:49:55.171 INFO:tasks.workunit.client.1.smithi196.stdout:9/707: chown d7/de/d74/d94/fa8 0 1 2024-03-13T14:49:55.174 INFO:tasks.workunit.client.1.smithi196.stdout:7/856: dread - d0/d9/d1e/d58/d72/da9/da3/fb9 zero size 2024-03-13T14:49:55.174 INFO:tasks.workunit.client.1.smithi196.stdout:9/708: chown d7/d24/d28/l73 19521 1 2024-03-13T14:49:55.176 INFO:tasks.workunit.client.0.smithi176.stdout:4/683: mknod d7/d8/d33/d96/cee 0 2024-03-13T14:49:55.176 INFO:tasks.workunit.client.1.smithi196.stdout:5/802: read d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/fb8 [148777,38649] 0 2024-03-13T14:49:55.177 INFO:tasks.workunit.client.0.smithi176.stdout:5/612: dwrite d5/d2e/f3f [0,4194304] 0 2024-03-13T14:49:55.178 INFO:tasks.workunit.client.1.smithi196.stdout:9/709: write d7/de/d4b/d1d/d93/ff0 [26332,59746] 0 2024-03-13T14:49:55.185 INFO:tasks.workunit.client.1.smithi196.stdout:1/823: mkdir d1/d9/d33/dd2/d11e 0 2024-03-13T14:49:55.187 INFO:tasks.workunit.client.0.smithi176.stdout:2/681: getdents d1/dc/d2d/d90/dcf/da0 0 2024-03-13T14:49:55.187 INFO:tasks.workunit.client.1.smithi196.stdout:1/824: fdatasync d1/d9/d4e/d55/d90/fff 0 2024-03-13T14:49:55.189 INFO:tasks.workunit.client.1.smithi196.stdout:6/913: sync 2024-03-13T14:49:55.189 INFO:tasks.workunit.client.1.smithi196.stdout:1/825: chown d1/d3f/d5b/d74/db5/c112 27272052 1 2024-03-13T14:49:55.190 INFO:tasks.workunit.client.1.smithi196.stdout:8/711: mknod d0/d16/d3d/d9c/d87/d8c/da9/db8/cf4 0 2024-03-13T14:49:55.190 INFO:tasks.workunit.client.0.smithi176.stdout:0/563: creat d2/d5/d28/dac/fae x:0 0 0 2024-03-13T14:49:55.192 INFO:tasks.workunit.client.0.smithi176.stdout:4/684: rename d7/d10/d2c/fca to d7/d8/d2f/fef 0 2024-03-13T14:49:55.193 INFO:tasks.workunit.client.1.smithi196.stdout:7/857: write d0/d6/d33/f39 [6115370,6908] 0 2024-03-13T14:49:55.194 INFO:tasks.workunit.client.0.smithi176.stdout:0/564: chown d2/db/f93 13092628 1 2024-03-13T14:49:55.195 INFO:tasks.workunit.client.1.smithi196.stdout:7/858: chown d0/d4d/fc4 5 1 2024-03-13T14:49:55.195 INFO:tasks.workunit.client.1.smithi196.stdout:1/826: dread d1/d9/d33/d4b/d66/d7b/f8d [0,4194304] 0 2024-03-13T14:49:55.196 INFO:tasks.workunit.client.0.smithi176.stdout:9/742: symlink d0/db/d89/d4d/d5d/de0/lf6 0 2024-03-13T14:49:55.196 INFO:tasks.workunit.client.0.smithi176.stdout:5/613: creat d5/d2c/d8a/fd4 x:0 0 0 2024-03-13T14:49:55.197 INFO:tasks.workunit.client.1.smithi196.stdout:2/692: getdents d0/d7/d9 0 2024-03-13T14:49:55.197 INFO:tasks.workunit.client.0.smithi176.stdout:0/565: stat d2/d5/d1e/l55 0 2024-03-13T14:49:55.198 INFO:tasks.workunit.client.1.smithi196.stdout:5/803: symlink d4/d1b/dc8/d14/d19/d26/d95/lfd 0 2024-03-13T14:49:55.198 INFO:tasks.workunit.client.0.smithi176.stdout:2/682: stat d1/dc/d2d/d59/d80/d7e/da3/fb9 0 2024-03-13T14:49:55.198 INFO:tasks.workunit.client.0.smithi176.stdout:9/743: stat d0/d40/fd2 0 2024-03-13T14:49:55.200 INFO:tasks.workunit.client.1.smithi196.stdout:9/710: symlink d7/de/d4b/d1d/d93/lf3 0 2024-03-13T14:49:55.201 INFO:tasks.workunit.client.0.smithi176.stdout:4/685: symlink d7/d8/d33/d96/lf0 0 2024-03-13T14:49:55.202 INFO:tasks.workunit.client.1.smithi196.stdout:0/777: symlink d8/da/d25/d26/d35/d3b/d5f/lff 0 2024-03-13T14:49:55.202 INFO:tasks.workunit.client.1.smithi196.stdout:3/771: creat db/d15/d44/ff5 x:0 0 0 2024-03-13T14:49:55.204 INFO:tasks.workunit.client.0.smithi176.stdout:8/600: sync 2024-03-13T14:49:55.204 INFO:tasks.workunit.client.1.smithi196.stdout:4/715: symlink d1/d3/d7/d35/d76/d58/lfd 0 2024-03-13T14:49:55.205 INFO:tasks.workunit.client.1.smithi196.stdout:4/716: readlink d1/l24 0 2024-03-13T14:49:55.208 INFO:tasks.workunit.client.1.smithi196.stdout:3/772: read db/d19/d50/db7/da9/fb9 [574759,10398] 0 2024-03-13T14:49:55.209 INFO:tasks.workunit.client.0.smithi176.stdout:1/625: dread d2/de/d3f/d50/d83/fae [0,4194304] 0 2024-03-13T14:49:55.210 INFO:tasks.workunit.client.1.smithi196.stdout:8/712: read d0/d16/d3d/d7f/fb3 [454296,97439] 0 2024-03-13T14:49:55.212 INFO:tasks.workunit.client.1.smithi196.stdout:6/914: rename d6/d2a/d23/d129 to d6/d9/d17/d19/d4e/d66/d102/d12a 0 2024-03-13T14:49:55.214 INFO:tasks.workunit.client.0.smithi176.stdout:4/686: mkdir d7/d10/d2c/df1 0 2024-03-13T14:49:55.216 INFO:tasks.workunit.client.1.smithi196.stdout:7/859: mkdir d0/d78/d81/d124 0 2024-03-13T14:49:55.217 INFO:tasks.workunit.client.1.smithi196.stdout:1/827: mkdir d1/d28/d34/d67/d11f 0 2024-03-13T14:49:55.220 INFO:tasks.workunit.client.0.smithi176.stdout:1/626: symlink d2/de/lc5 0 2024-03-13T14:49:55.220 INFO:tasks.workunit.client.0.smithi176.stdout:3/692: dread d4/f25 [0,4194304] 0 2024-03-13T14:49:55.221 INFO:tasks.workunit.client.1.smithi196.stdout:0/778: fsync d8/da/d25/d26/d35/d3b/d50/d55/f72 0 2024-03-13T14:49:55.223 INFO:tasks.workunit.client.0.smithi176.stdout:3/693: stat d4/d28/d34/f9e 0 2024-03-13T14:49:55.223 INFO:tasks.workunit.client.0.smithi176.stdout:1/627: fsync d2/de/d10/d16/d40/f77 0 2024-03-13T14:49:55.224 INFO:tasks.workunit.client.0.smithi176.stdout:8/601: dwrite d0/db/dc/fb8 [0,4194304] 0 2024-03-13T14:49:55.225 INFO:tasks.workunit.client.0.smithi176.stdout:1/628: dread - d2/de/dbf/fa5 zero size 2024-03-13T14:49:55.228 INFO:tasks.workunit.client.0.smithi176.stdout:8/602: chown d0/d5/c9b 3112 1 2024-03-13T14:49:55.229 INFO:tasks.workunit.client.1.smithi196.stdout:3/773: truncate db/d19/d4a/f94 235213 0 2024-03-13T14:49:55.234 INFO:tasks.workunit.client.1.smithi196.stdout:9/711: rename d7/l41 to d7/de/d4b/d1d/d35/lf4 0 2024-03-13T14:49:55.235 INFO:tasks.workunit.client.0.smithi176.stdout:9/744: dread d0/db/f31 [0,4194304] 0 2024-03-13T14:49:55.235 INFO:tasks.workunit.client.0.smithi176.stdout:3/694: truncate d4/d28/d34/f9e 3102062 0 2024-03-13T14:49:55.235 INFO:tasks.workunit.client.0.smithi176.stdout:0/566: write d2/db/d39/f47 [446405,20359] 0 2024-03-13T14:49:55.236 INFO:tasks.workunit.client.1.smithi196.stdout:1/828: dwrite d1/d9/d33/d4b/d66/fee [0,4194304] 0 2024-03-13T14:49:55.236 INFO:tasks.workunit.client.0.smithi176.stdout:5/614: write d5/d2e/fd2 [3533615,2838] 0 2024-03-13T14:49:55.237 INFO:tasks.workunit.client.1.smithi196.stdout:1/829: fdatasync d1/d9/d4e/f10c 0 2024-03-13T14:49:55.238 INFO:tasks.workunit.client.0.smithi176.stdout:7/622: sync 2024-03-13T14:49:55.238 INFO:tasks.workunit.client.0.smithi176.stdout:6/651: sync 2024-03-13T14:49:55.241 INFO:tasks.workunit.client.0.smithi176.stdout:6/652: chown d0/de/d38/d88/fc3 439887 1 2024-03-13T14:49:55.241 INFO:tasks.workunit.client.0.smithi176.stdout:4/687: mknod d7/d10/d2c/df1/cf2 0 2024-03-13T14:49:55.241 INFO:tasks.workunit.client.1.smithi196.stdout:5/804: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/ffe x:0 0 0 2024-03-13T14:49:55.242 INFO:tasks.workunit.client.0.smithi176.stdout:4/688: fsync d7/d8/d45/f76 0 2024-03-13T14:49:55.243 INFO:tasks.workunit.client.0.smithi176.stdout:2/683: write d1/dc/d2d/d59/d80/d7e/da3/fb9 [3452525,104619] 0 2024-03-13T14:49:55.247 INFO:tasks.workunit.client.0.smithi176.stdout:1/629: mknod d2/de/d3f/d3e/d86/db3/cc6 0 2024-03-13T14:49:55.249 INFO:tasks.workunit.client.1.smithi196.stdout:8/713: write d0/d1c/d29/f3c [7971098,22346] 0 2024-03-13T14:49:55.249 INFO:tasks.workunit.client.0.smithi176.stdout:3/695: mkdir d4/d28/d31/d52/de9 0 2024-03-13T14:49:55.250 INFO:tasks.workunit.client.1.smithi196.stdout:0/779: dread - d8/da/d25/d26/d35/d3b/d50/f99 zero size 2024-03-13T14:49:55.252 INFO:tasks.workunit.client.0.smithi176.stdout:4/689: truncate d7/d8/d45/fdb 661424 0 2024-03-13T14:49:55.253 INFO:tasks.workunit.client.1.smithi196.stdout:3/774: fsync db/d19/d4a/f8c 0 2024-03-13T14:49:55.253 INFO:tasks.workunit.client.0.smithi176.stdout:3/696: readlink d4/l22 0 2024-03-13T14:49:55.257 INFO:tasks.workunit.client.1.smithi196.stdout:6/915: write d6/d2a/de2/fd5 [464802,48972] 0 2024-03-13T14:49:55.258 INFO:tasks.workunit.client.0.smithi176.stdout:1/630: chown d2/de/d3f/d50/d83/d97 35 1 2024-03-13T14:49:55.259 INFO:tasks.workunit.client.0.smithi176.stdout:8/603: mknod d0/d5/d11/d47/db7/cbe 0 2024-03-13T14:49:55.259 INFO:tasks.workunit.client.1.smithi196.stdout:9/712: unlink d7/d24/c57 0 2024-03-13T14:49:55.260 INFO:tasks.workunit.client.1.smithi196.stdout:1/830: dwrite d1/d9/d33/d4b/d66/fa8 [0,4194304] 0 2024-03-13T14:49:55.260 INFO:tasks.workunit.client.0.smithi176.stdout:7/623: chown d1/d28/c74 6 1 2024-03-13T14:49:55.268 INFO:tasks.workunit.client.1.smithi196.stdout:5/805: read d4/d1b/dc8/d14/d19/d1c/f23 [803192,64319] 0 2024-03-13T14:49:55.269 INFO:tasks.workunit.client.1.smithi196.stdout:4/717: rmdir d1/d3/d7/d35/d3e/df9 0 2024-03-13T14:49:55.273 INFO:tasks.workunit.client.1.smithi196.stdout:0/780: mknod d8/da/d25/d26/d35/d3b/d5f/d17/c100 0 2024-03-13T14:49:55.273 INFO:tasks.workunit.client.0.smithi176.stdout:5/615: dwrite d5/d2e/fd2 [0,4194304] 0 2024-03-13T14:49:55.274 INFO:tasks.workunit.client.0.smithi176.stdout:4/690: unlink d7/d8/d45/f8d 0 2024-03-13T14:49:55.274 INFO:tasks.workunit.client.0.smithi176.stdout:0/567: dwrite d2/d5/dc/f49 [0,4194304] 0 2024-03-13T14:49:55.276 INFO:tasks.workunit.client.0.smithi176.stdout:8/604: stat d0/d5/d11/d47/ca4 0 2024-03-13T14:49:55.276 INFO:tasks.workunit.client.1.smithi196.stdout:1/831: read - d1/d9/d33/d4b/d88/fc7 zero size 2024-03-13T14:49:55.277 INFO:tasks.workunit.client.1.smithi196.stdout:7/860: getdents d0/d9/d31/d77 0 2024-03-13T14:49:55.277 INFO:tasks.workunit.client.1.smithi196.stdout:4/718: dread d1/d3/fa [0,4194304] 0 2024-03-13T14:49:55.278 INFO:tasks.workunit.client.0.smithi176.stdout:4/691: dread - d7/d2d/fa4 zero size 2024-03-13T14:49:55.281 INFO:tasks.workunit.client.1.smithi196.stdout:9/713: mknod d7/de/d4b/d1d/d93/dd7/df1/cf5 0 2024-03-13T14:49:55.284 INFO:tasks.workunit.client.0.smithi176.stdout:0/568: rmdir d2/db/d39 39 2024-03-13T14:49:55.284 INFO:tasks.workunit.client.0.smithi176.stdout:6/653: dwrite d0/de/d38/d88/fc3 [0,4194304] 0 2024-03-13T14:49:55.286 INFO:tasks.workunit.client.1.smithi196.stdout:8/714: rename d0/d1c/d29 to d0/d10/d12/df5 0 2024-03-13T14:49:55.289 INFO:tasks.workunit.client.1.smithi196.stdout:5/806: mknod d4/d1b/dc8/d14/d19/d26/d33/cff 0 2024-03-13T14:49:55.290 INFO:tasks.workunit.client.1.smithi196.stdout:8/715: stat d0/d10/d12/d1e/f34 0 2024-03-13T14:49:55.291 INFO:tasks.workunit.client.1.smithi196.stdout:7/861: fsync d0/d9/d1e/d3f/f60 0 2024-03-13T14:49:55.300 INFO:tasks.workunit.client.1.smithi196.stdout:9/714: readlink d7/d24/lb2 0 2024-03-13T14:49:55.302 INFO:tasks.workunit.client.1.smithi196.stdout:3/775: rename db/d19/d22/fc2 to db/d15/d44/d79/ff6 0 2024-03-13T14:49:55.304 INFO:tasks.workunit.client.1.smithi196.stdout:8/716: creat d0/d63/dcb/ff6 x:0 0 0 2024-03-13T14:49:55.304 INFO:tasks.workunit.client.0.smithi176.stdout:9/745: dwrite d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:55.306 INFO:tasks.workunit.client.0.smithi176.stdout:1/631: truncate d2/de/d10/d16/d47/f5c 188544 0 2024-03-13T14:49:55.313 INFO:tasks.workunit.client.0.smithi176.stdout:8/605: truncate d0/d24/d29/f72 838046 0 2024-03-13T14:49:55.313 INFO:tasks.workunit.client.0.smithi176.stdout:1/632: write d2/de/d3f/d3e/f5b [974224,63715] 0 2024-03-13T14:49:55.318 INFO:tasks.workunit.client.0.smithi176.stdout:3/697: dwrite d4/d13/d16/d3d/f4c [0,4194304] 0 2024-03-13T14:49:55.319 INFO:tasks.workunit.client.0.smithi176.stdout:9/746: write d0/db/fae [674103,121] 0 2024-03-13T14:49:55.320 INFO:tasks.workunit.client.1.smithi196.stdout:4/719: creat d1/d3/d7/d35/d76/d58/dc9/dd7/deb/ffe x:0 0 0 2024-03-13T14:49:55.325 INFO:tasks.workunit.client.1.smithi196.stdout:4/720: truncate d1/d3/d7/d35/f40 4677626 0 2024-03-13T14:49:55.326 INFO:tasks.workunit.client.0.smithi176.stdout:7/624: getdents d1/d28/d3a/d1b 0 2024-03-13T14:49:55.327 INFO:tasks.workunit.client.1.smithi196.stdout:6/916: dread d6/d9/f1f [0,4194304] 0 2024-03-13T14:49:55.329 INFO:tasks.workunit.client.1.smithi196.stdout:3/776: symlink db/d71/lf7 0 2024-03-13T14:49:55.333 INFO:tasks.workunit.client.0.smithi176.stdout:2/684: dwrite d1/dc/d2d/d59/d80/d77/f84 [0,4194304] 0 2024-03-13T14:49:55.333 INFO:tasks.workunit.client.0.smithi176.stdout:5/616: dwrite d5/d2c/d40/d76/f95 [0,4194304] 0 2024-03-13T14:49:55.339 INFO:tasks.workunit.client.0.smithi176.stdout:2/685: chown d1/dc/d2d/f97 39617311 1 2024-03-13T14:49:55.345 INFO:tasks.workunit.client.0.smithi176.stdout:2/686: chown d1/dc/d2d/d4b/l89 8109 1 2024-03-13T14:49:55.345 INFO:tasks.workunit.client.1.smithi196.stdout:0/781: link d8/da/d25/d26/d35/d3b/d50/d55/f64 d8/da/d25/d26/d54/d88/f101 0 2024-03-13T14:49:55.348 INFO:tasks.workunit.client.1.smithi196.stdout:1/832: dread d1/d9/d4e/f70 [0,4194304] 0 2024-03-13T14:49:55.351 INFO:tasks.workunit.client.1.smithi196.stdout:6/917: dwrite d6/d2a/d23/d28/d10a/d10d/f116 [0,4194304] 0 2024-03-13T14:49:55.355 INFO:tasks.workunit.client.1.smithi196.stdout:0/782: chown d8/da/d25/d26/d48/dba 16478381 1 2024-03-13T14:49:55.358 INFO:tasks.workunit.client.1.smithi196.stdout:8/717: fdatasync d0/d10/d12/df5/fba 0 2024-03-13T14:49:55.359 INFO:tasks.workunit.client.0.smithi176.stdout:1/633: dwrite d2/d30/d63/d22/f82 [0,4194304] 0 2024-03-13T14:49:55.359 INFO:tasks.workunit.client.0.smithi176.stdout:0/569: mkdir d2/db/d35/daa/daf 0 2024-03-13T14:49:55.362 INFO:tasks.workunit.client.0.smithi176.stdout:3/698: mknod d4/d13/d2d/d50/cea 0 2024-03-13T14:49:55.364 INFO:tasks.workunit.client.0.smithi176.stdout:9/747: mkdir d0/d1e/df7 0 2024-03-13T14:49:55.365 INFO:tasks.workunit.client.1.smithi196.stdout:9/715: creat d7/de/d74/ff6 x:0 0 0 2024-03-13T14:49:55.365 INFO:tasks.workunit.client.1.smithi196.stdout:7/862: dread d0/d6/faa [0,4194304] 0 2024-03-13T14:49:55.367 INFO:tasks.workunit.client.1.smithi196.stdout:0/783: readlink d8/da/l5e 0 2024-03-13T14:49:55.369 INFO:tasks.workunit.client.0.smithi176.stdout:9/748: chown d0/db/d89/d90 72613 1 2024-03-13T14:49:55.369 INFO:tasks.workunit.client.1.smithi196.stdout:8/718: mknod d0/d16/d3d/d9c/d87/d8c/da9/db8/cf7 0 2024-03-13T14:49:55.370 INFO:tasks.workunit.client.1.smithi196.stdout:8/719: chown d0/d10/f11 807921490 1 2024-03-13T14:49:55.372 INFO:tasks.workunit.client.1.smithi196.stdout:6/918: symlink d6/d2a/dc/d1c/da7/l12b 0 2024-03-13T14:49:55.373 INFO:tasks.workunit.client.1.smithi196.stdout:1/833: truncate d1/d28/d34/d67/fdf 1179842 0 2024-03-13T14:49:55.374 INFO:tasks.workunit.client.0.smithi176.stdout:1/634: rmdir d2/de/d10/d16/d4a/d5f/da4 39 2024-03-13T14:49:55.374 INFO:tasks.workunit.client.0.smithi176.stdout:9/749: write d0/d1e/d1f/feb [937608,69465] 0 2024-03-13T14:49:55.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:55 smithi176 ceph-mon[29064]: pgmap v16: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 57 MiB/s rd, 163 MiB/s wr, 334 op/s 2024-03-13T14:49:55.381 INFO:tasks.workunit.client.1.smithi196.stdout:9/716: mknod d7/de/d4b/d1d/d93/da7/cf7 0 2024-03-13T14:49:55.381 INFO:tasks.workunit.client.1.smithi196.stdout:2/693: sync 2024-03-13T14:49:55.381 INFO:tasks.workunit.client.1.smithi196.stdout:1/834: write f0 [3546162,16480] 0 2024-03-13T14:49:55.381 INFO:tasks.workunit.client.0.smithi176.stdout:3/699: link d4/d13/l26 d4/d28/d31/d52/de9/leb 0 2024-03-13T14:49:55.383 INFO:tasks.workunit.client.0.smithi176.stdout:9/750: chown d0/db/def/db8/lcb 225927 1 2024-03-13T14:49:55.384 INFO:tasks.workunit.client.0.smithi176.stdout:9/751: chown d0/d1e 102853402 1 2024-03-13T14:49:55.385 INFO:tasks.workunit.client.1.smithi196.stdout:7/863: link d0/d6/d33/f4c d0/d6/d33/d57/dae/de8/f125 0 2024-03-13T14:49:55.386 INFO:tasks.workunit.client.0.smithi176.stdout:3/700: mknod d4/d13/d16/cec 0 2024-03-13T14:49:55.388 INFO:tasks.workunit.client.0.smithi176.stdout:1/635: truncate d2/de/d10/d16/d47/fbb 515896 0 2024-03-13T14:49:55.392 INFO:tasks.workunit.client.1.smithi196.stdout:0/784: dwrite d8/da/d74/ff1 [0,4194304] 0 2024-03-13T14:49:55.394 INFO:tasks.workunit.client.1.smithi196.stdout:9/717: fdatasync d7/d24/d28/f82 0 2024-03-13T14:49:55.397 INFO:tasks.workunit.client.0.smithi176.stdout:4/692: truncate d7/d8/d15/d94/fa6 377798 0 2024-03-13T14:49:55.403 INFO:tasks.workunit.client.1.smithi196.stdout:8/720: dwrite d0/d10/da5/db2/dc8/fd4 [0,4194304] 0 2024-03-13T14:49:55.404 INFO:tasks.workunit.client.1.smithi196.stdout:9/718: chown d7/de/d4b/d1d/d35 24 1 2024-03-13T14:49:55.414 INFO:tasks.workunit.client.0.smithi176.stdout:8/606: dwrite d0/d5/d3b/f81 [0,4194304] 0 2024-03-13T14:49:55.416 INFO:tasks.workunit.client.0.smithi176.stdout:6/654: truncate d0/d10/d19/d9f/f74 2319423 0 2024-03-13T14:49:55.416 INFO:tasks.workunit.client.0.smithi176.stdout:5/617: dread d5/d22/f29 [0,4194304] 0 2024-03-13T14:49:55.421 INFO:tasks.workunit.client.0.smithi176.stdout:1/636: dwrite d2/d30/d63/d22/d71/fb7 [0,4194304] 0 2024-03-13T14:49:55.423 INFO:tasks.workunit.client.1.smithi196.stdout:4/721: dwrite d1/d3/f13 [0,4194304] 0 2024-03-13T14:49:55.423 INFO:tasks.workunit.client.0.smithi176.stdout:6/655: chown d0/d10/d19/d9f/f98 3818491 1 2024-03-13T14:49:55.431 INFO:tasks.workunit.client.1.smithi196.stdout:7/864: dwrite d0/d6/ddc/ffd [0,4194304] 0 2024-03-13T14:49:55.432 INFO:tasks.workunit.client.1.smithi196.stdout:8/721: symlink d0/d16/d3d/d9c/d87/d8c/da9/db8/lf8 0 2024-03-13T14:49:55.434 INFO:tasks.workunit.client.1.smithi196.stdout:5/807: dwrite d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/fb0 [0,4194304] 0 2024-03-13T14:49:55.437 INFO:tasks.workunit.client.0.smithi176.stdout:8/607: creat d0/db/d39/d56/d93/da7/fbf x:0 0 0 2024-03-13T14:49:55.437 INFO:tasks.workunit.client.1.smithi196.stdout:2/694: dread d0/d7/d9/d9a/daa/fb8 [0,4194304] 0 2024-03-13T14:49:55.440 INFO:tasks.workunit.client.0.smithi176.stdout:9/752: truncate d0/db/d89/d32/d58/d5f/d84/fb1 50736 0 2024-03-13T14:49:55.445 INFO:tasks.workunit.client.0.smithi176.stdout:3/701: truncate d4/d13/d16/d3d/fb7 2043338 0 2024-03-13T14:49:55.445 INFO:tasks.workunit.client.1.smithi196.stdout:7/865: read d0/d6/faa [194973,4451] 0 2024-03-13T14:49:55.446 INFO:tasks.workunit.client.1.smithi196.stdout:3/777: dread db/d19/d4a/f8c [0,4194304] 0 2024-03-13T14:49:55.446 INFO:tasks.workunit.client.0.smithi176.stdout:7/625: dread d1/d1c/d26/d41/d2b/d40/d64/d94/f83 [0,4194304] 0 2024-03-13T14:49:55.451 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:55 smithi196 ceph-mon[37486]: pgmap v16: 65 pgs: 65 active+clean; 2.4 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 57 MiB/s rd, 163 MiB/s wr, 334 op/s 2024-03-13T14:49:55.456 INFO:tasks.workunit.client.1.smithi196.stdout:9/719: dwrite f5 [0,4194304] 0 2024-03-13T14:49:55.458 INFO:tasks.workunit.client.0.smithi176.stdout:8/608: fsync d0/d24/d7b/dab/d86/fa8 0 2024-03-13T14:49:55.462 INFO:tasks.workunit.client.1.smithi196.stdout:6/919: dread d6/d9/d17/d19/f22 [0,4194304] 0 2024-03-13T14:49:55.464 INFO:tasks.workunit.client.1.smithi196.stdout:7/866: write d0/fa [1067236,45069] 0 2024-03-13T14:49:55.466 INFO:tasks.workunit.client.0.smithi176.stdout:7/626: fdatasync d1/d28/d3a/d1b/d8f/d91/fc0 0 2024-03-13T14:49:55.466 INFO:tasks.workunit.client.0.smithi176.stdout:1/637: dwrite d2/de/d10/d16/d4a/f8a [0,4194304] 0 2024-03-13T14:49:55.467 INFO:tasks.workunit.client.0.smithi176.stdout:5/618: read d5/d22/d15/f2d [1566635,109877] 0 2024-03-13T14:49:55.469 INFO:tasks.workunit.client.0.smithi176.stdout:7/627: dread - d1/d1c/d26/d41/fad zero size 2024-03-13T14:49:55.471 INFO:tasks.workunit.client.0.smithi176.stdout:5/619: write d5/f6 [4291386,62241] 0 2024-03-13T14:49:55.471 INFO:tasks.workunit.client.1.smithi196.stdout:1/835: dwrite d1/d3f/f43 [0,4194304] 0 2024-03-13T14:49:55.473 INFO:tasks.workunit.client.0.smithi176.stdout:4/693: truncate d7/dab/db5/fc6 4174977 0 2024-03-13T14:49:55.483 INFO:tasks.workunit.client.0.smithi176.stdout:3/702: symlink d4/d13/d16/ddd/led 0 2024-03-13T14:49:55.489 INFO:tasks.workunit.client.0.smithi176.stdout:9/753: rename d0/d17/f48 to d0/db/ff8 0 2024-03-13T14:49:55.492 INFO:tasks.workunit.client.1.smithi196.stdout:4/722: creat d1/d3/fff x:0 0 0 2024-03-13T14:49:55.492 INFO:tasks.workunit.client.0.smithi176.stdout:9/754: fdatasync d0/d1e/d8d/fe7 0 2024-03-13T14:49:55.494 INFO:tasks.workunit.client.0.smithi176.stdout:1/638: dwrite d2/d30/d63/d22/f2b [0,4194304] 0 2024-03-13T14:49:55.498 INFO:tasks.workunit.client.1.smithi196.stdout:2/695: write d0/d6/d90/fb0 [3558,32674] 0 2024-03-13T14:49:55.499 INFO:tasks.workunit.client.0.smithi176.stdout:6/656: getdents d0/d10/d19/d27/d2b/db1 0 2024-03-13T14:49:55.501 INFO:tasks.workunit.client.0.smithi176.stdout:6/657: stat d0/de/d38/d6b/l71 0 2024-03-13T14:49:55.503 INFO:tasks.workunit.client.1.smithi196.stdout:7/867: fdatasync d0/d6/faa 0 2024-03-13T14:49:55.507 INFO:tasks.workunit.client.0.smithi176.stdout:5/620: mknod d5/d31/cd5 0 2024-03-13T14:49:55.507 INFO:tasks.workunit.client.0.smithi176.stdout:8/609: getdents d0/db/dc/db9 0 2024-03-13T14:49:55.510 INFO:tasks.workunit.client.1.smithi196.stdout:1/836: stat d1/d3f/d5b/d74/ca2 0 2024-03-13T14:49:55.514 INFO:tasks.workunit.client.1.smithi196.stdout:8/722: dwrite d0/d10/d12/d1e/fe5 [0,4194304] 0 2024-03-13T14:49:55.514 INFO:tasks.workunit.client.1.smithi196.stdout:4/723: fsync d1/d3/f98 0 2024-03-13T14:49:55.516 INFO:tasks.workunit.client.0.smithi176.stdout:7/628: rename d1/d28/d7c to d1/d28/d3a/d1b/d8f/dc1 0 2024-03-13T14:49:55.519 INFO:tasks.workunit.client.0.smithi176.stdout:9/755: mkdir d0/dd8/d82/df9 0 2024-03-13T14:49:55.522 INFO:tasks.workunit.client.1.smithi196.stdout:5/808: getdents d4/d1b/dc8/d14/d19/d1c/d25 0 2024-03-13T14:49:55.524 INFO:tasks.workunit.client.1.smithi196.stdout:2/696: mknod d0/d6/d1d/ce2 0 2024-03-13T14:49:55.528 INFO:tasks.workunit.client.0.smithi176.stdout:6/658: creat d0/d10/d19/fd5 x:0 0 0 2024-03-13T14:49:55.530 INFO:tasks.workunit.client.1.smithi196.stdout:3/778: rmdir db/d19/dae 0 2024-03-13T14:49:55.531 INFO:tasks.workunit.client.1.smithi196.stdout:3/779: chown db 23665575 1 2024-03-13T14:49:55.533 INFO:tasks.workunit.client.1.smithi196.stdout:7/868: mkdir d0/d6/ddc/d126 0 2024-03-13T14:49:55.539 INFO:tasks.workunit.client.1.smithi196.stdout:4/724: mknod d1/d3/d7/d35/d76/d58/dc9/c100 0 2024-03-13T14:49:55.540 INFO:tasks.workunit.client.0.smithi176.stdout:0/570: sync 2024-03-13T14:49:55.540 INFO:tasks.workunit.client.0.smithi176.stdout:2/687: sync 2024-03-13T14:49:55.542 INFO:tasks.workunit.client.1.smithi196.stdout:5/809: fdatasync d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/f64 0 2024-03-13T14:49:55.545 INFO:tasks.workunit.client.1.smithi196.stdout:3/780: dwrite db/d37/f76 [0,4194304] 0 2024-03-13T14:49:55.549 INFO:tasks.workunit.client.1.smithi196.stdout:7/869: mkdir d0/d4d/d74/d127 0 2024-03-13T14:49:55.556 INFO:tasks.workunit.client.1.smithi196.stdout:4/725: readlink d1/d3/d7/d35/l97 0 2024-03-13T14:49:55.559 INFO:tasks.workunit.client.1.smithi196.stdout:9/720: getdents d7/d71 0 2024-03-13T14:49:55.561 INFO:tasks.workunit.client.1.smithi196.stdout:4/726: write d1/d3/d4c/d72/fea [760924,60156] 0 2024-03-13T14:49:55.562 INFO:tasks.workunit.client.1.smithi196.stdout:5/810: symlink d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/l100 0 2024-03-13T14:49:55.565 INFO:tasks.workunit.client.1.smithi196.stdout:3/781: dwrite db/d19/d50/db7/da9/fb9 [0,4194304] 0 2024-03-13T14:49:55.568 INFO:tasks.workunit.client.1.smithi196.stdout:6/920: rename d6/d2a/dc/d1c/d34/c46 to d6/c12c 0 2024-03-13T14:49:55.568 INFO:tasks.workunit.client.1.smithi196.stdout:5/811: stat d4/d1b/dc8/d14/d19/d26/dab/fcd 0 2024-03-13T14:49:55.575 INFO:tasks.workunit.client.1.smithi196.stdout:2/697: link d0/d7/f87 d0/d7/fe3 0 2024-03-13T14:49:55.575 INFO:tasks.workunit.client.1.smithi196.stdout:5/812: chown d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/fb8 9677 1 2024-03-13T14:49:55.579 INFO:tasks.workunit.client.1.smithi196.stdout:8/723: mkdir d0/d10/d70/dc9/df9 0 2024-03-13T14:49:55.581 INFO:tasks.workunit.client.1.smithi196.stdout:5/813: dread - d4/d1b/dc8/d14/d19/d1c/d25/d38/ff3 zero size 2024-03-13T14:49:55.581 INFO:tasks.workunit.client.1.smithi196.stdout:6/921: creat d6/d9/d11/f12d x:0 0 0 2024-03-13T14:49:55.583 INFO:tasks.workunit.client.0.smithi176.stdout:3/703: symlink d4/d28/d34/lee 0 2024-03-13T14:49:55.585 INFO:tasks.workunit.client.0.smithi176.stdout:5/621: creat d5/d31/d47/d80/fd6 x:0 0 0 2024-03-13T14:49:55.585 INFO:tasks.workunit.client.1.smithi196.stdout:9/721: dwrite d7/de/d16/f9b [0,4194304] 0 2024-03-13T14:49:55.587 INFO:tasks.workunit.client.0.smithi176.stdout:8/610: rmdir d0/db/d39/d56/d91 39 2024-03-13T14:49:55.591 INFO:tasks.workunit.client.0.smithi176.stdout:4/694: rename d7/d10/d2c/f3f to d7/d8/d45/ff3 0 2024-03-13T14:49:55.595 INFO:tasks.workunit.client.1.smithi196.stdout:2/698: rename d0/d7/d66 to d0/d7/d9/d42/d98/de4 0 2024-03-13T14:49:55.597 INFO:tasks.workunit.client.1.smithi196.stdout:8/724: stat d0/d16/d3d/d7f/fa8 0 2024-03-13T14:49:55.599 INFO:tasks.workunit.client.0.smithi176.stdout:9/756: mknod d0/db/d89/d32/d58/d5f/d84/d9a/cfa 0 2024-03-13T14:49:55.599 INFO:tasks.workunit.client.1.smithi196.stdout:4/727: dread d1/d3/d4c/d72/d7d/d80/fac [0,4194304] 0 2024-03-13T14:49:55.603 INFO:tasks.workunit.client.0.smithi176.stdout:1/639: mkdir d2/de/d10/d16/d47/db0/dc7 0 2024-03-13T14:49:55.605 INFO:tasks.workunit.client.1.smithi196.stdout:5/814: dread d4/d1b/dc8/d14/d37/fee [0,4194304] 0 2024-03-13T14:49:55.607 INFO:tasks.workunit.client.0.smithi176.stdout:6/659: fsync d0/de/f49 0 2024-03-13T14:49:55.613 INFO:tasks.workunit.client.1.smithi196.stdout:6/922: mknod d6/d7/df/d27/c12e 0 2024-03-13T14:49:55.615 INFO:tasks.workunit.client.1.smithi196.stdout:3/782: link db/d19/d50/db7/l33 db/d19/d50/db7/lf8 0 2024-03-13T14:49:55.621 INFO:tasks.workunit.client.0.smithi176.stdout:8/611: truncate d0/d24/d29/f67 3407572 0 2024-03-13T14:49:55.622 INFO:tasks.workunit.client.1.smithi196.stdout:2/699: dwrite d0/d7/d8e/fdd [0,4194304] 0 2024-03-13T14:49:55.623 INFO:tasks.workunit.client.0.smithi176.stdout:8/612: truncate d0/d24/f57 5028629 0 2024-03-13T14:49:55.630 INFO:tasks.workunit.client.0.smithi176.stdout:7/629: write d1/d1c/f5e [2876731,128684] 0 2024-03-13T14:49:55.631 INFO:tasks.workunit.client.0.smithi176.stdout:9/757: creat d0/db/d89/d4d/d6e/ffb x:0 0 0 2024-03-13T14:49:55.635 INFO:tasks.workunit.client.0.smithi176.stdout:1/640: symlink d2/de/d10/d16/d47/lc8 0 2024-03-13T14:49:55.638 INFO:tasks.workunit.client.0.smithi176.stdout:6/660: truncate d0/de/d38/d70/f8e 1427712 0 2024-03-13T14:49:55.638 INFO:tasks.workunit.client.0.smithi176.stdout:1/641: chown d2/de/d10/d16/d4a/d5f/c60 3125 1 2024-03-13T14:49:55.639 INFO:tasks.workunit.client.1.smithi196.stdout:2/700: fsync d0/d6/d1d/f5e 0 2024-03-13T14:49:55.641 INFO:tasks.workunit.client.0.smithi176.stdout:8/613: dwrite d0/db/d39/d56/f9f [0,4194304] 0 2024-03-13T14:49:55.642 INFO:tasks.workunit.client.1.smithi196.stdout:2/701: dread d0/d6/d90/fb0 [0,4194304] 0 2024-03-13T14:49:55.642 INFO:tasks.workunit.client.0.smithi176.stdout:0/571: creat d2/db/d39/d9c/fb0 x:0 0 0 2024-03-13T14:49:55.646 INFO:tasks.workunit.client.1.smithi196.stdout:9/722: write d7/de/d4b/d1d/d22/f67 [663765,82582] 0 2024-03-13T14:49:55.646 INFO:tasks.workunit.client.0.smithi176.stdout:6/661: truncate d0/d10/d19/d27/d2b/d2e/fad 732250 0 2024-03-13T14:49:55.654 INFO:tasks.workunit.client.1.smithi196.stdout:8/725: dwrite d0/d16/d71/f43 [0,4194304] 0 2024-03-13T14:49:55.655 INFO:tasks.workunit.client.1.smithi196.stdout:6/923: write d6/d7/df/f97 [1020264,99704] 0 2024-03-13T14:49:55.657 INFO:tasks.workunit.client.0.smithi176.stdout:2/688: dwrite d1/d2a/f82 [0,4194304] 0 2024-03-13T14:49:55.658 INFO:tasks.workunit.client.0.smithi176.stdout:3/704: dwrite d4/d13/d81/d89/f32 [0,4194304] 0 2024-03-13T14:49:55.663 INFO:tasks.workunit.client.1.smithi196.stdout:5/815: dwrite d4/d1b/dc8/d14/d19/d1c/f3f [0,4194304] 0 2024-03-13T14:49:55.663 INFO:tasks.workunit.client.0.smithi176.stdout:7/630: creat d1/d3f/d4b/fc2 x:0 0 0 2024-03-13T14:49:55.664 INFO:tasks.workunit.client.1.smithi196.stdout:2/702: dread d0/d6/d4b/f64 [0,4194304] 0 2024-03-13T14:49:55.665 INFO:tasks.workunit.client.1.smithi196.stdout:9/723: mkdir d7/de/d4b/d1d/d93/df8 0 2024-03-13T14:49:55.665 INFO:tasks.workunit.client.0.smithi176.stdout:9/758: mknod d0/db/d89/d32/cfc 0 2024-03-13T14:49:55.667 INFO:tasks.workunit.client.0.smithi176.stdout:4/695: dwrite d7/d8/d33/f7f [0,4194304] 0 2024-03-13T14:49:55.668 INFO:tasks.workunit.client.0.smithi176.stdout:5/622: truncate d5/d2e/f3f 1968383 0 2024-03-13T14:49:55.671 INFO:tasks.workunit.client.1.smithi196.stdout:5/816: chown d4/d1b/dc8/d14/d19/d1c/d25/d38/c79 27884 1 2024-03-13T14:49:55.674 INFO:tasks.workunit.client.1.smithi196.stdout:6/924: creat d6/d2a/dc/d1c/da7/d9e/f12f x:0 0 0 2024-03-13T14:49:55.674 INFO:tasks.workunit.client.0.smithi176.stdout:1/642: truncate d2/de/d3f/f52 4675142 0 2024-03-13T14:49:55.675 INFO:tasks.workunit.client.1.smithi196.stdout:3/783: write db/f2c [378730,67548] 0 2024-03-13T14:49:55.676 INFO:tasks.workunit.client.0.smithi176.stdout:9/759: dread - d0/db/d89/d4d/d6e/fea zero size 2024-03-13T14:49:55.677 INFO:tasks.workunit.client.1.smithi196.stdout:5/817: write d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/fdd [182908,63870] 0 2024-03-13T14:49:55.678 INFO:tasks.workunit.client.0.smithi176.stdout:0/572: creat d2/db/d35/daa/fb1 x:0 0 0 2024-03-13T14:49:55.682 INFO:tasks.workunit.client.0.smithi176.stdout:6/662: rename d0/de/d54/d99 to d0/d10/d19/d9f/dd6 0 2024-03-13T14:49:55.684 INFO:tasks.workunit.client.1.smithi196.stdout:3/784: dread db/d15/f1a [0,4194304] 0 2024-03-13T14:49:55.689 INFO:tasks.workunit.client.0.smithi176.stdout:3/705: mknod d4/db8/cef 0 2024-03-13T14:49:55.689 INFO:tasks.workunit.client.1.smithi196.stdout:2/703: fdatasync d0/d6/d1d/f5e 0 2024-03-13T14:49:55.690 INFO:tasks.workunit.client.1.smithi196.stdout:5/818: rmdir d4/d1b/dc8 39 2024-03-13T14:49:55.693 INFO:tasks.workunit.client.0.smithi176.stdout:5/623: unlink d5/d2c/f66 0 2024-03-13T14:49:55.701 INFO:tasks.workunit.client.1.smithi196.stdout:3/785: dread db/d15/d44/d51/fb4 [0,4194304] 0 2024-03-13T14:49:55.702 INFO:tasks.workunit.client.0.smithi176.stdout:7/631: symlink d1/d1c/d26/d41/d2b/d40/lc3 0 2024-03-13T14:49:55.703 INFO:tasks.workunit.client.1.smithi196.stdout:0/785: sync 2024-03-13T14:49:55.705 INFO:tasks.workunit.client.0.smithi176.stdout:3/706: mkdir d4/d13/d81/d89/d5b/dc5/df0 0 2024-03-13T14:49:55.705 INFO:tasks.workunit.client.0.smithi176.stdout:9/760: dwrite d0/d17/f19 [4194304,4194304] 0 2024-03-13T14:49:55.708 INFO:tasks.workunit.client.0.smithi176.stdout:0/573: dwrite d2/db/d39/fa2 [0,4194304] 0 2024-03-13T14:49:55.709 INFO:tasks.workunit.client.0.smithi176.stdout:5/624: chown d5/d22/d15/l1a 0 1 2024-03-13T14:49:55.709 INFO:tasks.workunit.client.0.smithi176.stdout:4/696: creat d7/d8/d15/ff4 x:0 0 0 2024-03-13T14:49:55.715 INFO:tasks.workunit.client.0.smithi176.stdout:5/625: fdatasync d5/d75/f8c 0 2024-03-13T14:49:55.718 INFO:tasks.workunit.client.0.smithi176.stdout:9/761: creat d0/db/d89/d7a/d8b/ffd x:0 0 0 2024-03-13T14:49:55.722 INFO:tasks.workunit.client.0.smithi176.stdout:5/626: symlink d5/d2c/d40/dc7/d89/ld7 0 2024-03-13T14:49:55.725 INFO:tasks.workunit.client.0.smithi176.stdout:7/632: dwrite d1/d3f/d4b/d63/fb3 [0,4194304] 0 2024-03-13T14:49:55.725 INFO:tasks.workunit.client.0.smithi176.stdout:9/762: creat d0/d1e/d8d/dbe/ffe x:0 0 0 2024-03-13T14:49:55.735 INFO:tasks.workunit.client.1.smithi196.stdout:8/726: truncate d0/d10/d12/df5/f65 4125258 0 2024-03-13T14:49:55.737 INFO:tasks.workunit.client.1.smithi196.stdout:6/925: truncate d6/d2a/d23/d28/f69 150864 0 2024-03-13T14:49:55.740 INFO:tasks.workunit.client.1.smithi196.stdout:8/727: symlink d0/d10/d79/lfa 0 2024-03-13T14:49:55.742 INFO:tasks.workunit.client.1.smithi196.stdout:2/704: dread d0/d7/f3f [0,4194304] 0 2024-03-13T14:49:55.743 INFO:tasks.workunit.client.0.smithi176.stdout:0/574: dwrite d2/f3 [0,4194304] 0 2024-03-13T14:49:55.743 INFO:tasks.workunit.client.1.smithi196.stdout:8/728: fdatasync d0/d10/d12/d1e/fe5 0 2024-03-13T14:49:55.744 INFO:tasks.workunit.client.0.smithi176.stdout:5/627: rename d5/d2c/d74/c9e to d5/cd8 0 2024-03-13T14:49:55.746 INFO:tasks.workunit.client.1.smithi196.stdout:6/926: truncate d6/d9/d17/dfc/f11b 552738 0 2024-03-13T14:49:55.747 INFO:tasks.workunit.client.0.smithi176.stdout:1/643: write d2/f21 [4368584,81874] 0 2024-03-13T14:49:55.747 INFO:tasks.workunit.client.1.smithi196.stdout:9/724: dwrite d7/d24/dbf/f5c [0,4194304] 0 2024-03-13T14:49:55.750 INFO:tasks.workunit.client.0.smithi176.stdout:0/575: dread - d2/d5/dc/f65 zero size 2024-03-13T14:49:55.753 INFO:tasks.workunit.client.1.smithi196.stdout:2/705: symlink d0/d7/d9/d42/d98/le5 0 2024-03-13T14:49:55.754 INFO:tasks.workunit.client.1.smithi196.stdout:5/819: write d4/d1b/dc8/d14/d37/f78 [999053,108046] 0 2024-03-13T14:49:55.756 INFO:tasks.workunit.client.0.smithi176.stdout:8/614: dwrite d0/d24/f6f [0,4194304] 0 2024-03-13T14:49:55.760 INFO:tasks.workunit.client.1.smithi196.stdout:0/786: write d8/da/fb0 [1282559,83314] 0 2024-03-13T14:49:55.765 INFO:tasks.workunit.client.0.smithi176.stdout:2/689: dread d1/dc/f2e [0,4194304] 0 2024-03-13T14:49:55.766 INFO:tasks.workunit.client.1.smithi196.stdout:3/786: dread db/d19/f1d [0,4194304] 0 2024-03-13T14:49:55.766 INFO:tasks.workunit.client.0.smithi176.stdout:6/663: dwrite d0/d10/d19/d27/d2b/d2e/f7a [0,4194304] 0 2024-03-13T14:49:55.768 INFO:tasks.workunit.client.1.smithi196.stdout:6/927: rename d6/d57 to d6/d7/df/d27/d39/d44/dc8/d130 0 2024-03-13T14:49:55.769 INFO:tasks.workunit.client.1.smithi196.stdout:0/787: readlink d8/da/d25/d26/d35/d3b/d5f/lff 0 2024-03-13T14:49:55.776 INFO:tasks.workunit.client.0.smithi176.stdout:4/697: write d7/d8/d2f/fcd [1786513,50726] 0 2024-03-13T14:49:55.778 INFO:tasks.workunit.client.1.smithi196.stdout:2/706: dwrite d0/d50/fd5 [0,4194304] 0 2024-03-13T14:49:55.783 INFO:tasks.workunit.client.0.smithi176.stdout:0/576: dread d2/d5/dc/d20/f34 [4194304,4194304] 0 2024-03-13T14:49:55.783 INFO:tasks.workunit.client.0.smithi176.stdout:5/628: link d5/d2e/d5f/f7c d5/d2e/d5f/fd9 0 2024-03-13T14:49:55.784 INFO:tasks.workunit.client.0.smithi176.stdout:3/707: dwrite d4/d13/d81/d89/f60 [0,4194304] 0 2024-03-13T14:49:55.792 INFO:tasks.workunit.client.1.smithi196.stdout:6/928: mkdir d6/d9/d17/d19/d4e/d66/d131 0 2024-03-13T14:49:55.793 INFO:tasks.workunit.client.0.smithi176.stdout:3/708: chown d4/d28/d31/d52/ce7 18 1 2024-03-13T14:49:55.794 INFO:tasks.workunit.client.1.smithi196.stdout:9/725: creat d7/de/d74/ff9 x:0 0 0 2024-03-13T14:49:55.798 INFO:tasks.workunit.client.0.smithi176.stdout:9/763: dwrite d0/d40/d53/fba [0,4194304] 0 2024-03-13T14:49:55.798 INFO:tasks.workunit.client.0.smithi176.stdout:6/664: creat d0/de/d38/d88/d9d/fd7 x:0 0 0 2024-03-13T14:49:55.804 INFO:tasks.workunit.client.0.smithi176.stdout:9/764: stat d0/d1e/d1f/fd6 0 2024-03-13T14:49:55.804 INFO:tasks.workunit.client.0.smithi176.stdout:6/665: write d0/d10/d19/f63 [3932740,89030] 0 2024-03-13T14:49:55.805 INFO:tasks.workunit.client.0.smithi176.stdout:1/644: link d2/de/d10/d16/d40/f77 d2/de/d3f/d3e/d86/db3/fc9 0 2024-03-13T14:49:55.807 INFO:tasks.workunit.client.0.smithi176.stdout:7/633: truncate d1/d1c/d26/d41/d2b/d40/d64/d94/f83 517478 0 2024-03-13T14:49:55.808 INFO:tasks.workunit.client.0.smithi176.stdout:3/709: read d4/d13/f73 [1151475,68423] 0 2024-03-13T14:49:55.809 INFO:tasks.workunit.client.1.smithi196.stdout:0/788: dread d8/da/d25/d26/d48/dba/fab [0,4194304] 0 2024-03-13T14:49:55.816 INFO:tasks.workunit.client.1.smithi196.stdout:8/729: dwrite d0/d16/d3d/d7f/fb3 [0,4194304] 0 2024-03-13T14:49:55.819 INFO:tasks.workunit.client.1.smithi196.stdout:8/730: chown d0/d16/fd9 27 1 2024-03-13T14:49:55.827 INFO:tasks.workunit.client.0.smithi176.stdout:7/634: dwrite d1/d3f/d4b/d63/fbd [0,4194304] 0 2024-03-13T14:49:55.829 INFO:tasks.workunit.client.0.smithi176.stdout:7/635: truncate d1/faf 772403 0 2024-03-13T14:49:55.830 INFO:tasks.workunit.client.0.smithi176.stdout:6/666: truncate d0/de/f49 245517 0 2024-03-13T14:49:55.832 INFO:tasks.workunit.client.0.smithi176.stdout:7/636: write d1/d1c/d26/fb0 [484260,69751] 0 2024-03-13T14:49:55.834 INFO:tasks.workunit.client.0.smithi176.stdout:1/645: truncate d2/de/f53 295868 0 2024-03-13T14:49:55.838 INFO:tasks.workunit.client.0.smithi176.stdout:7/637: truncate d1/d7/f3d 2072587 0 2024-03-13T14:49:55.840 INFO:tasks.workunit.client.0.smithi176.stdout:8/615: dread d0/db/d39/d56/f79 [0,4194304] 0 2024-03-13T14:49:55.841 INFO:tasks.workunit.client.0.smithi176.stdout:1/646: read d2/de/d10/d16/d4a/d5f/f81 [1055163,93328] 0 2024-03-13T14:49:55.843 INFO:tasks.workunit.client.0.smithi176.stdout:8/616: write d0/d5/d11/f5f [285655,109982] 0 2024-03-13T14:49:55.845 INFO:tasks.workunit.client.0.smithi176.stdout:8/617: stat d0/db/dc/l18 0 2024-03-13T14:49:55.846 INFO:tasks.workunit.client.0.smithi176.stdout:6/667: creat d0/d10/d19/d9f/dd6/fd8 x:0 0 0 2024-03-13T14:49:55.847 INFO:tasks.workunit.client.0.smithi176.stdout:2/690: truncate d1/dc/f52 843992 0 2024-03-13T14:49:55.847 INFO:tasks.workunit.client.1.smithi196.stdout:1/837: sync 2024-03-13T14:49:55.848 INFO:tasks.workunit.client.0.smithi176.stdout:7/638: stat d1/d3f/d65/c8c 0 2024-03-13T14:49:55.848 INFO:tasks.workunit.client.0.smithi176.stdout:6/668: write d0/d9/f55 [1404146,71279] 0 2024-03-13T14:49:55.849 INFO:tasks.workunit.client.1.smithi196.stdout:1/838: chown d1/d9/d4e/c8a 20634 1 2024-03-13T14:49:55.850 INFO:tasks.workunit.client.0.smithi176.stdout:0/577: creat d2/d5/fb2 x:0 0 0 2024-03-13T14:49:55.855 INFO:tasks.workunit.client.0.smithi176.stdout:1/647: symlink d2/de/d3f/d3e/lca 0 2024-03-13T14:49:55.855 INFO:tasks.workunit.client.0.smithi176.stdout:4/698: write d7/d8/d15/d94/fa6 [497339,12848] 0 2024-03-13T14:49:55.860 INFO:tasks.workunit.client.0.smithi176.stdout:4/699: write d7/d8/d15/fbc [1028210,121137] 0 2024-03-13T14:49:55.864 INFO:tasks.workunit.client.0.smithi176.stdout:7/639: write d1/d3f/d76/fb9 [3904137,84820] 0 2024-03-13T14:49:55.867 INFO:tasks.workunit.client.0.smithi176.stdout:0/578: creat d2/d5/dc/d59/fb3 x:0 0 0 2024-03-13T14:49:55.872 INFO:tasks.workunit.client.0.smithi176.stdout:2/691: creat d1/dc/d61/daa/dcc/fe7 x:0 0 0 2024-03-13T14:49:55.872 INFO:tasks.workunit.client.0.smithi176.stdout:1/648: dread d2/de/d3f/d3e/d86/db3/fc9 [0,4194304] 0 2024-03-13T14:49:55.875 INFO:tasks.workunit.client.0.smithi176.stdout:3/710: dwrite f2 [0,4194304] 0 2024-03-13T14:49:55.876 INFO:tasks.workunit.client.0.smithi176.stdout:5/629: dwrite d5/d31/f45 [0,4194304] 0 2024-03-13T14:49:55.879 INFO:tasks.workunit.client.0.smithi176.stdout:7/640: mknod d1/d3f/cc4 0 2024-03-13T14:49:55.882 INFO:tasks.workunit.client.0.smithi176.stdout:0/579: readlink d2/d5/dc/l67 0 2024-03-13T14:49:55.884 INFO:tasks.workunit.client.0.smithi176.stdout:6/669: write d0/d10/d19/d27/f4a [302250,46130] 0 2024-03-13T14:49:55.888 INFO:tasks.workunit.client.0.smithi176.stdout:3/711: symlink d4/d28/lf1 0 2024-03-13T14:49:55.889 INFO:tasks.workunit.client.1.smithi196.stdout:4/728: sync 2024-03-13T14:49:55.889 INFO:tasks.workunit.client.1.smithi196.stdout:7/870: sync 2024-03-13T14:49:55.891 INFO:tasks.workunit.client.1.smithi196.stdout:7/871: readlink d0/d6/d33/d57/d8a/lbc 0 2024-03-13T14:49:55.893 INFO:tasks.workunit.client.0.smithi176.stdout:1/649: rename d2/d30/d63/d22/d71 to d2/de/d3f/d50/d83/dcb 0 2024-03-13T14:49:55.896 INFO:tasks.workunit.client.1.smithi196.stdout:2/707: creat d0/d7/d9/d42/d98/de4/fe6 x:0 0 0 2024-03-13T14:49:55.896 INFO:tasks.workunit.client.0.smithi176.stdout:7/641: creat d1/d3f/d81/fc5 x:0 0 0 2024-03-13T14:49:55.898 INFO:tasks.workunit.client.1.smithi196.stdout:6/929: truncate d6/d9/f30 2913845 0 2024-03-13T14:49:55.902 INFO:tasks.workunit.client.0.smithi176.stdout:4/700: write d7/d10/d46/d4a/f79 [2663718,73437] 0 2024-03-13T14:49:55.903 INFO:tasks.workunit.client.0.smithi176.stdout:6/670: dwrite d0/d9/f55 [0,4194304] 0 2024-03-13T14:49:55.906 INFO:tasks.workunit.client.0.smithi176.stdout:1/650: creat d2/de/d3f/fcc x:0 0 0 2024-03-13T14:49:55.910 INFO:tasks.workunit.client.0.smithi176.stdout:7/642: mknod d1/d1c/d26/d41/d2b/d40/d64/d94/cc6 0 2024-03-13T14:49:55.911 INFO:tasks.workunit.client.1.smithi196.stdout:1/839: creat d1/d9/d33/dd2/f120 x:0 0 0 2024-03-13T14:49:55.912 INFO:tasks.workunit.client.0.smithi176.stdout:7/643: fsync d1/d3f/d76/f1e 0 2024-03-13T14:49:55.913 INFO:tasks.workunit.client.1.smithi196.stdout:4/729: unlink d1/d3/dbc/fe6 0 2024-03-13T14:49:55.916 INFO:tasks.workunit.client.0.smithi176.stdout:5/630: link d5/d2e/d7e/dad/f65 d5/d2c/d40/dc7/fda 0 2024-03-13T14:49:55.922 INFO:tasks.workunit.client.0.smithi176.stdout:1/651: dwrite d2/de/dbf/fa5 [0,4194304] 0 2024-03-13T14:49:55.926 INFO:tasks.workunit.client.0.smithi176.stdout:3/712: creat d4/d13/d16/ff2 x:0 0 0 2024-03-13T14:49:55.927 INFO:tasks.workunit.client.1.smithi196.stdout:5/820: getdents d4/d1b/dc8/d14/d19/d1c/d2d/d82 0 2024-03-13T14:49:55.929 INFO:tasks.workunit.client.1.smithi196.stdout:0/789: mknod d8/da/c102 0 2024-03-13T14:49:55.931 INFO:tasks.workunit.client.0.smithi176.stdout:2/692: truncate d1/dc/d2d/d59/d8c/f8f 835904 0 2024-03-13T14:49:55.932 INFO:tasks.workunit.client.0.smithi176.stdout:5/631: mknod d5/d2c/d40/d76/cdb 0 2024-03-13T14:49:55.933 INFO:tasks.workunit.client.0.smithi176.stdout:5/632: stat d5/d2c/d8a/lb0 0 2024-03-13T14:49:55.937 INFO:tasks.workunit.client.1.smithi196.stdout:4/730: fdatasync d1/d3/d7/d35/d76/d58/d69/d9f/fe0 0 2024-03-13T14:49:55.938 INFO:tasks.workunit.client.1.smithi196.stdout:3/787: getdents db/d19/d50/db7/da9 0 2024-03-13T14:49:55.941 INFO:tasks.workunit.client.0.smithi176.stdout:4/701: write d7/d10/f27 [2781421,112633] 0 2024-03-13T14:49:55.943 INFO:tasks.workunit.client.1.smithi196.stdout:9/726: rename d7/de/d4b/d1d/d22/l26 to d7/de/lfa 0 2024-03-13T14:49:55.944 INFO:tasks.workunit.client.0.smithi176.stdout:3/713: truncate d4/d13/d81/d89/fb4 800135 0 2024-03-13T14:49:55.948 INFO:tasks.workunit.client.1.smithi196.stdout:7/872: dwrite d0/f1c [4194304,4194304] 0 2024-03-13T14:49:55.949 INFO:tasks.workunit.client.1.smithi196.stdout:5/821: unlink d4/d1b/dc8/d14/d19/d26/f92 0 2024-03-13T14:49:55.951 INFO:tasks.workunit.client.0.smithi176.stdout:6/671: creat d0/de/d38/d6b/fd9 x:0 0 0 2024-03-13T14:49:55.952 INFO:tasks.workunit.client.1.smithi196.stdout:0/790: symlink d8/da/d25/d26/d48/l103 0 2024-03-13T14:49:55.953 INFO:tasks.workunit.client.0.smithi176.stdout:1/652: rename d2/f6 to d2/de/d10/d16/d40/fcd 0 2024-03-13T14:49:55.953 INFO:tasks.workunit.client.0.smithi176.stdout:6/672: write d0/d10/d19/d9f/d76/fab [958189,31858] 0 2024-03-13T14:49:55.956 INFO:tasks.workunit.client.1.smithi196.stdout:1/840: link d1/d3f/d5b/d74/db5/ffb d1/d9/d4e/d55/d63/da6/dc8/d11d/f121 0 2024-03-13T14:49:55.957 INFO:tasks.workunit.client.1.smithi196.stdout:1/841: chown d1/d9/d4e 164736 1 2024-03-13T14:49:55.959 INFO:tasks.workunit.client.1.smithi196.stdout:4/731: truncate d1/d1e/f6a 828926 0 2024-03-13T14:49:55.960 INFO:tasks.workunit.client.0.smithi176.stdout:3/714: truncate d4/d28/d34/f3f 366213 0 2024-03-13T14:49:55.960 INFO:tasks.workunit.client.1.smithi196.stdout:1/842: fdatasync d1/d9/d4e/d55/d63/db9/dbb/fc2 0 2024-03-13T14:49:55.961 INFO:tasks.workunit.client.0.smithi176.stdout:5/633: dwrite d5/d2c/d40/dc7/f6e [0,4194304] 0 2024-03-13T14:49:55.961 INFO:tasks.workunit.client.1.smithi196.stdout:6/930: creat d6/d7/f132 x:0 0 0 2024-03-13T14:49:55.962 INFO:tasks.workunit.client.1.smithi196.stdout:8/731: rename d0/d16/d71/d9a/dca to d0/d10/d12/df5/d47/dfb 0 2024-03-13T14:49:55.964 INFO:tasks.workunit.client.0.smithi176.stdout:7/644: getdents d1/d28 0 2024-03-13T14:49:55.968 INFO:tasks.workunit.client.1.smithi196.stdout:5/822: mkdir d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/d101 0 2024-03-13T14:49:55.968 INFO:tasks.workunit.client.1.smithi196.stdout:0/791: rmdir d8/da/d25/d26/d35 39 2024-03-13T14:49:55.969 INFO:tasks.workunit.client.0.smithi176.stdout:6/673: fdatasync d0/d10/f3f 0 2024-03-13T14:49:55.970 INFO:tasks.workunit.client.0.smithi176.stdout:2/693: creat d1/dc/d2d/d38/fe8 x:0 0 0 2024-03-13T14:49:55.972 INFO:tasks.workunit.client.1.smithi196.stdout:3/788: link db/d15/d44/d79/fe6 db/d19/d22/d3a/d53/ff9 0 2024-03-13T14:49:55.976 INFO:tasks.workunit.client.1.smithi196.stdout:5/823: read - d4/d1b/dc8/d14/d19/d26/fe6 zero size 2024-03-13T14:49:55.977 INFO:tasks.workunit.client.0.smithi176.stdout:1/653: creat d2/d29/fce x:0 0 0 2024-03-13T14:49:55.979 INFO:tasks.workunit.client.1.smithi196.stdout:6/931: dwrite d6/d9/daa/fc2 [0,4194304] 0 2024-03-13T14:49:55.982 INFO:tasks.workunit.client.1.smithi196.stdout:2/708: rename d0/d50/cac to d0/d7/d9/d5a/ce7 0 2024-03-13T14:49:55.984 INFO:tasks.workunit.client.1.smithi196.stdout:6/932: chown d6/d9/d11/f5e 1795067 1 2024-03-13T14:49:55.985 INFO:tasks.workunit.client.1.smithi196.stdout:8/732: mkdir d0/d16/d71/d9a/dfc 0 2024-03-13T14:49:55.989 INFO:tasks.workunit.client.0.smithi176.stdout:2/694: fsync d1/dc/d1c/f37 0 2024-03-13T14:49:55.991 INFO:tasks.workunit.client.1.smithi196.stdout:0/792: fsync d8/da/d25/f27 0 2024-03-13T14:49:55.992 INFO:tasks.workunit.client.0.smithi176.stdout:6/674: mknod d0/d10/d19/d9f/dd6/cda 0 2024-03-13T14:49:55.996 INFO:tasks.workunit.client.0.smithi176.stdout:3/715: mknod d4/d13/d81/d89/cf3 0 2024-03-13T14:49:55.998 INFO:tasks.workunit.client.1.smithi196.stdout:9/727: rename d7/d24 to d7/de/d74/d94/dfb 0 2024-03-13T14:49:55.999 INFO:tasks.workunit.client.0.smithi176.stdout:4/702: getdents d7/d10/d46/d4a/dcc 0 2024-03-13T14:49:56.001 INFO:tasks.workunit.client.1.smithi196.stdout:5/824: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/f102 x:0 0 0 2024-03-13T14:49:56.004 INFO:tasks.workunit.client.1.smithi196.stdout:2/709: fdatasync d0/d7/d9/d61/f93 0 2024-03-13T14:49:56.006 INFO:tasks.workunit.client.1.smithi196.stdout:6/933: creat d6/d9/d17/da1/d50/f133 x:0 0 0 2024-03-13T14:49:56.008 INFO:tasks.workunit.client.1.smithi196.stdout:6/934: readlink d6/d7/df/d27/d39/d44/dc8/d130/d91/lce 0 2024-03-13T14:49:56.009 INFO:tasks.workunit.client.0.smithi176.stdout:7/645: dread d1/d1c/f79 [0,4194304] 0 2024-03-13T14:49:56.010 INFO:tasks.workunit.client.0.smithi176.stdout:2/695: rmdir d1/d2a 39 2024-03-13T14:49:56.011 INFO:tasks.workunit.client.0.smithi176.stdout:5/634: dread f0 [0,4194304] 0 2024-03-13T14:49:56.012 INFO:tasks.workunit.client.0.smithi176.stdout:6/675: mknod d0/de/d38/d88/cdb 0 2024-03-13T14:49:56.014 INFO:tasks.workunit.client.0.smithi176.stdout:3/716: readlink d4/d13/d81/d89/d5b/ldb 0 2024-03-13T14:49:56.016 INFO:tasks.workunit.client.1.smithi196.stdout:4/732: dwrite d1/d3/d7/ddf/fd2 [0,4194304] 0 2024-03-13T14:49:56.020 INFO:tasks.workunit.client.0.smithi176.stdout:7/646: mknod d1/d28/d3a/d1b/d8f/dc1/cc7 0 2024-03-13T14:49:56.024 INFO:tasks.workunit.client.1.smithi196.stdout:1/843: rename d1/dfa to d1/d9/d4e/d55/d63/da6/dc8/d11d/d122 0 2024-03-13T14:49:56.027 INFO:tasks.workunit.client.0.smithi176.stdout:5/635: creat d5/d2e/d5f/fdc x:0 0 0 2024-03-13T14:49:56.028 INFO:tasks.workunit.client.0.smithi176.stdout:2/696: mknod d1/dc/d2d/d4b/ddb/ce9 0 2024-03-13T14:49:56.029 INFO:tasks.workunit.client.1.smithi196.stdout:7/873: dwrite d0/d78/d81/dfb/f107 [0,4194304] 0 2024-03-13T14:49:56.032 INFO:tasks.workunit.client.0.smithi176.stdout:3/717: symlink d4/d13/d81/d89/d5b/lf4 0 2024-03-13T14:49:56.037 INFO:tasks.workunit.client.0.smithi176.stdout:4/703: write d7/d8/d2f/f37 [207755,107722] 0 2024-03-13T14:49:56.039 INFO:tasks.workunit.client.1.smithi196.stdout:6/935: rmdir d6/d7/df 39 2024-03-13T14:49:56.041 INFO:tasks.workunit.client.0.smithi176.stdout:1/654: dwrite d2/de/d10/d16/d4a/d5f/f81 [0,4194304] 0 2024-03-13T14:49:56.042 INFO:tasks.workunit.client.1.smithi196.stdout:6/936: write d6/d6f/ff5 [1162098,87214] 0 2024-03-13T14:49:56.045 INFO:tasks.workunit.client.1.smithi196.stdout:3/789: creat db/d37/ffa x:0 0 0 2024-03-13T14:49:56.048 INFO:tasks.workunit.client.0.smithi176.stdout:2/697: mkdir d1/dc/d2d/d90/dcf/dea 0 2024-03-13T14:49:56.048 INFO:tasks.workunit.client.0.smithi176.stdout:1/655: fsync d2/de/d10/d16/d4a/f8a 0 2024-03-13T14:49:56.050 INFO:tasks.workunit.client.0.smithi176.stdout:4/704: creat d7/d10/d46/d4a/d5d/ff5 x:0 0 0 2024-03-13T14:49:56.051 INFO:tasks.workunit.client.1.smithi196.stdout:0/793: truncate d8/da/d25/d26/d48/d82/db3/dd5/fe5 4669283 0 2024-03-13T14:49:56.057 INFO:tasks.workunit.client.1.smithi196.stdout:9/728: symlink d7/de/d4b/d1d/d93/df8/lfc 0 2024-03-13T14:49:56.062 INFO:tasks.workunit.client.1.smithi196.stdout:5/825: mkdir d4/d1b/dc8/d3c/d103 0 2024-03-13T14:49:56.065 INFO:tasks.workunit.client.1.smithi196.stdout:7/874: creat d0/d6/d33/d57/dae/de8/f128 x:0 0 0 2024-03-13T14:49:56.066 INFO:tasks.workunit.client.0.smithi176.stdout:2/698: rmdir d1/dc/d2d/d59/d8b/daf 39 2024-03-13T14:49:56.066 INFO:tasks.workunit.client.1.smithi196.stdout:8/733: link d0/d16/d71/d68/c90 d0/d10/d12/d1e/dc5/cfd 0 2024-03-13T14:49:56.067 INFO:tasks.workunit.client.0.smithi176.stdout:3/718: write d4/d13/d81/d89/d5b/dc5/fa1 [229692,51536] 0 2024-03-13T14:49:56.069 INFO:tasks.workunit.client.1.smithi196.stdout:2/710: symlink d0/d6/d4b/le8 0 2024-03-13T14:49:56.071 INFO:tasks.workunit.client.0.smithi176.stdout:3/719: chown d4/d13/d81/d89/cf3 268667149 1 2024-03-13T14:49:56.075 INFO:tasks.workunit.client.0.smithi176.stdout:5/636: dwrite d5/d2c/d40/f8f [0,4194304] 0 2024-03-13T14:49:56.078 INFO:tasks.workunit.client.0.smithi176.stdout:5/637: fdatasync d5/d31/d47/d80/fd6 0 2024-03-13T14:49:56.078 INFO:tasks.workunit.client.0.smithi176.stdout:1/656: dread - d2/de/d3f/d50/f89 zero size 2024-03-13T14:49:56.080 INFO:tasks.workunit.client.0.smithi176.stdout:4/705: rename d7/d10/d46/d55 to d7/d8/df6 0 2024-03-13T14:49:56.085 INFO:tasks.workunit.client.0.smithi176.stdout:3/720: chown d4/d28/d31/d52/cab 10175769 1 2024-03-13T14:49:56.086 INFO:tasks.workunit.client.1.smithi196.stdout:5/826: symlink d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd/l104 0 2024-03-13T14:49:56.088 INFO:tasks.workunit.client.0.smithi176.stdout:8/618: sync 2024-03-13T14:49:56.088 INFO:tasks.workunit.client.0.smithi176.stdout:9/765: sync 2024-03-13T14:49:56.089 INFO:tasks.workunit.client.1.smithi196.stdout:8/734: mkdir d0/d16/d71/d68/dfe 0 2024-03-13T14:49:56.090 INFO:tasks.workunit.client.1.smithi196.stdout:8/735: stat d0/d10/d12/l39 0 2024-03-13T14:49:56.090 INFO:tasks.workunit.client.0.smithi176.stdout:8/619: chown d0/d24/f31 66672972 1 2024-03-13T14:49:56.092 INFO:tasks.workunit.client.0.smithi176.stdout:9/766: write d0/db/d89/d32/d58/d67/f7e [929051,51281] 0 2024-03-13T14:49:56.094 INFO:tasks.workunit.client.0.smithi176.stdout:4/706: creat d7/d10/d46/d4a/dcc/ff7 x:0 0 0 2024-03-13T14:49:56.094 INFO:tasks.workunit.client.1.smithi196.stdout:2/711: truncate d0/d6/d1d/f7a 454508 0 2024-03-13T14:49:56.099 INFO:tasks.workunit.client.1.smithi196.stdout:3/790: mkdir db/d15/dfb 0 2024-03-13T14:49:56.099 INFO:tasks.workunit.client.0.smithi176.stdout:3/721: rename d4/d13/d2d/d50 to d4/db8/df5 0 2024-03-13T14:49:56.100 INFO:tasks.workunit.client.1.smithi196.stdout:1/844: dwrite d1/d9/f9a [0,4194304] 0 2024-03-13T14:49:56.101 INFO:tasks.workunit.client.0.smithi176.stdout:1/657: creat d2/de/d3f/d50/d83/d97/fcf x:0 0 0 2024-03-13T14:49:56.103 INFO:tasks.workunit.client.0.smithi176.stdout:3/722: dread - d4/d13/d16/dad/fe6 zero size 2024-03-13T14:49:56.103 INFO:tasks.workunit.client.1.smithi196.stdout:0/794: unlink d8/da/d25/d26/d48/d82/db3/lec 0 2024-03-13T14:49:56.104 INFO:tasks.workunit.client.1.smithi196.stdout:5/827: mkdir d4/d1b/dc8/d14/d19/da7/d105 0 2024-03-13T14:49:56.107 INFO:tasks.workunit.client.0.smithi176.stdout:2/699: getdents d1/dc/d1c 0 2024-03-13T14:49:56.109 INFO:tasks.workunit.client.0.smithi176.stdout:8/620: dread d0/f30 [0,4194304] 0 2024-03-13T14:49:56.110 INFO:tasks.workunit.client.1.smithi196.stdout:8/736: mknod d0/d10/d12/d1e/cff 0 2024-03-13T14:49:56.111 INFO:tasks.workunit.client.0.smithi176.stdout:5/638: dwrite d5/d31/f3e [0,4194304] 0 2024-03-13T14:49:56.113 INFO:tasks.workunit.client.1.smithi196.stdout:8/737: readlink d0/l44 0 2024-03-13T14:49:56.116 INFO:tasks.workunit.client.1.smithi196.stdout:4/733: getdents d1/d3/d7/ddf 0 2024-03-13T14:49:56.117 INFO:tasks.workunit.client.1.smithi196.stdout:2/712: creat d0/d7/d9/d42/d98/de4/d67/fe9 x:0 0 0 2024-03-13T14:49:56.117 INFO:tasks.workunit.client.1.smithi196.stdout:1/845: dread - d1/d28/da3/fdc zero size 2024-03-13T14:49:56.119 INFO:tasks.workunit.client.1.smithi196.stdout:0/795: chown d8/da/d25/d26/d35/d3b/d59 188856047 1 2024-03-13T14:49:56.120 INFO:tasks.workunit.client.0.smithi176.stdout:8/621: truncate d0/d24/d29/fbd 342926 0 2024-03-13T14:49:56.122 INFO:tasks.workunit.client.0.smithi176.stdout:5/639: mknod d5/d2c/d40/dc7/d53/cdd 0 2024-03-13T14:49:56.122 INFO:tasks.workunit.client.0.smithi176.stdout:9/767: dwrite d0/d17/f19 [0,4194304] 0 2024-03-13T14:49:56.124 INFO:tasks.workunit.client.1.smithi196.stdout:0/796: readlink d8/da/d25/d26/d35/d3b/d50/d5d/d7e/l98 0 2024-03-13T14:49:56.125 INFO:tasks.workunit.client.1.smithi196.stdout:1/846: write d1/d9/d33/dd2/dde/ff2 [1118231,20427] 0 2024-03-13T14:49:56.126 INFO:tasks.workunit.client.1.smithi196.stdout:6/937: getdents d6/d9/d17 0 2024-03-13T14:49:56.127 INFO:tasks.workunit.client.1.smithi196.stdout:8/738: fdatasync d0/d16/d3d/d7f/fa8 0 2024-03-13T14:49:56.127 INFO:tasks.workunit.client.1.smithi196.stdout:7/875: link d0/d9/d1e/f51 d0/d9/d1e/d58/d72/da9/da3/d115/f129 0 2024-03-13T14:49:56.131 INFO:tasks.workunit.client.0.smithi176.stdout:9/768: chown d0/db/d89/d90/de6 27065 1 2024-03-13T14:49:56.131 INFO:tasks.workunit.client.0.smithi176.stdout:8/622: dread - d0/d5/f4d zero size 2024-03-13T14:49:56.132 INFO:tasks.workunit.client.0.smithi176.stdout:1/658: getdents d2/de/d10/d16/d4a/d5f/da8 0 2024-03-13T14:49:56.132 INFO:tasks.workunit.client.1.smithi196.stdout:7/876: write d0/d9/d1e/d3f/d6c/d106/f114 [493365,1100] 0 2024-03-13T14:49:56.134 INFO:tasks.workunit.client.1.smithi196.stdout:8/739: truncate d0/d63/dcb/ff6 187127 0 2024-03-13T14:49:56.136 INFO:tasks.workunit.client.0.smithi176.stdout:8/623: write d0/db/d39/d56/d93/da7/faa [922356,125700] 0 2024-03-13T14:49:56.136 INFO:tasks.workunit.client.0.smithi176.stdout:1/659: write d2/d30/f4c [2055282,43546] 0 2024-03-13T14:49:56.137 INFO:tasks.workunit.client.1.smithi196.stdout:2/713: rename d0/d7/d9/l2b to d0/d7/d9/d42/d98/de4/d67/lea 0 2024-03-13T14:49:56.140 INFO:tasks.workunit.client.1.smithi196.stdout:8/740: truncate d0/d16/f86 4476837 0 2024-03-13T14:49:56.141 INFO:tasks.workunit.client.0.smithi176.stdout:8/624: fdatasync d0/db/d39/d56/d93/da7/fbf 0 2024-03-13T14:49:56.145 INFO:tasks.workunit.client.1.smithi196.stdout:6/938: symlink d6/d7/dd/d60/l134 0 2024-03-13T14:49:56.148 INFO:tasks.workunit.client.1.smithi196.stdout:0/797: mknod d8/da/d25/d26/d48/d82/db3/c104 0 2024-03-13T14:49:56.154 INFO:tasks.workunit.client.0.smithi176.stdout:1/660: mkdir d2/d29/d73/dd0 0 2024-03-13T14:49:56.161 INFO:tasks.workunit.client.0.smithi176.stdout:1/661: mkdir d2/de/dd1 0 2024-03-13T14:49:56.162 INFO:tasks.workunit.client.0.smithi176.stdout:9/769: write d0/db/d89/d32/f95 [1091303,49843] 0 2024-03-13T14:49:56.162 INFO:tasks.workunit.client.0.smithi176.stdout:5/640: link d5/d2b/l35 d5/d22/lde 0 2024-03-13T14:49:56.162 INFO:tasks.workunit.client.0.smithi176.stdout:3/723: dread d4/d13/d2d/f51 [0,4194304] 0 2024-03-13T14:49:56.163 INFO:tasks.workunit.client.1.smithi196.stdout:5/828: dwrite d4/d1b/dc8/fd [0,4194304] 0 2024-03-13T14:49:56.164 INFO:tasks.workunit.client.0.smithi176.stdout:2/700: dwrite d1/dc/d2d/d59/d80/d5c/f5f [0,4194304] 0 2024-03-13T14:49:56.166 INFO:tasks.workunit.client.0.smithi176.stdout:4/707: dwrite d7/d8/df6/f68 [0,4194304] 0 2024-03-13T14:49:56.167 INFO:tasks.workunit.client.1.smithi196.stdout:4/734: dwrite d1/d1e/f33 [0,4194304] 0 2024-03-13T14:49:56.167 INFO:tasks.workunit.client.0.smithi176.stdout:1/662: fsync d2/de/d3f/d50/d83/fad 0 2024-03-13T14:49:56.169 INFO:tasks.workunit.client.1.smithi196.stdout:0/798: rename d8/da/d25/d26/d35/d3b/d5f/d17/c100 to d8/da/d25/d26/d35/d3b/d50/d55/c105 0 2024-03-13T14:49:56.170 INFO:tasks.workunit.client.1.smithi196.stdout:5/829: stat d4/d1b/dc8/d3c/d6a 0 2024-03-13T14:49:56.170 INFO:tasks.workunit.client.1.smithi196.stdout:1/847: write d1/f37 [4963070,121719] 0 2024-03-13T14:49:56.171 INFO:tasks.workunit.client.1.smithi196.stdout:8/741: fdatasync d0/d1c/f7b 0 2024-03-13T14:49:56.173 INFO:tasks.workunit.client.0.smithi176.stdout:9/770: truncate d0/d1e/d8d/fe7 816644 0 2024-03-13T14:49:56.176 INFO:tasks.workunit.client.1.smithi196.stdout:7/877: creat d0/d6/f12a x:0 0 0 2024-03-13T14:49:56.177 INFO:tasks.workunit.client.0.smithi176.stdout:5/641: write d5/f24 [4582550,23048] 0 2024-03-13T14:49:56.182 INFO:tasks.workunit.client.1.smithi196.stdout:4/735: symlink d1/d3/d7/d35/d76/d58/dc9/l101 0 2024-03-13T14:49:56.182 INFO:tasks.workunit.client.1.smithi196.stdout:0/799: unlink d8/da/l5e 0 2024-03-13T14:49:56.186 INFO:tasks.workunit.client.1.smithi196.stdout:4/736: read d1/d3/d4c/d72/fea [687974,11760] 0 2024-03-13T14:49:56.187 INFO:tasks.workunit.client.1.smithi196.stdout:8/742: creat d0/d10/d79/f100 x:0 0 0 2024-03-13T14:49:56.188 INFO:tasks.workunit.client.0.smithi176.stdout:9/771: rename d0/d1e/d6f/fac to d0/db/d89/d7a/fff 0 2024-03-13T14:49:56.192 INFO:tasks.workunit.client.0.smithi176.stdout:1/663: mkdir d2/de/d10/d16/d4a/d5f/da4/dd2 0 2024-03-13T14:49:56.195 INFO:tasks.workunit.client.1.smithi196.stdout:2/714: mkdir d0/d7/d9/d5a/dbc/deb 0 2024-03-13T14:49:56.196 INFO:tasks.workunit.client.1.smithi196.stdout:0/800: unlink d8/da/d25/d26/d35/ca6 0 2024-03-13T14:49:56.198 INFO:tasks.workunit.client.0.smithi176.stdout:3/724: link d4/d13/d16/db6/fc4 d4/d13/d16/ff6 0 2024-03-13T14:49:56.200 INFO:tasks.workunit.client.0.smithi176.stdout:4/708: dwrite d7/d10/d2c/f66 [0,4194304] 0 2024-03-13T14:49:56.200 INFO:tasks.workunit.client.1.smithi196.stdout:6/939: rename d6/d7/df/dcb to d6/d7/d135 0 2024-03-13T14:49:56.203 INFO:tasks.workunit.client.0.smithi176.stdout:4/709: fdatasync d7/d8/d15/d94/fa6 0 2024-03-13T14:49:56.203 INFO:tasks.workunit.client.0.smithi176.stdout:9/772: mkdir d0/db/d89/d4d/d6e/d100 0 2024-03-13T14:49:56.204 INFO:tasks.workunit.client.1.smithi196.stdout:1/848: creat d1/d9/d33/d4b/f123 x:0 0 0 2024-03-13T14:49:56.209 INFO:tasks.workunit.client.1.smithi196.stdout:2/715: creat d0/d7/d9/fec x:0 0 0 2024-03-13T14:49:56.211 INFO:tasks.workunit.client.1.smithi196.stdout:6/940: symlink d6/d7/df/d27/d39/d44/l136 0 2024-03-13T14:49:56.213 INFO:tasks.workunit.client.1.smithi196.stdout:6/941: chown d6/d2a/de2 26099 1 2024-03-13T14:49:56.214 INFO:tasks.workunit.client.0.smithi176.stdout:3/725: creat d4/d13/d81/d89/ff7 x:0 0 0 2024-03-13T14:49:56.216 INFO:tasks.workunit.client.1.smithi196.stdout:1/849: mkdir d1/d9/d4e/d55/d63/da6/dc8/d11d/d124 0 2024-03-13T14:49:56.216 INFO:tasks.workunit.client.1.smithi196.stdout:4/737: dwrite d1/d3/d4c/d72/fea [0,4194304] 0 2024-03-13T14:49:56.220 INFO:tasks.workunit.client.1.smithi196.stdout:4/738: read d1/d3/d7/d35/f63 [525321,34282] 0 2024-03-13T14:49:56.221 INFO:tasks.workunit.client.0.smithi176.stdout:3/726: creat d4/db8/df5/ff8 x:0 0 0 2024-03-13T14:49:56.223 INFO:tasks.workunit.client.1.smithi196.stdout:2/716: unlink d0/d7/d9/d42/d98/fde 0 2024-03-13T14:49:56.225 INFO:tasks.workunit.client.0.smithi176.stdout:9/773: dread d0/d40/d53/fba [0,4194304] 0 2024-03-13T14:49:56.227 INFO:tasks.workunit.client.0.smithi176.stdout:2/701: dread d1/d2a/d33/fa2 [0,4194304] 0 2024-03-13T14:49:56.229 INFO:tasks.workunit.client.0.smithi176.stdout:3/727: rename d4/d13/d5e/fa0 to d4/d13/d81/d89/d5b/ff9 0 2024-03-13T14:49:56.229 INFO:tasks.workunit.client.1.smithi196.stdout:4/739: rmdir d1/d1e 39 2024-03-13T14:49:56.230 INFO:tasks.workunit.client.1.smithi196.stdout:6/942: fdatasync d6/d2a/dc/d1c/da7/fb5 0 2024-03-13T14:49:56.230 INFO:tasks.workunit.client.1.smithi196.stdout:2/717: read - d0/d7/d9/d9a/fd8 zero size 2024-03-13T14:49:56.231 INFO:tasks.workunit.client.1.smithi196.stdout:6/943: stat d6/d2a/c1a 0 2024-03-13T14:49:56.231 INFO:tasks.workunit.client.0.smithi176.stdout:9/774: chown d0/db/d89/d32/d58/d5f/d84/d9a/f9b 620 1 2024-03-13T14:49:56.233 INFO:tasks.workunit.client.1.smithi196.stdout:1/850: creat d1/d3f/d5b/d74/f125 x:0 0 0 2024-03-13T14:49:56.234 INFO:tasks.workunit.client.0.smithi176.stdout:2/702: creat d1/dc/d61/feb x:0 0 0 2024-03-13T14:49:56.234 INFO:tasks.workunit.client.0.smithi176.stdout:3/728: mknod d4/cfa 0 2024-03-13T14:49:56.237 INFO:tasks.workunit.client.1.smithi196.stdout:6/944: write d6/d7/df/d27/d39/d44/d74/f87 [2555915,94534] 0 2024-03-13T14:49:56.239 INFO:tasks.workunit.client.1.smithi196.stdout:4/740: rename d1/d3/d4c/d72/fa0 to d1/d3/f102 0 2024-03-13T14:49:56.242 INFO:tasks.workunit.client.1.smithi196.stdout:1/851: mkdir d1/d126 0 2024-03-13T14:49:56.244 INFO:tasks.workunit.client.0.smithi176.stdout:2/703: truncate d1/dc/fae 870008 0 2024-03-13T14:49:56.245 INFO:tasks.workunit.client.1.smithi196.stdout:1/852: write d1/d9/d33/f42 [991204,75877] 0 2024-03-13T14:49:56.246 INFO:tasks.workunit.client.0.smithi176.stdout:2/704: write d1/dc/d61/feb [230865,42133] 0 2024-03-13T14:49:56.247 INFO:tasks.workunit.client.1.smithi196.stdout:6/945: creat d6/d9/d11/f137 x:0 0 0 2024-03-13T14:49:56.247 INFO:tasks.workunit.client.0.smithi176.stdout:5/642: dwrite f0 [0,4194304] 0 2024-03-13T14:49:56.249 INFO:tasks.workunit.client.0.smithi176.stdout:9/775: mknod d0/db/d89/d4d/d6e/d100/c101 0 2024-03-13T14:49:56.251 INFO:tasks.workunit.client.1.smithi196.stdout:2/718: dwrite d0/d7/d9/fec [0,4194304] 0 2024-03-13T14:49:56.254 INFO:tasks.workunit.client.1.smithi196.stdout:4/741: fsync d1/f29 0 2024-03-13T14:49:56.254 INFO:tasks.workunit.client.1.smithi196.stdout:6/946: truncate d6/d2a/dc/d1c/f123 167453 0 2024-03-13T14:49:56.257 INFO:tasks.workunit.client.1.smithi196.stdout:1/853: read d1/d9/d6c/f77 [5068219,81730] 0 2024-03-13T14:49:56.258 INFO:tasks.workunit.client.0.smithi176.stdout:3/729: rename d4/d28/d34/d38/f88 to d4/d13/d81/d89/d5b/ffb 0 2024-03-13T14:49:56.265 INFO:tasks.workunit.client.1.smithi196.stdout:8/743: write d0/d10/d12/df5/f65 [2933446,45514] 0 2024-03-13T14:49:56.269 INFO:tasks.workunit.client.0.smithi176.stdout:9/776: rename d0/l21 to d0/db/d89/d32/d58/l102 0 2024-03-13T14:49:56.270 INFO:tasks.workunit.client.1.smithi196.stdout:0/801: write d8/da/d25/d26/d35/d3b/d50/f99 [668896,45581] 0 2024-03-13T14:49:56.270 INFO:tasks.workunit.client.1.smithi196.stdout:2/719: mknod d0/d7/d9/d42/d98/de4/d67/ced 0 2024-03-13T14:49:56.274 INFO:tasks.workunit.client.1.smithi196.stdout:7/878: dwrite d0/d9/d1e/d58/d72/d6e/fba [0,4194304] 0 2024-03-13T14:49:56.274 INFO:tasks.workunit.client.1.smithi196.stdout:2/720: write d0/f1a [47003,103676] 0 2024-03-13T14:49:56.274 INFO:tasks.workunit.client.1.smithi196.stdout:4/742: creat d1/d3/d7/d35/d3e/d56/f103 x:0 0 0 2024-03-13T14:49:56.278 INFO:tasks.workunit.client.1.smithi196.stdout:2/721: write d0/d6/d4b/d5f/f81 [5140725,12770] 0 2024-03-13T14:49:56.281 INFO:tasks.workunit.client.0.smithi176.stdout:4/710: dwrite d7/d8/f39 [0,4194304] 0 2024-03-13T14:49:56.287 INFO:tasks.workunit.client.0.smithi176.stdout:9/777: creat d0/d1e/d8d/da7/f103 x:0 0 0 2024-03-13T14:49:56.291 INFO:tasks.workunit.client.1.smithi196.stdout:8/744: symlink d0/d10/da5/db2/l101 0 2024-03-13T14:49:56.297 INFO:tasks.workunit.client.0.smithi176.stdout:2/705: getdents d1/dc/d2d/d59/d80/d5c 0 2024-03-13T14:49:56.301 INFO:tasks.workunit.client.1.smithi196.stdout:8/745: read d0/d16/d3d/d9c/f9d [64629,30684] 0 2024-03-13T14:49:56.302 INFO:tasks.workunit.client.1.smithi196.stdout:1/854: dwrite d1/d3f/d5b/ff7 [0,4194304] 0 2024-03-13T14:49:56.303 INFO:tasks.workunit.client.0.smithi176.stdout:3/730: getdents d4/d13/d81/d89/d5b 0 2024-03-13T14:49:56.306 INFO:tasks.workunit.client.0.smithi176.stdout:2/706: dread d1/f96 [0,4194304] 0 2024-03-13T14:49:56.310 INFO:tasks.workunit.client.0.smithi176.stdout:5/643: truncate d5/d2e/fd2 578218 0 2024-03-13T14:49:56.312 INFO:tasks.workunit.client.0.smithi176.stdout:2/707: read d1/dc/d2d/d4b/d69/f94 [810537,107238] 0 2024-03-13T14:49:56.312 INFO:tasks.workunit.client.1.smithi196.stdout:0/802: creat d8/da/d25/d26/d48/d82/da0/df2/f106 x:0 0 0 2024-03-13T14:49:56.313 INFO:tasks.workunit.client.0.smithi176.stdout:9/778: getdents d0/d40/d53/d6d/dee 0 2024-03-13T14:49:56.317 INFO:tasks.workunit.client.1.smithi196.stdout:4/743: mknod d1/c104 0 2024-03-13T14:49:56.319 INFO:tasks.workunit.client.0.smithi176.stdout:2/708: mknod d1/dc/d61/daa/cec 0 2024-03-13T14:49:56.320 INFO:tasks.workunit.client.0.smithi176.stdout:9/779: creat d0/d40/d53/d6d/de1/f104 x:0 0 0 2024-03-13T14:49:56.327 INFO:tasks.workunit.client.0.smithi176.stdout:5/644: dwrite d5/d22/d15/f2d [4194304,4194304] 0 2024-03-13T14:49:56.328 INFO:tasks.workunit.client.1.smithi196.stdout:6/947: getdents d6/d9/d17/d19 0 2024-03-13T14:49:56.330 INFO:tasks.workunit.client.0.smithi176.stdout:3/731: write d4/d28/fc0 [633083,14195] 0 2024-03-13T14:49:56.332 INFO:tasks.workunit.client.1.smithi196.stdout:2/722: dwrite d0/d7/f87 [0,4194304] 0 2024-03-13T14:49:56.333 INFO:tasks.workunit.client.1.smithi196.stdout:7/879: rename d0/d6/d35/c52 to d0/c12b 0 2024-03-13T14:49:56.334 INFO:tasks.workunit.client.1.smithi196.stdout:8/746: rmdir d0/d10/d70/d8d 39 2024-03-13T14:49:56.337 INFO:tasks.workunit.client.1.smithi196.stdout:2/723: truncate d0/d7/d9/d42/d98/de4/d67/fbd 1013113 0 2024-03-13T14:49:56.337 INFO:tasks.workunit.client.0.smithi176.stdout:0/580: sync 2024-03-13T14:49:56.341 INFO:tasks.workunit.client.1.smithi196.stdout:4/744: mkdir d1/d1e/d5c/d105 0 2024-03-13T14:49:56.343 INFO:tasks.workunit.client.0.smithi176.stdout:9/780: creat d0/db/d89/d4d/dd5/f105 x:0 0 0 2024-03-13T14:49:56.343 INFO:tasks.workunit.client.0.smithi176.stdout:3/732: mkdir d4/d13/d81/d89/d5b/dfc 0 2024-03-13T14:49:56.347 INFO:tasks.workunit.client.0.smithi176.stdout:9/781: write d0/db/d89/d4d/d6e/ffb [1016014,17516] 0 2024-03-13T14:49:56.348 INFO:tasks.workunit.client.1.smithi196.stdout:2/724: creat d0/d6/d1d/fee x:0 0 0 2024-03-13T14:49:56.349 INFO:tasks.workunit.client.0.smithi176.stdout:2/709: dwrite d1/dc/d2d/d90/fc9 [0,4194304] 0 2024-03-13T14:49:56.350 INFO:tasks.workunit.client.1.smithi196.stdout:0/803: dwrite d8/da/d25/d26/d48/dba/ff6 [0,4194304] 0 2024-03-13T14:49:56.350 INFO:tasks.workunit.client.1.smithi196.stdout:6/948: link d6/d9/d11/f137 d6/d9/d17/f138 0 2024-03-13T14:49:56.351 INFO:tasks.workunit.client.1.smithi196.stdout:2/725: chown d0/d6/d1d/f5e 141066 1 2024-03-13T14:49:56.353 INFO:tasks.workunit.client.1.smithi196.stdout:2/726: chown d0/d7/d40/c49 14419 1 2024-03-13T14:49:56.358 INFO:tasks.workunit.client.0.smithi176.stdout:0/581: mknod d2/db/d31/cb4 0 2024-03-13T14:49:56.362 INFO:tasks.workunit.client.0.smithi176.stdout:5/645: write d5/d31/fa2 [5782598,60857] 0 2024-03-13T14:49:56.362 INFO:tasks.workunit.client.1.smithi196.stdout:4/745: dread - d1/d3/d7/d35/d76/d58/d69/d9f/fd8 zero size 2024-03-13T14:49:56.366 INFO:tasks.workunit.client.0.smithi176.stdout:9/782: read d0/d40/f78 [80486,59978] 0 2024-03-13T14:49:56.367 INFO:tasks.workunit.client.0.smithi176.stdout:3/733: unlink d4/d13/d16/ff6 0 2024-03-13T14:49:56.368 INFO:tasks.workunit.client.1.smithi196.stdout:1/855: rename d1/d28/d34/d67/l6b to d1/d9/d33/d4b/l127 0 2024-03-13T14:49:56.370 INFO:tasks.workunit.client.0.smithi176.stdout:2/710: creat d1/dc/d2d/d59/d80/d5c/d78/fed x:0 0 0 2024-03-13T14:49:56.372 INFO:tasks.workunit.client.0.smithi176.stdout:2/711: write d1/dc/d2d/d59/d80/d77/f84 [463234,49620] 0 2024-03-13T14:49:56.377 INFO:tasks.workunit.client.0.smithi176.stdout:5/646: fdatasync d5/d2e/d7e/dad/f65 0 2024-03-13T14:49:56.381 INFO:tasks.workunit.client.1.smithi196.stdout:8/747: dwrite d0/d10/d12/df5/f59 [0,4194304] 0 2024-03-13T14:49:56.382 INFO:tasks.workunit.client.1.smithi196.stdout:4/746: mkdir d1/d3/d4c/d72/d106 0 2024-03-13T14:49:56.388 INFO:tasks.workunit.client.1.smithi196.stdout:7/880: rename d0/d78/d81/fd3 to d0/d9/d31/f12c 0 2024-03-13T14:49:56.389 INFO:tasks.workunit.client.0.smithi176.stdout:0/582: write d2/d5/f87 [932825,14804] 0 2024-03-13T14:49:56.392 INFO:tasks.workunit.client.0.smithi176.stdout:2/712: unlink d1/dc/d2d/d4f/l83 0 2024-03-13T14:49:56.392 INFO:tasks.workunit.client.1.smithi196.stdout:7/881: chown d0/d6/ddc/ffd 60475382 1 2024-03-13T14:49:56.394 INFO:tasks.workunit.client.1.smithi196.stdout:6/949: unlink d6/d2a/dc/d1c/cb4 0 2024-03-13T14:49:56.396 INFO:tasks.workunit.client.0.smithi176.stdout:5/647: fsync d5/d75/f8c 0 2024-03-13T14:49:56.396 INFO:tasks.workunit.client.0.smithi176.stdout:9/783: dread d0/db/d89/d32/d58/d5f/d84/d9a/f9b [0,4194304] 0 2024-03-13T14:49:56.402 INFO:tasks.workunit.client.0.smithi176.stdout:2/713: creat d1/dc/de0/fee x:0 0 0 2024-03-13T14:49:56.406 INFO:tasks.workunit.client.1.smithi196.stdout:8/748: unlink d0/d16/d3d/d9c/le6 0 2024-03-13T14:49:56.407 INFO:tasks.workunit.client.1.smithi196.stdout:9/729: sync 2024-03-13T14:49:56.408 INFO:tasks.workunit.client.0.smithi176.stdout:3/734: write d4/d28/d34/f9e [3851957,44769] 0 2024-03-13T14:49:56.410 INFO:tasks.workunit.client.1.smithi196.stdout:0/804: truncate d8/da/d25/d26/f60 1029017 0 2024-03-13T14:49:56.411 INFO:tasks.workunit.client.1.smithi196.stdout:4/747: truncate d1/d3/d7/d35/d76/d58/d69/d6c/f7e 1531530 0 2024-03-13T14:49:56.411 INFO:tasks.workunit.client.0.smithi176.stdout:5/648: creat d5/d2e/d5f/fdf x:0 0 0 2024-03-13T14:49:56.413 INFO:tasks.workunit.client.0.smithi176.stdout:5/649: chown d5/d31/d86/fb4 4 1 2024-03-13T14:49:56.413 INFO:tasks.workunit.client.1.smithi196.stdout:1/856: dwrite d1/d9/d4e/d55/d90/fb1 [0,4194304] 0 2024-03-13T14:49:56.415 INFO:tasks.workunit.client.1.smithi196.stdout:7/882: rename d0/d9/d31/d77/ca1 to d0/d9/d31/d77/dcf/de0/c12d 0 2024-03-13T14:49:56.417 INFO:tasks.workunit.client.0.smithi176.stdout:0/583: creat d2/d5/d28/fb5 x:0 0 0 2024-03-13T14:49:56.419 INFO:tasks.workunit.client.0.smithi176.stdout:3/735: creat d4/d28/d4f/ffd x:0 0 0 2024-03-13T14:49:56.421 INFO:tasks.workunit.client.1.smithi196.stdout:8/749: symlink d0/d10/d12/d17/d42/dbe/l102 0 2024-03-13T14:49:56.421 INFO:tasks.workunit.client.1.smithi196.stdout:7/883: stat d0/d6/ddc/d126 0 2024-03-13T14:49:56.421 INFO:tasks.workunit.client.1.smithi196.stdout:2/727: dread d0/d7/d9/d5a/f7c [0,4194304] 0 2024-03-13T14:49:56.421 INFO:tasks.workunit.client.0.smithi176.stdout:9/784: getdents d0/dd8/d82/df9 0 2024-03-13T14:49:56.422 INFO:tasks.workunit.client.1.smithi196.stdout:9/730: creat d7/de/d4b/d1d/d93/ffd x:0 0 0 2024-03-13T14:49:56.425 INFO:tasks.workunit.client.1.smithi196.stdout:4/748: write d1/d3/d7/d35/d76/d58/d69/de3/df4/fcc [4235,47648] 0 2024-03-13T14:49:56.425 INFO:tasks.workunit.client.1.smithi196.stdout:1/857: creat d1/d28/f128 x:0 0 0 2024-03-13T14:49:56.426 INFO:tasks.workunit.client.0.smithi176.stdout:0/584: mknod d2/d5/d28/d8c/cb6 0 2024-03-13T14:49:56.427 INFO:tasks.workunit.client.1.smithi196.stdout:9/731: write d7/de/d16/fe1 [431553,110852] 0 2024-03-13T14:49:56.429 INFO:tasks.workunit.client.0.smithi176.stdout:5/650: dwrite d5/d31/fa2 [0,4194304] 0 2024-03-13T14:49:56.436 INFO:tasks.workunit.client.1.smithi196.stdout:6/950: dread d6/d2a/de2/fbe [0,4194304] 0 2024-03-13T14:49:56.436 INFO:tasks.workunit.client.1.smithi196.stdout:8/750: mknod d0/d10/da5/c103 0 2024-03-13T14:49:56.442 INFO:tasks.workunit.client.0.smithi176.stdout:9/785: rename d0/d40/f47 to d0/db/d89/d32/d58/d67/f106 0 2024-03-13T14:49:56.443 INFO:tasks.workunit.client.0.smithi176.stdout:2/714: write d1/dc/fae [145800,115402] 0 2024-03-13T14:49:56.444 INFO:tasks.workunit.client.1.smithi196.stdout:6/951: fsync d6/d2a/d23/d28/f119 0 2024-03-13T14:49:56.445 INFO:tasks.workunit.client.0.smithi176.stdout:3/736: creat d4/d13/d81/d89/ffe x:0 0 0 2024-03-13T14:49:56.446 INFO:tasks.workunit.client.0.smithi176.stdout:0/585: mknod d2/db/d35/daa/daf/cb7 0 2024-03-13T14:49:56.446 INFO:tasks.workunit.client.1.smithi196.stdout:4/749: mknod d1/d3/d4c/d72/d7d/d80/c107 0 2024-03-13T14:49:56.448 INFO:tasks.workunit.client.0.smithi176.stdout:0/586: fsync d2/d5/dc/d59/f5a 0 2024-03-13T14:49:56.449 INFO:tasks.workunit.client.0.smithi176.stdout:0/587: stat d2/f99 0 2024-03-13T14:49:56.450 INFO:tasks.workunit.client.1.smithi196.stdout:1/858: truncate d1/d9/d6c/f77 4863027 0 2024-03-13T14:49:56.453 INFO:tasks.workunit.client.0.smithi176.stdout:5/651: rename d5/d22/l1f to d5/d2c/d40/dc7/d89/le0 0 2024-03-13T14:49:56.453 INFO:tasks.workunit.client.1.smithi196.stdout:0/805: truncate d8/da/d25/d26/d54/d88/fc5 5327077 0 2024-03-13T14:49:56.456 INFO:tasks.workunit.client.0.smithi176.stdout:3/737: mkdir d4/d28/d31/d52/dff 0 2024-03-13T14:49:56.459 INFO:tasks.workunit.client.1.smithi196.stdout:7/884: link d0/d78/d81/dfb/f107 d0/d78/d81/dfb/f12e 0 2024-03-13T14:49:56.462 INFO:tasks.workunit.client.1.smithi196.stdout:4/750: creat d1/d3/d4c/f108 x:0 0 0 2024-03-13T14:49:56.463 INFO:tasks.workunit.client.0.smithi176.stdout:9/786: dwrite d0/d1e/d8d/dbe/ffe [0,4194304] 0 2024-03-13T14:49:56.463 INFO:tasks.workunit.client.1.smithi196.stdout:6/952: mkdir d6/d7/df/d27/d39/d44/dc8/d130/d91/d139 0 2024-03-13T14:49:56.464 INFO:tasks.workunit.client.1.smithi196.stdout:9/732: dwrite d7/de/d4b/d1d/f54 [0,4194304] 0 2024-03-13T14:49:56.465 INFO:tasks.workunit.client.0.smithi176.stdout:0/588: truncate d2/db/d35/f3f 30149 0 2024-03-13T14:49:56.465 INFO:tasks.workunit.client.0.smithi176.stdout:9/787: chown d0/db/d89/d4d/d6e/d100 2000 1 2024-03-13T14:49:56.468 INFO:tasks.workunit.client.0.smithi176.stdout:3/738: mkdir d4/d13/d81/d89/d5b/dc5/df0/d100 0 2024-03-13T14:49:56.469 INFO:tasks.workunit.client.1.smithi196.stdout:9/733: write d7/de/d4b/d1d/d22/f67 [1280369,61310] 0 2024-03-13T14:49:56.471 INFO:tasks.workunit.client.0.smithi176.stdout:9/788: creat d0/db/d89/d32/f107 x:0 0 0 2024-03-13T14:49:56.473 INFO:tasks.workunit.client.1.smithi196.stdout:9/734: truncate d7/d9/fc8 365234 0 2024-03-13T14:49:56.474 INFO:tasks.workunit.client.1.smithi196.stdout:0/806: creat d8/da/d25/d26/d35/d3b/d5f/d33/d44/f107 x:0 0 0 2024-03-13T14:49:56.478 INFO:tasks.workunit.client.0.smithi176.stdout:0/589: truncate d2/d5/dc/d20/f34 4425367 0 2024-03-13T14:49:56.480 INFO:tasks.workunit.client.0.smithi176.stdout:0/590: chown d2/d5/dc/d20/f37 13222 1 2024-03-13T14:49:56.482 INFO:tasks.workunit.client.0.smithi176.stdout:3/739: rename d4/d13/d16/cec to d4/d13/d81/c101 0 2024-03-13T14:49:56.487 INFO:tasks.workunit.client.0.smithi176.stdout:6/676: sync 2024-03-13T14:49:56.487 INFO:tasks.workunit.client.0.smithi176.stdout:9/789: symlink d0/d1e/d8d/d9f/de3/l108 0 2024-03-13T14:49:56.487 INFO:tasks.workunit.client.0.smithi176.stdout:5/652: write d5/d31/d47/d80/f62 [2648139,74451] 0 2024-03-13T14:49:56.489 INFO:tasks.workunit.client.1.smithi196.stdout:1/859: dread d1/f2a [0,4194304] 0 2024-03-13T14:49:56.491 INFO:tasks.workunit.client.1.smithi196.stdout:1/860: stat d1/d9/d33/d4b/d66/d7b/fa9 0 2024-03-13T14:49:56.492 INFO:tasks.workunit.client.0.smithi176.stdout:3/740: mkdir d4/d28/d31/d45/d82/d87/d102 0 2024-03-13T14:49:56.494 INFO:tasks.workunit.client.0.smithi176.stdout:9/790: read d0/db/d89/d32/d58/f83 [1308950,115112] 0 2024-03-13T14:49:56.497 INFO:tasks.workunit.client.1.smithi196.stdout:1/861: fsync d1/d9/d33/d4b/d66/d7b/fa9 0 2024-03-13T14:49:56.501 INFO:tasks.workunit.client.0.smithi176.stdout:6/677: mkdir d0/d10/d19/d27/d2b/ddc 0 2024-03-13T14:49:56.502 INFO:tasks.workunit.client.1.smithi196.stdout:2/728: link d0/d7/la3 d0/d7/d9/d61/d83/dd9/lef 0 2024-03-13T14:49:56.505 INFO:tasks.workunit.client.0.smithi176.stdout:9/791: mknod d0/d1e/d1f/d68/c109 0 2024-03-13T14:49:56.505 INFO:tasks.workunit.client.1.smithi196.stdout:7/885: symlink d0/d9/d1e/d58/d72/da9/l12f 0 2024-03-13T14:49:56.511 INFO:tasks.workunit.client.0.smithi176.stdout:6/678: unlink d0/d10/d19/d27/f4a 0 2024-03-13T14:49:56.513 INFO:tasks.workunit.client.0.smithi176.stdout:9/792: write d0/d1e/d1f/d68/faf [1658959,16837] 0 2024-03-13T14:49:56.514 INFO:tasks.workunit.client.0.smithi176.stdout:5/653: dwrite d5/f24 [0,4194304] 0 2024-03-13T14:49:56.517 INFO:tasks.workunit.client.0.smithi176.stdout:9/793: chown d0/db/d89/d32/d58/d67/lb2 254 1 2024-03-13T14:49:56.518 INFO:tasks.workunit.client.1.smithi196.stdout:1/862: rmdir d1/d28 39 2024-03-13T14:49:56.522 INFO:tasks.workunit.client.1.smithi196.stdout:4/751: dwrite d1/d3/d4c/f59 [0,4194304] 0 2024-03-13T14:49:56.526 INFO:tasks.workunit.client.0.smithi176.stdout:0/591: dwrite d2/db/d31/f57 [0,4194304] 0 2024-03-13T14:49:56.530 INFO:tasks.workunit.client.1.smithi196.stdout:6/953: link d6/d2a/d23/d104/c124 d6/d7/df/d27/d39/d44/dc8/d130/d113/c13a 0 2024-03-13T14:49:56.537 INFO:tasks.workunit.client.1.smithi196.stdout:9/735: creat d7/de/d74/ffe x:0 0 0 2024-03-13T14:49:56.540 INFO:tasks.workunit.client.1.smithi196.stdout:1/863: mkdir d1/d9/d4e/d55/d63/d129 0 2024-03-13T14:49:56.545 INFO:tasks.workunit.client.1.smithi196.stdout:4/752: rename d1/d3/d4c/d72/d7d/fcd to d1/d3/d7/ddf/f109 0 2024-03-13T14:49:56.552 INFO:tasks.workunit.client.1.smithi196.stdout:0/807: dwrite d8/da/d25/d26/d54/f7f [0,4194304] 0 2024-03-13T14:49:56.553 INFO:tasks.workunit.client.1.smithi196.stdout:7/886: mknod d0/d6/db4/dca/d103/c130 0 2024-03-13T14:49:56.555 INFO:tasks.workunit.client.1.smithi196.stdout:9/736: truncate d7/f69 3513088 0 2024-03-13T14:49:56.559 INFO:tasks.workunit.client.1.smithi196.stdout:9/737: write d7/de/d16/f99 [3043257,49910] 0 2024-03-13T14:49:56.569 INFO:tasks.workunit.client.1.smithi196.stdout:6/954: write d6/d7/df/d27/f64 [450491,6430] 0 2024-03-13T14:49:56.570 INFO:tasks.workunit.client.1.smithi196.stdout:7/887: creat d0/d6/d33/d57/d8a/f131 x:0 0 0 2024-03-13T14:49:56.570 INFO:tasks.workunit.client.1.smithi196.stdout:0/808: mkdir d8/da/d25/d26/d48/d82/da0/df2/d108 0 2024-03-13T14:49:56.578 INFO:tasks.workunit.client.1.smithi196.stdout:2/729: link d0/d7/d9/l10 d0/d6/d1d/lf0 0 2024-03-13T14:49:56.578 INFO:tasks.workunit.client.1.smithi196.stdout:6/955: read d6/d2a/d23/d28/f3f [424036,85512] 0 2024-03-13T14:49:56.581 INFO:tasks.workunit.client.1.smithi196.stdout:6/956: dread - d6/d7/d135/f11d zero size 2024-03-13T14:49:56.589 INFO:tasks.workunit.client.1.smithi196.stdout:0/809: dwrite d8/da/d74/ff1 [0,4194304] 0 2024-03-13T14:49:56.591 INFO:tasks.workunit.client.1.smithi196.stdout:4/753: link d1/d3/c6d d1/d3/d7/d35/d76/d58/dc9/c10a 0 2024-03-13T14:49:56.593 INFO:tasks.workunit.client.1.smithi196.stdout:0/810: fdatasync d8/da/d25/d26/d35/d3b/d59/d6a/f8d 0 2024-03-13T14:49:56.594 INFO:tasks.workunit.client.1.smithi196.stdout:9/738: write d7/de/d74/d94/fa8 [2370714,15758] 0 2024-03-13T14:49:56.596 INFO:tasks.workunit.client.1.smithi196.stdout:1/864: dread d1/d9/d6c/f77 [0,4194304] 0 2024-03-13T14:49:56.597 INFO:tasks.workunit.client.1.smithi196.stdout:6/957: creat d6/d9/d17/da1/f13b x:0 0 0 2024-03-13T14:49:56.598 INFO:tasks.workunit.client.1.smithi196.stdout:3/791: sync 2024-03-13T14:49:56.598 INFO:tasks.workunit.client.1.smithi196.stdout:5/830: sync 2024-03-13T14:49:56.601 INFO:tasks.workunit.client.1.smithi196.stdout:7/888: write d0/d6/f1d [956956,27996] 0 2024-03-13T14:49:56.602 INFO:tasks.workunit.client.1.smithi196.stdout:5/831: read d4/d1b/dc8/d14/d19/d26/d33/fdb [3951765,44849] 0 2024-03-13T14:49:56.605 INFO:tasks.workunit.client.1.smithi196.stdout:2/730: creat d0/d6/d4b/d5f/dce/ff1 x:0 0 0 2024-03-13T14:49:56.606 INFO:tasks.workunit.client.0.smithi176.stdout:9/794: write d0/d40/d53/f6b [1371188,80485] 0 2024-03-13T14:49:56.607 INFO:tasks.workunit.client.1.smithi196.stdout:2/731: dread - d0/d6/d1d/fee zero size 2024-03-13T14:49:56.607 INFO:tasks.workunit.client.1.smithi196.stdout:5/832: chown d4/d1b/dc8/d14/d19/l34 1516 1 2024-03-13T14:49:56.613 INFO:tasks.workunit.client.0.smithi176.stdout:0/592: symlink d2/db/d39/lb8 0 2024-03-13T14:49:56.619 INFO:tasks.workunit.client.0.smithi176.stdout:6/679: creat d0/d10/d19/d27/d2b/d2e/d3e/db4/fdd x:0 0 0 2024-03-13T14:49:56.620 INFO:tasks.workunit.client.0.smithi176.stdout:6/680: fdatasync d0/f4e 0 2024-03-13T14:49:56.622 INFO:tasks.workunit.client.0.smithi176.stdout:6/681: chown d0/de/d8d/ca4 102714 1 2024-03-13T14:49:56.623 INFO:tasks.workunit.client.0.smithi176.stdout:9/795: symlink d0/db/d89/d4d/l10a 0 2024-03-13T14:49:56.623 INFO:tasks.workunit.client.1.smithi196.stdout:4/754: write d1/d3/d7/d35/d76/d58/d69/d9f/fd8 [1045436,48001] 0 2024-03-13T14:49:56.623 INFO:tasks.workunit.client.1.smithi196.stdout:2/732: rmdir d0/d7/d40 39 2024-03-13T14:49:56.624 INFO:tasks.workunit.client.0.smithi176.stdout:6/682: write d0/f4e [685443,57358] 0 2024-03-13T14:49:56.626 INFO:tasks.workunit.client.0.smithi176.stdout:0/593: mknod d2/db/d39/cb9 0 2024-03-13T14:49:56.626 INFO:tasks.workunit.client.0.smithi176.stdout:6/683: chown d0/d10/d19/d27/d2b/d2e/d3e 336 1 2024-03-13T14:49:56.626 INFO:tasks.workunit.client.0.smithi176.stdout:6/684: chown d0 19 1 2024-03-13T14:49:56.626 INFO:tasks.workunit.client.0.smithi176.stdout:0/594: read - d2/f99 zero size 2024-03-13T14:49:56.628 INFO:tasks.workunit.client.1.smithi196.stdout:0/811: creat d8/da/dd8/f109 x:0 0 0 2024-03-13T14:49:56.629 INFO:tasks.workunit.client.1.smithi196.stdout:7/889: dread d0/d6/d33/f3c [0,4194304] 0 2024-03-13T14:49:56.630 INFO:tasks.workunit.client.1.smithi196.stdout:0/812: chown d8/da/d25/d26/d54 701374 1 2024-03-13T14:49:56.630 INFO:tasks.workunit.client.1.smithi196.stdout:7/890: readlink d0/l2a 0 2024-03-13T14:49:56.632 INFO:tasks.workunit.client.1.smithi196.stdout:5/833: rename d4/d1b/dc8/d14/d19/da7/dc7 to d4/d1b/dc8/d3c/d6a/d106 0 2024-03-13T14:49:56.637 INFO:tasks.workunit.client.1.smithi196.stdout:6/958: unlink d6/d7/df/fe6 0 2024-03-13T14:49:56.638 INFO:tasks.workunit.client.1.smithi196.stdout:3/792: mknod db/d19/d4a/dd5/cfc 0 2024-03-13T14:49:56.640 INFO:tasks.workunit.client.1.smithi196.stdout:6/959: chown d6/d7/df/d27/d39/d44/d74/fac 11001606 1 2024-03-13T14:49:56.640 INFO:tasks.workunit.client.0.smithi176.stdout:5/654: rename d5/d22/d15/f67 to d5/d2e/d7e/dad/fe1 0 2024-03-13T14:49:56.643 INFO:tasks.workunit.client.0.smithi176.stdout:6/685: fsync d0/de/f49 0 2024-03-13T14:49:56.644 INFO:tasks.workunit.client.0.smithi176.stdout:0/595: truncate d2/d5/dc/d20/f34 2788605 0 2024-03-13T14:49:56.645 INFO:tasks.workunit.client.0.smithi176.stdout:6/686: readlink d0/d10/d19/d9f/d76/d7c/l96 0 2024-03-13T14:49:56.647 INFO:tasks.workunit.client.0.smithi176.stdout:9/796: creat d0/d1e/f10b x:0 0 0 2024-03-13T14:49:56.649 INFO:tasks.workunit.client.1.smithi196.stdout:5/834: mknod d4/d1b/dc8/d14/d19/da7/c107 0 2024-03-13T14:49:56.649 INFO:tasks.workunit.client.0.smithi176.stdout:9/797: write d0/db/d89/d7a/ff3 [453497,13504] 0 2024-03-13T14:49:56.649 INFO:tasks.workunit.client.0.smithi176.stdout:9/798: read - d0/dd8/ff0 zero size 2024-03-13T14:49:56.651 INFO:tasks.workunit.client.1.smithi196.stdout:0/813: dwrite d8/da/d25/d26/d35/d3b/f7d [0,4194304] 0 2024-03-13T14:49:56.652 INFO:tasks.workunit.client.0.smithi176.stdout:5/655: creat d5/d75/fe2 x:0 0 0 2024-03-13T14:49:56.656 INFO:tasks.workunit.client.0.smithi176.stdout:5/656: dread d5/d22/fa6 [0,4194304] 0 2024-03-13T14:49:56.658 INFO:tasks.workunit.client.1.smithi196.stdout:6/960: dwrite d6/d2a/dc/d1c/d34/fff [0,4194304] 0 2024-03-13T14:49:56.661 INFO:tasks.workunit.client.1.smithi196.stdout:4/755: mknod d1/d3/d4c/d72/d7d/c10b 0 2024-03-13T14:49:56.665 INFO:tasks.workunit.client.0.smithi176.stdout:1/664: sync 2024-03-13T14:49:56.665 INFO:tasks.workunit.client.0.smithi176.stdout:7/647: sync 2024-03-13T14:49:56.665 INFO:tasks.workunit.client.0.smithi176.stdout:4/711: sync 2024-03-13T14:49:56.665 INFO:tasks.workunit.client.1.smithi196.stdout:6/961: chown d6/d9/d17/dfc/c108 1051 1 2024-03-13T14:49:56.666 INFO:tasks.workunit.client.0.smithi176.stdout:8/625: sync 2024-03-13T14:49:56.669 INFO:tasks.workunit.client.1.smithi196.stdout:3/793: truncate db/d19/d22/d3a/d68/d8e/fe0 770452 0 2024-03-13T14:49:56.669 INFO:tasks.workunit.client.0.smithi176.stdout:4/712: fsync d7/d8/d2f/f37 0 2024-03-13T14:49:56.671 INFO:tasks.workunit.client.0.smithi176.stdout:5/657: chown d5/d2e/d7e/dad/d7b/lc6 246212 1 2024-03-13T14:49:56.671 INFO:tasks.workunit.client.0.smithi176.stdout:0/596: unlink d2/d5/d28/c8b 0 2024-03-13T14:49:56.682 INFO:tasks.workunit.client.1.smithi196.stdout:9/739: link d7/de/d74/d94/dfb/d28/f8a d7/d9/d17/fff 0 2024-03-13T14:49:56.683 INFO:tasks.workunit.client.1.smithi196.stdout:4/756: fdatasync d1/d3/d7/d35/d3e/d56/ff5 0 2024-03-13T14:49:56.684 INFO:tasks.workunit.client.1.smithi196.stdout:7/891: write d0/f76 [5212878,28194] 0 2024-03-13T14:49:56.686 INFO:tasks.workunit.client.0.smithi176.stdout:7/648: symlink d1/d3f/d65/lc8 0 2024-03-13T14:49:56.687 INFO:tasks.workunit.client.1.smithi196.stdout:7/892: chown d0/d9/d31/cb6 413755 1 2024-03-13T14:49:56.689 INFO:tasks.workunit.client.1.smithi196.stdout:0/814: mknod d8/da/c10a 0 2024-03-13T14:49:56.689 INFO:tasks.workunit.client.1.smithi196.stdout:4/757: write d1/d3/d4c/d72/d7d/d80/f86 [3522339,68729] 0 2024-03-13T14:49:56.690 INFO:tasks.workunit.client.0.smithi176.stdout:8/626: symlink d0/d24/d7b/lc0 0 2024-03-13T14:49:56.691 INFO:tasks.workunit.client.1.smithi196.stdout:6/962: dread d6/d2a/d23/d28/f69 [0,4194304] 0 2024-03-13T14:49:56.694 INFO:tasks.workunit.client.1.smithi196.stdout:3/794: symlink db/d19/d4a/dd5/lfd 0 2024-03-13T14:49:56.695 INFO:tasks.workunit.client.1.smithi196.stdout:6/963: readlink d6/d7/df/d27/d39/d44/lcf 0 2024-03-13T14:49:56.697 INFO:tasks.workunit.client.0.smithi176.stdout:1/665: mkdir d2/d30/dd3 0 2024-03-13T14:49:56.701 INFO:tasks.workunit.client.1.smithi196.stdout:9/740: dwrite d7/de/f40 [0,4194304] 0 2024-03-13T14:49:56.702 INFO:tasks.workunit.client.0.smithi176.stdout:5/658: mknod d5/d2b/ce3 0 2024-03-13T14:49:56.704 INFO:tasks.workunit.client.1.smithi196.stdout:7/893: fdatasync d0/d9/d1e/f4b 0 2024-03-13T14:49:56.707 INFO:tasks.workunit.client.0.smithi176.stdout:9/799: rmdir d0/d40/d53/d6d/dee 0 2024-03-13T14:49:56.709 INFO:tasks.workunit.client.0.smithi176.stdout:9/800: chown d0/d1e/d8d/dbe/fde 53123 1 2024-03-13T14:49:56.713 INFO:tasks.workunit.client.1.smithi196.stdout:6/964: creat d6/d9/d17/da1/d50/f13c x:0 0 0 2024-03-13T14:49:56.713 INFO:tasks.workunit.client.0.smithi176.stdout:4/713: truncate d7/d10/d46/d4a/d5d/da1/f8c 321323 0 2024-03-13T14:49:56.717 INFO:tasks.workunit.client.1.smithi196.stdout:5/835: dwrite d4/d1b/dc8/d3c/fc0 [0,4194304] 0 2024-03-13T14:49:56.721 INFO:tasks.workunit.client.0.smithi176.stdout:7/649: symlink d1/d28/lc9 0 2024-03-13T14:49:56.723 INFO:tasks.workunit.client.0.smithi176.stdout:9/801: dwrite d0/d40/d53/f6b [0,4194304] 0 2024-03-13T14:49:56.726 INFO:tasks.workunit.client.0.smithi176.stdout:6/687: write d0/de/d54/f75 [555353,126949] 0 2024-03-13T14:49:56.730 INFO:tasks.workunit.client.0.smithi176.stdout:3/741: sync 2024-03-13T14:49:56.730 INFO:tasks.workunit.client.0.smithi176.stdout:2/715: sync 2024-03-13T14:49:56.734 INFO:tasks.workunit.client.1.smithi196.stdout:0/815: dwrite d8/da/d25/d26/d35/d3b/d5f/d33/fb2 [0,4194304] 0 2024-03-13T14:49:56.734 INFO:tasks.workunit.client.0.smithi176.stdout:2/716: chown d1/dc/d61/daa/cec 1 1 2024-03-13T14:49:56.743 INFO:tasks.workunit.client.0.smithi176.stdout:8/627: dread d0/d5/f20 [0,4194304] 0 2024-03-13T14:49:56.755 INFO:tasks.workunit.client.1.smithi196.stdout:6/965: dread d6/d9/d17/d19/d4e/f6e [0,4194304] 0 2024-03-13T14:49:56.778 INFO:tasks.workunit.client.0.smithi176.stdout:1/666: read d2/d30/f65 [513541,125290] 0 2024-03-13T14:49:56.789 INFO:tasks.workunit.client.1.smithi196.stdout:8/751: sync 2024-03-13T14:49:56.791 INFO:tasks.workunit.client.1.smithi196.stdout:8/752: write d0/d10/d12/d1e/fbb [615652,74922] 0 2024-03-13T14:49:56.798 INFO:tasks.workunit.client.1.smithi196.stdout:8/753: chown d0/d10/d12/df5/d47/f52 31006425 1 2024-03-13T14:49:56.824 INFO:tasks.workunit.client.1.smithi196.stdout:4/758: truncate d1/d1e/f6a 788334 0 2024-03-13T14:49:56.832 INFO:tasks.workunit.client.1.smithi196.stdout:9/741: link d7/de/d74/fd4 d7/de/d74/d94/dfb/dbf/f100 0 2024-03-13T14:49:56.834 INFO:tasks.workunit.client.1.smithi196.stdout:9/742: chown d7/f45 16 1 2024-03-13T14:49:56.838 INFO:tasks.workunit.client.1.smithi196.stdout:5/836: rename d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/ca1 to d4/d1b/dc8/d3c/d6a/d91/c108 0 2024-03-13T14:49:56.842 INFO:tasks.workunit.client.0.smithi176.stdout:4/714: truncate d7/d8/d15/d23/f7d 746465 0 2024-03-13T14:49:56.849 INFO:tasks.workunit.client.1.smithi196.stdout:4/759: dread d1/d3/d7/f1d [0,4194304] 0 2024-03-13T14:49:56.849 INFO:tasks.workunit.client.0.smithi176.stdout:6/688: rmdir d0/d10/d19 39 2024-03-13T14:49:56.851 INFO:tasks.workunit.client.1.smithi196.stdout:4/760: chown d1/d3/d7/d35/d3e/l7b 1517606616 1 2024-03-13T14:49:56.852 INFO:tasks.workunit.client.1.smithi196.stdout:4/761: chown d1/d3/f98 26378 1 2024-03-13T14:49:56.854 INFO:tasks.workunit.client.0.smithi176.stdout:9/802: rename d0/d17/l1a to d0/db/d89/d32/d58/d5f/d84/d9a/l10c 0 2024-03-13T14:49:56.856 INFO:tasks.workunit.client.0.smithi176.stdout:3/742: rmdir d4/d28/d31/d45/d82/d87 39 2024-03-13T14:49:56.856 INFO:tasks.workunit.client.0.smithi176.stdout:6/689: dread d0/de/f49 [0,4194304] 0 2024-03-13T14:49:56.862 INFO:tasks.workunit.client.0.smithi176.stdout:8/628: dread - d0/d24/d7b/dab/f92 zero size 2024-03-13T14:49:56.870 INFO:tasks.workunit.client.1.smithi196.stdout:4/762: dwrite d1/d3/d7/d35/d76/fed [0,4194304] 0 2024-03-13T14:49:56.870 INFO:tasks.workunit.client.0.smithi176.stdout:1/667: read d2/de/d10/d16/f27 [4427825,80544] 0 2024-03-13T14:49:56.870 INFO:tasks.workunit.client.0.smithi176.stdout:5/659: symlink d5/d2b/dce/le4 0 2024-03-13T14:49:56.870 INFO:tasks.workunit.client.0.smithi176.stdout:1/668: fsync d2/d30/d63/d22/f2b 0 2024-03-13T14:49:56.872 INFO:tasks.workunit.client.0.smithi176.stdout:5/660: chown d5/d2e/d43/f50 238330 1 2024-03-13T14:49:56.878 INFO:tasks.workunit.client.0.smithi176.stdout:4/715: rename d7/d8/d33/d96/cb1 to d7/d8/d33/cf8 0 2024-03-13T14:49:56.881 INFO:tasks.workunit.client.1.smithi196.stdout:4/763: dread d1/d3/d7/f1d [0,4194304] 0 2024-03-13T14:49:56.884 INFO:tasks.workunit.client.0.smithi176.stdout:9/803: dwrite d0/d1e/d1f/f38 [0,4194304] 0 2024-03-13T14:49:56.885 INFO:tasks.workunit.client.0.smithi176.stdout:4/716: truncate d7/d10/d46/d4a/d5d/ff5 221081 0 2024-03-13T14:49:56.892 INFO:tasks.workunit.client.0.smithi176.stdout:6/690: chown d0/d10/d19/l66 360973365 1 2024-03-13T14:49:56.894 INFO:tasks.workunit.client.0.smithi176.stdout:5/661: mknod d5/d2c/d74/ce5 0 2024-03-13T14:49:56.896 INFO:tasks.workunit.client.0.smithi176.stdout:7/650: read - d1/d28/d3a/d1b/d8f/f99 zero size 2024-03-13T14:49:56.906 INFO:tasks.workunit.client.0.smithi176.stdout:4/717: mknod d7/d10/d2c/d9d/dbb/cf9 0 2024-03-13T14:49:56.912 INFO:tasks.workunit.client.0.smithi176.stdout:5/662: read - d5/d31/d86/f9d zero size 2024-03-13T14:49:56.920 INFO:tasks.workunit.client.0.smithi176.stdout:9/804: creat d0/db/d89/d90/de6/f10d x:0 0 0 2024-03-13T14:49:56.920 INFO:tasks.workunit.client.0.smithi176.stdout:3/743: creat d4/d28/d31/f103 x:0 0 0 2024-03-13T14:49:56.923 INFO:tasks.workunit.client.0.smithi176.stdout:6/691: rename d0/de/d38/d70/f8e to d0/d10/d19/d27/d2b/d2e/dc4/dc5/fde 0 2024-03-13T14:49:56.928 INFO:tasks.workunit.client.1.smithi196.stdout:1/865: sync 2024-03-13T14:49:56.934 INFO:tasks.workunit.client.0.smithi176.stdout:9/805: link d0/d17/f19 d0/d1e/d8d/dbe/f10e 0 2024-03-13T14:49:56.937 INFO:tasks.workunit.client.0.smithi176.stdout:5/663: dwrite d5/d2e/f54 [4194304,4194304] 0 2024-03-13T14:49:56.946 INFO:tasks.workunit.client.0.smithi176.stdout:1/669: dread d2/de/d3f/d19/f32 [0,4194304] 0 2024-03-13T14:49:56.950 INFO:tasks.workunit.client.0.smithi176.stdout:6/692: getdents d0/d10/d19/d27/d2b/d2e/d3e 0 2024-03-13T14:49:56.956 INFO:tasks.workunit.client.1.smithi196.stdout:9/743: creat d7/f101 x:0 0 0 2024-03-13T14:49:56.958 INFO:tasks.workunit.client.1.smithi196.stdout:8/754: rename d0/d16/d3d/d9c/d87/laa to d0/d10/d12/d17/l104 0 2024-03-13T14:49:56.964 INFO:tasks.workunit.client.1.smithi196.stdout:7/894: mkdir d0/d9/d31/d77/d93/d94/d132 0 2024-03-13T14:49:56.965 INFO:tasks.workunit.client.0.smithi176.stdout:9/806: dwrite d0/d17/f19 [4194304,4194304] 0 2024-03-13T14:49:56.968 INFO:tasks.workunit.client.0.smithi176.stdout:1/670: dwrite d2/d29/f70 [4194304,4194304] 0 2024-03-13T14:49:56.969 INFO:tasks.workunit.client.0.smithi176.stdout:5/664: dwrite d5/d2c/d40/d76/f95 [0,4194304] 0 2024-03-13T14:49:56.972 INFO:tasks.workunit.client.1.smithi196.stdout:9/744: dwrite d7/d71/f9e [0,4194304] 0 2024-03-13T14:49:56.976 INFO:tasks.workunit.client.0.smithi176.stdout:1/671: truncate d2/de/d10/d16/d47/fbb 824410 0 2024-03-13T14:49:56.977 INFO:tasks.workunit.client.1.smithi196.stdout:7/895: rename d0/d6/db4/dca/d103 to d0/d6/d33/d57/d8a/d9f/d133 0 2024-03-13T14:49:56.980 INFO:tasks.workunit.client.0.smithi176.stdout:5/665: unlink d5/d2b/dce/le4 0 2024-03-13T14:49:56.992 INFO:tasks.workunit.client.1.smithi196.stdout:9/745: rename d7/de/de5/le6 to d7/de/d74/d94/dfb/d28/d6c/d7b/l102 0 2024-03-13T14:49:56.997 INFO:tasks.workunit.client.1.smithi196.stdout:7/896: unlink d0/d6/d33/d57/dae/de8/ff2 0 2024-03-13T14:49:57.000 INFO:tasks.workunit.client.0.smithi176.stdout:9/807: link d0/db/d89/d32/d58/d67/fbb d0/dd8/d82/f10f 0 2024-03-13T14:49:57.010 INFO:tasks.workunit.client.1.smithi196.stdout:9/746: read d7/de/d74/d94/dfb/f38 [3091621,44266] 0 2024-03-13T14:49:57.021 INFO:tasks.workunit.client.0.smithi176.stdout:7/651: creat d1/d3f/d81/fca x:0 0 0 2024-03-13T14:49:57.022 INFO:tasks.workunit.client.0.smithi176.stdout:2/717: symlink d1/dc/d61/lef 0 2024-03-13T14:49:57.025 INFO:tasks.workunit.client.0.smithi176.stdout:2/718: dread - d1/dc/d1c/fa4 zero size 2024-03-13T14:49:57.028 INFO:tasks.workunit.client.0.smithi176.stdout:9/808: dwrite d0/d1e/d8d/dbe/ffe [0,4194304] 0 2024-03-13T14:49:57.029 INFO:tasks.workunit.client.1.smithi196.stdout:0/816: write d8/da/d74/f8f [2115224,98043] 0 2024-03-13T14:49:57.031 INFO:tasks.workunit.client.0.smithi176.stdout:7/652: chown d1/d28/d3a/d1b/d8f/fac 47632 1 2024-03-13T14:49:57.034 INFO:tasks.workunit.client.0.smithi176.stdout:7/653: dread - d1/d3f/d4b/fc2 zero size 2024-03-13T14:49:57.035 INFO:tasks.workunit.client.1.smithi196.stdout:0/817: dread - d8/da/d25/d26/d35/d3b/fe0 zero size 2024-03-13T14:49:57.036 INFO:tasks.workunit.client.0.smithi176.stdout:2/719: symlink d1/dc/d2d/d90/dcf/da0/lf0 0 2024-03-13T14:49:57.038 INFO:tasks.workunit.client.1.smithi196.stdout:6/966: dwrite d6/d9/d17/d19/f42 [0,4194304] 0 2024-03-13T14:49:57.041 INFO:tasks.workunit.client.1.smithi196.stdout:6/967: chown d6/d9/d17/d19/cd0 116217451 1 2024-03-13T14:49:57.042 INFO:tasks.workunit.client.0.smithi176.stdout:9/809: mkdir d0/db/d89/d4d/d6e/d100/d110 0 2024-03-13T14:49:57.042 INFO:tasks.workunit.client.0.smithi176.stdout:0/597: dwrite d2/d5/d1e/f52 [0,4194304] 0 2024-03-13T14:49:57.042 INFO:tasks.workunit.client.1.smithi196.stdout:0/818: mknod d8/da/d25/d26/d35/d3b/d50/d5d/c10b 0 2024-03-13T14:49:57.045 INFO:tasks.workunit.client.0.smithi176.stdout:0/598: chown d2/db/d31/l5b 1349428 1 2024-03-13T14:49:57.045 INFO:tasks.workunit.client.1.smithi196.stdout:4/764: creat d1/d3/d7/d35/f10c x:0 0 0 2024-03-13T14:49:57.045 INFO:tasks.workunit.client.1.smithi196.stdout:6/968: readlink d6/d2a/dc/d1c/l7d 0 2024-03-13T14:49:57.046 INFO:tasks.workunit.client.0.smithi176.stdout:2/720: rmdir d1/dc/d2d/d4b 39 2024-03-13T14:49:57.046 INFO:tasks.workunit.client.0.smithi176.stdout:9/810: symlink d0/d1e/d6f/l111 0 2024-03-13T14:49:57.047 INFO:tasks.workunit.client.1.smithi196.stdout:0/819: mknod d8/da/d25/d26/d48/d82/db3/dd5/c10c 0 2024-03-13T14:49:57.048 INFO:tasks.workunit.client.0.smithi176.stdout:7/654: truncate d1/d1c/d26/d41/d2b/d40/f8a 2047409 0 2024-03-13T14:49:57.052 INFO:tasks.workunit.client.0.smithi176.stdout:0/599: write d2/db/f3b [3987577,63862] 0 2024-03-13T14:49:57.053 INFO:tasks.workunit.client.0.smithi176.stdout:2/721: mkdir d1/dc/d2d/d4f/df1 0 2024-03-13T14:49:57.059 INFO:tasks.workunit.client.0.smithi176.stdout:9/811: creat d0/d1e/d8d/da7/f112 x:0 0 0 2024-03-13T14:49:57.060 INFO:tasks.workunit.client.0.smithi176.stdout:1/672: dread d2/d30/f4c [0,4194304] 0 2024-03-13T14:49:57.063 INFO:tasks.workunit.client.0.smithi176.stdout:0/600: symlink d2/db/d39/lba 0 2024-03-13T14:49:57.065 INFO:tasks.workunit.client.1.smithi196.stdout:4/765: dwrite d1/d3/d7/d35/d76/f94 [0,4194304] 0 2024-03-13T14:49:57.067 INFO:tasks.workunit.client.0.smithi176.stdout:9/812: creat d0/d40/f113 x:0 0 0 2024-03-13T14:49:57.068 INFO:tasks.workunit.client.1.smithi196.stdout:4/766: dread - d1/d3/d7/d35/d3e/d56/f103 zero size 2024-03-13T14:49:57.071 INFO:tasks.workunit.client.1.smithi196.stdout:6/969: creat d6/d9/d17/d19/f13d x:0 0 0 2024-03-13T14:49:57.078 INFO:tasks.workunit.client.0.smithi176.stdout:0/601: mknod d2/d5/d1e/cbb 0 2024-03-13T14:49:57.080 INFO:tasks.workunit.client.0.smithi176.stdout:7/655: dwrite d1/d1c/f79 [0,4194304] 0 2024-03-13T14:49:57.086 INFO:tasks.workunit.client.0.smithi176.stdout:1/673: dwrite d2/de/d10/d16/d4a/f8a [0,4194304] 0 2024-03-13T14:49:57.088 INFO:tasks.workunit.client.0.smithi176.stdout:0/602: truncate d2/db/d39/d9c/fb0 376360 0 2024-03-13T14:49:57.088 INFO:tasks.workunit.client.1.smithi196.stdout:7/897: creat d0/d9/d31/d77/dcf/f134 x:0 0 0 2024-03-13T14:49:57.091 INFO:tasks.workunit.client.1.smithi196.stdout:7/898: rmdir d0/d9/d1e/d58/d72 39 2024-03-13T14:49:57.091 INFO:tasks.workunit.client.0.smithi176.stdout:1/674: dread - d2/de/d3f/d50/d83/d97/fcf zero size 2024-03-13T14:49:57.093 INFO:tasks.workunit.client.0.smithi176.stdout:0/603: write d2/d5/d1e/f71 [4698624,57913] 0 2024-03-13T14:49:57.098 INFO:tasks.workunit.client.0.smithi176.stdout:7/656: truncate d1/d28/d3a/f5c 1473423 0 2024-03-13T14:49:57.106 INFO:tasks.workunit.client.0.smithi176.stdout:0/604: chown d2/db/d39/d9c/ca1 31105 1 2024-03-13T14:49:57.106 INFO:tasks.workunit.client.0.smithi176.stdout:4/718: sync 2024-03-13T14:49:57.110 INFO:tasks.workunit.client.0.smithi176.stdout:8/629: dwrite d0/db/d39/d56/d91/f9c [0,4194304] 0 2024-03-13T14:49:57.111 INFO:tasks.workunit.client.1.smithi196.stdout:2/733: sync 2024-03-13T14:49:57.111 INFO:tasks.workunit.client.1.smithi196.stdout:3/795: sync 2024-03-13T14:49:57.116 INFO:tasks.workunit.client.0.smithi176.stdout:7/657: fsync d1/d28/f2a 0 2024-03-13T14:49:57.117 INFO:tasks.workunit.client.0.smithi176.stdout:2/722: dread d1/dc/d2d/d59/f8a [0,4194304] 0 2024-03-13T14:49:57.119 INFO:tasks.workunit.client.0.smithi176.stdout:0/605: mknod d2/db/cbc 0 2024-03-13T14:49:57.120 INFO:tasks.workunit.client.1.smithi196.stdout:5/837: write d4/d1b/dc8/d14/d19/d26/f51 [835625,53696] 0 2024-03-13T14:49:57.122 INFO:tasks.workunit.client.0.smithi176.stdout:2/723: dread d1/f96 [0,4194304] 0 2024-03-13T14:49:57.125 INFO:tasks.workunit.client.0.smithi176.stdout:8/630: symlink d0/db/d53/lc1 0 2024-03-13T14:49:57.125 INFO:tasks.workunit.client.0.smithi176.stdout:0/606: fsync d2/d5/f64 0 2024-03-13T14:49:57.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:56 smithi176 ceph-mon[29064]: pgmap v17: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 94 MiB/s rd, 249 MiB/s wr, 520 op/s 2024-03-13T14:49:57.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:56 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:57.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:56 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:57.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:56 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:49:57.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:56 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:49:57.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:56 smithi176 ceph-mon[29064]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:49:57.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:56 smithi176 ceph-mon[29064]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:49:57.131 INFO:tasks.workunit.client.0.smithi176.stdout:0/607: read d2/d5/f87 [56968,101438] 0 2024-03-13T14:49:57.137 INFO:tasks.workunit.client.0.smithi176.stdout:4/719: dwrite d7/db9/fe7 [0,4194304] 0 2024-03-13T14:49:57.139 INFO:tasks.workunit.client.1.smithi196.stdout:5/838: dwrite d4/d1b/dc8/d4a/faf [0,4194304] 0 2024-03-13T14:49:57.141 INFO:tasks.workunit.client.0.smithi176.stdout:6/693: truncate d0/d10/d19/d27/fbe 2030845 0 2024-03-13T14:49:57.144 INFO:tasks.workunit.client.0.smithi176.stdout:0/608: rmdir d2/d5/dc/d20 39 2024-03-13T14:49:57.149 INFO:tasks.workunit.client.0.smithi176.stdout:3/744: sync 2024-03-13T14:49:57.150 INFO:tasks.workunit.client.0.smithi176.stdout:8/631: mkdir d0/d5/dc2 0 2024-03-13T14:49:57.152 INFO:tasks.workunit.client.0.smithi176.stdout:4/720: truncate d7/d8/d45/f62 1097485 0 2024-03-13T14:49:57.153 INFO:tasks.workunit.client.0.smithi176.stdout:6/694: creat d0/d10/d19/d7d/fdf x:0 0 0 2024-03-13T14:49:57.154 INFO:tasks.workunit.client.0.smithi176.stdout:9/813: dread d0/d1e/d8d/dbe/fde [0,4194304] 0 2024-03-13T14:49:57.158 INFO:tasks.workunit.client.0.smithi176.stdout:7/658: dwrite d1/d1c/d26/d41/d2b/f96 [0,4194304] 0 2024-03-13T14:49:57.160 INFO:tasks.workunit.client.1.smithi196.stdout:1/866: dwrite d1/d28/da3/fdc [0,4194304] 0 2024-03-13T14:49:57.165 INFO:tasks.workunit.client.1.smithi196.stdout:3/796: dread db/d19/d22/f31 [0,4194304] 0 2024-03-13T14:49:57.169 INFO:tasks.workunit.client.0.smithi176.stdout:8/632: mkdir d0/d24/dc3 0 2024-03-13T14:49:57.175 INFO:tasks.workunit.client.0.smithi176.stdout:9/814: readlink d0/d1e/d1f/l2a 0 2024-03-13T14:49:57.176 INFO:tasks.workunit.client.1.smithi196.stdout:3/797: symlink db/d15/da1/lfe 0 2024-03-13T14:49:57.176 INFO:tasks.workunit.client.0.smithi176.stdout:7/659: unlink d1/d1c/d26/d41/fad 0 2024-03-13T14:49:57.177 INFO:tasks.workunit.client.0.smithi176.stdout:0/609: mknod d2/d74/d61/cbd 0 2024-03-13T14:49:57.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:56 smithi196 ceph-mon[37486]: pgmap v17: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 94 MiB/s rd, 249 MiB/s wr, 520 op/s 2024-03-13T14:49:57.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:56 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:57.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:56 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:49:57.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:56 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:49:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:56 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:49:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:56 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:49:57.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:56 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:49:57.180 INFO:tasks.workunit.client.1.smithi196.stdout:1/867: dread d1/d9/f9a [0,4194304] 0 2024-03-13T14:49:57.180 INFO:tasks.workunit.client.0.smithi176.stdout:4/721: symlink d7/d2d/da7/db3/lfa 0 2024-03-13T14:49:57.181 INFO:tasks.workunit.client.0.smithi176.stdout:5/666: dwrite d5/d2e/fd2 [0,4194304] 0 2024-03-13T14:49:57.182 INFO:tasks.workunit.client.0.smithi176.stdout:6/695: truncate d0/de/d8d/fbd 4250124 0 2024-03-13T14:49:57.182 INFO:tasks.workunit.client.1.smithi196.stdout:3/798: fsync db/fe 0 2024-03-13T14:49:57.184 INFO:tasks.workunit.client.1.smithi196.stdout:0/820: rename d8/da/d25/l2e to d8/da/d25/d26/d48/d82/db3/l10d 0 2024-03-13T14:49:57.187 INFO:tasks.workunit.client.0.smithi176.stdout:6/696: dread d0/de/f49 [0,4194304] 0 2024-03-13T14:49:57.194 INFO:tasks.workunit.client.1.smithi196.stdout:3/799: mkdir db/d19/d50/db7/dff 0 2024-03-13T14:49:57.196 INFO:tasks.workunit.client.0.smithi176.stdout:6/697: read d0/d10/d19/d27/d2b/d2e/f40 [240039,109516] 0 2024-03-13T14:49:57.196 INFO:tasks.workunit.client.0.smithi176.stdout:6/698: stat d0/d10 0 2024-03-13T14:49:57.199 INFO:tasks.workunit.client.0.smithi176.stdout:7/660: unlink d1/l53 0 2024-03-13T14:49:57.200 INFO:tasks.workunit.client.1.smithi196.stdout:5/839: rmdir d4/d60 39 2024-03-13T14:49:57.202 INFO:tasks.workunit.client.1.smithi196.stdout:0/821: creat d8/da/d25/dd7/f10e x:0 0 0 2024-03-13T14:49:57.204 INFO:tasks.workunit.client.0.smithi176.stdout:9/815: dread d0/dd8/d82/fb7 [0,4194304] 0 2024-03-13T14:49:57.204 INFO:tasks.workunit.client.1.smithi196.stdout:1/868: dwrite d1/d9/d4e/d55/d90/fff [0,4194304] 0 2024-03-13T14:49:57.205 INFO:tasks.workunit.client.0.smithi176.stdout:4/722: dwrite d7/d2d/da7/fae [0,4194304] 0 2024-03-13T14:49:57.205 INFO:tasks.workunit.client.1.smithi196.stdout:1/869: chown d1/d94 5853330 1 2024-03-13T14:49:57.210 INFO:tasks.workunit.client.1.smithi196.stdout:8/755: sync 2024-03-13T14:49:57.213 INFO:tasks.workunit.client.0.smithi176.stdout:8/633: symlink d0/db/d39/d56/lc4 0 2024-03-13T14:49:57.217 INFO:tasks.workunit.client.0.smithi176.stdout:8/634: fsync d0/db/dc/f6b 0 2024-03-13T14:49:57.218 INFO:tasks.workunit.client.1.smithi196.stdout:5/840: mknod d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/c109 0 2024-03-13T14:49:57.220 INFO:tasks.workunit.client.1.smithi196.stdout:1/870: symlink d1/d9/d33/d4b/d66/d7b/l12a 0 2024-03-13T14:49:57.222 INFO:tasks.workunit.client.0.smithi176.stdout:6/699: rmdir d0/d10/d19/d9f/d76 39 2024-03-13T14:49:57.222 INFO:tasks.workunit.client.0.smithi176.stdout:0/610: mknod d2/d5/d1e/d97/cbe 0 2024-03-13T14:49:57.224 INFO:tasks.workunit.client.1.smithi196.stdout:1/871: dread d1/d9/d33/d4b/d66/d7b/f8d [0,4194304] 0 2024-03-13T14:49:57.225 INFO:tasks.workunit.client.0.smithi176.stdout:7/661: unlink d1/d1c/d26/d41/d2b/d40/d64/f95 0 2024-03-13T14:49:57.226 INFO:tasks.workunit.client.1.smithi196.stdout:8/756: truncate d0/d10/d12/df5/d47/f52 492951 0 2024-03-13T14:49:57.229 INFO:tasks.workunit.client.1.smithi196.stdout:6/970: write d6/d9/d17/d19/f13d [782418,13258] 0 2024-03-13T14:49:57.230 INFO:tasks.workunit.client.1.smithi196.stdout:7/899: rename d0/d78/c79 to d0/d9/c135 0 2024-03-13T14:49:57.231 INFO:tasks.workunit.client.0.smithi176.stdout:9/816: dwrite d0/d1e/f2e [8388608,4194304] 0 2024-03-13T14:49:57.232 INFO:tasks.workunit.client.1.smithi196.stdout:1/872: truncate d1/d9/d6c/fc1 3788056 0 2024-03-13T14:49:57.234 INFO:tasks.workunit.client.0.smithi176.stdout:4/723: dwrite d7/d8/d33/f6d [4194304,4194304] 0 2024-03-13T14:49:57.235 INFO:tasks.workunit.client.0.smithi176.stdout:9/817: read d0/d1e/d1f/d68/fb4 [552193,115412] 0 2024-03-13T14:49:57.236 INFO:tasks.workunit.client.1.smithi196.stdout:8/757: mknod d0/d10/d12/d17/d42/c105 0 2024-03-13T14:49:57.237 INFO:tasks.workunit.client.0.smithi176.stdout:9/818: dread - d0/d1e/d8d/da7/f103 zero size 2024-03-13T14:49:57.239 INFO:tasks.workunit.client.1.smithi196.stdout:1/873: truncate d1/d38/fef 395171 0 2024-03-13T14:49:57.240 INFO:tasks.workunit.client.1.smithi196.stdout:3/800: rename db/d19/d4a/d9b/cf1 to db/d19/d22/d3a/d68/d8e/c100 0 2024-03-13T14:49:57.241 INFO:tasks.workunit.client.0.smithi176.stdout:8/635: stat d0/d24/d29/f72 0 2024-03-13T14:49:57.243 INFO:tasks.workunit.client.0.smithi176.stdout:8/636: stat d0/d5/d11/f1b 0 2024-03-13T14:49:57.248 INFO:tasks.workunit.client.0.smithi176.stdout:6/700: rename d0/d10/d19/d9f to d0/d10/d19/d27/d2b/de0 0 2024-03-13T14:49:57.251 INFO:tasks.workunit.client.1.smithi196.stdout:4/767: dwrite d1/d3/dbc/fdb [0,4194304] 0 2024-03-13T14:49:57.251 INFO:tasks.workunit.client.1.smithi196.stdout:8/758: dread d0/d16/d3d/d9c/f9d [0,4194304] 0 2024-03-13T14:49:57.253 INFO:tasks.workunit.client.1.smithi196.stdout:2/734: truncate d0/d7/d9/d61/fb6 910583 0 2024-03-13T14:49:57.254 INFO:tasks.workunit.client.0.smithi176.stdout:6/701: dread - d0/de/d38/d88/d9d/fd4 zero size 2024-03-13T14:49:57.254 INFO:tasks.workunit.client.0.smithi176.stdout:7/662: symlink d1/d3f/d65/lcb 0 2024-03-13T14:49:57.261 INFO:tasks.workunit.client.1.smithi196.stdout:6/971: symlink d6/d2a/dc/d1c/l13e 0 2024-03-13T14:49:57.263 INFO:tasks.workunit.client.1.smithi196.stdout:1/874: dread - d1/d9/d4e/d55/d63/db9/dbb/fce zero size 2024-03-13T14:49:57.263 INFO:tasks.workunit.client.0.smithi176.stdout:2/724: write d1/dc/f2e [3168388,48402] 0 2024-03-13T14:49:57.264 INFO:tasks.workunit.client.0.smithi176.stdout:1/675: dwrite d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:49:57.266 INFO:tasks.workunit.client.0.smithi176.stdout:9/819: dwrite d0/db/d89/d32/f107 [0,4194304] 0 2024-03-13T14:49:57.266 INFO:tasks.workunit.client.0.smithi176.stdout:0/611: creat d2/d74/d9f/fbf x:0 0 0 2024-03-13T14:49:57.268 INFO:tasks.workunit.client.0.smithi176.stdout:0/612: readlink d2/d5/d28/l70 0 2024-03-13T14:49:57.270 INFO:tasks.workunit.client.0.smithi176.stdout:1/676: fsync d2/de/d10/d16/d59/f6e 0 2024-03-13T14:49:57.272 INFO:tasks.workunit.client.1.smithi196.stdout:5/841: dread d4/d1b/dc8/d14/d19/d26/d68/f72 [0,4194304] 0 2024-03-13T14:49:57.273 INFO:tasks.workunit.client.0.smithi176.stdout:6/702: mkdir d0/d10/dca/de1 0 2024-03-13T14:49:57.276 INFO:tasks.workunit.client.0.smithi176.stdout:1/677: chown d2/de/d10/d16/d47/lc8 250 1 2024-03-13T14:49:57.282 INFO:tasks.workunit.client.0.smithi176.stdout:3/745: write d4/d28/d31/f75 [240128,65613] 0 2024-03-13T14:49:57.282 INFO:tasks.workunit.client.1.smithi196.stdout:4/768: creat d1/d3/d7/d35/d76/d58/d69/d9f/f10d x:0 0 0 2024-03-13T14:49:57.284 INFO:tasks.workunit.client.1.smithi196.stdout:9/747: sync 2024-03-13T14:49:57.286 INFO:tasks.workunit.client.0.smithi176.stdout:3/746: write d4/d28/d31/f62 [926553,27920] 0 2024-03-13T14:49:57.288 INFO:tasks.workunit.client.0.smithi176.stdout:8/637: mkdir d0/d24/d98/dc5 0 2024-03-13T14:49:57.298 INFO:tasks.workunit.client.1.smithi196.stdout:7/900: truncate d0/d9/d1e/d58/d72/da9/da3/ffa 826900 0 2024-03-13T14:49:57.300 INFO:tasks.workunit.client.1.smithi196.stdout:6/972: dwrite d6/f8 [0,4194304] 0 2024-03-13T14:49:57.301 INFO:tasks.workunit.client.0.smithi176.stdout:0/613: creat d2/d5/d28/dac/fc0 x:0 0 0 2024-03-13T14:49:57.302 INFO:tasks.workunit.client.0.smithi176.stdout:6/703: unlink d0/f69 0 2024-03-13T14:49:57.305 INFO:tasks.workunit.client.1.smithi196.stdout:6/973: chown d6/d7/d135/c128 53 1 2024-03-13T14:49:57.308 INFO:tasks.workunit.client.1.smithi196.stdout:3/801: dread db/d4c/feb [0,4194304] 0 2024-03-13T14:49:57.311 INFO:tasks.workunit.client.0.smithi176.stdout:4/724: creat d7/ffb x:0 0 0 2024-03-13T14:49:57.311 INFO:tasks.workunit.client.1.smithi196.stdout:9/748: mknod d7/d71/c103 0 2024-03-13T14:49:57.314 INFO:tasks.workunit.client.1.smithi196.stdout:4/769: creat d1/d3/d7/d35/d76/d58/d69/d9f/f10e x:0 0 0 2024-03-13T14:49:57.315 INFO:tasks.workunit.client.0.smithi176.stdout:7/663: rename d1/d28/f2d to d1/d1c/d26/d41/fcc 0 2024-03-13T14:49:57.315 INFO:tasks.workunit.client.0.smithi176.stdout:5/667: write d5/d2c/d40/dc7/d89/fa1 [953783,66772] 0 2024-03-13T14:49:57.316 INFO:tasks.workunit.client.1.smithi196.stdout:0/822: write d8/da/d25/d26/d48/d82/f8b [4536671,64437] 0 2024-03-13T14:49:57.321 INFO:tasks.workunit.client.1.smithi196.stdout:2/735: dread d0/d6/d1d/f23 [0,4194304] 0 2024-03-13T14:49:57.322 INFO:tasks.workunit.client.0.smithi176.stdout:3/747: truncate d4/d28/d34/da4/fa9 4081795 0 2024-03-13T14:49:57.323 INFO:tasks.workunit.client.1.smithi196.stdout:2/736: chown d0/d7/d9/d42/d98/de4/d67 515983739 1 2024-03-13T14:49:57.325 INFO:tasks.workunit.client.0.smithi176.stdout:0/614: dread - d2/db/d39/f73 zero size 2024-03-13T14:49:57.326 INFO:tasks.workunit.client.1.smithi196.stdout:6/974: dread - d6/d7/df/d27/d39/d44/d74/f107 zero size 2024-03-13T14:49:57.327 INFO:tasks.workunit.client.1.smithi196.stdout:0/823: dread - d8/da/d25/d26/d54/d88/f101 zero size 2024-03-13T14:49:57.328 INFO:tasks.workunit.client.0.smithi176.stdout:6/704: chown d0/d10/d19/d27/d2b/d2e/d3e/l4f 1240917 1 2024-03-13T14:49:57.332 INFO:tasks.workunit.client.1.smithi196.stdout:1/875: dread d1/d9/d33/d4b/d66/f7c [4194304,4194304] 0 2024-03-13T14:49:57.332 INFO:tasks.workunit.client.0.smithi176.stdout:1/678: creat d2/de/dd1/fd4 x:0 0 0 2024-03-13T14:49:57.333 INFO:tasks.workunit.client.0.smithi176.stdout:6/705: fsync d0/de/d38/d70/f9e 0 2024-03-13T14:49:57.334 INFO:tasks.workunit.client.1.smithi196.stdout:2/737: mkdir d0/d7/d9/d5a/df2 0 2024-03-13T14:49:57.335 INFO:tasks.workunit.client.1.smithi196.stdout:6/975: dread d6/d2a/dc/d1c/f123 [0,4194304] 0 2024-03-13T14:49:57.339 INFO:tasks.workunit.client.0.smithi176.stdout:8/638: rename d0/db/dc to d0/d24/d98/dc6 0 2024-03-13T14:49:57.342 INFO:tasks.workunit.client.1.smithi196.stdout:3/802: mkdir db/d101 0 2024-03-13T14:49:57.342 INFO:tasks.workunit.client.1.smithi196.stdout:9/749: dread d7/d9/d83/f9d [0,4194304] 0 2024-03-13T14:49:57.343 INFO:tasks.workunit.client.0.smithi176.stdout:8/639: write d0/d5/d11/f7f [268271,10104] 0 2024-03-13T14:49:57.345 INFO:tasks.workunit.client.1.smithi196.stdout:3/803: stat db/d19/d22/d3a/d53/laa 0 2024-03-13T14:49:57.355 INFO:tasks.workunit.client.1.smithi196.stdout:0/824: creat d8/da/d25/f10f x:0 0 0 2024-03-13T14:49:57.355 INFO:tasks.workunit.client.1.smithi196.stdout:8/759: write d0/d16/d3d/d8b/d3f/f76 [931198,80617] 0 2024-03-13T14:49:57.357 INFO:tasks.workunit.client.1.smithi196.stdout:1/876: mknod d1/d9/d33/d4b/d66/c12b 0 2024-03-13T14:49:57.363 INFO:tasks.workunit.client.0.smithi176.stdout:6/706: dread d0/d10/d19/d27/fa9 [0,4194304] 0 2024-03-13T14:49:57.364 INFO:tasks.workunit.client.1.smithi196.stdout:5/842: write d4/d1b/dc8/d14/d19/d26/d68/fd3 [553493,115156] 0 2024-03-13T14:49:57.366 INFO:tasks.workunit.client.0.smithi176.stdout:3/748: rename d4/d13/d2d/fd3 to d4/d13/d16/ddf/f104 0 2024-03-13T14:49:57.368 INFO:tasks.workunit.client.1.smithi196.stdout:4/770: creat d1/d3/d7/d35/d76/f10f x:0 0 0 2024-03-13T14:49:57.369 INFO:tasks.workunit.client.0.smithi176.stdout:9/820: write d0/d17/fa3 [26645,15086] 0 2024-03-13T14:49:57.369 INFO:tasks.workunit.client.1.smithi196.stdout:1/877: dread d1/d9/d33/d4b/d66/d7b/fa9 [0,4194304] 0 2024-03-13T14:49:57.371 INFO:tasks.workunit.client.0.smithi176.stdout:8/640: fdatasync d0/db/d39/d56/f5e 0 2024-03-13T14:49:57.373 INFO:tasks.workunit.client.0.smithi176.stdout:7/664: mknod d1/d28/d3a/d1b/d8f/dc1/db7/ccd 0 2024-03-13T14:49:57.376 INFO:tasks.workunit.client.0.smithi176.stdout:7/665: write d1/d28/d3a/d1b/d8f/d91/fc0 [2612700,34151] 0 2024-03-13T14:49:57.378 INFO:tasks.workunit.client.0.smithi176.stdout:2/725: link d1/dc/fa5 d1/dc/d2d/d59/d80/d5c/d78/ff2 0 2024-03-13T14:49:57.379 INFO:tasks.workunit.client.1.smithi196.stdout:0/825: mknod d8/da/d25/d26/d35/d3b/d59/c110 0 2024-03-13T14:49:57.384 INFO:tasks.workunit.client.1.smithi196.stdout:9/750: symlink d7/de/l104 0 2024-03-13T14:49:57.386 INFO:tasks.workunit.client.1.smithi196.stdout:5/843: mknod d4/d1b/dc8/d14/d19/d1c/d2d/dc5/c10a 0 2024-03-13T14:49:57.388 INFO:tasks.workunit.client.1.smithi196.stdout:2/738: dread d0/d6/d4b/d5f/f3b [0,4194304] 0 2024-03-13T14:49:57.391 INFO:tasks.workunit.client.0.smithi176.stdout:3/749: dread d4/d13/d81/d89/d5b/dc5/fc7 [0,4194304] 0 2024-03-13T14:49:57.393 INFO:tasks.workunit.client.1.smithi196.stdout:7/901: dwrite d0/f6a [0,4194304] 0 2024-03-13T14:49:57.394 INFO:tasks.workunit.client.1.smithi196.stdout:3/804: mknod db/d37/c102 0 2024-03-13T14:49:57.396 INFO:tasks.workunit.client.1.smithi196.stdout:3/805: readlink db/d19/d50/ld3 0 2024-03-13T14:49:57.400 INFO:tasks.workunit.client.0.smithi176.stdout:6/707: truncate d0/f8 900089 0 2024-03-13T14:49:57.404 INFO:tasks.workunit.client.1.smithi196.stdout:8/760: fsync d0/d10/d12/df5/d47/f80 0 2024-03-13T14:49:57.407 INFO:tasks.workunit.client.1.smithi196.stdout:6/976: link d6/d2a/de2/fbe d6/d7/df/f13f 0 2024-03-13T14:49:57.409 INFO:tasks.workunit.client.0.smithi176.stdout:5/668: creat d5/d2e/d7e/dad/fe6 x:0 0 0 2024-03-13T14:49:57.410 INFO:tasks.workunit.client.0.smithi176.stdout:2/726: dread d1/dc/d2d/f2f [0,4194304] 0 2024-03-13T14:49:57.410 INFO:tasks.workunit.client.0.smithi176.stdout:5/669: chown d5/lc5 30 1 2024-03-13T14:49:57.412 INFO:tasks.workunit.client.1.smithi196.stdout:9/751: symlink d7/de/d74/d94/l105 0 2024-03-13T14:49:57.415 INFO:tasks.workunit.client.1.smithi196.stdout:9/752: write d7/de/d4b/f1a [3550253,116789] 0 2024-03-13T14:49:57.415 INFO:tasks.workunit.client.1.smithi196.stdout:0/826: truncate d8/da/d25/d26/d48/d82/db3/dd5/fe6 751052 0 2024-03-13T14:49:57.417 INFO:tasks.workunit.client.1.smithi196.stdout:4/771: fsync d1/d3/d7/ddf/fd2 0 2024-03-13T14:49:57.418 INFO:tasks.workunit.client.0.smithi176.stdout:7/666: chown d1/d1c/l69 3980864 1 2024-03-13T14:49:57.419 INFO:tasks.workunit.client.0.smithi176.stdout:7/667: read d1/d1c/d4a/f98 [352535,17249] 0 2024-03-13T14:49:57.422 INFO:tasks.workunit.client.0.smithi176.stdout:4/725: getdents d7/d10/d46/d4a/dcc 0 2024-03-13T14:49:57.423 INFO:tasks.workunit.client.1.smithi196.stdout:2/739: stat d0/d7/d9/d42/d98/de4/lb2 0 2024-03-13T14:49:57.424 INFO:tasks.workunit.client.0.smithi176.stdout:4/726: read d7/d8/f39 [1687219,111212] 0 2024-03-13T14:49:57.426 INFO:tasks.workunit.client.1.smithi196.stdout:7/902: mknod d0/d9/d31/d77/d93/d94/c136 0 2024-03-13T14:49:57.426 INFO:tasks.workunit.client.1.smithi196.stdout:2/740: chown d0/d7/d9/d9a/fb4 30976 1 2024-03-13T14:49:57.429 INFO:tasks.workunit.client.0.smithi176.stdout:6/708: mknod d0/d10/d19/d27/d2b/d2e/dc4/ce2 0 2024-03-13T14:49:57.429 INFO:tasks.workunit.client.0.smithi176.stdout:0/615: dwrite d2/db/f1c [0,4194304] 0 2024-03-13T14:49:57.434 INFO:tasks.workunit.client.0.smithi176.stdout:0/616: chown d2/d5/dc/d59/d79/c9a 721016115 1 2024-03-13T14:49:57.434 INFO:tasks.workunit.client.0.smithi176.stdout:0/617: fdatasync d2/d5/d28/f54 0 2024-03-13T14:49:57.437 INFO:tasks.workunit.client.1.smithi196.stdout:1/878: dwrite d1/d28/d34/f41 [4194304,4194304] 0 2024-03-13T14:49:57.439 INFO:tasks.workunit.client.1.smithi196.stdout:6/977: creat d6/d7/df/d27/d39/d44/dc8/d130/d82/f140 x:0 0 0 2024-03-13T14:49:57.443 INFO:tasks.workunit.client.0.smithi176.stdout:3/750: write d4/d13/d5e/f79 [220575,80929] 0 2024-03-13T14:49:57.447 INFO:tasks.workunit.client.0.smithi176.stdout:9/821: dwrite d0/d40/d53/f6b [4194304,4194304] 0 2024-03-13T14:49:57.448 INFO:tasks.workunit.client.0.smithi176.stdout:3/751: write d4/d28/d31/d52/f6a [4000865,120753] 0 2024-03-13T14:49:57.451 INFO:tasks.workunit.client.0.smithi176.stdout:9/822: chown d0/d17/l18 0 1 2024-03-13T14:49:57.454 INFO:tasks.workunit.client.0.smithi176.stdout:9/823: chown d0/db/f3f 5617 1 2024-03-13T14:49:57.455 INFO:tasks.workunit.client.0.smithi176.stdout:2/727: dwrite d1/f96 [0,4194304] 0 2024-03-13T14:49:57.456 INFO:tasks.workunit.client.1.smithi196.stdout:4/772: symlink d1/d3/d7/d35/d76/d58/d69/de3/df4/l110 0 2024-03-13T14:49:57.457 INFO:tasks.workunit.client.0.smithi176.stdout:7/668: dread d1/d28/d3a/f14 [0,4194304] 0 2024-03-13T14:49:57.460 INFO:tasks.workunit.client.1.smithi196.stdout:3/806: dwrite db/d19/f2b [4194304,4194304] 0 2024-03-13T14:49:57.461 INFO:tasks.workunit.client.0.smithi176.stdout:2/728: dread - d1/dc/d61/daa/fb8 zero size 2024-03-13T14:49:57.461 INFO:tasks.workunit.client.0.smithi176.stdout:0/618: fsync d2/d5/f87 0 2024-03-13T14:49:57.462 INFO:tasks.workunit.client.0.smithi176.stdout:8/641: creat d0/fc7 x:0 0 0 2024-03-13T14:49:57.463 INFO:tasks.workunit.client.1.smithi196.stdout:4/773: read d1/f2e [603044,36125] 0 2024-03-13T14:49:57.463 INFO:tasks.workunit.client.1.smithi196.stdout:8/761: dwrite d0/d16/d3d/d8b/f4a [0,4194304] 0 2024-03-13T14:49:57.464 INFO:tasks.workunit.client.0.smithi176.stdout:2/729: stat d1/dc/d2d/d4f/fac 0 2024-03-13T14:49:57.466 INFO:tasks.workunit.client.0.smithi176.stdout:0/619: truncate d2/d74/d9f/fbf 871852 0 2024-03-13T14:49:57.466 INFO:tasks.workunit.client.0.smithi176.stdout:9/824: dread d0/db/d89/d7a/ff3 [0,4194304] 0 2024-03-13T14:49:57.468 INFO:tasks.workunit.client.0.smithi176.stdout:9/825: stat d0/d40/d53/d6d/l9e 0 2024-03-13T14:49:57.468 INFO:tasks.workunit.client.1.smithi196.stdout:7/903: dwrite d0/d6/ddc/ffd [0,4194304] 0 2024-03-13T14:49:57.468 INFO:tasks.workunit.client.1.smithi196.stdout:1/879: mkdir d1/d28/d12c 0 2024-03-13T14:49:57.476 INFO:tasks.workunit.client.1.smithi196.stdout:9/753: mknod d7/de/d74/d94/dfb/c106 0 2024-03-13T14:49:57.478 INFO:tasks.workunit.client.1.smithi196.stdout:1/880: stat d1/d9/d4e/d55 0 2024-03-13T14:49:57.480 INFO:tasks.workunit.client.1.smithi196.stdout:5/844: truncate d4/d1b/dc8/d14/d19/d26/d33/fdb 3588499 0 2024-03-13T14:49:57.482 INFO:tasks.workunit.client.1.smithi196.stdout:0/827: dread d8/f9 [0,4194304] 0 2024-03-13T14:49:57.485 INFO:tasks.workunit.client.1.smithi196.stdout:1/881: read - d1/d9/d33/dd2/f120 zero size 2024-03-13T14:49:57.486 INFO:tasks.workunit.client.1.smithi196.stdout:8/762: fdatasync d0/d16/d3d/d7f/fa8 0 2024-03-13T14:49:57.488 INFO:tasks.workunit.client.0.smithi176.stdout:5/670: creat d5/d2c/fe7 x:0 0 0 2024-03-13T14:49:57.490 INFO:tasks.workunit.client.0.smithi176.stdout:3/752: read d4/d13/d81/d89/f60 [1775216,19729] 0 2024-03-13T14:49:57.492 INFO:tasks.workunit.client.0.smithi176.stdout:3/753: chown d4/c9d 9 1 2024-03-13T14:49:57.494 INFO:tasks.workunit.client.1.smithi196.stdout:1/882: read - d1/d9/d4e/d55/d63/da6/dc8/fe6 zero size 2024-03-13T14:49:57.494 INFO:tasks.workunit.client.1.smithi196.stdout:4/774: symlink d1/d3/d7/d35/d76/l111 0 2024-03-13T14:49:57.494 INFO:tasks.workunit.client.0.smithi176.stdout:3/754: write d4/f97 [2016146,113696] 0 2024-03-13T14:49:57.494 INFO:tasks.workunit.client.0.smithi176.stdout:7/669: symlink d1/d7/lce 0 2024-03-13T14:49:57.500 INFO:tasks.workunit.client.0.smithi176.stdout:2/730: mkdir d1/dc/d2d/d59/df3 0 2024-03-13T14:49:57.502 INFO:tasks.workunit.client.1.smithi196.stdout:6/978: mknod d6/d9/d17/d19/d4e/c141 0 2024-03-13T14:49:57.503 INFO:tasks.workunit.client.0.smithi176.stdout:2/731: chown d1/dc/d2d/d4f/c67 1 1 2024-03-13T14:49:57.507 INFO:tasks.workunit.client.0.smithi176.stdout:9/826: creat d0/d40/d53/f114 x:0 0 0 2024-03-13T14:49:57.508 INFO:tasks.workunit.client.1.smithi196.stdout:5/845: creat d4/d1b/dc8/d14/d19/d26/dab/f10b x:0 0 0 2024-03-13T14:49:57.510 INFO:tasks.workunit.client.1.smithi196.stdout:7/904: link d0/d6/db4/f112 d0/d9/d31/d77/d93/d94/d132/f137 0 2024-03-13T14:49:57.513 INFO:tasks.workunit.client.1.smithi196.stdout:3/807: creat db/d19/f103 x:0 0 0 2024-03-13T14:49:57.517 INFO:tasks.workunit.client.0.smithi176.stdout:7/670: dwrite d1/d28/d3a/d1b/fa6 [0,4194304] 0 2024-03-13T14:49:57.522 INFO:tasks.workunit.client.1.smithi196.stdout:6/979: write d6/d7/df/d27/d39/d44/dc8/d130/ffe [318519,61689] 0 2024-03-13T14:49:57.523 INFO:tasks.workunit.client.1.smithi196.stdout:9/754: link d7/de/d4b/d1d/d35/fb9 d7/de/d4b/d1d/d35/dcc/f107 0 2024-03-13T14:49:57.526 INFO:tasks.workunit.client.1.smithi196.stdout:0/828: dwrite d8/da/f13 [0,4194304] 0 2024-03-13T14:49:57.526 INFO:tasks.workunit.client.1.smithi196.stdout:8/763: dwrite d0/d10/d12/df5/f65 [0,4194304] 0 2024-03-13T14:49:57.531 INFO:tasks.workunit.client.0.smithi176.stdout:8/642: symlink d0/db/d39/d56/lc8 0 2024-03-13T14:49:57.531 INFO:tasks.workunit.client.0.smithi176.stdout:3/755: read d4/d13/d81/d89/fb4 [165722,4832] 0 2024-03-13T14:49:57.535 INFO:tasks.workunit.client.0.smithi176.stdout:3/756: read d4/d13/f73 [157381,112197] 0 2024-03-13T14:49:57.536 INFO:tasks.workunit.client.0.smithi176.stdout:3/757: stat d4/d28/d4f 0 2024-03-13T14:49:57.539 INFO:tasks.workunit.client.1.smithi196.stdout:2/741: truncate d0/d7/d9/d42/d98/de4/d67/fbd 964370 0 2024-03-13T14:49:57.540 INFO:tasks.workunit.client.1.smithi196.stdout:7/905: creat d0/d4d/dd1/dd6/f138 x:0 0 0 2024-03-13T14:49:57.541 INFO:tasks.workunit.client.0.smithi176.stdout:2/732: mkdir d1/dc/d61/df4 0 2024-03-13T14:49:57.543 INFO:tasks.workunit.client.0.smithi176.stdout:2/733: fsync d1/dc/d61/daa/fe4 0 2024-03-13T14:49:57.543 INFO:tasks.workunit.client.1.smithi196.stdout:7/906: stat d0/d6/c12 0 2024-03-13T14:49:57.544 INFO:tasks.workunit.client.1.smithi196.stdout:2/742: dread - d0/fbb zero size 2024-03-13T14:49:57.548 INFO:tasks.workunit.client.1.smithi196.stdout:7/907: chown d0/d9/d1e/d3f/lbe 7 1 2024-03-13T14:49:57.553 INFO:tasks.workunit.client.0.smithi176.stdout:4/727: getdents d7/d2d/da7/db3 0 2024-03-13T14:49:57.557 INFO:tasks.workunit.client.1.smithi196.stdout:7/908: dread - d0/d6/d33/d57/f10f zero size 2024-03-13T14:49:57.558 INFO:tasks.workunit.client.1.smithi196.stdout:5/846: dwrite d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/fbf [4194304,4194304] 0 2024-03-13T14:49:57.560 INFO:tasks.workunit.client.1.smithi196.stdout:3/808: creat db/d15/dfb/f104 x:0 0 0 2024-03-13T14:49:57.561 INFO:tasks.workunit.client.1.smithi196.stdout:0/829: readlink d8/da/d25/d26/d48/d82/db3/l10d 0 2024-03-13T14:49:57.567 INFO:tasks.workunit.client.0.smithi176.stdout:4/728: dwrite d7/d8/df6/f68 [4194304,4194304] 0 2024-03-13T14:49:57.571 INFO:tasks.workunit.client.0.smithi176.stdout:1/679: sync 2024-03-13T14:49:57.572 INFO:tasks.workunit.client.1.smithi196.stdout:0/830: chown d8/da/d25/d26/d35/d3b/d5f/f7a 27549 1 2024-03-13T14:49:57.572 INFO:tasks.workunit.client.1.smithi196.stdout:2/743: mknod d0/d7/d9/d42/d98/cf3 0 2024-03-13T14:49:57.575 INFO:tasks.workunit.client.0.smithi176.stdout:5/671: dread d5/d31/f61 [0,4194304] 0 2024-03-13T14:49:57.579 INFO:tasks.workunit.client.1.smithi196.stdout:8/764: dwrite d0/d16/d3d/d8b/f4b [0,4194304] 0 2024-03-13T14:49:57.590 INFO:tasks.workunit.client.1.smithi196.stdout:3/809: stat db/lb8 0 2024-03-13T14:49:57.591 INFO:tasks.workunit.client.1.smithi196.stdout:5/847: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/f10c x:0 0 0 2024-03-13T14:49:57.596 INFO:tasks.workunit.client.1.smithi196.stdout:0/831: symlink d8/da/d25/d26/d35/d3b/d59/d6a/l111 0 2024-03-13T14:49:57.598 INFO:tasks.workunit.client.0.smithi176.stdout:6/709: dwrite d0/f8 [0,4194304] 0 2024-03-13T14:49:57.602 INFO:tasks.workunit.client.1.smithi196.stdout:6/980: write d6/d2a/de2/feb [531846,85134] 0 2024-03-13T14:49:57.609 INFO:tasks.workunit.client.0.smithi176.stdout:9/827: dread d0/db/fae [0,4194304] 0 2024-03-13T14:49:57.609 INFO:tasks.workunit.client.1.smithi196.stdout:7/909: write d0/d4d/dd1/dd6/fde [516313,34020] 0 2024-03-13T14:49:57.620 INFO:tasks.workunit.client.1.smithi196.stdout:9/755: dwrite d7/de/d74/d94/dfb/f98 [0,4194304] 0 2024-03-13T14:49:57.624 INFO:tasks.workunit.client.1.smithi196.stdout:6/981: rename d6/d9/d17/d19/d4e/cef to d6/d2a/d23/c142 0 2024-03-13T14:49:57.630 INFO:tasks.workunit.client.1.smithi196.stdout:2/744: creat d0/d7/d9/d9a/ff4 x:0 0 0 2024-03-13T14:49:57.635 INFO:tasks.workunit.client.1.smithi196.stdout:3/810: dwrite db/d19/d50/db7/fbe [0,4194304] 0 2024-03-13T14:49:57.644 INFO:tasks.workunit.client.1.smithi196.stdout:2/745: creat d0/d7/d9/d61/d83/ff5 x:0 0 0 2024-03-13T14:49:57.655 INFO:tasks.workunit.client.1.smithi196.stdout:7/910: dwrite d0/d6/d33/f40 [0,4194304] 0 2024-03-13T14:49:57.658 INFO:tasks.workunit.client.0.smithi176.stdout:8/643: chown d0/d24/d29/f67 354028688 1 2024-03-13T14:49:57.660 INFO:tasks.workunit.client.1.smithi196.stdout:5/848: getdents d4/d1b/dc8/d14/d37 0 2024-03-13T14:49:57.662 INFO:tasks.workunit.client.1.smithi196.stdout:2/746: symlink d0/d6/d4b/d5f/lf6 0 2024-03-13T14:49:57.664 INFO:tasks.workunit.client.1.smithi196.stdout:2/747: chown d0/f77 3826 1 2024-03-13T14:49:57.666 INFO:tasks.workunit.client.1.smithi196.stdout:0/832: getdents d8/da/d25/d26/d35/d3b/d5f/d33 0 2024-03-13T14:49:57.669 INFO:tasks.workunit.client.1.smithi196.stdout:3/811: mknod db/d19/c105 0 2024-03-13T14:49:57.673 INFO:tasks.workunit.client.1.smithi196.stdout:3/812: read db/d71/fa5 [2014072,46403] 0 2024-03-13T14:49:57.673 INFO:tasks.workunit.client.1.smithi196.stdout:7/911: creat d0/d6/d33/d57/dae/de8/f139 x:0 0 0 2024-03-13T14:49:57.675 INFO:tasks.workunit.client.1.smithi196.stdout:5/849: rename d4/f11 to d4/d1b/dc8/d14/d19/da7/f10d 0 2024-03-13T14:49:57.676 INFO:tasks.workunit.client.1.smithi196.stdout:9/756: getdents d7/de/d16 0 2024-03-13T14:49:57.677 INFO:tasks.workunit.client.0.smithi176.stdout:5/672: symlink d5/d2c/d8a/le8 0 2024-03-13T14:49:57.679 INFO:tasks.workunit.client.1.smithi196.stdout:6/982: getdents d6/d2a/d23/d28/d10a/d10d 0 2024-03-13T14:49:57.682 INFO:tasks.workunit.client.1.smithi196.stdout:0/833: creat d8/da/d25/d26/d35/d3b/d50/d5d/da5/f112 x:0 0 0 2024-03-13T14:49:57.683 INFO:tasks.workunit.client.1.smithi196.stdout:3/813: mknod db/d19/d50/db7/c106 0 2024-03-13T14:49:57.684 INFO:tasks.workunit.client.0.smithi176.stdout:7/671: mknod d1/d28/ccf 0 2024-03-13T14:49:57.684 INFO:tasks.workunit.client.1.smithi196.stdout:0/834: readlink d8/da/d25/d26/d35/d3b/d59/l77 0 2024-03-13T14:49:57.688 INFO:tasks.workunit.client.0.smithi176.stdout:3/758: mkdir d4/d13/d2d/d105 0 2024-03-13T14:49:57.689 INFO:tasks.workunit.client.1.smithi196.stdout:9/757: creat d7/de/d4b/d1d/d22/f108 x:0 0 0 2024-03-13T14:49:57.692 INFO:tasks.workunit.client.0.smithi176.stdout:0/620: link d2/d5/d1e/l24 d2/d74/d9f/lc1 0 2024-03-13T14:49:57.693 INFO:tasks.workunit.client.1.smithi196.stdout:6/983: creat d6/d9/d17/d19/d4e/d66/d102/d12a/f143 x:0 0 0 2024-03-13T14:49:57.699 INFO:tasks.workunit.client.1.smithi196.stdout:3/814: write db/d4c/feb [2428401,98209] 0 2024-03-13T14:49:57.700 INFO:tasks.workunit.client.0.smithi176.stdout:5/673: mkdir d5/d2c/d40/dc7/d53/de9 0 2024-03-13T14:49:57.700 INFO:tasks.workunit.client.0.smithi176.stdout:9/828: dwrite d0/d40/d53/fb5 [0,4194304] 0 2024-03-13T14:49:57.701 INFO:tasks.workunit.client.1.smithi196.stdout:5/850: mknod d4/d1b/dc8/d14/d19/da7/d105/c10e 0 2024-03-13T14:49:57.704 INFO:tasks.workunit.client.1.smithi196.stdout:0/835: fdatasync d8/da/d25/d26/d35/d3b/d50/d55/f64 0 2024-03-13T14:49:57.707 INFO:tasks.workunit.client.0.smithi176.stdout:6/710: creat d0/d10/dca/dd2/fe3 x:0 0 0 2024-03-13T14:49:57.707 INFO:tasks.workunit.client.0.smithi176.stdout:8/644: write d0/d24/f7d [737171,14629] 0 2024-03-13T14:49:57.709 INFO:tasks.workunit.client.1.smithi196.stdout:7/912: write d0/d9/d31/d77/d93/fc6 [378442,96846] 0 2024-03-13T14:49:57.710 INFO:tasks.workunit.client.1.smithi196.stdout:6/984: creat d6/d7/df/d27/d39/d44/dc8/d130/d91/f144 x:0 0 0 2024-03-13T14:49:57.711 INFO:tasks.workunit.client.0.smithi176.stdout:3/759: mknod d4/d13/d16/db6/c106 0 2024-03-13T14:49:57.713 INFO:tasks.workunit.client.0.smithi176.stdout:6/711: chown d0/d10/d19/d27/d2b/d2e/d3e/db4/fdd 28826 1 2024-03-13T14:49:57.714 INFO:tasks.workunit.client.0.smithi176.stdout:6/712: stat d0/de 0 2024-03-13T14:49:57.714 INFO:tasks.workunit.client.0.smithi176.stdout:4/729: write d7/d8/d15/d23/f7d [397584,111944] 0 2024-03-13T14:49:57.715 INFO:tasks.workunit.client.0.smithi176.stdout:1/680: write d2/de/d3f/d50/fa3 [1499611,91096] 0 2024-03-13T14:49:57.719 INFO:tasks.workunit.client.0.smithi176.stdout:5/674: unlink d5/d31/d47/d80/c7a 0 2024-03-13T14:49:57.721 INFO:tasks.workunit.client.0.smithi176.stdout:5/675: chown d5/d2e/d7e/d9f/f5a 0 1 2024-03-13T14:49:57.721 INFO:tasks.workunit.client.1.smithi196.stdout:0/836: mkdir d8/da/d25/d26/d54/d88/d113 0 2024-03-13T14:49:57.723 INFO:tasks.workunit.client.1.smithi196.stdout:9/758: write d7/d9/d17/f9c [175127,100273] 0 2024-03-13T14:49:57.724 INFO:tasks.workunit.client.0.smithi176.stdout:9/829: rename d0/db/d89/d32/d58/d5f/l8c to d0/d1e/df7/l115 0 2024-03-13T14:49:57.724 INFO:tasks.workunit.client.0.smithi176.stdout:0/621: dread d2/f6 [4194304,4194304] 0 2024-03-13T14:49:57.724 INFO:tasks.workunit.client.0.smithi176.stdout:9/830: chown d0/d40/d53 9441 1 2024-03-13T14:49:57.726 INFO:tasks.workunit.client.0.smithi176.stdout:7/672: fdatasync d1/d3f/fab 0 2024-03-13T14:49:57.727 INFO:tasks.workunit.client.1.smithi196.stdout:6/985: dread - d6/d7/df/fdd zero size 2024-03-13T14:49:57.728 INFO:tasks.workunit.client.0.smithi176.stdout:7/673: fdatasync d1/d1c/f5e 0 2024-03-13T14:49:57.729 INFO:tasks.workunit.client.1.smithi196.stdout:7/913: creat d0/d6/d33/d57/dae/de8/f13a x:0 0 0 2024-03-13T14:49:57.729 INFO:tasks.workunit.client.0.smithi176.stdout:3/760: mkdir d4/d13/d81/d107 0 2024-03-13T14:49:57.729 INFO:tasks.workunit.client.0.smithi176.stdout:2/734: link d1/d2a/l72 d1/lf5 0 2024-03-13T14:49:57.731 INFO:tasks.workunit.client.0.smithi176.stdout:2/735: stat d1/dc/d2d/d90 0 2024-03-13T14:49:57.732 INFO:tasks.workunit.client.0.smithi176.stdout:9/831: truncate d0/d40/f78 4893387 0 2024-03-13T14:49:57.733 INFO:tasks.workunit.client.0.smithi176.stdout:6/713: mkdir d0/d10/d19/d27/d2b/d2e/db7/de4 0 2024-03-13T14:49:57.733 INFO:tasks.workunit.client.0.smithi176.stdout:5/676: rmdir d5/d2c/d40 39 2024-03-13T14:49:57.734 INFO:tasks.workunit.client.0.smithi176.stdout:8/645: symlink d0/d5/d11/lc9 0 2024-03-13T14:49:57.737 INFO:tasks.workunit.client.1.smithi196.stdout:2/748: dwrite d0/d7/d40/f9c [0,4194304] 0 2024-03-13T14:49:57.739 INFO:tasks.workunit.client.1.smithi196.stdout:9/759: truncate d7/de/d4b/d1d/d93/da7/fae 2953562 0 2024-03-13T14:49:57.748 INFO:tasks.workunit.client.1.smithi196.stdout:7/914: creat d0/d9/d31/d77/d93/f13b x:0 0 0 2024-03-13T14:49:57.749 INFO:tasks.workunit.client.0.smithi176.stdout:4/730: write d7/d10/d2c/f32 [4323067,16755] 0 2024-03-13T14:49:57.750 INFO:tasks.workunit.client.0.smithi176.stdout:1/681: write d2/de/d10/d16/d40/f77 [325225,23884] 0 2024-03-13T14:49:57.750 INFO:tasks.workunit.client.1.smithi196.stdout:9/760: symlink d7/de/d4b/d1d/d22/l109 0 2024-03-13T14:49:57.752 INFO:tasks.workunit.client.1.smithi196.stdout:0/837: dwrite d8/da/d25/d26/d48/dba/ff6 [0,4194304] 0 2024-03-13T14:49:57.752 INFO:tasks.workunit.client.1.smithi196.stdout:7/915: write d0/f76 [1567773,75067] 0 2024-03-13T14:49:57.755 INFO:tasks.workunit.client.1.smithi196.stdout:9/761: stat d7/de/d74/d94/l105 0 2024-03-13T14:49:57.757 INFO:tasks.workunit.client.0.smithi176.stdout:8/646: mkdir d0/d24/d29/d70/dca 0 2024-03-13T14:49:57.758 INFO:tasks.workunit.client.0.smithi176.stdout:4/731: readlink d7/d8/d45/lde 0 2024-03-13T14:49:57.758 INFO:tasks.workunit.client.0.smithi176.stdout:0/622: dwrite d2/d74/f91 [0,4194304] 0 2024-03-13T14:49:57.758 INFO:tasks.workunit.client.1.smithi196.stdout:3/815: dread db/d19/d22/d3a/d53/f5c [0,4194304] 0 2024-03-13T14:49:57.761 INFO:tasks.workunit.client.0.smithi176.stdout:4/732: chown d7/d8/d45/f5c 3519 1 2024-03-13T14:49:57.761 INFO:tasks.workunit.client.0.smithi176.stdout:6/714: dwrite d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/fc2 [0,4194304] 0 2024-03-13T14:49:57.763 INFO:tasks.workunit.client.1.smithi196.stdout:0/838: stat d8/da/f10 0 2024-03-13T14:49:57.763 INFO:tasks.workunit.client.1.smithi196.stdout:5/851: dread d4/d1b/dc8/d4a/fa3 [0,4194304] 0 2024-03-13T14:49:57.764 INFO:tasks.workunit.client.0.smithi176.stdout:1/682: chown d2/de/d10/d16/d4a/d5f/cb9 0 1 2024-03-13T14:49:57.768 INFO:tasks.workunit.client.0.smithi176.stdout:7/674: write d1/d28/f2a [2834625,57039] 0 2024-03-13T14:49:57.769 INFO:tasks.workunit.client.1.smithi196.stdout:2/749: dread d0/d7/f21 [0,4194304] 0 2024-03-13T14:49:57.771 INFO:tasks.workunit.client.1.smithi196.stdout:9/762: rmdir d7/de/d74/d94 39 2024-03-13T14:49:57.774 INFO:tasks.workunit.client.1.smithi196.stdout:6/986: link d6/d7/df/d27/d39/c103 d6/d9/d17/da1/d50/c145 0 2024-03-13T14:49:57.774 INFO:tasks.workunit.client.1.smithi196.stdout:7/916: creat d0/d9/d1e/d58/d61/f13c x:0 0 0 2024-03-13T14:49:57.775 INFO:tasks.workunit.client.0.smithi176.stdout:9/832: unlink d0/db/d89/d32/d58/l102 0 2024-03-13T14:49:57.775 INFO:tasks.workunit.client.0.smithi176.stdout:2/736: truncate d1/dc/d2d/d4b/d69/f94 2608768 0 2024-03-13T14:49:57.776 INFO:tasks.workunit.client.1.smithi196.stdout:3/816: unlink db/d19/d4a/dc5/fd7 0 2024-03-13T14:49:57.784 INFO:tasks.workunit.client.0.smithi176.stdout:5/677: dwrite d5/d2c/d40/d76/f95 [0,4194304] 0 2024-03-13T14:49:57.784 INFO:tasks.workunit.client.0.smithi176.stdout:7/675: readlink d1/d28/d3a/d1b/d8f/dc1/dbb/l9b 0 2024-03-13T14:49:57.785 INFO:tasks.workunit.client.0.smithi176.stdout:8/647: mknod d0/db/d39/d56/d93/da7/ccb 0 2024-03-13T14:49:57.786 INFO:tasks.workunit.client.0.smithi176.stdout:3/761: dwrite d4/d13/f57 [0,4194304] 0 2024-03-13T14:49:57.787 INFO:tasks.workunit.client.1.smithi196.stdout:5/852: symlink d4/d1b/dc8/d3c/dd0/l10f 0 2024-03-13T14:49:57.794 INFO:tasks.workunit.client.0.smithi176.stdout:9/833: truncate d0/d40/d53/d6d/de1/f104 419200 0 2024-03-13T14:49:57.797 INFO:tasks.workunit.client.1.smithi196.stdout:7/917: mkdir d0/d9/d1e/d58/d72/da9/d47/d13d 0 2024-03-13T14:49:57.804 INFO:tasks.workunit.client.0.smithi176.stdout:7/676: creat d1/d28/d3a/d1b/d8f/dc1/db7/fd0 x:0 0 0 2024-03-13T14:49:57.809 INFO:tasks.workunit.client.0.smithi176.stdout:6/715: mkdir d0/de/d38/d6b/da3/de5 0 2024-03-13T14:49:57.810 INFO:tasks.workunit.client.0.smithi176.stdout:8/648: creat d0/db/d39/d56/d91/fcc x:0 0 0 2024-03-13T14:49:57.815 INFO:tasks.workunit.client.0.smithi176.stdout:8/649: chown d0/db/d39/d56/lc8 800 1 2024-03-13T14:49:57.818 INFO:tasks.workunit.client.0.smithi176.stdout:5/678: dread d5/d2c/d40/d76/f95 [0,4194304] 0 2024-03-13T14:49:57.819 INFO:tasks.workunit.client.0.smithi176.stdout:1/683: dread d2/de/d3f/d3e/f5b [0,4194304] 0 2024-03-13T14:49:57.820 INFO:tasks.workunit.client.0.smithi176.stdout:3/762: rename d4/d13/d5e/lc1 to d4/db8/dcc/l108 0 2024-03-13T14:49:57.820 INFO:tasks.workunit.client.0.smithi176.stdout:7/677: truncate d1/d1c/d4a/f8d 4750756 0 2024-03-13T14:49:57.820 INFO:tasks.workunit.client.1.smithi196.stdout:7/918: dwrite d0/d6/d33/d57/d8a/f131 [0,4194304] 0 2024-03-13T14:49:57.823 INFO:tasks.workunit.client.0.smithi176.stdout:8/650: creat d0/d24/d98/dc6/d40/fcd x:0 0 0 2024-03-13T14:49:57.823 INFO:tasks.workunit.client.0.smithi176.stdout:1/684: fsync d2/de/dbf/fa5 0 2024-03-13T14:49:57.825 INFO:tasks.workunit.client.0.smithi176.stdout:9/834: link d0/db/d89/d7a/ff3 d0/db/def/db8/f116 0 2024-03-13T14:49:57.826 INFO:tasks.workunit.client.0.smithi176.stdout:0/623: dwrite d2/d5/d1e/f92 [0,4194304] 0 2024-03-13T14:49:57.827 INFO:tasks.workunit.client.0.smithi176.stdout:3/763: readlink d4/d28/d31/d52/de9/leb 0 2024-03-13T14:49:57.832 INFO:tasks.workunit.client.0.smithi176.stdout:8/651: mkdir d0/d24/d29/dce 0 2024-03-13T14:49:57.833 INFO:tasks.workunit.client.0.smithi176.stdout:0/624: write d2/d5/dc/f49 [3547960,13144] 0 2024-03-13T14:49:57.836 INFO:tasks.workunit.client.0.smithi176.stdout:0/625: chown d2/d5/l8a 59 1 2024-03-13T14:49:57.838 INFO:tasks.workunit.client.0.smithi176.stdout:3/764: rmdir d4/d13/d16/ddd 39 2024-03-13T14:49:57.839 INFO:tasks.workunit.client.0.smithi176.stdout:0/626: chown d2/d5/dc/l67 128469 1 2024-03-13T14:49:57.843 INFO:tasks.workunit.client.1.smithi196.stdout:6/987: write d6/d2a/dc/da3/fc3 [1022306,15169] 0 2024-03-13T14:49:57.846 INFO:tasks.workunit.client.0.smithi176.stdout:4/733: write d7/d10/d46/d4a/d5d/f69 [1190424,67506] 0 2024-03-13T14:49:57.853 INFO:tasks.workunit.client.0.smithi176.stdout:3/765: symlink d4/d13/d16/dad/l109 0 2024-03-13T14:49:57.859 INFO:tasks.workunit.client.0.smithi176.stdout:4/734: fsync d7/d8/d45/ff3 0 2024-03-13T14:49:57.859 INFO:tasks.workunit.client.0.smithi176.stdout:2/737: write d1/dc/d61/d9e/fcd [3178682,41535] 0 2024-03-13T14:49:57.863 INFO:tasks.workunit.client.0.smithi176.stdout:5/679: dread d5/f63 [0,4194304] 0 2024-03-13T14:49:57.864 INFO:tasks.workunit.client.0.smithi176.stdout:3/766: write d4/d13/d16/ddf/f104 [275148,31860] 0 2024-03-13T14:49:57.866 INFO:tasks.workunit.client.0.smithi176.stdout:0/627: dwrite d2/f6 [0,4194304] 0 2024-03-13T14:49:57.869 INFO:tasks.workunit.client.0.smithi176.stdout:0/628: readlink d2/db/d31/l88 0 2024-03-13T14:49:57.874 INFO:tasks.workunit.client.1.smithi196.stdout:9/763: mkdir d7/de/d74/d94/dfb/d28/de4/d10a 0 2024-03-13T14:49:57.874 INFO:tasks.workunit.client.0.smithi176.stdout:2/738: symlink d1/dc/d2d/d59/lf6 0 2024-03-13T14:49:57.875 INFO:tasks.workunit.client.0.smithi176.stdout:6/716: write d0/de/d8d/fbd [4377612,2510] 0 2024-03-13T14:49:57.880 INFO:tasks.workunit.client.1.smithi196.stdout:7/919: mkdir d0/d6/d33/d57/dae/de8/d13e 0 2024-03-13T14:49:57.881 INFO:tasks.workunit.client.0.smithi176.stdout:2/739: stat d1/dc/d2d/d38/fe8 0 2024-03-13T14:49:57.882 INFO:tasks.workunit.client.0.smithi176.stdout:0/629: creat d2/d5/d28/dac/fc2 x:0 0 0 2024-03-13T14:49:57.884 INFO:tasks.workunit.client.1.smithi196.stdout:6/988: rename d6/d7/dd/faf to d6/d2a/d23/d28/f146 0 2024-03-13T14:49:57.886 INFO:tasks.workunit.client.0.smithi176.stdout:6/717: write d0/de/d38/d70/f9e [3348,102231] 0 2024-03-13T14:49:57.889 INFO:tasks.workunit.client.0.smithi176.stdout:9/835: dread d0/f59 [0,4194304] 0 2024-03-13T14:49:57.890 INFO:tasks.workunit.client.1.smithi196.stdout:9/764: symlink d7/de/d4b/d1d/d22/l10b 0 2024-03-13T14:49:57.891 INFO:tasks.workunit.client.0.smithi176.stdout:6/718: fdatasync d0/d10/fc0 0 2024-03-13T14:49:57.894 INFO:tasks.workunit.client.1.smithi196.stdout:7/920: mkdir d0/d6/db4/d13f 0 2024-03-13T14:49:57.896 INFO:tasks.workunit.client.1.smithi196.stdout:5/853: getdents d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0 0 2024-03-13T14:49:57.898 INFO:tasks.workunit.client.0.smithi176.stdout:0/630: symlink d2/db/d35/daa/daf/lc3 0 2024-03-13T14:49:57.900 INFO:tasks.workunit.client.1.smithi196.stdout:9/765: symlink d7/de/d16/l10c 0 2024-03-13T14:49:57.904 INFO:tasks.workunit.client.1.smithi196.stdout:7/921: getdents d0/d6/ddc/d126 0 2024-03-13T14:49:57.904 INFO:tasks.workunit.client.0.smithi176.stdout:7/678: dread d1/d1c/d26/d41/d2b/d40/d64/f60 [0,4194304] 0 2024-03-13T14:49:57.904 INFO:tasks.workunit.client.1.smithi196.stdout:6/989: dwrite d6/d2a/d23/d28/f119 [0,4194304] 0 2024-03-13T14:49:57.906 INFO:tasks.workunit.client.1.smithi196.stdout:7/922: chown d0/d9/d1e/d3f/d6c/fb0 1419302 1 2024-03-13T14:49:57.907 INFO:tasks.workunit.client.0.smithi176.stdout:6/719: chown d0/d10/d19/d27/c46 69 1 2024-03-13T14:49:57.911 INFO:tasks.workunit.client.1.smithi196.stdout:9/766: rename d7/l13 to d7/de/d74/d94/dfb/d28/l10d 0 2024-03-13T14:49:57.911 INFO:tasks.workunit.client.0.smithi176.stdout:4/735: dread d7/d8/d33/f7f [0,4194304] 0 2024-03-13T14:49:57.913 INFO:tasks.workunit.client.0.smithi176.stdout:4/736: stat d7/d10/d46/d4a/fe6 0 2024-03-13T14:49:57.913 INFO:tasks.workunit.client.0.smithi176.stdout:8/652: dwrite d0/d5/f4d [0,4194304] 0 2024-03-13T14:49:57.914 INFO:tasks.workunit.client.1.smithi196.stdout:6/990: mknod d6/d2a/dc/de3/c147 0 2024-03-13T14:49:57.916 INFO:tasks.workunit.client.0.smithi176.stdout:3/767: getdents d4/d13/d81/d89/d5b/dc5 0 2024-03-13T14:49:57.917 INFO:tasks.workunit.client.0.smithi176.stdout:0/631: mkdir d2/db/d31/dc4 0 2024-03-13T14:49:57.919 INFO:tasks.workunit.client.0.smithi176.stdout:3/768: write d4/d28/d4f/ffd [387303,68947] 0 2024-03-13T14:49:57.919 INFO:tasks.workunit.client.1.smithi196.stdout:9/767: creat d7/de/d4b/d1d/d93/df8/f10e x:0 0 0 2024-03-13T14:49:57.921 INFO:tasks.workunit.client.0.smithi176.stdout:7/679: mkdir d1/d28/d3a/d1b/d8f/dd1 0 2024-03-13T14:49:57.927 INFO:tasks.workunit.client.1.smithi196.stdout:7/923: unlink d0/d9/d1e/d58/d72/da9/da3/d115/f129 0 2024-03-13T14:49:57.927 INFO:tasks.workunit.client.0.smithi176.stdout:2/740: creat d1/dc/d2d/ff7 x:0 0 0 2024-03-13T14:49:57.929 INFO:tasks.workunit.client.0.smithi176.stdout:9/836: dread - d0/d1e/d8d/da7/f112 zero size 2024-03-13T14:49:57.930 INFO:tasks.workunit.client.1.smithi196.stdout:6/991: mknod d6/d9/d17/d19/d5f/c148 0 2024-03-13T14:49:57.933 INFO:tasks.workunit.client.0.smithi176.stdout:6/720: truncate d0/d10/d19/d27/f4d 1210889 0 2024-03-13T14:49:57.934 INFO:tasks.workunit.client.1.smithi196.stdout:9/768: mkdir d7/de/d74/d94/dfb/d10f 0 2024-03-13T14:49:57.936 INFO:tasks.workunit.client.0.smithi176.stdout:4/737: mknod d7/d10/d2c/d9d/cfc 0 2024-03-13T14:49:57.936 INFO:tasks.workunit.client.1.smithi196.stdout:9/769: dread - d7/de/d4b/d1d/d93/da7/fb6 zero size 2024-03-13T14:49:57.937 INFO:tasks.workunit.client.1.smithi196.stdout:7/924: truncate d0/d9/d1e/d58/d72/da9/da3/fb9 1002266 0 2024-03-13T14:49:57.937 INFO:tasks.workunit.client.1.smithi196.stdout:9/770: chown d7/d71/f9e 76474 1 2024-03-13T14:49:57.938 INFO:tasks.workunit.client.0.smithi176.stdout:8/653: read - d0/f6c zero size 2024-03-13T14:49:57.939 INFO:tasks.workunit.client.0.smithi176.stdout:8/654: read - d0/d5/fa5 zero size 2024-03-13T14:49:57.940 INFO:tasks.workunit.client.1.smithi196.stdout:6/992: creat d6/d7/df/d27/d39/f149 x:0 0 0 2024-03-13T14:49:57.944 INFO:tasks.workunit.client.0.smithi176.stdout:5/680: write d5/f9a [657654,68335] 0 2024-03-13T14:49:57.947 INFO:tasks.workunit.client.1.smithi196.stdout:6/993: creat d6/d9/d17/d19/d4e/d66/f14a x:0 0 0 2024-03-13T14:49:57.948 INFO:tasks.workunit.client.0.smithi176.stdout:2/741: unlink d1/dc/d2d/d4f/c50 0 2024-03-13T14:49:57.949 INFO:tasks.workunit.client.1.smithi196.stdout:9/771: getdents d7/de/d4b/d1d/d93/de0 0 2024-03-13T14:49:57.950 INFO:tasks.workunit.client.0.smithi176.stdout:2/742: write d1/dc/d61/d9e/fcd [2122642,83200] 0 2024-03-13T14:49:57.953 INFO:tasks.workunit.client.0.smithi176.stdout:6/721: mknod d0/d10/d19/d7d/ce6 0 2024-03-13T14:49:57.954 INFO:tasks.workunit.client.0.smithi176.stdout:6/722: fdatasync d0/d10/dca/dd2/fe3 0 2024-03-13T14:49:57.956 INFO:tasks.workunit.client.0.smithi176.stdout:4/738: read d7/d8/d45/f62 [535699,33316] 0 2024-03-13T14:49:57.958 INFO:tasks.workunit.client.0.smithi176.stdout:0/632: creat d2/d5/dc/d20/da8/fc5 x:0 0 0 2024-03-13T14:49:57.959 INFO:tasks.workunit.client.0.smithi176.stdout:7/680: unlink d1/d3f/d4b/d63/c9e 0 2024-03-13T14:49:57.962 INFO:tasks.workunit.client.1.smithi196.stdout:9/772: rmdir d7/de/d74/d94/dfb/d10f 0 2024-03-13T14:49:57.965 INFO:tasks.workunit.client.0.smithi176.stdout:1/685: sync 2024-03-13T14:49:57.966 INFO:tasks.workunit.client.0.smithi176.stdout:9/837: unlink d0/l2b 0 2024-03-13T14:49:57.967 INFO:tasks.workunit.client.1.smithi196.stdout:9/773: mknod d7/c110 0 2024-03-13T14:49:57.968 INFO:tasks.workunit.client.0.smithi176.stdout:1/686: truncate d2/de/dd1/fd4 958136 0 2024-03-13T14:49:57.969 INFO:tasks.workunit.client.0.smithi176.stdout:3/769: write d4/d13/d16/d3d/fb2 [989807,86059] 0 2024-03-13T14:49:57.970 INFO:tasks.workunit.client.0.smithi176.stdout:8/655: mknod d0/ccf 0 2024-03-13T14:49:57.972 INFO:tasks.workunit.client.1.smithi196.stdout:6/994: write d6/d2a/dc/d1c/da7/fd1 [646587,20683] 0 2024-03-13T14:49:57.973 INFO:tasks.workunit.client.1.smithi196.stdout:9/774: rename d7/de/d16/c63 to d7/de/d16/c111 0 2024-03-13T14:49:57.975 INFO:tasks.workunit.client.0.smithi176.stdout:6/723: creat d0/d10/d19/d27/d2b/d2e/dc4/dc5/fe7 x:0 0 0 2024-03-13T14:49:57.975 INFO:tasks.workunit.client.1.smithi196.stdout:9/775: chown d7/d9/d83/f9d 612824 1 2024-03-13T14:49:57.979 INFO:tasks.workunit.client.0.smithi176.stdout:7/681: dwrite d1/d3f/d4b/d63/fb4 [0,4194304] 0 2024-03-13T14:49:57.983 INFO:tasks.workunit.client.0.smithi176.stdout:6/724: chown d0/d10/d19/d27/d2b/de0/dd6/cda 59268120 1 2024-03-13T14:49:57.983 INFO:tasks.workunit.client.0.smithi176.stdout:5/681: write d5/d22/d15/f16 [1406045,68110] 0 2024-03-13T14:49:57.983 INFO:tasks.workunit.client.0.smithi176.stdout:9/838: rename d0/f54 to d0/dd8/f117 0 2024-03-13T14:49:57.991 INFO:tasks.workunit.client.0.smithi176.stdout:1/687: mkdir d2/d30/d63/dd5 0 2024-03-13T14:49:57.992 INFO:tasks.workunit.client.0.smithi176.stdout:4/739: write d7/d8/d15/d23/f5a [94220,27352] 0 2024-03-13T14:49:57.994 INFO:tasks.workunit.client.1.smithi196.stdout:6/995: rename d6/d9/d17/d19/fa8 to d6/f14b 0 2024-03-13T14:49:57.997 INFO:tasks.workunit.client.1.smithi196.stdout:9/776: dread d7/d9/d83/fcb [0,4194304] 0 2024-03-13T14:49:58.004 INFO:tasks.workunit.client.0.smithi176.stdout:7/682: rmdir d1/d1c/d26/d41/d2b/d40 39 2024-03-13T14:49:58.004 INFO:tasks.workunit.client.0.smithi176.stdout:8/656: dwrite d0/d5/d3b/f81 [0,4194304] 0 2024-03-13T14:49:58.008 INFO:tasks.workunit.client.0.smithi176.stdout:0/633: dwrite d2/d5/dc/f65 [0,4194304] 0 2024-03-13T14:49:58.010 INFO:tasks.workunit.client.1.smithi196.stdout:6/996: creat d6/d2a/dc/d1c/f14c x:0 0 0 2024-03-13T14:49:58.010 INFO:tasks.workunit.client.0.smithi176.stdout:9/839: mknod d0/d1e/d8d/d9f/c118 0 2024-03-13T14:49:58.011 INFO:tasks.workunit.client.1.smithi196.stdout:9/777: write d7/de/d4b/f59 [1304296,87137] 0 2024-03-13T14:49:58.013 INFO:tasks.workunit.client.0.smithi176.stdout:3/770: dread d4/d28/f5c [0,4194304] 0 2024-03-13T14:49:58.016 INFO:tasks.workunit.client.1.smithi196.stdout:6/997: fsync d6/d7/f95 0 2024-03-13T14:49:58.018 INFO:tasks.workunit.client.0.smithi176.stdout:0/634: truncate d2/d5/d1e/f92 5584183 0 2024-03-13T14:49:58.020 INFO:tasks.workunit.client.1.smithi196.stdout:9/778: readlink d7/de/d74/d94/dfb/d28/l10d 0 2024-03-13T14:49:58.020 INFO:tasks.workunit.client.0.smithi176.stdout:4/740: rename d7/d8/c97 to d7/dab/db5/cfd 0 2024-03-13T14:49:58.021 INFO:tasks.workunit.client.1.smithi196.stdout:6/998: write d6/d9/d17/dfc/f11b [440061,106701] 0 2024-03-13T14:49:58.021 INFO:tasks.workunit.client.0.smithi176.stdout:5/682: truncate d5/d2e/d5f/f7c 390675 0 2024-03-13T14:49:58.023 INFO:tasks.workunit.client.0.smithi176.stdout:6/725: fsync d0/d10/d19/d27/fac 0 2024-03-13T14:49:58.024 INFO:tasks.workunit.client.0.smithi176.stdout:8/657: fsync d0/db/f1a 0 2024-03-13T14:49:58.024 INFO:tasks.workunit.client.0.smithi176.stdout:4/741: chown d7/c54 21 1 2024-03-13T14:49:58.027 INFO:tasks.workunit.client.0.smithi176.stdout:7/683: read d1/d28/d3a/d1b/d8f/d91/fc0 [1374655,21142] 0 2024-03-13T14:49:58.028 INFO:tasks.workunit.client.0.smithi176.stdout:3/771: stat d4/d13/d81/d89/d5b/lf4 0 2024-03-13T14:49:58.032 INFO:tasks.workunit.client.0.smithi176.stdout:0/635: symlink d2/d5/d28/lc6 0 2024-03-13T14:49:58.032 INFO:tasks.workunit.client.0.smithi176.stdout:2/743: dwrite d1/dc/d2d/d59/d8c/f8f [0,4194304] 0 2024-03-13T14:49:58.035 INFO:tasks.workunit.client.0.smithi176.stdout:9/840: mkdir d0/db/d89/d32/d58/d67/d119 0 2024-03-13T14:49:58.035 INFO:tasks.workunit.client.0.smithi176.stdout:5/683: creat d5/d2e/d43/fea x:0 0 0 2024-03-13T14:49:58.037 INFO:tasks.workunit.client.1.smithi196.stdout:1/883: sync 2024-03-13T14:49:58.038 INFO:tasks.workunit.client.1.smithi196.stdout:4/775: sync 2024-03-13T14:49:58.038 INFO:tasks.workunit.client.1.smithi196.stdout:8/765: sync 2024-03-13T14:49:58.038 INFO:tasks.workunit.client.0.smithi176.stdout:0/636: read - d2/db/d39/d9c/f9e zero size 2024-03-13T14:49:58.038 INFO:tasks.workunit.client.0.smithi176.stdout:6/726: creat d0/fe8 x:0 0 0 2024-03-13T14:49:58.042 INFO:tasks.workunit.client.1.smithi196.stdout:9/779: write d7/de/d74/d94/dfb/d28/d6c/fc2 [696885,32493] 0 2024-03-13T14:49:58.044 INFO:tasks.workunit.client.1.smithi196.stdout:1/884: creat d1/d9/d33/dd2/f12d x:0 0 0 2024-03-13T14:49:58.044 INFO:tasks.workunit.client.0.smithi176.stdout:5/684: truncate d5/d2e/d43/fea 979279 0 2024-03-13T14:49:58.044 INFO:tasks.workunit.client.0.smithi176.stdout:7/684: mknod d1/d3f/d81/cd2 0 2024-03-13T14:49:58.045 INFO:tasks.workunit.client.1.smithi196.stdout:6/999: write d6/f14b [7819032,71196] 0 2024-03-13T14:49:58.046 INFO:tasks.workunit.client.0.smithi176.stdout:9/841: symlink d0/db/d89/d4d/d6e/l11a 0 2024-03-13T14:49:58.047 INFO:tasks.workunit.client.1.smithi196.stdout:4/776: creat d1/d3/d7/d35/d3e/d56/f112 x:0 0 0 2024-03-13T14:49:58.047 INFO:tasks.workunit.client.0.smithi176.stdout:5/685: dread - d5/d2e/d5f/fdc zero size 2024-03-13T14:49:58.047 INFO:tasks.workunit.client.0.smithi176.stdout:1/688: link d2/de/d3f/d19/cac d2/de/d10/d16/d4a/d5f/cd6 0 2024-03-13T14:49:58.050 INFO:tasks.workunit.client.0.smithi176.stdout:5/686: write d5/f24 [2276086,96528] 0 2024-03-13T14:49:58.051 INFO:tasks.workunit.client.0.smithi176.stdout:8/658: creat d0/d24/d29/d82/fd0 x:0 0 0 2024-03-13T14:49:58.053 INFO:tasks.workunit.client.1.smithi196.stdout:4/777: truncate d1/fd0 788746 0 2024-03-13T14:49:58.055 INFO:tasks.workunit.client.1.smithi196.stdout:4/778: chown d1/d3/fcf 18190475 1 2024-03-13T14:49:58.055 INFO:tasks.workunit.client.0.smithi176.stdout:4/742: unlink d7/d8/d33/cf8 0 2024-03-13T14:49:58.056 INFO:tasks.workunit.client.0.smithi176.stdout:3/772: mknod d4/d28/d31/d52/de9/c10a 0 2024-03-13T14:49:58.056 INFO:tasks.workunit.client.0.smithi176.stdout:5/687: write d5/d2e/d5f/fcb [610047,1553] 0 2024-03-13T14:49:58.056 INFO:tasks.workunit.client.0.smithi176.stdout:7/685: mknod d1/d3f/d4b/d63/d7a/cd3 0 2024-03-13T14:49:58.057 INFO:tasks.workunit.client.1.smithi196.stdout:1/885: creat d1/d28/d34/d100/f12e x:0 0 0 2024-03-13T14:49:58.058 INFO:tasks.workunit.client.0.smithi176.stdout:9/842: fsync d0/d1e/d8d/dbe/fde 0 2024-03-13T14:49:58.059 INFO:tasks.workunit.client.0.smithi176.stdout:1/689: truncate d2/de/d3f/d50/d83/dcb/fb7 5180819 0 2024-03-13T14:49:58.059 INFO:tasks.workunit.client.0.smithi176.stdout:8/659: read d0/d24/d29/fbd [146348,103161] 0 2024-03-13T14:49:58.061 INFO:tasks.workunit.client.1.smithi196.stdout:1/886: creat d1/d28/d34/d100/f12f x:0 0 0 2024-03-13T14:49:58.062 INFO:tasks.workunit.client.0.smithi176.stdout:0/637: creat d2/d5/d28/d8c/d9d/fc7 x:0 0 0 2024-03-13T14:49:58.064 INFO:tasks.workunit.client.0.smithi176.stdout:7/686: readlink d1/d28/d3a/d1b/d8f/dc1/dbb/l9b 0 2024-03-13T14:49:58.065 INFO:tasks.workunit.client.0.smithi176.stdout:2/744: dwrite d1/dc/d2d/d59/d80/d7e/fd5 [0,4194304] 0 2024-03-13T14:49:58.066 INFO:tasks.workunit.client.1.smithi196.stdout:4/779: creat d1/d1e/f113 x:0 0 0 2024-03-13T14:49:58.066 INFO:tasks.workunit.client.1.smithi196.stdout:9/780: write d7/de/d4b/f6a [1355525,127028] 0 2024-03-13T14:49:58.066 INFO:tasks.workunit.client.1.smithi196.stdout:1/887: dread - d1/d9/d4e/fc5 zero size 2024-03-13T14:49:58.068 INFO:tasks.workunit.client.0.smithi176.stdout:4/743: rmdir d7/dab/db5 39 2024-03-13T14:49:58.069 INFO:tasks.workunit.client.1.smithi196.stdout:1/888: chown d1/d3f/d5b/d74/dd4/fd7 321981163 1 2024-03-13T14:49:58.070 INFO:tasks.workunit.client.1.smithi196.stdout:8/766: truncate d0/d16/d3d/d7f/fb3 3006469 0 2024-03-13T14:49:58.071 INFO:tasks.workunit.client.1.smithi196.stdout:4/780: dread d1/f2e [0,4194304] 0 2024-03-13T14:49:58.071 INFO:tasks.workunit.client.0.smithi176.stdout:3/773: creat d4/d13/d81/f10b x:0 0 0 2024-03-13T14:49:58.074 INFO:tasks.workunit.client.1.smithi196.stdout:8/767: chown d0/d10/d12/df5/d47/f98 505732 1 2024-03-13T14:49:58.074 INFO:tasks.workunit.client.0.smithi176.stdout:5/688: symlink d5/d2c/d40/dc7/d89/leb 0 2024-03-13T14:49:58.075 INFO:tasks.workunit.client.1.smithi196.stdout:8/768: chown d0/cc1 52150030 1 2024-03-13T14:49:58.076 INFO:tasks.workunit.client.0.smithi176.stdout:8/660: symlink d0/d24/d98/ld1 0 2024-03-13T14:49:58.077 INFO:tasks.workunit.client.0.smithi176.stdout:8/661: chown d0/f6c 940 1 2024-03-13T14:49:58.078 INFO:tasks.workunit.client.0.smithi176.stdout:8/662: chown d0/db/d39/d56/f5e 0 1 2024-03-13T14:49:58.081 INFO:tasks.workunit.client.1.smithi196.stdout:1/889: symlink d1/d9/d33/d4b/d88/l130 0 2024-03-13T14:49:58.083 INFO:tasks.workunit.client.0.smithi176.stdout:0/638: creat d2/db/d39/fc8 x:0 0 0 2024-03-13T14:49:58.090 INFO:tasks.workunit.client.1.smithi196.stdout:8/769: rename d0/d10/d12/d1e/d2d/dd8 to d0/d10/d12/d17/d42/dbe/d106 0 2024-03-13T14:49:58.091 INFO:tasks.workunit.client.0.smithi176.stdout:6/727: truncate d0/d10/d19/d27/fa9 3563033 0 2024-03-13T14:49:58.093 INFO:tasks.workunit.client.1.smithi196.stdout:4/781: dwrite d1/d3/d7/d35/d76/d58/d69/de3/df4/fcc [0,4194304] 0 2024-03-13T14:49:58.094 INFO:tasks.workunit.client.1.smithi196.stdout:1/890: unlink d1/d38/fb2 0 2024-03-13T14:49:58.095 INFO:tasks.workunit.client.0.smithi176.stdout:4/744: read - d7/d8/d15/d23/f8b zero size 2024-03-13T14:49:58.098 INFO:tasks.workunit.client.0.smithi176.stdout:4/745: read d7/d10/d46/d4a/d5d/ff5 [126576,86172] 0 2024-03-13T14:49:58.102 INFO:tasks.workunit.client.0.smithi176.stdout:2/745: creat d1/dc/d2d/d4b/ddb/ff8 x:0 0 0 2024-03-13T14:49:58.102 INFO:tasks.workunit.client.0.smithi176.stdout:2/746: write d1/f96 [2373273,45310] 0 2024-03-13T14:49:58.104 INFO:tasks.workunit.client.0.smithi176.stdout:5/689: creat d5/d22/d15/fec x:0 0 0 2024-03-13T14:49:58.106 INFO:tasks.workunit.client.0.smithi176.stdout:5/690: write d5/d2e/d43/fea [13411,21161] 0 2024-03-13T14:49:58.112 INFO:tasks.workunit.client.0.smithi176.stdout:9/843: symlink d0/db/d89/d32/d58/d67/d119/l11b 0 2024-03-13T14:49:58.114 INFO:tasks.workunit.client.0.smithi176.stdout:7/687: mknod d1/d28/d3a/d1b/d8f/dd1/cd4 0 2024-03-13T14:49:58.115 INFO:tasks.workunit.client.0.smithi176.stdout:7/688: chown d1/d28 360047402 1 2024-03-13T14:49:58.117 INFO:tasks.workunit.client.1.smithi196.stdout:9/781: dwrite d7/d9/d17/dbc/fa9 [0,4194304] 0 2024-03-13T14:49:58.117 INFO:tasks.workunit.client.0.smithi176.stdout:7/689: chown d1/d3f/d4b/d63/fb3 112026528 1 2024-03-13T14:49:58.123 INFO:tasks.workunit.client.1.smithi196.stdout:4/782: link d1/d3/d7/c3d d1/d3/d7/d35/d76/d58/d69/d9f/dd5/c114 0 2024-03-13T14:49:58.123 INFO:tasks.workunit.client.0.smithi176.stdout:8/663: write d0/d24/d29/f67 [603641,92805] 0 2024-03-13T14:49:58.124 INFO:tasks.workunit.client.1.smithi196.stdout:4/783: chown d1/d1e/c8b 132749287 1 2024-03-13T14:49:58.126 INFO:tasks.workunit.client.0.smithi176.stdout:6/728: creat d0/de/d38/fe9 x:0 0 0 2024-03-13T14:49:58.127 INFO:tasks.workunit.client.1.smithi196.stdout:1/891: truncate d1/d9/d33/d4b/d66/fa8 8296344 0 2024-03-13T14:49:58.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:57 smithi176 ceph-mon[29064]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:49:58.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:57 smithi176 ceph-mon[29064]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:49:58.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:57 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:58.136 INFO:tasks.workunit.client.0.smithi176.stdout:1/690: truncate d2/de/d3f/d50/d83/f98 298806 0 2024-03-13T14:49:58.137 INFO:tasks.workunit.client.1.smithi196.stdout:4/784: symlink d1/d3/d7/d35/d76/d58/d69/de3/df4/l115 0 2024-03-13T14:49:58.138 INFO:tasks.workunit.client.0.smithi176.stdout:1/691: readlink d2/de/d3f/d3e/lca 0 2024-03-13T14:49:58.145 INFO:tasks.workunit.client.1.smithi196.stdout:9/782: write d7/de/d4b/f97 [2252052,105018] 0 2024-03-13T14:49:58.145 INFO:tasks.workunit.client.1.smithi196.stdout:4/785: chown d1/d3/d4c/fb8 59025 1 2024-03-13T14:49:58.145 INFO:tasks.workunit.client.0.smithi176.stdout:7/690: dwrite d1/d3f/d76/fb9 [0,4194304] 0 2024-03-13T14:49:58.149 INFO:tasks.workunit.client.0.smithi176.stdout:4/746: write d7/d10/d2c/fb2 [789824,71644] 0 2024-03-13T14:49:58.150 INFO:tasks.workunit.client.0.smithi176.stdout:4/747: chown d7/d10 891391 1 2024-03-13T14:49:58.151 INFO:tasks.workunit.client.1.smithi196.stdout:9/783: chown d7/de/d74/d94/dfb/d28/d6c/l8d 3 1 2024-03-13T14:49:58.162 INFO:tasks.workunit.client.0.smithi176.stdout:3/774: dwrite d4/d13/d16/db6/fc4 [0,4194304] 0 2024-03-13T14:49:58.163 INFO:tasks.workunit.client.0.smithi176.stdout:1/692: dread d2/d30/f65 [0,4194304] 0 2024-03-13T14:49:58.164 INFO:tasks.workunit.client.0.smithi176.stdout:8/664: symlink d0/d5/d11/ld2 0 2024-03-13T14:49:58.165 INFO:tasks.workunit.client.1.smithi196.stdout:1/892: dwrite d1/d9/d4e/d55/fc3 [0,4194304] 0 2024-03-13T14:49:58.166 INFO:tasks.workunit.client.0.smithi176.stdout:3/775: readlink d4/d13/d16/db6/ld4 0 2024-03-13T14:49:58.171 INFO:tasks.workunit.client.1.smithi196.stdout:1/893: creat d1/d3f/d5b/d74/ded/f131 x:0 0 0 2024-03-13T14:49:58.172 INFO:tasks.workunit.client.1.smithi196.stdout:9/784: getdents d7/de/d74/d94/dfb/d28/d6c/d7b 0 2024-03-13T14:49:58.172 INFO:tasks.workunit.client.0.smithi176.stdout:0/639: link d2/c78 d2/d5/dc/d59/cc9 0 2024-03-13T14:49:58.175 INFO:tasks.workunit.client.0.smithi176.stdout:9/844: mknod d0/d1e/d1f/c11c 0 2024-03-13T14:49:58.175 INFO:tasks.workunit.client.0.smithi176.stdout:2/747: dwrite d1/dc/d2d/d38/f3f [0,4194304] 0 2024-03-13T14:49:58.176 INFO:tasks.workunit.client.0.smithi176.stdout:2/748: chown d1/lc7 4033675 1 2024-03-13T14:49:58.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:57 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:49:58.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:57 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:49:58.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:57 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:49:58.180 INFO:tasks.workunit.client.0.smithi176.stdout:2/749: write d1/dc/d2d/d4b/ddb/ff8 [188442,73073] 0 2024-03-13T14:49:58.182 INFO:tasks.workunit.client.0.smithi176.stdout:5/691: truncate d5/d2b/f70 73664 0 2024-03-13T14:49:58.185 INFO:tasks.workunit.client.1.smithi196.stdout:1/894: creat d1/d9/d33/dd2/dde/f132 x:0 0 0 2024-03-13T14:49:58.187 INFO:tasks.workunit.client.0.smithi176.stdout:8/665: creat d0/d24/d98/dc6/d40/fd3 x:0 0 0 2024-03-13T14:49:58.188 INFO:tasks.workunit.client.1.smithi196.stdout:4/786: dread d1/f29 [0,4194304] 0 2024-03-13T14:49:58.190 INFO:tasks.workunit.client.1.smithi196.stdout:4/787: readlink d1/d1e/d43/l9b 0 2024-03-13T14:49:58.196 INFO:tasks.workunit.client.0.smithi176.stdout:5/692: creat d5/d31/d47/fed x:0 0 0 2024-03-13T14:49:58.196 INFO:tasks.workunit.client.0.smithi176.stdout:1/693: getdents d2/d29/d73 0 2024-03-13T14:49:58.199 INFO:tasks.workunit.client.0.smithi176.stdout:9/845: creat d0/f11d x:0 0 0 2024-03-13T14:49:58.201 INFO:tasks.workunit.client.0.smithi176.stdout:0/640: creat d2/d5/dc/fca x:0 0 0 2024-03-13T14:49:58.201 INFO:tasks.workunit.client.1.smithi196.stdout:4/788: getdents d1/d1e/d5c/d105 0 2024-03-13T14:49:58.201 INFO:tasks.workunit.client.0.smithi176.stdout:4/748: truncate d7/d10/d46/d4a/d5d/f69 3742514 0 2024-03-13T14:49:58.202 INFO:tasks.workunit.client.0.smithi176.stdout:5/693: chown d5/d2e/d7e/dad/d7b/l87 1748678955 1 2024-03-13T14:49:58.206 INFO:tasks.workunit.client.0.smithi176.stdout:5/694: chown d5/d31/d47/f85 40896 1 2024-03-13T14:49:58.207 INFO:tasks.workunit.client.0.smithi176.stdout:6/729: dwrite d0/d10/d19/d27/f4d [0,4194304] 0 2024-03-13T14:49:58.207 INFO:tasks.workunit.client.1.smithi196.stdout:1/895: dwrite d1/f2a [0,4194304] 0 2024-03-13T14:49:58.211 INFO:tasks.workunit.client.1.smithi196.stdout:4/789: symlink d1/d3/d7/d35/d76/d58/dc9/l116 0 2024-03-13T14:49:58.212 INFO:tasks.workunit.client.1.smithi196.stdout:9/785: truncate d7/d71/f9e 761431 0 2024-03-13T14:49:58.213 INFO:tasks.workunit.client.0.smithi176.stdout:1/694: creat d2/de/d10/d16/d47/fd7 x:0 0 0 2024-03-13T14:49:58.213 INFO:tasks.workunit.client.0.smithi176.stdout:0/641: mknod d2/db/d31/ccb 0 2024-03-13T14:49:58.215 INFO:tasks.workunit.client.0.smithi176.stdout:5/695: write d5/d2e/d7e/dad/d7b/daa/fb8 [216867,60620] 0 2024-03-13T14:49:58.216 INFO:tasks.workunit.client.1.smithi196.stdout:9/786: dread d7/d71/faf [0,4194304] 0 2024-03-13T14:49:58.223 INFO:tasks.workunit.client.0.smithi176.stdout:1/695: write d2/de/dd1/fd4 [1190136,81040] 0 2024-03-13T14:49:58.224 INFO:tasks.workunit.client.1.smithi196.stdout:1/896: rename d1/d9/d4e/d55/d63/fea to d1/d9/d6c/f133 0 2024-03-13T14:49:58.226 INFO:tasks.workunit.client.0.smithi176.stdout:8/666: link d0/d5/cb5 d0/db/d39/d56/d91/cd4 0 2024-03-13T14:49:58.229 INFO:tasks.workunit.client.0.smithi176.stdout:7/691: dwrite d1/d1c/d4a/f62 [0,4194304] 0 2024-03-13T14:49:58.230 INFO:tasks.workunit.client.0.smithi176.stdout:6/730: mknod d0/d10/d19/d7d/cea 0 2024-03-13T14:49:58.231 INFO:tasks.workunit.client.0.smithi176.stdout:9/846: write d0/db/fd9 [346698,22198] 0 2024-03-13T14:49:58.232 INFO:tasks.workunit.client.0.smithi176.stdout:3/776: dwrite d4/d28/d4f/f53 [0,4194304] 0 2024-03-13T14:49:58.235 INFO:tasks.workunit.client.1.smithi196.stdout:4/790: creat d1/f117 x:0 0 0 2024-03-13T14:49:58.236 INFO:tasks.workunit.client.0.smithi176.stdout:7/692: chown d1/d1c/d4a 53079 1 2024-03-13T14:49:58.236 INFO:tasks.workunit.client.0.smithi176.stdout:1/696: chown d2/de/d10/d16/d4a/l75 108 1 2024-03-13T14:49:58.237 INFO:tasks.workunit.client.0.smithi176.stdout:4/749: fsync d7/d10/f2a 0 2024-03-13T14:49:58.238 INFO:tasks.workunit.client.1.smithi196.stdout:4/791: chown d1/d3/d7/d35/d3e/d56/fb6 94351 1 2024-03-13T14:49:58.239 INFO:tasks.workunit.client.1.smithi196.stdout:9/787: dwrite d7/de/d74/d94/dfb/d28/d6c/fc2 [0,4194304] 0 2024-03-13T14:49:58.240 INFO:tasks.workunit.client.0.smithi176.stdout:5/696: symlink d5/d2e/d7e/dad/d7b/daa/lee 0 2024-03-13T14:49:58.242 INFO:tasks.workunit.client.1.smithi196.stdout:9/788: dread - d7/de/d4b/d1d/d93/df8/f10e zero size 2024-03-13T14:49:58.246 INFO:tasks.workunit.client.0.smithi176.stdout:3/777: read - d4/d13/d81/d89/ff7 zero size 2024-03-13T14:49:58.247 INFO:tasks.workunit.client.0.smithi176.stdout:2/750: dwrite d1/dc/d1c/f37 [0,4194304] 0 2024-03-13T14:49:58.252 INFO:tasks.workunit.client.0.smithi176.stdout:8/667: mknod d0/db/d39/d56/cd5 0 2024-03-13T14:49:58.253 INFO:tasks.workunit.client.0.smithi176.stdout:4/750: fdatasync d7/db9/dd8/fdf 0 2024-03-13T14:49:58.258 INFO:tasks.workunit.client.0.smithi176.stdout:5/697: mkdir d5/d31/def 0 2024-03-13T14:49:58.260 INFO:tasks.workunit.client.0.smithi176.stdout:3/778: write d4/d28/d31/f103 [727736,117300] 0 2024-03-13T14:49:58.261 INFO:tasks.workunit.client.0.smithi176.stdout:8/668: mknod d0/db/cd6 0 2024-03-13T14:49:58.261 INFO:tasks.workunit.client.0.smithi176.stdout:1/697: write d2/de/d3f/d50/d83/dcb/fb7 [3144442,40567] 0 2024-03-13T14:49:58.263 INFO:tasks.workunit.client.0.smithi176.stdout:7/693: write d1/d28/d3a/d1b/d8f/d91/fc0 [1770219,69943] 0 2024-03-13T14:49:58.263 INFO:tasks.workunit.client.0.smithi176.stdout:9/847: read d0/d17/f3a [237367,63740] 0 2024-03-13T14:49:58.264 INFO:tasks.workunit.client.0.smithi176.stdout:0/642: getdents d2/d5 0 2024-03-13T14:49:58.265 INFO:tasks.workunit.client.1.smithi196.stdout:1/897: truncate d1/f2a 2278326 0 2024-03-13T14:49:58.266 INFO:tasks.workunit.client.0.smithi176.stdout:6/731: truncate d0/de/d54/f5e 5114942 0 2024-03-13T14:49:58.268 INFO:tasks.workunit.client.0.smithi176.stdout:2/751: rename d1/dc/d2d/d38/f3f to d1/d2a/d33/ff9 0 2024-03-13T14:49:58.270 INFO:tasks.workunit.client.0.smithi176.stdout:3/779: symlink d4/d28/d34/da4/l10c 0 2024-03-13T14:49:58.276 INFO:tasks.workunit.client.0.smithi176.stdout:5/698: creat d5/d2c/d40/dc7/ff0 x:0 0 0 2024-03-13T14:49:58.277 INFO:tasks.workunit.client.0.smithi176.stdout:0/643: creat d2/d5/d28/d8c/fcc x:0 0 0 2024-03-13T14:49:58.279 INFO:tasks.workunit.client.1.smithi196.stdout:9/789: write d7/de/d4b/d1d/d93/da7/fb6 [213948,85583] 0 2024-03-13T14:49:58.281 INFO:tasks.workunit.client.0.smithi176.stdout:9/848: truncate d0/d17/f3a 5817962 0 2024-03-13T14:49:58.282 INFO:tasks.workunit.client.0.smithi176.stdout:2/752: mknod d1/dc/d2d/d59/d80/d77/cfa 0 2024-03-13T14:49:58.284 INFO:tasks.workunit.client.1.smithi196.stdout:0/839: sync 2024-03-13T14:49:58.284 INFO:tasks.workunit.client.1.smithi196.stdout:2/750: sync 2024-03-13T14:49:58.286 INFO:tasks.workunit.client.0.smithi176.stdout:7/694: fdatasync d1/d28/d3a/f5c 0 2024-03-13T14:49:58.287 INFO:tasks.workunit.client.0.smithi176.stdout:7/695: chown d1/d3f/d81 125306 1 2024-03-13T14:49:58.288 INFO:tasks.workunit.client.0.smithi176.stdout:3/780: truncate d4/d13/d81/d89/f9c 3457127 0 2024-03-13T14:49:58.288 INFO:tasks.workunit.client.1.smithi196.stdout:0/840: chown d8/da/d25/d26/d35/d3b/d5f/d17/c2c 1607019 1 2024-03-13T14:49:58.289 INFO:tasks.workunit.client.1.smithi196.stdout:4/792: dwrite d1/d3/d4c/d72/d7d/d80/d55/fc1 [0,4194304] 0 2024-03-13T14:49:58.290 INFO:tasks.workunit.client.1.smithi196.stdout:9/790: truncate d7/de/d16/f49 897285 0 2024-03-13T14:49:58.291 INFO:tasks.workunit.client.0.smithi176.stdout:5/699: rmdir d5/d2e/d7e/d9f 39 2024-03-13T14:49:58.293 INFO:tasks.workunit.client.1.smithi196.stdout:4/793: chown d1/d3/d7/d35/d76/d58/dc6 264416 1 2024-03-13T14:49:58.294 INFO:tasks.workunit.client.0.smithi176.stdout:4/751: symlink d7/d10/d46/d4a/d5d/da1/de8/lfe 0 2024-03-13T14:49:58.294 INFO:tasks.workunit.client.0.smithi176.stdout:8/669: write d0/d5/f87 [155418,13143] 0 2024-03-13T14:49:58.295 INFO:tasks.workunit.client.1.smithi196.stdout:9/791: write d7/de/d4b/d1d/d93/da7/fb6 [232874,78777] 0 2024-03-13T14:49:58.296 INFO:tasks.workunit.client.0.smithi176.stdout:4/752: readlink d7/d10/d46/l5b 0 2024-03-13T14:49:58.296 INFO:tasks.workunit.client.1.smithi196.stdout:4/794: chown d1/d1e/f113 4 1 2024-03-13T14:49:58.297 INFO:tasks.workunit.client.1.smithi196.stdout:0/841: creat d8/f114 x:0 0 0 2024-03-13T14:49:58.299 INFO:tasks.workunit.client.0.smithi176.stdout:4/753: chown d7/d8/d45/f9f 198699844 1 2024-03-13T14:49:58.300 INFO:tasks.workunit.client.1.smithi196.stdout:1/898: dread d1/f2a [0,4194304] 0 2024-03-13T14:49:58.300 INFO:tasks.workunit.client.0.smithi176.stdout:0/644: rename d2/d5/l17 to d2/d5/dc/d59/lcd 0 2024-03-13T14:49:58.302 INFO:tasks.workunit.client.1.smithi196.stdout:1/899: read - d1/d9/d4e/fc5 zero size 2024-03-13T14:49:58.302 INFO:tasks.workunit.client.0.smithi176.stdout:6/732: write d0/de/f12 [5810280,47999] 0 2024-03-13T14:49:58.306 INFO:tasks.workunit.client.1.smithi196.stdout:2/751: symlink d0/d7/d9/d5a/dbc/deb/lf7 0 2024-03-13T14:49:58.307 INFO:tasks.workunit.client.0.smithi176.stdout:8/670: dread - d0/db/d39/fae zero size 2024-03-13T14:49:58.307 INFO:tasks.workunit.client.0.smithi176.stdout:9/849: symlink d0/db/def/l11e 0 2024-03-13T14:49:58.308 INFO:tasks.workunit.client.0.smithi176.stdout:9/850: chown d0/d1e/d8d/dbe/de8 166828 1 2024-03-13T14:49:58.309 INFO:tasks.workunit.client.0.smithi176.stdout:3/781: dwrite d4/d13/d16/ddf/f104 [0,4194304] 0 2024-03-13T14:49:58.315 INFO:tasks.workunit.client.0.smithi176.stdout:5/700: dwrite d5/d31/d47/fed [0,4194304] 0 2024-03-13T14:49:58.316 INFO:tasks.workunit.client.1.smithi196.stdout:9/792: rmdir d7 39 2024-03-13T14:49:58.322 INFO:tasks.workunit.client.1.smithi196.stdout:2/752: unlink d0/d6/d1d/l4e 0 2024-03-13T14:49:58.327 INFO:tasks.workunit.client.1.smithi196.stdout:4/795: fdatasync d1/d1e/f6a 0 2024-03-13T14:49:58.330 INFO:tasks.workunit.client.0.smithi176.stdout:7/696: mknod d1/d28/d3a/d1b/d8f/dc1/dbb/cd5 0 2024-03-13T14:49:58.338 INFO:tasks.workunit.client.0.smithi176.stdout:2/753: mknod d1/dc/d2d/d4b/d8d/cfb 0 2024-03-13T14:49:58.340 INFO:tasks.workunit.client.0.smithi176.stdout:5/701: rmdir d5/d2b 39 2024-03-13T14:49:58.342 INFO:tasks.workunit.client.0.smithi176.stdout:1/698: truncate d2/de/d10/d16/d4a/d5f/f81 608430 0 2024-03-13T14:49:58.344 INFO:tasks.workunit.client.0.smithi176.stdout:0/645: mknod d2/db/cce 0 2024-03-13T14:49:58.348 INFO:tasks.workunit.client.0.smithi176.stdout:8/671: symlink d0/db/d39/d56/ld7 0 2024-03-13T14:49:58.349 INFO:tasks.workunit.client.1.smithi196.stdout:4/796: dwrite d1/d1e/fb7 [0,4194304] 0 2024-03-13T14:49:58.349 INFO:tasks.workunit.client.0.smithi176.stdout:5/702: dread d5/d2c/d40/dc7/d89/fa1 [0,4194304] 0 2024-03-13T14:49:58.354 INFO:tasks.workunit.client.0.smithi176.stdout:4/754: write d7/d8/d15/dd6/fdc [668499,116635] 0 2024-03-13T14:49:58.354 INFO:tasks.workunit.client.1.smithi196.stdout:9/793: getdents d7/d71 0 2024-03-13T14:49:58.359 INFO:tasks.workunit.client.1.smithi196.stdout:4/797: write d1/d3/d7/d35/d76/d58/d69/d9f/fd8 [427920,78424] 0 2024-03-13T14:49:58.361 INFO:tasks.workunit.client.1.smithi196.stdout:1/900: dwrite d1/d28/d34/d67/f72 [0,4194304] 0 2024-03-13T14:49:58.362 INFO:tasks.workunit.client.0.smithi176.stdout:6/733: dwrite d0/de/d38/f72 [0,4194304] 0 2024-03-13T14:49:58.363 INFO:tasks.workunit.client.1.smithi196.stdout:0/842: dread d8/da/d25/fd1 [0,4194304] 0 2024-03-13T14:49:58.365 INFO:tasks.workunit.client.0.smithi176.stdout:1/699: write d2/de/d3f/f6f [451111,74869] 0 2024-03-13T14:49:58.373 INFO:tasks.workunit.client.0.smithi176.stdout:8/672: truncate d0/d5/d11/f68 2474731 0 2024-03-13T14:49:58.374 INFO:tasks.workunit.client.1.smithi196.stdout:2/753: dwrite d0/d6/d1d/f27 [0,4194304] 0 2024-03-13T14:49:58.377 INFO:tasks.workunit.client.0.smithi176.stdout:9/851: write d0/db/d89/d32/d58/d67/f106 [960977,35797] 0 2024-03-13T14:49:58.378 INFO:tasks.workunit.client.0.smithi176.stdout:3/782: link d4/d28/d34/d38/ca7 d4/d28/d31/d52/dff/c10d 0 2024-03-13T14:49:58.379 INFO:tasks.workunit.client.1.smithi196.stdout:4/798: read - d1/d3/d7/d35/d3e/d56/fb6 zero size 2024-03-13T14:49:58.379 INFO:tasks.workunit.client.1.smithi196.stdout:7/925: sync 2024-03-13T14:49:58.379 INFO:tasks.workunit.client.1.smithi196.stdout:8/770: sync 2024-03-13T14:49:58.379 INFO:tasks.workunit.client.0.smithi176.stdout:4/755: unlink d7/d10/d46/d4a/f79 0 2024-03-13T14:49:58.382 INFO:tasks.workunit.client.1.smithi196.stdout:5/854: sync 2024-03-13T14:49:58.382 INFO:tasks.workunit.client.1.smithi196.stdout:3/817: sync 2024-03-13T14:49:58.382 INFO:tasks.workunit.client.0.smithi176.stdout:9/852: dread - d0/db/d89/d4d/dd5/f105 zero size 2024-03-13T14:49:58.383 INFO:tasks.workunit.client.0.smithi176.stdout:5/703: read d5/d22/d15/f7d [1489553,83857] 0 2024-03-13T14:49:58.383 INFO:tasks.workunit.client.0.smithi176.stdout:9/853: readlink d0/db/d89/d7a/lc6 0 2024-03-13T14:49:58.386 INFO:tasks.workunit.client.1.smithi196.stdout:1/901: rename d1/d28/d34/d67/f72 to d1/d9/d4e/d55/d63/f134 0 2024-03-13T14:49:58.386 INFO:tasks.workunit.client.1.smithi196.stdout:3/818: read db/d15/d44/d79/ff6 [366974,13653] 0 2024-03-13T14:49:58.391 INFO:tasks.workunit.client.1.smithi196.stdout:9/794: creat d7/de/d74/f112 x:0 0 0 2024-03-13T14:49:58.393 INFO:tasks.workunit.client.1.smithi196.stdout:9/795: write d7/de/d16/f99 [1385306,44152] 0 2024-03-13T14:49:58.394 INFO:tasks.workunit.client.0.smithi176.stdout:5/704: dread d5/d2e/d7e/dad/d7b/daa/fb8 [0,4194304] 0 2024-03-13T14:49:58.394 INFO:tasks.workunit.client.0.smithi176.stdout:2/754: symlink d1/dc/d2d/d38/lfc 0 2024-03-13T14:49:58.395 INFO:tasks.workunit.client.1.smithi196.stdout:4/799: unlink d1/d3/lc2 0 2024-03-13T14:49:58.398 INFO:tasks.workunit.client.0.smithi176.stdout:1/700: truncate d2/de/d3f/d50/d83/dcb/f9d 28958 0 2024-03-13T14:49:58.399 INFO:tasks.workunit.client.0.smithi176.stdout:8/673: creat d0/db/d39/d56/d91/fd8 x:0 0 0 2024-03-13T14:49:58.402 INFO:tasks.workunit.client.0.smithi176.stdout:5/705: truncate d5/d2c/d40/dc7/fb9 997076 0 2024-03-13T14:49:58.403 INFO:tasks.workunit.client.1.smithi196.stdout:1/902: mknod d1/d9/d4e/d55/d63/da6/dc8/c135 0 2024-03-13T14:49:58.408 INFO:tasks.workunit.client.1.smithi196.stdout:9/796: write d7/de/f87 [5212909,50884] 0 2024-03-13T14:49:58.408 INFO:tasks.workunit.client.0.smithi176.stdout:4/756: getdents d7/d8/d15/dd6/dba 0 2024-03-13T14:49:58.410 INFO:tasks.workunit.client.1.smithi196.stdout:5/855: mkdir d4/d1b/dc8/d14/d19/d26/d33/d110 0 2024-03-13T14:49:58.410 INFO:tasks.workunit.client.0.smithi176.stdout:8/674: dread - d0/f5c zero size 2024-03-13T14:49:58.412 INFO:tasks.workunit.client.0.smithi176.stdout:5/706: creat d5/d2e/d5f/ff1 x:0 0 0 2024-03-13T14:49:58.416 INFO:tasks.workunit.client.0.smithi176.stdout:2/755: rename d1/dc/d2d/d59/d80/d7e/l8e to d1/d2a/lfd 0 2024-03-13T14:49:58.416 INFO:tasks.workunit.client.1.smithi196.stdout:2/754: creat d0/d7/d9/d42/ff8 x:0 0 0 2024-03-13T14:49:58.419 INFO:tasks.workunit.client.0.smithi176.stdout:1/701: mknod d2/de/d3f/d50/cd8 0 2024-03-13T14:49:58.419 INFO:tasks.workunit.client.0.smithi176.stdout:7/697: dwrite d1/d1c/d4a/f98 [0,4194304] 0 2024-03-13T14:49:58.421 INFO:tasks.workunit.client.1.smithi196.stdout:8/771: link d0/d10/d12/df5/d47/ca3 d0/d16/d3d/d7f/dda/c107 0 2024-03-13T14:49:58.421 INFO:tasks.workunit.client.0.smithi176.stdout:1/702: fsync d2/fb8 0 2024-03-13T14:49:58.425 INFO:tasks.workunit.client.0.smithi176.stdout:7/698: dread - d1/d3f/d76/da5/fb8 zero size 2024-03-13T14:49:58.426 INFO:tasks.workunit.client.1.smithi196.stdout:7/926: rename d0/d9/d31/d77 to d0/d6/d140 0 2024-03-13T14:49:58.428 INFO:tasks.workunit.client.1.smithi196.stdout:7/927: fsync d0/f11 0 2024-03-13T14:49:58.429 INFO:tasks.workunit.client.1.smithi196.stdout:5/856: unlink d4/d1b/dc8/d3c/d6a/l76 0 2024-03-13T14:49:58.430 INFO:tasks.workunit.client.1.smithi196.stdout:7/928: chown d0/d6/d140/dcf/de0/de7 169545408 1 2024-03-13T14:49:58.431 INFO:tasks.workunit.client.1.smithi196.stdout:9/797: truncate d7/de/d74/d94/fb8 3264355 0 2024-03-13T14:49:58.434 INFO:tasks.workunit.client.1.smithi196.stdout:8/772: dread d0/d1c/f7c [0,4194304] 0 2024-03-13T14:49:58.435 INFO:tasks.workunit.client.1.smithi196.stdout:3/819: link db/d19/d4a/fd1 db/d15/da1/dc8/f107 0 2024-03-13T14:49:58.436 INFO:tasks.workunit.client.1.smithi196.stdout:9/798: chown d7/de/d74/d94/dfb/d28/d6c/fee 16 1 2024-03-13T14:49:58.439 INFO:tasks.workunit.client.0.smithi176.stdout:0/646: rename d2/d5/dc/d59/d79 to d2/d5/d1e/d97/dcf 0 2024-03-13T14:49:58.441 INFO:tasks.workunit.client.0.smithi176.stdout:6/734: dwrite d0/d9/f6f [0,4194304] 0 2024-03-13T14:49:58.441 INFO:tasks.workunit.client.1.smithi196.stdout:1/903: creat d1/d9/d6c/d101/f136 x:0 0 0 2024-03-13T14:49:58.441 INFO:tasks.workunit.client.1.smithi196.stdout:8/773: read d0/d16/d3d/d8b/d3f/f76 [63470,3032] 0 2024-03-13T14:49:58.442 INFO:tasks.workunit.client.0.smithi176.stdout:3/783: getdents d4/d13/d81/d89/d5b 0 2024-03-13T14:49:58.444 INFO:tasks.workunit.client.1.smithi196.stdout:0/843: truncate d8/da/d25/d26/d35/d3b/d59/d6a/f8d 1388770 0 2024-03-13T14:49:58.448 INFO:tasks.workunit.client.0.smithi176.stdout:2/756: truncate d1/dc/d61/f6d 227768 0 2024-03-13T14:49:58.448 INFO:tasks.workunit.client.0.smithi176.stdout:5/707: dread d5/d2c/d40/dc7/fda [0,4194304] 0 2024-03-13T14:49:58.449 INFO:tasks.workunit.client.1.smithi196.stdout:2/755: creat d0/d6/d4b/d5f/dce/ff9 x:0 0 0 2024-03-13T14:49:58.451 INFO:tasks.workunit.client.0.smithi176.stdout:1/703: mknod d2/de/d10/d16/d47/db4/cd9 0 2024-03-13T14:49:58.453 INFO:tasks.workunit.client.0.smithi176.stdout:7/699: mknod d1/d28/d3a/d1b/d8f/dc1/db7/cd6 0 2024-03-13T14:49:58.455 INFO:tasks.workunit.client.0.smithi176.stdout:1/704: dread - d2/de/d3f/d50/f89 zero size 2024-03-13T14:49:58.456 INFO:tasks.workunit.client.1.smithi196.stdout:7/929: chown d0/d6/d33/f4c 5569 1 2024-03-13T14:49:58.458 INFO:tasks.workunit.client.1.smithi196.stdout:7/930: chown d0/d6/ddc/d126 3 1 2024-03-13T14:49:58.461 INFO:tasks.workunit.client.0.smithi176.stdout:6/735: dread d0/f8 [0,4194304] 0 2024-03-13T14:49:58.462 INFO:tasks.workunit.client.0.smithi176.stdout:9/854: dwrite d0/dd8/d82/f10f [0,4194304] 0 2024-03-13T14:49:58.463 INFO:tasks.workunit.client.1.smithi196.stdout:3/820: chown db/d19/d50/db7/l33 11317 1 2024-03-13T14:49:58.466 INFO:tasks.workunit.client.0.smithi176.stdout:4/757: rename d7/d10/d2c/d9d to d7/d10/d46/d4a/dcc/dff 0 2024-03-13T14:49:58.468 INFO:tasks.workunit.client.0.smithi176.stdout:4/758: readlink d7/d8/d15/l22 0 2024-03-13T14:49:58.478 INFO:tasks.workunit.client.1.smithi196.stdout:4/800: getdents d1 0 2024-03-13T14:49:58.480 INFO:tasks.workunit.client.0.smithi176.stdout:3/784: mknod d4/d13/d16/dad/c10e 0 2024-03-13T14:49:58.483 INFO:tasks.workunit.client.1.smithi196.stdout:3/821: dwrite db/d37/f76 [0,4194304] 0 2024-03-13T14:49:58.485 INFO:tasks.workunit.client.0.smithi176.stdout:2/757: mknod d1/dc/d2d/d59/d80/d7e/cfe 0 2024-03-13T14:49:58.486 INFO:tasks.workunit.client.1.smithi196.stdout:3/822: chown db/d19/d22/d3a/c91 59655 1 2024-03-13T14:49:58.486 INFO:tasks.workunit.client.0.smithi176.stdout:1/705: dwrite d2/de/f17 [0,4194304] 0 2024-03-13T14:49:58.489 INFO:tasks.workunit.client.1.smithi196.stdout:0/844: dread - d8/da/d25/d26/d35/d3b/d5f/d33/fe4 zero size 2024-03-13T14:49:58.490 INFO:tasks.workunit.client.0.smithi176.stdout:5/708: mkdir d5/d2c/d40/dc7/d89/df2 0 2024-03-13T14:49:58.495 INFO:tasks.workunit.client.1.smithi196.stdout:1/904: dread d1/d9/d33/d4b/d66/f95 [0,4194304] 0 2024-03-13T14:49:58.496 INFO:tasks.workunit.client.0.smithi176.stdout:7/700: rmdir d1/d28/d3a/d1b/d8f/d91 39 2024-03-13T14:49:58.500 INFO:tasks.workunit.client.1.smithi196.stdout:5/857: fdatasync d4/d1b/dc8/d14/d19/d26/d33/fdb 0 2024-03-13T14:49:58.509 INFO:tasks.workunit.client.0.smithi176.stdout:8/675: rename d0/d5/c74 to d0/d24/d98/dc5/cd9 0 2024-03-13T14:49:58.510 INFO:tasks.workunit.client.0.smithi176.stdout:2/758: rename d1/dc/d2d/d4b to d1/dc/d2d/d4b/d69/dff 22 2024-03-13T14:49:58.512 INFO:tasks.workunit.client.1.smithi196.stdout:9/799: creat d7/de/d74/d94/dda/f113 x:0 0 0 2024-03-13T14:49:58.514 INFO:tasks.workunit.client.1.smithi196.stdout:4/801: fdatasync d1/d3/d4c/d72/daf/fd6 0 2024-03-13T14:49:58.515 INFO:tasks.workunit.client.1.smithi196.stdout:8/774: write d0/d10/d12/d17/d42/fd2 [958107,63000] 0 2024-03-13T14:49:58.517 INFO:tasks.workunit.client.1.smithi196.stdout:8/775: readlink d0/d63/l8a 0 2024-03-13T14:49:58.521 INFO:tasks.workunit.client.1.smithi196.stdout:3/823: creat db/d19/d4a/f108 x:0 0 0 2024-03-13T14:49:58.525 INFO:tasks.workunit.client.0.smithi176.stdout:0/647: dwrite d2/db/d31/f7f [0,4194304] 0 2024-03-13T14:49:58.525 INFO:tasks.workunit.client.1.smithi196.stdout:0/845: creat d8/da/d25/d26/d35/d3b/d5f/d33/f115 x:0 0 0 2024-03-13T14:49:58.526 INFO:tasks.workunit.client.0.smithi176.stdout:9/855: dread d0/d17/f19 [4194304,4194304] 0 2024-03-13T14:49:58.529 INFO:tasks.workunit.client.0.smithi176.stdout:5/709: chown d5/d22/f1e 153 1 2024-03-13T14:49:58.531 INFO:tasks.workunit.client.1.smithi196.stdout:1/905: rename d1/d9/d4e/fc5 to d1/d38/f137 0 2024-03-13T14:49:58.532 INFO:tasks.workunit.client.1.smithi196.stdout:7/931: fsync d0/d6/d33/d57/f95 0 2024-03-13T14:49:58.535 INFO:tasks.workunit.client.1.smithi196.stdout:5/858: fsync d4/d1b/dc8/d14/d19/d26/d33/fdb 0 2024-03-13T14:49:58.537 INFO:tasks.workunit.client.0.smithi176.stdout:2/759: creat d1/d2a/f100 x:0 0 0 2024-03-13T14:49:58.538 INFO:tasks.workunit.client.0.smithi176.stdout:6/736: dwrite d0/d10/d19/fa8 [0,4194304] 0 2024-03-13T14:49:58.539 INFO:tasks.workunit.client.0.smithi176.stdout:6/737: readlink d0/d10/d19/d27/ld0 0 2024-03-13T14:49:58.540 INFO:tasks.workunit.client.1.smithi196.stdout:9/800: read - d7/d9/d17/fff zero size 2024-03-13T14:49:58.543 INFO:tasks.workunit.client.0.smithi176.stdout:5/710: dread d5/d31/fa2 [0,4194304] 0 2024-03-13T14:49:58.550 INFO:tasks.workunit.client.1.smithi196.stdout:0/846: symlink d8/da/d25/d26/d35/d3b/d5f/d33/l116 0 2024-03-13T14:49:58.554 INFO:tasks.workunit.client.1.smithi196.stdout:7/932: unlink d0/d6/d140/d93/lcd 0 2024-03-13T14:49:58.556 INFO:tasks.workunit.client.1.smithi196.stdout:5/859: creat d4/d1b/dc8/d14/d19/da7/f111 x:0 0 0 2024-03-13T14:49:58.558 INFO:tasks.workunit.client.1.smithi196.stdout:9/801: dread - d7/de/d74/fad zero size 2024-03-13T14:49:58.560 INFO:tasks.workunit.client.1.smithi196.stdout:8/776: getdents d0/d10/d12/d1e/db1 0 2024-03-13T14:49:58.561 INFO:tasks.workunit.client.1.smithi196.stdout:5/860: chown d4/d1b/dc8/d14/d19/d26/d33/d61/fd9 3 1 2024-03-13T14:49:58.564 INFO:tasks.workunit.client.1.smithi196.stdout:2/756: getdents d0/d7/d40 0 2024-03-13T14:49:58.566 INFO:tasks.workunit.client.1.smithi196.stdout:1/906: dread d1/d9/d4e/d55/d63/db9/dbb/fc2 [0,4194304] 0 2024-03-13T14:49:58.567 INFO:tasks.workunit.client.1.smithi196.stdout:0/847: symlink d8/da/d25/dd7/l117 0 2024-03-13T14:49:58.569 INFO:tasks.workunit.client.1.smithi196.stdout:4/802: creat d1/d3/d7/f118 x:0 0 0 2024-03-13T14:49:58.572 INFO:tasks.workunit.client.1.smithi196.stdout:8/777: creat d0/d10/d12/d17/d42/dbe/f108 x:0 0 0 2024-03-13T14:49:58.575 INFO:tasks.workunit.client.1.smithi196.stdout:2/757: dread - d0/d7/d9/fdf zero size 2024-03-13T14:49:58.576 INFO:tasks.workunit.client.1.smithi196.stdout:1/907: rename d1/d28/f35 to d1/d3f/f138 0 2024-03-13T14:49:58.578 INFO:tasks.workunit.client.1.smithi196.stdout:7/933: symlink d0/l141 0 2024-03-13T14:49:58.581 INFO:tasks.workunit.client.1.smithi196.stdout:7/934: truncate d0/d6/d140/d93/f13b 497856 0 2024-03-13T14:49:58.583 INFO:tasks.workunit.client.1.smithi196.stdout:3/824: link db/d19/f28 db/d19/f109 0 2024-03-13T14:49:58.583 INFO:tasks.workunit.client.1.smithi196.stdout:7/935: chown d0/d6/faa 1682792 1 2024-03-13T14:49:58.588 INFO:tasks.workunit.client.1.smithi196.stdout:2/758: creat d0/d7/d56/dc8/ffa x:0 0 0 2024-03-13T14:49:58.591 INFO:tasks.workunit.client.1.smithi196.stdout:4/803: rename d1/d1e/f113 to d1/d3/d4c/d72/daf/f119 0 2024-03-13T14:49:58.592 INFO:tasks.workunit.client.0.smithi176.stdout:1/706: creat d2/de/d3f/d6a/fda x:0 0 0 2024-03-13T14:49:58.593 INFO:tasks.workunit.client.0.smithi176.stdout:3/785: truncate d4/d13/d16/d80/faf 2035519 0 2024-03-13T14:49:58.595 INFO:tasks.workunit.client.0.smithi176.stdout:0/648: chown d2/d5/d28/d8c/cad 62 1 2024-03-13T14:49:58.596 INFO:tasks.workunit.client.0.smithi176.stdout:0/649: chown d2/d74/d61/c6c 891 1 2024-03-13T14:49:58.597 INFO:tasks.workunit.client.0.smithi176.stdout:7/701: truncate d1/d1c/d4a/f8d 3460041 0 2024-03-13T14:49:58.602 INFO:tasks.workunit.client.1.smithi196.stdout:7/936: mkdir d0/d9/d1e/d58/d72/da9/da3/d142 0 2024-03-13T14:49:58.608 INFO:tasks.workunit.client.0.smithi176.stdout:3/786: dwrite d4/d13/d16/d3d/f4c [4194304,4194304] 0 2024-03-13T14:49:58.608 INFO:tasks.workunit.client.0.smithi176.stdout:1/707: mkdir d2/d29/d73/ddb 0 2024-03-13T14:49:58.610 INFO:tasks.workunit.client.1.smithi196.stdout:0/848: creat d8/da/d25/d26/d54/f118 x:0 0 0 2024-03-13T14:49:58.612 INFO:tasks.workunit.client.1.smithi196.stdout:9/802: getdents d7/de/d74/d94/dfb/d28 0 2024-03-13T14:49:58.614 INFO:tasks.workunit.client.0.smithi176.stdout:9/856: mkdir d0/d1e/df7/d11f 0 2024-03-13T14:49:58.616 INFO:tasks.workunit.client.0.smithi176.stdout:2/760: dread d1/dc/d2d/d38/d5d/f62 [0,4194304] 0 2024-03-13T14:49:58.617 INFO:tasks.workunit.client.0.smithi176.stdout:0/650: mknod d2/d74/d61/cd0 0 2024-03-13T14:49:58.618 INFO:tasks.workunit.client.1.smithi196.stdout:5/861: getdents d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd 0 2024-03-13T14:49:58.618 INFO:tasks.workunit.client.1.smithi196.stdout:3/825: dwrite db/d15/d44/d79/ff6 [0,4194304] 0 2024-03-13T14:49:58.620 INFO:tasks.workunit.client.0.smithi176.stdout:7/702: creat d1/d28/d3a/d1b/d8f/dc1/dbb/fd7 x:0 0 0 2024-03-13T14:49:58.620 INFO:tasks.workunit.client.0.smithi176.stdout:2/761: readlink d1/dc/d2d/d38/l74 0 2024-03-13T14:49:58.623 INFO:tasks.workunit.client.1.smithi196.stdout:9/803: read d7/de/d16/d70/f89 [2198617,95653] 0 2024-03-13T14:49:58.623 INFO:tasks.workunit.client.0.smithi176.stdout:7/703: dread - d1/d28/d3a/d1b/d8f/dc1/db7/fd0 zero size 2024-03-13T14:49:58.627 INFO:tasks.workunit.client.0.smithi176.stdout:1/708: dwrite d2/d29/fce [0,4194304] 0 2024-03-13T14:49:58.627 INFO:tasks.workunit.client.0.smithi176.stdout:6/738: mknod d0/de/d54/ceb 0 2024-03-13T14:49:58.633 INFO:tasks.workunit.client.0.smithi176.stdout:3/787: symlink d4/d13/d5e/daa/l10f 0 2024-03-13T14:49:58.634 INFO:tasks.workunit.client.1.smithi196.stdout:4/804: mknod d1/d1e/c11a 0 2024-03-13T14:49:58.634 INFO:tasks.workunit.client.1.smithi196.stdout:8/778: write d0/d1c/f7c [65298,94909] 0 2024-03-13T14:49:58.640 INFO:tasks.workunit.client.1.smithi196.stdout:0/849: creat d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f119 x:0 0 0 2024-03-13T14:49:58.641 INFO:tasks.workunit.client.1.smithi196.stdout:8/779: dread d0/d10/f77 [0,4194304] 0 2024-03-13T14:49:58.643 INFO:tasks.workunit.client.0.smithi176.stdout:9/857: unlink d0/d17/c46 0 2024-03-13T14:49:58.648 INFO:tasks.workunit.client.1.smithi196.stdout:5/862: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/f112 x:0 0 0 2024-03-13T14:49:58.652 INFO:tasks.workunit.client.0.smithi176.stdout:8/676: link d0/d24/f31 d0/d5/d11/fda 0 2024-03-13T14:49:58.656 INFO:tasks.workunit.client.1.smithi196.stdout:9/804: readlink d7/de/d74/l96 0 2024-03-13T14:49:58.657 INFO:tasks.workunit.client.0.smithi176.stdout:1/709: mknod d2/d29/d73/cdc 0 2024-03-13T14:49:58.659 INFO:tasks.workunit.client.1.smithi196.stdout:9/805: write d7/de/d16/d70/f7a [3152186,66433] 0 2024-03-13T14:49:58.661 INFO:tasks.workunit.client.0.smithi176.stdout:6/739: mknod d0/d9/cec 0 2024-03-13T14:49:58.662 INFO:tasks.workunit.client.1.smithi196.stdout:9/806: truncate d7/fde 104125 0 2024-03-13T14:49:58.664 INFO:tasks.workunit.client.1.smithi196.stdout:9/807: stat d7/de/d4b/d1d/d93/da7 0 2024-03-13T14:49:58.666 INFO:tasks.workunit.client.1.smithi196.stdout:4/805: symlink d1/d3/d4c/d72/daf/l11b 0 2024-03-13T14:49:58.667 INFO:tasks.workunit.client.1.smithi196.stdout:1/908: truncate d1/d9/d4e/d55/d63/f134 1877254 0 2024-03-13T14:49:58.669 INFO:tasks.workunit.client.1.smithi196.stdout:5/863: dwrite d4/d1b/dc8/d4a/faf [0,4194304] 0 2024-03-13T14:49:58.674 INFO:tasks.workunit.client.1.smithi196.stdout:2/759: write d0/d7/d9/d42/d98/de4/d67/fbd [722349,90641] 0 2024-03-13T14:49:58.682 INFO:tasks.workunit.client.1.smithi196.stdout:3/826: symlink db/d19/d22/d3a/l10a 0 2024-03-13T14:49:58.685 INFO:tasks.workunit.client.1.smithi196.stdout:7/937: creat d0/d6/f143 x:0 0 0 2024-03-13T14:49:58.686 INFO:tasks.workunit.client.1.smithi196.stdout:7/938: stat d0/d4d/d74/la5 0 2024-03-13T14:49:58.687 INFO:tasks.workunit.client.1.smithi196.stdout:9/808: mknod d7/de/d4b/d1d/d93/dd7/c114 0 2024-03-13T14:49:58.687 INFO:tasks.workunit.client.1.smithi196.stdout:1/909: fdatasync d1/d9/d6c/fab 0 2024-03-13T14:49:58.689 INFO:tasks.workunit.client.1.smithi196.stdout:9/809: chown d7/de/d74/d94/fd5 2841416 1 2024-03-13T14:49:58.690 INFO:tasks.workunit.client.1.smithi196.stdout:4/806: rename d1/d3/d7/d35/d3e/d56 to d1/dfb/d11c 0 2024-03-13T14:49:58.693 INFO:tasks.workunit.client.1.smithi196.stdout:9/810: read d7/de/d74/d94/dfb/d28/d6c/fc2 [3883909,119258] 0 2024-03-13T14:49:58.695 INFO:tasks.workunit.client.1.smithi196.stdout:9/811: write d7/d71/ff2 [638520,88] 0 2024-03-13T14:49:58.697 INFO:tasks.workunit.client.1.smithi196.stdout:3/827: fdatasync db/d19/d22/d3a/d53/f59 0 2024-03-13T14:49:58.700 INFO:tasks.workunit.client.1.smithi196.stdout:3/828: chown db/d15/c1e 3699825 1 2024-03-13T14:49:58.702 INFO:tasks.workunit.client.1.smithi196.stdout:1/910: dread d1/d9/d33/d4b/d66/d7b/fa9 [0,4194304] 0 2024-03-13T14:49:58.703 INFO:tasks.workunit.client.0.smithi176.stdout:6/740: chown d0/d10/d19/d27/d2b/d2e/fad 62270872 1 2024-03-13T14:49:58.704 INFO:tasks.workunit.client.0.smithi176.stdout:5/711: rename d5/d22/d15/f7d to d5/d2c/d40/ff3 0 2024-03-13T14:49:58.707 INFO:tasks.workunit.client.1.smithi196.stdout:2/760: link d0/d7/d9/d42/d98/le5 d0/d50/lfb 0 2024-03-13T14:49:58.708 INFO:tasks.workunit.client.0.smithi176.stdout:1/710: symlink d2/de/d10/d16/d47/db0/ldd 0 2024-03-13T14:49:58.709 INFO:tasks.workunit.client.0.smithi176.stdout:1/711: readlink l0 0 2024-03-13T14:49:58.710 INFO:tasks.workunit.client.0.smithi176.stdout:5/712: dread - d5/d2c/d40/dc7/ff0 zero size 2024-03-13T14:49:58.711 INFO:tasks.workunit.client.1.smithi196.stdout:7/939: creat d0/d4d/f144 x:0 0 0 2024-03-13T14:49:58.714 INFO:tasks.workunit.client.0.smithi176.stdout:0/651: creat d2/d5/fd1 x:0 0 0 2024-03-13T14:49:58.714 INFO:tasks.workunit.client.0.smithi176.stdout:2/762: write d1/dc/f52 [921029,115088] 0 2024-03-13T14:49:58.716 INFO:tasks.workunit.client.0.smithi176.stdout:8/677: rename d0/db/d39/d56/f95 to d0/d24/d98/dc6/db9/fdb 0 2024-03-13T14:49:58.717 INFO:tasks.workunit.client.1.smithi196.stdout:8/780: write d0/d1c/f7b [989975,66151] 0 2024-03-13T14:49:58.719 INFO:tasks.workunit.client.1.smithi196.stdout:5/864: getdents d4/d1b/dc8/d3c/d6a 0 2024-03-13T14:49:58.722 INFO:tasks.workunit.client.1.smithi196.stdout:1/911: mknod d1/d9/d4e/d55/d63/da6/dc8/d11d/c139 0 2024-03-13T14:49:58.722 INFO:tasks.workunit.client.0.smithi176.stdout:6/741: dwrite d0/d10/d19/d27/f4d [0,4194304] 0 2024-03-13T14:49:58.723 INFO:tasks.workunit.client.1.smithi196.stdout:3/829: dwrite db/d19/d50/fca [4194304,4194304] 0 2024-03-13T14:49:58.724 INFO:tasks.workunit.client.0.smithi176.stdout:6/742: chown d0/de/d38 2 1 2024-03-13T14:49:58.727 INFO:tasks.workunit.client.1.smithi196.stdout:2/761: fdatasync d0/d6/d4b/d5f/fb7 0 2024-03-13T14:49:58.728 INFO:tasks.workunit.client.0.smithi176.stdout:1/712: read - d2/de/d10/d16/d47/f54 zero size 2024-03-13T14:49:58.730 INFO:tasks.workunit.client.1.smithi196.stdout:4/807: rmdir d1/d3 39 2024-03-13T14:49:58.732 INFO:tasks.workunit.client.1.smithi196.stdout:9/812: dread d7/de/d74/d94/dfb/dbf/f100 [4194304,4194304] 0 2024-03-13T14:49:58.732 INFO:tasks.workunit.client.1.smithi196.stdout:8/781: creat d0/d63/f109 x:0 0 0 2024-03-13T14:49:58.732 INFO:tasks.workunit.client.0.smithi176.stdout:3/788: dread d4/d13/f61 [0,4194304] 0 2024-03-13T14:49:58.737 INFO:tasks.workunit.client.1.smithi196.stdout:5/865: unlink d4/d1b/dc8/d3c/fc0 0 2024-03-13T14:49:58.739 INFO:tasks.workunit.client.0.smithi176.stdout:0/652: chown d2/d5/c3c 16073 1 2024-03-13T14:49:58.742 INFO:tasks.workunit.client.1.smithi196.stdout:1/912: creat d1/d9/d6c/d101/f13a x:0 0 0 2024-03-13T14:49:58.743 INFO:tasks.workunit.client.1.smithi196.stdout:7/940: mkdir d0/d9/d1e/d58/d72/d6e/d145 0 2024-03-13T14:49:58.744 INFO:tasks.workunit.client.0.smithi176.stdout:2/763: symlink d1/dc/d2d/d59/d80/d77/l101 0 2024-03-13T14:49:58.745 INFO:tasks.workunit.client.1.smithi196.stdout:0/850: dwrite d8/da/d25/d26/f60 [0,4194304] 0 2024-03-13T14:49:58.748 INFO:tasks.workunit.client.0.smithi176.stdout:2/764: truncate d1/dc/d61/daa/dcc/fe7 483069 0 2024-03-13T14:49:58.749 INFO:tasks.workunit.client.0.smithi176.stdout:1/713: creat d2/de/d3f/d50/d83/d97/fde x:0 0 0 2024-03-13T14:49:58.751 INFO:tasks.workunit.client.0.smithi176.stdout:1/714: chown d2/de/c8c 197 1 2024-03-13T14:49:58.757 INFO:tasks.workunit.client.1.smithi196.stdout:9/813: mkdir d7/de/d4b/d1d/d93/dd7/df1/d115 0 2024-03-13T14:49:58.758 INFO:tasks.workunit.client.1.smithi196.stdout:5/866: creat d4/d1b/dc8/d3c/d6a/d106/f113 x:0 0 0 2024-03-13T14:49:58.758 INFO:tasks.workunit.client.1.smithi196.stdout:8/782: dwrite d0/f7e [0,4194304] 0 2024-03-13T14:49:58.762 INFO:tasks.workunit.client.0.smithi176.stdout:0/653: link d2/cf d2/d5/d1e/d97/dcf/cd2 0 2024-03-13T14:49:58.762 INFO:tasks.workunit.client.1.smithi196.stdout:1/913: rmdir d1/d9/d33 39 2024-03-13T14:49:58.763 INFO:tasks.workunit.client.1.smithi196.stdout:5/867: fdatasync d4/d1b/dc8/d14/d19/da7/f111 0 2024-03-13T14:49:58.767 INFO:tasks.workunit.client.0.smithi176.stdout:0/654: fsync d2/d5/d1e/f72 0 2024-03-13T14:49:58.767 INFO:tasks.workunit.client.0.smithi176.stdout:6/743: dread d0/d10/d19/d27/d2b/de0/f74 [0,4194304] 0 2024-03-13T14:49:58.767 INFO:tasks.workunit.client.0.smithi176.stdout:1/715: dread d2/de/d10/d16/d59/fa1 [0,4194304] 0 2024-03-13T14:49:58.769 INFO:tasks.workunit.client.0.smithi176.stdout:2/765: getdents d1/dc 0 2024-03-13T14:49:58.769 INFO:tasks.workunit.client.0.smithi176.stdout:3/789: dread d4/d28/f29 [0,4194304] 0 2024-03-13T14:49:58.771 INFO:tasks.workunit.client.1.smithi196.stdout:9/814: symlink d7/d9/d17/dbc/l116 0 2024-03-13T14:49:58.771 INFO:tasks.workunit.client.1.smithi196.stdout:4/808: read d1/d1e/d5c/ff1 [2772150,128719] 0 2024-03-13T14:49:58.772 INFO:tasks.workunit.client.1.smithi196.stdout:5/868: rmdir d4/d1b/dc8/d14/d19/d1c/d94 39 2024-03-13T14:49:58.773 INFO:tasks.workunit.client.0.smithi176.stdout:1/716: creat d2/de/dd1/fdf x:0 0 0 2024-03-13T14:49:58.774 INFO:tasks.workunit.client.1.smithi196.stdout:2/762: link d0/d7/d9/d42/d98/cf3 d0/d7/d9/d61/dd3/cfc 0 2024-03-13T14:49:58.774 INFO:tasks.workunit.client.0.smithi176.stdout:6/744: symlink d0/d10/led 0 2024-03-13T14:49:58.776 INFO:tasks.workunit.client.0.smithi176.stdout:6/745: dread - d0/de/d38/d6b/fd9 zero size 2024-03-13T14:49:58.776 INFO:tasks.workunit.client.1.smithi196.stdout:7/941: unlink d0/d9/d1e/d58/d72/da9/da3/ffa 0 2024-03-13T14:49:58.777 INFO:tasks.workunit.client.0.smithi176.stdout:2/766: truncate d1/dc/d2d/d90/dcf/da0/fd8 840316 0 2024-03-13T14:49:58.781 INFO:tasks.workunit.client.1.smithi196.stdout:5/869: rmdir d4/d1b/dc8/d14 39 2024-03-13T14:49:58.783 INFO:tasks.workunit.client.0.smithi176.stdout:2/767: creat d1/dc/d1c/f102 x:0 0 0 2024-03-13T14:49:58.783 INFO:tasks.workunit.client.1.smithi196.stdout:2/763: truncate d0/d7/d9/d9a/fca 1002193 0 2024-03-13T14:49:58.784 INFO:tasks.workunit.client.0.smithi176.stdout:0/655: getdents d2/d5/d28/d8c 0 2024-03-13T14:49:58.784 INFO:tasks.workunit.client.1.smithi196.stdout:2/764: readlink d0/d7/l59 0 2024-03-13T14:49:58.785 INFO:tasks.workunit.client.1.smithi196.stdout:1/914: creat d1/d9/f13b x:0 0 0 2024-03-13T14:49:58.786 INFO:tasks.workunit.client.0.smithi176.stdout:6/746: mknod d0/d10/d19/d27/d2b/de0/cee 0 2024-03-13T14:49:58.787 INFO:tasks.workunit.client.1.smithi196.stdout:8/783: link d0/d10/d12/d17/d42/dbe/d106/le3 d0/d16/d3d/d7f/l10a 0 2024-03-13T14:49:58.789 INFO:tasks.workunit.client.1.smithi196.stdout:4/809: creat d1/d1e/d5c/d105/f11d x:0 0 0 2024-03-13T14:49:58.790 INFO:tasks.workunit.client.0.smithi176.stdout:3/790: dread d4/d13/d2d/f30 [8388608,4194304] 0 2024-03-13T14:49:58.791 INFO:tasks.workunit.client.1.smithi196.stdout:2/765: chown d0/d6/d1d/lf0 879173 1 2024-03-13T14:49:58.792 INFO:tasks.workunit.client.1.smithi196.stdout:1/915: mkdir d1/d28/d34/d13c 0 2024-03-13T14:49:58.793 INFO:tasks.workunit.client.0.smithi176.stdout:6/747: creat d0/d10/d19/d27/d2b/ddc/fef x:0 0 0 2024-03-13T14:49:58.794 INFO:tasks.workunit.client.0.smithi176.stdout:3/791: fdatasync d4/d13/d2d/f51 0 2024-03-13T14:49:58.795 INFO:tasks.workunit.client.1.smithi196.stdout:8/784: creat d0/d16/d71/d9a/f10b x:0 0 0 2024-03-13T14:49:58.796 INFO:tasks.workunit.client.0.smithi176.stdout:6/748: dread - d0/de/d38/d88/d9d/fd4 zero size 2024-03-13T14:49:58.796 INFO:tasks.workunit.client.1.smithi196.stdout:9/815: dread d7/de/d4b/d1d/d93/da7/fae [0,4194304] 0 2024-03-13T14:49:58.798 INFO:tasks.workunit.client.1.smithi196.stdout:5/870: read d4/d1b/dc8/d14/d19/da7/f10d [2373713,10549] 0 2024-03-13T14:49:58.799 INFO:tasks.workunit.client.1.smithi196.stdout:7/942: rmdir d0/d6/ddc/d126 0 2024-03-13T14:49:58.800 INFO:tasks.workunit.client.0.smithi176.stdout:2/768: dwrite d1/dc/d2d/d59/d8c/fd4 [0,4194304] 0 2024-03-13T14:49:58.802 INFO:tasks.workunit.client.1.smithi196.stdout:7/943: write d0/d6/d33/f96 [1163608,67407] 0 2024-03-13T14:49:58.803 INFO:tasks.workunit.client.0.smithi176.stdout:6/749: getdents d0/d10/dca/de1 0 2024-03-13T14:49:58.803 INFO:tasks.workunit.client.1.smithi196.stdout:5/871: stat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/f42 0 2024-03-13T14:49:58.806 INFO:tasks.workunit.client.1.smithi196.stdout:4/810: mknod d1/d3/d4c/d72/d7d/d80/d55/c11e 0 2024-03-13T14:49:58.807 INFO:tasks.workunit.client.0.smithi176.stdout:6/750: mknod d0/d10/d19/d27/d2b/cf0 0 2024-03-13T14:49:58.809 INFO:tasks.workunit.client.0.smithi176.stdout:6/751: chown d0/d10/d19/d27/d2b/d2e/d3e/db4 2958 1 2024-03-13T14:49:58.809 INFO:tasks.workunit.client.0.smithi176.stdout:3/792: dwrite f2 [0,4194304] 0 2024-03-13T14:49:58.813 INFO:tasks.workunit.client.1.smithi196.stdout:9/816: creat d7/de/d4b/d1d/d35/f117 x:0 0 0 2024-03-13T14:49:58.816 INFO:tasks.workunit.client.1.smithi196.stdout:9/817: chown d7/de/d4b/d1d/d93/dd7 6336249 1 2024-03-13T14:49:58.819 INFO:tasks.workunit.client.1.smithi196.stdout:9/818: dread - d7/de/d74/fad zero size 2024-03-13T14:49:58.821 INFO:tasks.workunit.client.1.smithi196.stdout:5/872: read - d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/f102 zero size 2024-03-13T14:49:58.821 INFO:tasks.workunit.client.0.smithi176.stdout:4/759: sync 2024-03-13T14:49:58.825 INFO:tasks.workunit.client.0.smithi176.stdout:1/717: dread d2/de/f53 [0,4194304] 0 2024-03-13T14:49:58.826 INFO:tasks.workunit.client.1.smithi196.stdout:8/785: dread d0/d10/d12/df5/d47/f52 [0,4194304] 0 2024-03-13T14:49:58.830 INFO:tasks.workunit.client.1.smithi196.stdout:8/786: fdatasync d0/d63/f109 0 2024-03-13T14:49:58.830 INFO:tasks.workunit.client.0.smithi176.stdout:4/760: creat d7/d8/d2f/f100 x:0 0 0 2024-03-13T14:49:58.831 INFO:tasks.workunit.client.1.smithi196.stdout:5/873: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/ffe 923334 0 2024-03-13T14:49:58.834 INFO:tasks.workunit.client.0.smithi176.stdout:6/752: dwrite d0/d10/d19/d27/d2b/d2e/f2d [0,4194304] 0 2024-03-13T14:49:58.838 INFO:tasks.workunit.client.1.smithi196.stdout:4/811: dwrite d1/d1e/f33 [4194304,4194304] 0 2024-03-13T14:49:58.843 INFO:tasks.workunit.client.0.smithi176.stdout:6/753: rename d0/de/d38/d6b/lcb to d0/d10/d19/d27/d2b/d2e/d3e/d9a/lf1 0 2024-03-13T14:49:58.845 INFO:tasks.workunit.client.1.smithi196.stdout:4/812: write d1/d3/d7/f65 [3884134,118260] 0 2024-03-13T14:49:58.845 INFO:tasks.workunit.client.1.smithi196.stdout:2/766: creat d0/ffd x:0 0 0 2024-03-13T14:49:58.846 INFO:tasks.workunit.client.0.smithi176.stdout:1/718: link d2/de/d3f/d50/f8e d2/de/d3f/d3e/d86/fe0 0 2024-03-13T14:49:58.846 INFO:tasks.workunit.client.0.smithi176.stdout:4/761: truncate d7/d10/d46/d4a/d5d/da1/f9e 488144 0 2024-03-13T14:49:58.847 INFO:tasks.workunit.client.1.smithi196.stdout:7/944: symlink d0/d4d/dd1/dd6/l146 0 2024-03-13T14:49:58.852 INFO:tasks.workunit.client.0.smithi176.stdout:6/754: mkdir d0/d10/d19/d27/d2b/d2e/d3e/d9a/df2 0 2024-03-13T14:49:58.854 INFO:tasks.workunit.client.1.smithi196.stdout:8/787: mknod d0/d10/da5/db2/c10c 0 2024-03-13T14:49:58.856 INFO:tasks.workunit.client.1.smithi196.stdout:5/874: symlink d4/d1b/dc8/d14/d19/d26/dab/l114 0 2024-03-13T14:49:58.858 INFO:tasks.workunit.client.1.smithi196.stdout:8/788: read d0/d10/f77 [124191,82977] 0 2024-03-13T14:49:58.859 INFO:tasks.workunit.client.0.smithi176.stdout:4/762: mknod d7/d10/d46/d4a/d5d/c101 0 2024-03-13T14:49:58.859 INFO:tasks.workunit.client.1.smithi196.stdout:5/875: chown d4/d1b/dc8/d14/d37 166 1 2024-03-13T14:49:58.864 INFO:tasks.workunit.client.0.smithi176.stdout:8/678: write d0/d24/d7b/dab/f92 [746659,6489] 0 2024-03-13T14:49:58.867 INFO:tasks.workunit.client.1.smithi196.stdout:4/813: creat d1/d3/d7/d35/d76/d58/dc6/f11f x:0 0 0 2024-03-13T14:49:58.871 INFO:tasks.workunit.client.1.smithi196.stdout:2/767: mkdir d0/d7/d9/d5a/dbc/dfe 0 2024-03-13T14:49:58.872 INFO:tasks.workunit.client.1.smithi196.stdout:9/819: link d7/de/d4b/d1d/d93/da7/fb6 d7/de/d4b/d1d/d93/dd7/f118 0 2024-03-13T14:49:58.874 INFO:tasks.workunit.client.1.smithi196.stdout:7/945: fdatasync d0/d9/d1e/d58/d72/da9/d47/f10e 0 2024-03-13T14:49:58.876 INFO:tasks.workunit.client.1.smithi196.stdout:7/946: dread - d0/d6/f12a zero size 2024-03-13T14:49:58.878 INFO:tasks.workunit.client.0.smithi176.stdout:6/755: dwrite d0/d10/d19/d27/d2b/de0/dd6/fcd [0,4194304] 0 2024-03-13T14:49:58.878 INFO:tasks.workunit.client.1.smithi196.stdout:7/947: chown d0/d6/faa 2 1 2024-03-13T14:49:58.880 INFO:tasks.workunit.client.0.smithi176.stdout:1/719: dwrite d2/d29/f70 [0,4194304] 0 2024-03-13T14:49:58.885 INFO:tasks.workunit.client.0.smithi176.stdout:2/769: dread d1/d2a/f82 [0,4194304] 0 2024-03-13T14:49:58.886 INFO:tasks.workunit.client.1.smithi196.stdout:3/830: truncate db/d19/f2f 2252538 0 2024-03-13T14:49:58.888 INFO:tasks.workunit.client.0.smithi176.stdout:5/713: dwrite d5/d2c/d40/ff3 [0,4194304] 0 2024-03-13T14:49:58.888 INFO:tasks.workunit.client.1.smithi196.stdout:4/814: read - d1/d3/d7/ddf/fef zero size 2024-03-13T14:49:58.890 INFO:tasks.workunit.client.0.smithi176.stdout:1/720: write d2/de/d10/d16/d40/f77 [406740,103633] 0 2024-03-13T14:49:58.892 INFO:tasks.workunit.client.0.smithi176.stdout:1/721: fsync d2/d30/d63/d22/f2b 0 2024-03-13T14:49:58.892 INFO:tasks.workunit.client.1.smithi196.stdout:5/876: mkdir d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/db2/d115 0 2024-03-13T14:49:58.892 INFO:tasks.workunit.client.1.smithi196.stdout:0/851: write d8/da/d25/f2f [2753911,108693] 0 2024-03-13T14:49:58.896 INFO:tasks.workunit.client.0.smithi176.stdout:1/722: write d2/de/f17 [3100196,27187] 0 2024-03-13T14:49:58.898 INFO:tasks.workunit.client.0.smithi176.stdout:1/723: chown d2/de/d3f/d19 233552739 1 2024-03-13T14:49:58.902 INFO:tasks.workunit.client.1.smithi196.stdout:2/768: dwrite d0/d6/d1d/fee [0,4194304] 0 2024-03-13T14:49:58.902 INFO:tasks.workunit.client.1.smithi196.stdout:0/852: read d8/da/f13 [3126546,37234] 0 2024-03-13T14:49:58.905 INFO:tasks.workunit.client.1.smithi196.stdout:0/853: fdatasync d8/da/dd8/f109 0 2024-03-13T14:49:58.908 INFO:tasks.workunit.client.0.smithi176.stdout:6/756: dwrite d0/de/d8d/fbd [0,4194304] 0 2024-03-13T14:49:58.912 INFO:tasks.workunit.client.1.smithi196.stdout:7/948: symlink d0/d6/d140/d93/d94/l147 0 2024-03-13T14:49:58.913 INFO:tasks.workunit.client.0.smithi176.stdout:5/714: dwrite d5/d2c/d40/f8f [4194304,4194304] 0 2024-03-13T14:49:58.927 INFO:tasks.workunit.client.0.smithi176.stdout:0/656: write d2/f82 [797739,53954] 0 2024-03-13T14:49:58.931 INFO:tasks.workunit.client.0.smithi176.stdout:0/657: chown d2/db/f1c 13716 1 2024-03-13T14:49:58.932 INFO:tasks.workunit.client.0.smithi176.stdout:1/724: creat d2/de/d10/d16/d4a/d5f/d66/fe1 x:0 0 0 2024-03-13T14:49:58.933 INFO:tasks.workunit.client.0.smithi176.stdout:0/658: chown d2/f3 22 1 2024-03-13T14:49:58.935 INFO:tasks.workunit.client.0.smithi176.stdout:0/659: stat d2/l76 0 2024-03-13T14:49:58.939 INFO:tasks.workunit.client.0.smithi176.stdout:6/757: creat d0/ff3 x:0 0 0 2024-03-13T14:49:58.939 INFO:tasks.workunit.client.0.smithi176.stdout:2/770: truncate d1/dc/d2d/d4b/d69/f94 270261 0 2024-03-13T14:49:58.950 INFO:tasks.workunit.client.1.smithi196.stdout:7/949: unlink d0/d9/d31/d50/l5e 0 2024-03-13T14:49:58.951 INFO:tasks.workunit.client.1.smithi196.stdout:1/916: dwrite d1/d38/f25 [0,4194304] 0 2024-03-13T14:49:58.955 INFO:tasks.workunit.client.0.smithi176.stdout:0/660: symlink d2/d74/d61/ld3 0 2024-03-13T14:49:58.956 INFO:tasks.workunit.client.0.smithi176.stdout:1/725: dread d2/de/d3f/d3e/d86/fbc [0,4194304] 0 2024-03-13T14:49:58.956 INFO:tasks.workunit.client.0.smithi176.stdout:3/793: truncate d4/d13/d81/d89/f60 631833 0 2024-03-13T14:49:58.957 INFO:tasks.workunit.client.0.smithi176.stdout:6/758: symlink d0/dc1/lf4 0 2024-03-13T14:49:58.959 INFO:tasks.workunit.client.1.smithi196.stdout:4/815: symlink d1/d3/d7/ddf/l120 0 2024-03-13T14:49:58.959 INFO:tasks.workunit.client.0.smithi176.stdout:3/794: write d4/d13/d5e/fc2 [1059180,83658] 0 2024-03-13T14:49:58.960 INFO:tasks.workunit.client.0.smithi176.stdout:2/771: creat d1/dc/d2d/d59/d80/d77/f103 x:0 0 0 2024-03-13T14:49:58.961 INFO:tasks.workunit.client.1.smithi196.stdout:2/769: dread d0/d50/fd5 [0,4194304] 0 2024-03-13T14:49:58.964 INFO:tasks.workunit.client.1.smithi196.stdout:0/854: truncate d8/da/d74/fae 173053 0 2024-03-13T14:49:58.967 INFO:tasks.workunit.client.1.smithi196.stdout:2/770: write d0/d7/d8e/fdd [1752424,51593] 0 2024-03-13T14:49:58.967 INFO:tasks.workunit.client.0.smithi176.stdout:1/726: creat d2/de/d3f/d6a/fe2 x:0 0 0 2024-03-13T14:49:58.969 INFO:tasks.workunit.client.1.smithi196.stdout:9/820: getdents d7/de/d74/d94/dfb/d28 0 2024-03-13T14:49:58.976 INFO:tasks.workunit.client.1.smithi196.stdout:9/821: dread d7/de/d74/d94/dfb/d28/d6c/d7b/fb4 [0,4194304] 0 2024-03-13T14:49:58.977 INFO:tasks.workunit.client.0.smithi176.stdout:6/759: dwrite d0/de/d38/d88/d9d/fd7 [0,4194304] 0 2024-03-13T14:49:58.980 INFO:tasks.workunit.client.0.smithi176.stdout:3/795: read d4/d28/d34/f3f [356403,17279] 0 2024-03-13T14:49:58.983 INFO:tasks.workunit.client.0.smithi176.stdout:2/772: mkdir d1/dc/d61/d104 0 2024-03-13T14:49:58.985 INFO:tasks.workunit.client.0.smithi176.stdout:3/796: readlink d4/d28/d34/da4/la8 0 2024-03-13T14:49:58.986 INFO:tasks.workunit.client.0.smithi176.stdout:1/727: dwrite d2/de/f17 [4194304,4194304] 0 2024-03-13T14:49:58.990 INFO:tasks.workunit.client.0.smithi176.stdout:1/728: chown d2/de/d10/d16/d59/fa1 80 1 2024-03-13T14:49:58.993 INFO:tasks.workunit.client.0.smithi176.stdout:1/729: chown d2/de/d3f/d50/cd8 467 1 2024-03-13T14:49:58.999 INFO:tasks.workunit.client.1.smithi196.stdout:4/816: creat d1/dfb/f121 x:0 0 0 2024-03-13T14:49:59.000 INFO:tasks.workunit.client.1.smithi196.stdout:0/855: symlink d8/da/d25/d26/d35/d3b/d5f/l11a 0 2024-03-13T14:49:59.001 INFO:tasks.workunit.client.0.smithi176.stdout:0/661: truncate d2/d5/dc/d59/f5a 855555 0 2024-03-13T14:49:59.002 INFO:tasks.workunit.client.0.smithi176.stdout:2/773: dwrite d1/dc/d2d/d59/d80/fe5 [0,4194304] 0 2024-03-13T14:49:59.005 INFO:tasks.workunit.client.0.smithi176.stdout:6/760: rmdir d0/d10/d19/d27/d2b/ddc 39 2024-03-13T14:49:59.009 INFO:tasks.workunit.client.1.smithi196.stdout:8/789: write d0/d63/f85 [462669,16222] 0 2024-03-13T14:49:59.015 INFO:tasks.workunit.client.0.smithi176.stdout:4/763: dwrite d7/d10/d46/d4a/d5d/fc9 [0,4194304] 0 2024-03-13T14:49:59.015 INFO:tasks.workunit.client.0.smithi176.stdout:7/704: sync 2024-03-13T14:49:59.019 INFO:tasks.workunit.client.0.smithi176.stdout:1/730: symlink d2/d30/d63/d22/le3 0 2024-03-13T14:49:59.020 INFO:tasks.workunit.client.1.smithi196.stdout:7/950: link d0/d78/l86 d0/d6/d33/d57/dae/l148 0 2024-03-13T14:49:59.024 INFO:tasks.workunit.client.1.smithi196.stdout:4/817: unlink d1/f2e 0 2024-03-13T14:49:59.028 INFO:tasks.workunit.client.0.smithi176.stdout:0/662: symlink d2/db/d39/ld4 0 2024-03-13T14:49:59.028 INFO:tasks.workunit.client.0.smithi176.stdout:2/774: stat d1/dc/d2d/d59/d8b/daf/db2 0 2024-03-13T14:49:59.029 INFO:tasks.workunit.client.1.smithi196.stdout:2/771: link d0/d6/d1d/ld0 d0/d6/d34/d5b/lff 0 2024-03-13T14:49:59.030 INFO:tasks.workunit.client.0.smithi176.stdout:2/775: readlink d1/dc/l1f 0 2024-03-13T14:49:59.033 INFO:tasks.workunit.client.1.smithi196.stdout:0/856: mknod d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe/c11b 0 2024-03-13T14:49:59.035 INFO:tasks.workunit.client.1.smithi196.stdout:3/831: dwrite db/d15/d44/d51/fc6 [4194304,4194304] 0 2024-03-13T14:49:59.038 INFO:tasks.workunit.client.0.smithi176.stdout:3/797: creat d4/d13/d81/d89/d5b/dc5/df0/d100/f110 x:0 0 0 2024-03-13T14:49:59.044 INFO:tasks.workunit.client.0.smithi176.stdout:4/764: dread d7/d10/d46/d4a/d5d/ff5 [0,4194304] 0 2024-03-13T14:49:59.052 INFO:tasks.workunit.client.1.smithi196.stdout:2/772: write d0/d7/d8e/fa5 [581387,1006] 0 2024-03-13T14:49:59.054 INFO:tasks.workunit.client.0.smithi176.stdout:5/715: write d5/d2e/f3f [2081038,8477] 0 2024-03-13T14:49:59.056 INFO:tasks.workunit.client.0.smithi176.stdout:3/798: symlink d4/d13/d81/d89/d5b/dc5/df0/l111 0 2024-03-13T14:49:59.057 INFO:tasks.workunit.client.0.smithi176.stdout:1/731: mknod d2/d30/dd3/ce4 0 2024-03-13T14:49:59.058 INFO:tasks.workunit.client.0.smithi176.stdout:3/799: stat d4/d28/d31/d52/de9/leb 0 2024-03-13T14:49:59.058 INFO:tasks.workunit.client.0.smithi176.stdout:7/705: dread d1/d1c/d26/d41/d2b/f96 [0,4194304] 0 2024-03-13T14:49:59.059 INFO:tasks.workunit.client.1.smithi196.stdout:0/857: dwrite d8/ff9 [0,4194304] 0 2024-03-13T14:49:59.063 INFO:tasks.workunit.client.0.smithi176.stdout:7/706: write d1/d1c/d26/d41/d2b/f96 [1954484,116027] 0 2024-03-13T14:49:59.063 INFO:tasks.workunit.client.0.smithi176.stdout:4/765: readlink d7/d10/d2c/l88 0 2024-03-13T14:49:59.067 INFO:tasks.workunit.client.0.smithi176.stdout:2/776: mkdir d1/dc/d61/d104/d105 0 2024-03-13T14:49:59.077 INFO:tasks.workunit.client.1.smithi196.stdout:2/773: fsync d0/d7/d9/d5a/f6f 0 2024-03-13T14:49:59.078 INFO:tasks.workunit.client.0.smithi176.stdout:1/732: mkdir d2/de/d3f/d3e/d86/db3/de5 0 2024-03-13T14:49:59.080 INFO:tasks.workunit.client.0.smithi176.stdout:5/716: dread d5/f81 [0,4194304] 0 2024-03-13T14:49:59.082 INFO:tasks.workunit.client.1.smithi196.stdout:7/951: dread d0/d4d/fc8 [0,4194304] 0 2024-03-13T14:49:59.082 INFO:tasks.workunit.client.0.smithi176.stdout:7/707: symlink d1/d3f/d76/ld8 0 2024-03-13T14:49:59.091 INFO:tasks.workunit.client.1.smithi196.stdout:8/790: truncate d0/d1c/f7b 70082 0 2024-03-13T14:49:59.091 INFO:tasks.workunit.client.1.smithi196.stdout:9/822: dwrite d7/f20 [0,4194304] 0 2024-03-13T14:49:59.092 INFO:tasks.workunit.client.0.smithi176.stdout:3/800: truncate d4/d13/d16/f84 1326342 0 2024-03-13T14:49:59.094 INFO:tasks.workunit.client.0.smithi176.stdout:6/761: dwrite d0/de/d38/d70/faf [4194304,4194304] 0 2024-03-13T14:49:59.097 INFO:tasks.workunit.client.1.smithi196.stdout:7/952: truncate d0/d6/faa 1371687 0 2024-03-13T14:49:59.098 INFO:tasks.workunit.client.0.smithi176.stdout:5/717: dwrite d5/d2c/d8a/fd4 [0,4194304] 0 2024-03-13T14:49:59.101 INFO:tasks.workunit.client.1.smithi196.stdout:3/832: write db/d19/f1d [2953399,22139] 0 2024-03-13T14:49:59.105 INFO:tasks.workunit.client.1.smithi196.stdout:7/953: symlink d0/d6/d35/dff/l149 0 2024-03-13T14:49:59.106 INFO:tasks.workunit.client.0.smithi176.stdout:7/708: creat d1/d28/d3a/d1b/fd9 x:0 0 0 2024-03-13T14:49:59.110 INFO:tasks.workunit.client.0.smithi176.stdout:7/709: truncate d1/d3f/d76/da5/fba 39636 0 2024-03-13T14:49:59.110 INFO:tasks.workunit.client.1.smithi196.stdout:7/954: dread - d0/d9/d1e/d58/d61/fd4 zero size 2024-03-13T14:49:59.114 INFO:tasks.workunit.client.1.smithi196.stdout:4/818: dwrite d1/f29 [0,4194304] 0 2024-03-13T14:49:59.118 INFO:tasks.workunit.client.1.smithi196.stdout:9/823: symlink d7/de/d74/d94/dfb/d28/d6c/d7b/l119 0 2024-03-13T14:49:59.123 INFO:tasks.workunit.client.1.smithi196.stdout:9/824: chown d7/d9/d83/fcb 16096 1 2024-03-13T14:49:59.123 INFO:tasks.workunit.client.1.smithi196.stdout:3/833: truncate db/f3e 888674 0 2024-03-13T14:49:59.125 INFO:tasks.workunit.client.1.smithi196.stdout:9/825: write d7/de/d16/f9b [3910363,128422] 0 2024-03-13T14:49:59.129 INFO:tasks.workunit.client.1.smithi196.stdout:7/955: mkdir d0/d9/d1e/d58/d72/da9/da3/ddf/d14a 0 2024-03-13T14:49:59.134 INFO:tasks.workunit.client.1.smithi196.stdout:0/858: dwrite d8/da/f10 [0,4194304] 0 2024-03-13T14:49:59.141 INFO:tasks.workunit.client.1.smithi196.stdout:2/774: dwrite d0/d7/d40/f46 [0,4194304] 0 2024-03-13T14:49:59.141 INFO:tasks.workunit.client.0.smithi176.stdout:1/733: unlink d2/c93 0 2024-03-13T14:49:59.148 INFO:tasks.workunit.client.0.smithi176.stdout:3/801: mknod d4/d13/d81/d89/d5b/dc5/df0/d100/c112 0 2024-03-13T14:49:59.150 INFO:tasks.workunit.client.0.smithi176.stdout:9/858: sync 2024-03-13T14:49:59.150 INFO:tasks.workunit.client.0.smithi176.stdout:8/679: sync 2024-03-13T14:49:59.150 INFO:tasks.workunit.client.1.smithi196.stdout:3/834: mknod db/d19/d22/d3a/d68/c10b 0 2024-03-13T14:49:59.150 INFO:tasks.workunit.client.1.smithi196.stdout:9/826: symlink d7/de/d4b/d1d/d22/l11a 0 2024-03-13T14:49:59.150 INFO:tasks.workunit.client.1.smithi196.stdout:3/835: stat db/d15/lb1 0 2024-03-13T14:49:59.155 INFO:tasks.workunit.client.0.smithi176.stdout:6/762: mknod d0/d10/d19/d27/d2b/d2e/d3e/cf5 0 2024-03-13T14:49:59.157 INFO:tasks.workunit.client.1.smithi196.stdout:8/791: dread d0/d10/d12/df5/fba [0,4194304] 0 2024-03-13T14:49:59.162 INFO:tasks.workunit.client.1.smithi196.stdout:0/859: fsync d8/da/f4e 0 2024-03-13T14:49:59.167 INFO:tasks.workunit.client.1.smithi196.stdout:4/819: write d1/d3/d4c/f83 [2306386,119281] 0 2024-03-13T14:49:59.168 INFO:tasks.workunit.client.1.smithi196.stdout:2/775: chown d0/d7/d9/l47 1 1 2024-03-13T14:49:59.169 INFO:tasks.workunit.client.0.smithi176.stdout:1/734: rmdir d2/de/d10/d16/d47 39 2024-03-13T14:49:59.173 INFO:tasks.workunit.client.1.smithi196.stdout:9/827: mkdir d7/de/d74/d94/d11b 0 2024-03-13T14:49:59.176 INFO:tasks.workunit.client.1.smithi196.stdout:2/776: truncate d0/d7/d9/d42/d98/de4/d67/fda 979659 0 2024-03-13T14:49:59.176 INFO:tasks.workunit.client.0.smithi176.stdout:3/802: read d4/d28/d34/f6e [908357,117962] 0 2024-03-13T14:49:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:58 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:49:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:58 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:49:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:58 smithi196 ceph-mon[37486]: pgmap v18: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 71 MiB/s rd, 178 MiB/s wr, 376 op/s 2024-03-13T14:49:59.178 INFO:tasks.workunit.client.0.smithi176.stdout:9/859: creat d0/d40/d53/d6d/f120 x:0 0 0 2024-03-13T14:49:59.183 INFO:tasks.workunit.client.0.smithi176.stdout:2/777: truncate d1/dc/d2d/d4b/d69/f94 706221 0 2024-03-13T14:49:59.184 INFO:tasks.workunit.client.0.smithi176.stdout:4/766: getdents d7/d8/d15/d94 0 2024-03-13T14:49:59.185 INFO:tasks.workunit.client.1.smithi196.stdout:4/820: fdatasync d1/d3/d7/d35/d76/f38 0 2024-03-13T14:49:59.186 INFO:tasks.workunit.client.0.smithi176.stdout:7/710: dread d1/d3f/d76/f6d [0,4194304] 0 2024-03-13T14:49:59.186 INFO:tasks.workunit.client.0.smithi176.stdout:2/778: dread - d1/dc/de0/fee zero size 2024-03-13T14:49:59.187 INFO:tasks.workunit.client.0.smithi176.stdout:1/735: rmdir d2/de/d10/d16/d4a/d5f 39 2024-03-13T14:49:59.188 INFO:tasks.workunit.client.1.smithi196.stdout:9/828: creat d7/de/d74/d94/dfb/d28/de4/f11c x:0 0 0 2024-03-13T14:49:59.188 INFO:tasks.workunit.client.0.smithi176.stdout:4/767: chown d7/d10/d46/d4a/fe6 9822619 1 2024-03-13T14:49:59.190 INFO:tasks.workunit.client.0.smithi176.stdout:6/763: write d0/de/d38/d7e/f84 [7797094,62817] 0 2024-03-13T14:49:59.190 INFO:tasks.workunit.client.0.smithi176.stdout:5/718: dwrite d5/d2c/d40/dc7/d89/fa1 [0,4194304] 0 2024-03-13T14:49:59.190 INFO:tasks.workunit.client.0.smithi176.stdout:9/860: fdatasync d0/d1e/d8d/dbe/f10e 0 2024-03-13T14:49:59.190 INFO:tasks.workunit.client.0.smithi176.stdout:4/768: read - d7/ffb zero size 2024-03-13T14:49:59.192 INFO:tasks.workunit.client.0.smithi176.stdout:6/764: dread - d0/fe8 zero size 2024-03-13T14:49:59.198 INFO:tasks.workunit.client.0.smithi176.stdout:1/736: chown d2/de/d3f/d50/f78 7726505 1 2024-03-13T14:49:59.201 INFO:tasks.workunit.client.1.smithi196.stdout:2/777: dread d0/d7/d40/f60 [0,4194304] 0 2024-03-13T14:49:59.204 INFO:tasks.workunit.client.0.smithi176.stdout:2/779: creat d1/dc/d2d/d90/f106 x:0 0 0 2024-03-13T14:49:59.207 INFO:tasks.workunit.client.0.smithi176.stdout:0/663: sync 2024-03-13T14:49:59.207 INFO:tasks.workunit.client.1.smithi196.stdout:8/792: rename d0/d16/d3d/d8b/f4a to d0/d10/d70/f10d 0 2024-03-13T14:49:59.210 INFO:tasks.workunit.client.1.smithi196.stdout:7/956: truncate d0/d6/faa 1925662 0 2024-03-13T14:49:59.210 INFO:tasks.workunit.client.1.smithi196.stdout:0/860: write d8/da/d25/d26/d35/d3b/d5f/d33/f34 [2732135,115140] 0 2024-03-13T14:49:59.211 INFO:tasks.workunit.client.0.smithi176.stdout:5/719: creat d5/d2c/d40/d76/ff4 x:0 0 0 2024-03-13T14:49:59.212 INFO:tasks.workunit.client.0.smithi176.stdout:6/765: symlink d0/d10/d19/d7d/lf6 0 2024-03-13T14:49:59.212 INFO:tasks.workunit.client.0.smithi176.stdout:5/720: chown d5/d2c/d74/ce5 3698 1 2024-03-13T14:49:59.213 INFO:tasks.workunit.client.0.smithi176.stdout:6/766: stat d0/de/d38/d88/fc3 0 2024-03-13T14:49:59.214 INFO:tasks.workunit.client.1.smithi196.stdout:3/836: link db/d19/f28 db/d19/d22/d3a/d53/f10c 0 2024-03-13T14:49:59.214 INFO:tasks.workunit.client.0.smithi176.stdout:5/721: chown d5/d2c/d40/dc7/d89 11390908 1 2024-03-13T14:49:59.217 INFO:tasks.workunit.client.0.smithi176.stdout:3/803: rename d4/d13/d5e/daa to d4/d13/d16/d113 0 2024-03-13T14:49:59.217 INFO:tasks.workunit.client.0.smithi176.stdout:1/737: symlink d2/de/d3f/d50/d83/dcb/le6 0 2024-03-13T14:49:59.224 INFO:tasks.workunit.client.1.smithi196.stdout:8/793: unlink d0/d16/d3d/d8b/lf0 0 2024-03-13T14:49:59.225 INFO:tasks.workunit.client.0.smithi176.stdout:8/680: getdents d0/d24/d29/d70 0 2024-03-13T14:49:59.226 INFO:tasks.workunit.client.1.smithi196.stdout:2/778: dwrite d0/d6/d1d/f27 [0,4194304] 0 2024-03-13T14:49:59.227 INFO:tasks.workunit.client.0.smithi176.stdout:0/664: truncate d2/db/d31/f44 4915034 0 2024-03-13T14:49:59.229 INFO:tasks.workunit.client.0.smithi176.stdout:5/722: dwrite d5/d2e/d5f/fcb [0,4194304] 0 2024-03-13T14:49:59.231 INFO:tasks.workunit.client.0.smithi176.stdout:9/861: symlink d0/db/d89/d4d/d6e/d100/d110/l121 0 2024-03-13T14:49:59.234 INFO:tasks.workunit.client.1.smithi196.stdout:0/861: mknod d8/da/d25/d26/d48/d82/db3/dd5/c11c 0 2024-03-13T14:49:59.236 INFO:tasks.workunit.client.1.smithi196.stdout:4/821: creat d1/d3/d7/ddf/f122 x:0 0 0 2024-03-13T14:49:59.237 INFO:tasks.workunit.client.0.smithi176.stdout:6/767: creat d0/d10/d19/d27/d2b/d2e/d3e/db4/ff7 x:0 0 0 2024-03-13T14:49:59.238 INFO:tasks.workunit.client.1.smithi196.stdout:9/829: dread d7/de/d74/f85 [0,4194304] 0 2024-03-13T14:49:59.239 INFO:tasks.workunit.client.1.smithi196.stdout:7/957: symlink d0/d9/d1e/d58/d72/da9/l14b 0 2024-03-13T14:49:59.240 INFO:tasks.workunit.client.0.smithi176.stdout:3/804: symlink d4/d13/d81/d89/d5b/dc5/df0/d100/l114 0 2024-03-13T14:49:59.240 INFO:tasks.workunit.client.0.smithi176.stdout:6/768: stat d0/de/d38/d88/d9d/lb0 0 2024-03-13T14:49:59.241 INFO:tasks.workunit.client.1.smithi196.stdout:2/779: unlink d0/d7/d40/c6e 0 2024-03-13T14:49:59.244 INFO:tasks.workunit.client.1.smithi196.stdout:0/862: stat d8/da/d25/d26/d35/d3b/d5f/l4b 0 2024-03-13T14:49:59.245 INFO:tasks.workunit.client.0.smithi176.stdout:4/769: dread d7/d8/d15/d23/fa8 [0,4194304] 0 2024-03-13T14:49:59.246 INFO:tasks.workunit.client.1.smithi196.stdout:2/780: stat d0/d7/d9/d61/dc7/lcb 0 2024-03-13T14:49:59.247 INFO:tasks.workunit.client.0.smithi176.stdout:8/681: rename d0/d5/fa5 to d0/db/fdc 0 2024-03-13T14:49:59.250 INFO:tasks.workunit.client.0.smithi176.stdout:0/665: mkdir d2/db/d39/d9c/dd5 0 2024-03-13T14:49:59.253 INFO:tasks.workunit.client.0.smithi176.stdout:7/711: getdents d1/d3f/d4b/d63/d7a 0 2024-03-13T14:49:59.253 INFO:tasks.workunit.client.0.smithi176.stdout:9/862: write d0/db/d89/d90/fe5 [1166204,85832] 0 2024-03-13T14:49:59.256 INFO:tasks.workunit.client.1.smithi196.stdout:9/830: creat d7/de/d4b/dd1/f11d x:0 0 0 2024-03-13T14:49:59.257 INFO:tasks.workunit.client.0.smithi176.stdout:9/863: dread - d0/db/d89/d4d/dd5/f105 zero size 2024-03-13T14:49:59.259 INFO:tasks.workunit.client.0.smithi176.stdout:3/805: dwrite d4/d13/f73 [4194304,4194304] 0 2024-03-13T14:49:59.259 INFO:tasks.workunit.client.0.smithi176.stdout:2/780: getdents d1/dc/d61 0 2024-03-13T14:49:59.259 INFO:tasks.workunit.client.1.smithi196.stdout:7/958: mkdir d0/d78/d14c 0 2024-03-13T14:49:59.260 INFO:tasks.workunit.client.0.smithi176.stdout:7/712: dread - d1/d28/d3a/d1b/fd9 zero size 2024-03-13T14:49:59.261 INFO:tasks.workunit.client.0.smithi176.stdout:1/738: truncate d2/de/d10/d16/d47/f5c 990944 0 2024-03-13T14:49:59.264 INFO:tasks.workunit.client.0.smithi176.stdout:1/739: chown d2/d30/dd3/ce4 11 1 2024-03-13T14:49:59.267 INFO:tasks.workunit.client.0.smithi176.stdout:0/666: fsync d2/db/f25 0 2024-03-13T14:49:59.270 INFO:tasks.workunit.client.0.smithi176.stdout:3/806: unlink d4/d28/d4f/f53 0 2024-03-13T14:49:59.270 INFO:tasks.workunit.client.0.smithi176.stdout:0/667: write d2/d74/d9f/fbf [519988,79445] 0 2024-03-13T14:49:59.272 INFO:tasks.workunit.client.0.smithi176.stdout:8/682: link d0/d5/d11/cba d0/d24/d29/dce/cdd 0 2024-03-13T14:49:59.275 INFO:tasks.workunit.client.1.smithi196.stdout:3/837: dread db/f13 [0,4194304] 0 2024-03-13T14:49:59.277 INFO:tasks.workunit.client.0.smithi176.stdout:2/781: rename d1/dc/d2d/d4f/df1 to d1/dc/d1c/d107 0 2024-03-13T14:49:59.277 INFO:tasks.workunit.client.0.smithi176.stdout:6/769: getdents d0/d10/d19/d27/d2b/d2e/db7 0 2024-03-13T14:49:59.280 INFO:tasks.workunit.client.1.smithi196.stdout:0/863: dwrite d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 [0,4194304] 0 2024-03-13T14:49:59.292 INFO:tasks.workunit.client.1.smithi196.stdout:0/864: symlink d8/da/d25/d26/d35/d3b/d59/l11d 0 2024-03-13T14:49:59.295 INFO:tasks.workunit.client.0.smithi176.stdout:2/782: mknod d1/dc/d2d/d59/c108 0 2024-03-13T14:49:59.296 INFO:tasks.workunit.client.0.smithi176.stdout:4/770: write d7/d8/d15/f7b [64627,69789] 0 2024-03-13T14:49:59.299 INFO:tasks.workunit.client.1.smithi196.stdout:0/865: chown d8/da/d25/d26/d35/d3b/d5f/d17/fcc 239375456 1 2024-03-13T14:49:59.302 INFO:tasks.workunit.client.0.smithi176.stdout:3/807: fsync d4/d13/d81/d89/f9c 0 2024-03-13T14:49:59.303 INFO:tasks.workunit.client.1.smithi196.stdout:8/794: dwrite d0/d16/d3d/d8b/f91 [0,4194304] 0 2024-03-13T14:49:59.305 INFO:tasks.workunit.client.1.smithi196.stdout:4/822: dwrite d1/d3/d7/f1a [0,4194304] 0 2024-03-13T14:49:59.318 INFO:tasks.workunit.client.1.smithi196.stdout:8/795: dread d0/d63/f85 [0,4194304] 0 2024-03-13T14:49:59.319 INFO:tasks.workunit.client.0.smithi176.stdout:9/864: truncate d0/db/d89/d90/fe5 1718507 0 2024-03-13T14:49:59.321 INFO:tasks.workunit.client.1.smithi196.stdout:8/796: readlink d0/d10/d12/df5/l4e 0 2024-03-13T14:49:59.323 INFO:tasks.workunit.client.1.smithi196.stdout:9/831: dwrite d7/de/d4b/d1d/d22/f5a [0,4194304] 0 2024-03-13T14:49:59.324 INFO:tasks.workunit.client.1.smithi196.stdout:7/959: dwrite d0/d9/f24 [0,4194304] 0 2024-03-13T14:49:59.326 INFO:tasks.workunit.client.0.smithi176.stdout:7/713: dwrite d1/d1c/d26/d41/f5b [0,4194304] 0 2024-03-13T14:49:59.330 INFO:tasks.workunit.client.0.smithi176.stdout:3/808: symlink d4/d13/d16/d80/l115 0 2024-03-13T14:49:59.330 INFO:tasks.workunit.client.0.smithi176.stdout:0/668: truncate d2/db/d39/fa2 3035183 0 2024-03-13T14:49:59.330 INFO:tasks.workunit.client.0.smithi176.stdout:1/740: write d2/de/d10/d16/d4a/d5f/f81 [286090,378] 0 2024-03-13T14:49:59.336 INFO:tasks.workunit.client.1.smithi196.stdout:8/797: write d0/d10/d12/d1e/dc5/ff2 [803725,25964] 0 2024-03-13T14:49:59.336 INFO:tasks.workunit.client.1.smithi196.stdout:9/832: read d7/de/f87 [1162229,55002] 0 2024-03-13T14:49:59.337 INFO:tasks.workunit.client.0.smithi176.stdout:6/770: creat d0/de/d38/d6b/ff8 x:0 0 0 2024-03-13T14:49:59.337 INFO:tasks.workunit.client.1.smithi196.stdout:4/823: rename d1/d3/d7/d35/d76/d58/f87 to d1/d3/d7/d35/d76/f123 0 2024-03-13T14:49:59.340 INFO:tasks.workunit.client.0.smithi176.stdout:8/683: dwrite d0/db/d39/d56/f5e [0,4194304] 0 2024-03-13T14:49:59.342 INFO:tasks.workunit.client.1.smithi196.stdout:9/833: chown d7/de/d74 2385 1 2024-03-13T14:49:59.343 INFO:tasks.workunit.client.0.smithi176.stdout:4/771: dread d7/d10/fcb [0,4194304] 0 2024-03-13T14:49:59.345 INFO:tasks.workunit.client.0.smithi176.stdout:2/783: rename d1/dc/f2e to d1/dc/d2d/d59/d8b/f109 0 2024-03-13T14:49:59.348 INFO:tasks.workunit.client.0.smithi176.stdout:7/714: getdents d1/d3f/d4b/d63/d7a 0 2024-03-13T14:49:59.348 INFO:tasks.workunit.client.1.smithi196.stdout:4/824: dread - d1/d3/d4c/d72/daf/f119 zero size 2024-03-13T14:49:59.351 INFO:tasks.workunit.client.0.smithi176.stdout:5/723: sync 2024-03-13T14:49:59.352 INFO:tasks.workunit.client.0.smithi176.stdout:1/741: read d2/de/d3f/d3e/f5b [793724,56068] 0 2024-03-13T14:49:59.353 INFO:tasks.workunit.client.1.smithi196.stdout:4/825: dread - d1/d1e/d43/f85 zero size 2024-03-13T14:49:59.355 INFO:tasks.workunit.client.1.smithi196.stdout:0/866: dwrite d8/da/d25/d26/d35/d3b/d5f/d17/f4d [0,4194304] 0 2024-03-13T14:49:59.357 INFO:tasks.workunit.client.0.smithi176.stdout:3/809: dread d4/d28/d4f/ffd [0,4194304] 0 2024-03-13T14:49:59.358 INFO:tasks.workunit.client.1.smithi196.stdout:4/826: readlink d1/d3/d4c/d72/d7d/d80/lec 0 2024-03-13T14:49:59.358 INFO:tasks.workunit.client.0.smithi176.stdout:3/810: dread - d4/d13/d16/dad/fe6 zero size 2024-03-13T14:49:59.360 INFO:tasks.workunit.client.1.smithi196.stdout:9/834: mknod d7/de/d74/d94/dfb/d28/d6c/c11e 0 2024-03-13T14:49:59.361 INFO:tasks.workunit.client.0.smithi176.stdout:2/784: dread - d1/dc/d2d/d4b/ddb/fdc zero size 2024-03-13T14:49:59.363 INFO:tasks.workunit.client.1.smithi196.stdout:9/835: chown d7/de/d4b/d1d/d93/dd7 353 1 2024-03-13T14:49:59.364 INFO:tasks.workunit.client.1.smithi196.stdout:0/867: stat d8/da/d25/d26/d35/d3b/d59 0 2024-03-13T14:49:59.367 INFO:tasks.workunit.client.0.smithi176.stdout:9/865: rename d0/d1e/d1f to d0/db/d89/d32/d122 0 2024-03-13T14:49:59.372 INFO:tasks.workunit.client.1.smithi196.stdout:8/798: dwrite d0/d10/d12/d1e/fe5 [0,4194304] 0 2024-03-13T14:49:59.373 INFO:tasks.workunit.client.0.smithi176.stdout:7/715: mkdir d1/d1c/d4a/dda 0 2024-03-13T14:49:59.377 INFO:tasks.workunit.client.1.smithi196.stdout:9/836: mknod d7/de/d4b/d1d/d35/c11f 0 2024-03-13T14:49:59.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:58 smithi176 ceph-mon[29064]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:49:59.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:58 smithi176 ceph-mon[29064]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:49:59.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:58 smithi176 ceph-mon[29064]: pgmap v18: 65 pgs: 65 active+clean; 2.8 GiB data, 8.5 GiB used, 528 GiB / 536 GiB avail; 71 MiB/s rd, 178 MiB/s wr, 376 op/s 2024-03-13T14:49:59.380 INFO:tasks.workunit.client.1.smithi196.stdout:9/837: write d7/de/d16/d70/f7a [1540190,92454] 0 2024-03-13T14:49:59.383 INFO:tasks.workunit.client.1.smithi196.stdout:4/827: mknod d1/d3/d7/d35/d76/d58/d69/de3/df4/c124 0 2024-03-13T14:49:59.384 INFO:tasks.workunit.client.0.smithi176.stdout:2/785: mknod d1/dc/d2d/d59/d80/d7e/da3/c10a 0 2024-03-13T14:49:59.386 INFO:tasks.workunit.client.1.smithi196.stdout:0/868: symlink d8/da/d25/d26/d48/d82/l11e 0 2024-03-13T14:49:59.388 INFO:tasks.workunit.client.0.smithi176.stdout:0/669: write d2/d5/d1e/f72 [570569,20899] 0 2024-03-13T14:49:59.389 INFO:tasks.workunit.client.0.smithi176.stdout:0/670: stat d2 0 2024-03-13T14:49:59.390 INFO:tasks.workunit.client.1.smithi196.stdout:8/799: creat d0/d10/d12/d1e/d2d/f10e x:0 0 0 2024-03-13T14:49:59.394 INFO:tasks.workunit.client.1.smithi196.stdout:9/838: rename d7/d71/c8e to d7/de/de5/c120 0 2024-03-13T14:49:59.396 INFO:tasks.workunit.client.0.smithi176.stdout:2/786: dread d1/dc/d1c/f37 [0,4194304] 0 2024-03-13T14:49:59.399 INFO:tasks.workunit.client.1.smithi196.stdout:0/869: symlink d8/da/d25/d26/d35/d3b/d50/d55/l11f 0 2024-03-13T14:49:59.403 INFO:tasks.workunit.client.1.smithi196.stdout:9/839: symlink d7/de/d74/d94/dda/l121 0 2024-03-13T14:49:59.403 INFO:tasks.workunit.client.0.smithi176.stdout:4/772: write d7/d8/d15/d23/fa8 [1002727,102141] 0 2024-03-13T14:49:59.405 INFO:tasks.workunit.client.0.smithi176.stdout:5/724: creat d5/d2e/d7e/ff5 x:0 0 0 2024-03-13T14:49:59.407 INFO:tasks.workunit.client.0.smithi176.stdout:4/773: read d7/d8/f39 [2916670,130518] 0 2024-03-13T14:49:59.407 INFO:tasks.workunit.client.1.smithi196.stdout:7/960: dread d0/d9/d1e/d58/d72/da9/da3/fad [0,4194304] 0 2024-03-13T14:49:59.408 INFO:tasks.workunit.client.0.smithi176.stdout:0/671: symlink d2/d5/d1e/d97/dcf/ld6 0 2024-03-13T14:49:59.413 INFO:tasks.workunit.client.0.smithi176.stdout:3/811: rename d4/d13/f48 to d4/d13/d81/d107/f116 0 2024-03-13T14:49:59.416 INFO:tasks.workunit.client.1.smithi196.stdout:9/840: read - d7/de/d4b/d1d/fca zero size 2024-03-13T14:49:59.419 INFO:tasks.workunit.client.1.smithi196.stdout:0/870: dwrite d8/da/d25/d26/d54/d88/f45 [0,4194304] 0 2024-03-13T14:49:59.419 INFO:tasks.workunit.client.0.smithi176.stdout:8/684: getdents d0/d5/d11/d47/db7 0 2024-03-13T14:49:59.419 INFO:tasks.workunit.client.1.smithi196.stdout:8/800: link d0/d10/d12/d1e/f5e d0/d10/d12/d17/f10f 0 2024-03-13T14:49:59.420 INFO:tasks.workunit.client.0.smithi176.stdout:8/685: chown d0/db/cd6 123962014 1 2024-03-13T14:49:59.422 INFO:tasks.workunit.client.1.smithi196.stdout:8/801: write d0/d10/d12/d1e/fbb [3365873,17009] 0 2024-03-13T14:49:59.425 INFO:tasks.workunit.client.0.smithi176.stdout:2/787: write d1/dc/d2d/f2f [3220537,48514] 0 2024-03-13T14:49:59.425 INFO:tasks.workunit.client.1.smithi196.stdout:7/961: symlink d0/d6/d140/d93/d94/l14d 0 2024-03-13T14:49:59.426 INFO:tasks.workunit.client.1.smithi196.stdout:4/828: dread d1/d1e/f7f [0,4194304] 0 2024-03-13T14:49:59.427 INFO:tasks.workunit.client.0.smithi176.stdout:1/742: dwrite d2/de/d3f/d50/f8e [0,4194304] 0 2024-03-13T14:49:59.427 INFO:tasks.workunit.client.1.smithi196.stdout:9/841: mkdir d7/de/d4b/d1d/d93/dd7/d122 0 2024-03-13T14:49:59.431 INFO:tasks.workunit.client.0.smithi176.stdout:7/716: mkdir d1/d28/d3a/d1b/d8f/dc1/ddb 0 2024-03-13T14:49:59.432 INFO:tasks.workunit.client.1.smithi196.stdout:0/871: mkdir d8/da/d25/d26/d48/d82/db3/dd5/d120 0 2024-03-13T14:49:59.434 INFO:tasks.workunit.client.0.smithi176.stdout:0/672: dread d2/d5/d1e/f4e [0,4194304] 0 2024-03-13T14:49:59.438 INFO:tasks.workunit.client.1.smithi196.stdout:7/962: creat d0/d6/d33/d57/f14e x:0 0 0 2024-03-13T14:49:59.439 INFO:tasks.workunit.client.0.smithi176.stdout:4/774: fdatasync d7/d8/d15/d23/f8b 0 2024-03-13T14:49:59.442 INFO:tasks.workunit.client.0.smithi176.stdout:8/686: dwrite d0/db/d39/d56/d93/da7/fbf [0,4194304] 0 2024-03-13T14:49:59.443 INFO:tasks.workunit.client.0.smithi176.stdout:4/775: chown d7/d8/d33/fd2 1977606 1 2024-03-13T14:49:59.447 INFO:tasks.workunit.client.1.smithi196.stdout:9/842: dread d7/de/d74/d94/dfb/dbf/f90 [0,4194304] 0 2024-03-13T14:49:59.449 INFO:tasks.workunit.client.1.smithi196.stdout:7/963: truncate d0/d4d/fc4 40959 0 2024-03-13T14:49:59.450 INFO:tasks.workunit.client.0.smithi176.stdout:6/771: sync 2024-03-13T14:49:59.452 INFO:tasks.workunit.client.0.smithi176.stdout:2/788: symlink d1/dc/d61/d9e/l10b 0 2024-03-13T14:49:59.453 INFO:tasks.workunit.client.1.smithi196.stdout:0/872: dwrite d8/da/d25/d26/d48/d82/db3/ffe [0,4194304] 0 2024-03-13T14:49:59.455 INFO:tasks.workunit.client.0.smithi176.stdout:1/743: rename d2/de/d10/d16/d4a/d5f/da4/dd2 to d2/d30/d63/d22/de7 0 2024-03-13T14:49:59.455 INFO:tasks.workunit.client.0.smithi176.stdout:5/725: write d5/d22/f1e [1599118,16320] 0 2024-03-13T14:49:59.456 INFO:tasks.workunit.client.0.smithi176.stdout:0/673: rmdir d2/d5/d1e/d97/dcf 39 2024-03-13T14:49:59.456 INFO:tasks.workunit.client.1.smithi196.stdout:4/829: link d1/d3/d7/c88 d1/d3/d7/c125 0 2024-03-13T14:49:59.456 INFO:tasks.workunit.client.1.smithi196.stdout:9/843: read d7/de/d4b/d1d/f54 [1080314,88809] 0 2024-03-13T14:49:59.461 INFO:tasks.workunit.client.0.smithi176.stdout:3/812: creat d4/d28/d34/dde/f117 x:0 0 0 2024-03-13T14:49:59.461 INFO:tasks.workunit.client.1.smithi196.stdout:0/873: chown d8/da/d25/d26/d35/d3b/d5f/d33/d44 163240 1 2024-03-13T14:49:59.462 INFO:tasks.workunit.client.0.smithi176.stdout:4/776: mknod d7/d10/d2c/df1/c102 0 2024-03-13T14:49:59.462 INFO:tasks.workunit.client.0.smithi176.stdout:5/726: dread - d5/d2e/d7e/ff5 zero size 2024-03-13T14:49:59.464 INFO:tasks.workunit.client.0.smithi176.stdout:0/674: write d2/d5/fd1 [171761,52851] 0 2024-03-13T14:49:59.464 INFO:tasks.workunit.client.0.smithi176.stdout:6/772: rmdir d0/d10/dca 39 2024-03-13T14:49:59.464 INFO:tasks.workunit.client.1.smithi196.stdout:0/874: write d8/ff9 [3524988,32284] 0 2024-03-13T14:49:59.465 INFO:tasks.workunit.client.1.smithi196.stdout:8/802: dwrite d0/f7e [4194304,4194304] 0 2024-03-13T14:49:59.465 INFO:tasks.workunit.client.0.smithi176.stdout:2/789: rename d1/d2a/d33/l47 to d1/dc/d1c/d107/l10c 0 2024-03-13T14:49:59.467 INFO:tasks.workunit.client.1.smithi196.stdout:9/844: mknod d7/de/d4b/dd1/c123 0 2024-03-13T14:49:59.470 INFO:tasks.workunit.client.1.smithi196.stdout:4/830: mknod d1/d3/d7/d35/d76/d58/c126 0 2024-03-13T14:49:59.471 INFO:tasks.workunit.client.0.smithi176.stdout:6/773: mknod d0/d10/d19/d7d/cf9 0 2024-03-13T14:49:59.474 INFO:tasks.workunit.client.0.smithi176.stdout:9/866: dwrite d0/d40/d53/fba [0,4194304] 0 2024-03-13T14:49:59.477 INFO:tasks.workunit.client.0.smithi176.stdout:1/744: truncate d2/de/d3f/d19/f32 3868471 0 2024-03-13T14:49:59.480 INFO:tasks.workunit.client.0.smithi176.stdout:9/867: chown d0/db/def/db8/lcb 1 1 2024-03-13T14:49:59.481 INFO:tasks.workunit.client.1.smithi196.stdout:8/803: rename d0/d16/d3d/d8b/d3f to d0/d10/da5/d110 0 2024-03-13T14:49:59.484 INFO:tasks.workunit.client.0.smithi176.stdout:1/745: write d2/de/f17 [5290570,84517] 0 2024-03-13T14:49:59.486 INFO:tasks.workunit.client.1.smithi196.stdout:7/964: write d0/d4d/d74/fd9 [851271,119875] 0 2024-03-13T14:49:59.492 INFO:tasks.workunit.client.1.smithi196.stdout:8/804: rmdir d0/d16/d71 39 2024-03-13T14:49:59.493 INFO:tasks.workunit.client.1.smithi196.stdout:4/831: read d1/d3/d4c/fa5 [3639322,6333] 0 2024-03-13T14:49:59.494 INFO:tasks.workunit.client.0.smithi176.stdout:3/813: creat d4/d13/d81/d89/d5b/dc5/df0/d100/f118 x:0 0 0 2024-03-13T14:49:59.495 INFO:tasks.workunit.client.1.smithi196.stdout:9/845: creat d7/de/d74/d94/dfb/d28/de4/d10a/f124 x:0 0 0 2024-03-13T14:49:59.496 INFO:tasks.workunit.client.0.smithi176.stdout:8/687: truncate d0/db/d39/d56/d93/da7/fbf 2783003 0 2024-03-13T14:49:59.504 INFO:tasks.workunit.client.0.smithi176.stdout:9/868: truncate d0/db/f3f 578350 0 2024-03-13T14:49:59.506 INFO:tasks.workunit.client.1.smithi196.stdout:9/846: rmdir d7 39 2024-03-13T14:49:59.507 INFO:tasks.workunit.client.0.smithi176.stdout:7/717: dread d1/d3f/d65/f97 [0,4194304] 0 2024-03-13T14:49:59.507 INFO:tasks.workunit.client.1.smithi196.stdout:1/917: sync 2024-03-13T14:49:59.507 INFO:tasks.workunit.client.1.smithi196.stdout:5/877: sync 2024-03-13T14:49:59.507 INFO:tasks.workunit.client.0.smithi176.stdout:1/746: rmdir d2/de/d10/d16/d4a/d5f 39 2024-03-13T14:49:59.508 INFO:tasks.workunit.client.0.smithi176.stdout:7/718: dread - d1/d28/d3a/d1b/d8f/fac zero size 2024-03-13T14:49:59.510 INFO:tasks.workunit.client.0.smithi176.stdout:1/747: write d2/d30/d63/d22/f82 [2010382,73132] 0 2024-03-13T14:49:59.514 INFO:tasks.workunit.client.0.smithi176.stdout:8/688: rename d0/d24/d29/d70/l84 to d0/d24/d98/dc6/lde 0 2024-03-13T14:49:59.515 INFO:tasks.workunit.client.1.smithi196.stdout:9/847: chown d7/le2 1307566 1 2024-03-13T14:49:59.515 INFO:tasks.workunit.client.0.smithi176.stdout:0/675: write d2/d5/d28/f5e [4133890,92588] 0 2024-03-13T14:49:59.518 INFO:tasks.workunit.client.0.smithi176.stdout:6/774: creat d0/d10/d19/d27/d2b/de0/ffa x:0 0 0 2024-03-13T14:49:59.523 INFO:tasks.workunit.client.1.smithi196.stdout:4/832: rename d1/d3/f98 to d1/d3/d7/ddf/f127 0 2024-03-13T14:49:59.525 INFO:tasks.workunit.client.1.smithi196.stdout:9/848: write d7/de/d74/d94/dfb/dbf/f100 [1907113,15758] 0 2024-03-13T14:49:59.525 INFO:tasks.workunit.client.1.smithi196.stdout:4/833: chown d1/d1e/f7f 13145 1 2024-03-13T14:49:59.526 INFO:tasks.workunit.client.0.smithi176.stdout:3/814: symlink d4/d13/l119 0 2024-03-13T14:49:59.526 INFO:tasks.workunit.client.0.smithi176.stdout:4/777: dread d7/fe [0,4194304] 0 2024-03-13T14:49:59.530 INFO:tasks.workunit.client.1.smithi196.stdout:0/875: truncate d8/da/d25/d26/d35/d3b/d6c/fc6 3901655 0 2024-03-13T14:49:59.533 INFO:tasks.workunit.client.0.smithi176.stdout:5/727: truncate d5/d31/f45 1438378 0 2024-03-13T14:49:59.538 INFO:tasks.workunit.client.1.smithi196.stdout:8/805: dread d0/d10/d12/df5/f3c [0,4194304] 0 2024-03-13T14:49:59.538 INFO:tasks.workunit.client.1.smithi196.stdout:1/918: dwrite d1/d9/d33/d4b/d66/d7b/fb4 [0,4194304] 0 2024-03-13T14:49:59.538 INFO:tasks.workunit.client.1.smithi196.stdout:8/806: dread - d0/d63/f109 zero size 2024-03-13T14:49:59.542 INFO:tasks.workunit.client.1.smithi196.stdout:9/849: link d7/de/d16/d70/feb d7/de/d74/d94/d11b/f125 0 2024-03-13T14:49:59.547 INFO:tasks.workunit.client.1.smithi196.stdout:1/919: symlink d1/d9/d33/d4b/d66/l13d 0 2024-03-13T14:49:59.552 INFO:tasks.workunit.client.1.smithi196.stdout:5/878: dwrite d4/d1b/dc8/d4a/fa3 [0,4194304] 0 2024-03-13T14:49:59.555 INFO:tasks.workunit.client.0.smithi176.stdout:6/775: dread d0/d10/d19/d27/d2b/de0/f98 [0,4194304] 0 2024-03-13T14:49:59.556 INFO:tasks.workunit.client.1.smithi196.stdout:9/850: creat d7/de/d4b/d1d/d22/f126 x:0 0 0 2024-03-13T14:49:59.559 INFO:tasks.workunit.client.0.smithi176.stdout:9/869: dread d0/d1e/f4a [0,4194304] 0 2024-03-13T14:49:59.561 INFO:tasks.workunit.client.1.smithi196.stdout:4/834: getdents d1/dfb/d11c 0 2024-03-13T14:49:59.565 INFO:tasks.workunit.client.1.smithi196.stdout:1/920: creat d1/d126/f13e x:0 0 0 2024-03-13T14:49:59.569 INFO:tasks.workunit.client.1.smithi196.stdout:1/921: dread - d1/d9/d4e/d55/d63/db9/dbb/fce zero size 2024-03-13T14:49:59.569 INFO:tasks.workunit.client.1.smithi196.stdout:0/876: dwrite d8/da/d25/d26/f2d [0,4194304] 0 2024-03-13T14:49:59.575 INFO:tasks.workunit.client.1.smithi196.stdout:4/835: rmdir d1/d3/d7/ddf 39 2024-03-13T14:49:59.576 INFO:tasks.workunit.client.0.smithi176.stdout:2/790: sync 2024-03-13T14:49:59.577 INFO:tasks.workunit.client.1.smithi196.stdout:8/807: rename d0/cd to d0/d1c/c111 0 2024-03-13T14:49:59.580 INFO:tasks.workunit.client.1.smithi196.stdout:5/879: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/f42 1492278 0 2024-03-13T14:49:59.590 INFO:tasks.workunit.client.1.smithi196.stdout:5/880: dread d4/d1b/dc8/d14/d19/f96 [0,4194304] 0 2024-03-13T14:49:59.592 INFO:tasks.workunit.client.1.smithi196.stdout:1/922: rename d1/d9/ca4 to d1/d28/d34/d100/c13f 0 2024-03-13T14:49:59.595 INFO:tasks.workunit.client.1.smithi196.stdout:5/881: write d4/d1b/dc8/d3c/f57 [3650786,50103] 0 2024-03-13T14:49:59.609 INFO:tasks.workunit.client.1.smithi196.stdout:9/851: dread d7/de/d16/f99 [0,4194304] 0 2024-03-13T14:49:59.611 INFO:tasks.workunit.client.1.smithi196.stdout:0/877: dread d8/da/d25/d26/d35/d3b/d5f/f7a [0,4194304] 0 2024-03-13T14:49:59.613 INFO:tasks.workunit.client.1.smithi196.stdout:1/923: creat d1/d28/d34/d67/d11f/f140 x:0 0 0 2024-03-13T14:49:59.616 INFO:tasks.workunit.client.1.smithi196.stdout:0/878: chown d8/c58 1 1 2024-03-13T14:49:59.616 INFO:tasks.workunit.client.1.smithi196.stdout:5/882: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/db2/f116 x:0 0 0 2024-03-13T14:49:59.621 INFO:tasks.workunit.client.1.smithi196.stdout:4/836: dwrite d1/dfb/d11c/f9c [0,4194304] 0 2024-03-13T14:49:59.628 INFO:tasks.workunit.client.1.smithi196.stdout:8/808: getdents d0/d10/d12/d1e/dc5 0 2024-03-13T14:49:59.629 INFO:tasks.workunit.client.1.smithi196.stdout:8/809: readlink d0/d63/dcb/ld3 0 2024-03-13T14:49:59.631 INFO:tasks.workunit.client.0.smithi176.stdout:1/748: mkdir d2/de/d10/de8 0 2024-03-13T14:49:59.634 INFO:tasks.workunit.client.1.smithi196.stdout:0/879: creat d8/da/d25/d26/d48/d82/da0/f121 x:0 0 0 2024-03-13T14:49:59.637 INFO:tasks.workunit.client.0.smithi176.stdout:8/689: truncate d0/d24/d98/dc6/d40/f77 873673 0 2024-03-13T14:49:59.644 INFO:tasks.workunit.client.0.smithi176.stdout:0/676: truncate d2/db/d31/f44 5253553 0 2024-03-13T14:49:59.647 INFO:tasks.workunit.client.1.smithi196.stdout:0/880: rename d8/da/d25/d26/d35/d3b/d5f/d33/fb2 to d8/da/d25/d26/d48/d82/da0/f122 0 2024-03-13T14:49:59.648 INFO:tasks.workunit.client.1.smithi196.stdout:9/852: dwrite d7/de/d4b/f7c [4194304,4194304] 0 2024-03-13T14:49:59.652 INFO:tasks.workunit.client.0.smithi176.stdout:6/776: mkdir d0/d10/d19/d27/d2b/d2e/dc4/dc5/dfb 0 2024-03-13T14:49:59.654 INFO:tasks.workunit.client.1.smithi196.stdout:8/810: symlink d0/d10/da5/de4/l112 0 2024-03-13T14:49:59.655 INFO:tasks.workunit.client.1.smithi196.stdout:4/837: write d1/d1e/d5c/f5f [1879122,49218] 0 2024-03-13T14:49:59.656 INFO:tasks.workunit.client.0.smithi176.stdout:6/777: write d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/fc2 [3215479,7763] 0 2024-03-13T14:49:59.658 INFO:tasks.workunit.client.0.smithi176.stdout:6/778: stat d0/de/d38/d6b/fd9 0 2024-03-13T14:49:59.658 INFO:tasks.workunit.client.0.smithi176.stdout:4/778: dread d7/d8/d2f/f58 [0,4194304] 0 2024-03-13T14:49:59.661 INFO:tasks.workunit.client.1.smithi196.stdout:4/838: stat d1/d3/d4c/d72/d7d/d80/d55/c11e 0 2024-03-13T14:49:59.666 INFO:tasks.workunit.client.1.smithi196.stdout:0/881: creat d8/da/d25/d26/d35/d3b/d5f/d33/f123 x:0 0 0 2024-03-13T14:49:59.667 INFO:tasks.workunit.client.1.smithi196.stdout:5/883: dwrite d4/d1b/dc8/d14/d19/fac [0,4194304] 0 2024-03-13T14:49:59.669 INFO:tasks.workunit.client.1.smithi196.stdout:4/839: chown d1/d1e/d43/f85 8 1 2024-03-13T14:49:59.669 INFO:tasks.workunit.client.0.smithi176.stdout:8/690: rmdir d0 39 2024-03-13T14:49:59.678 INFO:tasks.workunit.client.1.smithi196.stdout:1/924: dwrite d1/d3f/d5b/d74/f8f [0,4194304] 0 2024-03-13T14:49:59.679 INFO:tasks.workunit.client.0.smithi176.stdout:0/677: creat d2/db/d31/fd7 x:0 0 0 2024-03-13T14:49:59.680 INFO:tasks.workunit.client.1.smithi196.stdout:9/853: dwrite d7/de/d16/f9b [4194304,4194304] 0 2024-03-13T14:49:59.691 INFO:tasks.workunit.client.1.smithi196.stdout:5/884: rename d4/d1b/dc8/d14/d19/da7/c107 to d4/d1b/dc8/d14/d19/da7/d105/c117 0 2024-03-13T14:49:59.692 INFO:tasks.workunit.client.0.smithi176.stdout:2/791: link d1/dc/d2d/d4b/ddb/ff8 d1/dc/d2d/d59/f10d 0 2024-03-13T14:49:59.695 INFO:tasks.workunit.client.1.smithi196.stdout:0/882: unlink d8/da/d25/d26/d35/d3b/d50/d55/l8c 0 2024-03-13T14:49:59.701 INFO:tasks.workunit.client.1.smithi196.stdout:0/883: readlink d8/da/d25/d26/d35/d3b/d5f/l4b 0 2024-03-13T14:49:59.703 INFO:tasks.workunit.client.0.smithi176.stdout:4/779: dread - d7/d10/d46/d4a/dcc/dff/fb8 zero size 2024-03-13T14:49:59.705 INFO:tasks.workunit.client.1.smithi196.stdout:5/885: mkdir d4/d1b/dc8/d14/d19/d26/d95/d118 0 2024-03-13T14:49:59.705 INFO:tasks.workunit.client.0.smithi176.stdout:7/719: rename d1/d28/d3a/d1b/d8f/d91 to d1/d1c/d26/d41/d2b/ddc 0 2024-03-13T14:49:59.707 INFO:tasks.workunit.client.0.smithi176.stdout:5/728: creat d5/d22/ff6 x:0 0 0 2024-03-13T14:49:59.707 INFO:tasks.workunit.client.0.smithi176.stdout:3/815: link d4/d28/d31/d52/ca3 d4/d13/d81/d89/d5b/dfc/c11a 0 2024-03-13T14:49:59.709 INFO:tasks.workunit.client.1.smithi196.stdout:4/840: rename d1/d3/d7/d35/d76/d58/d69/de3/df4/c124 to d1/d3/d7/d35/d76/d58/c128 0 2024-03-13T14:49:59.710 INFO:tasks.workunit.client.0.smithi176.stdout:9/870: link d0/db/d89/l3d d0/db/d89/d32/d58/d5f/l123 0 2024-03-13T14:49:59.711 INFO:tasks.workunit.client.1.smithi196.stdout:4/841: dread - d1/dfb/d11c/f112 zero size 2024-03-13T14:49:59.711 INFO:tasks.workunit.client.0.smithi176.stdout:2/792: creat d1/dc/d2d/d59/d8b/daf/db2/f10e x:0 0 0 2024-03-13T14:49:59.712 INFO:tasks.workunit.client.0.smithi176.stdout:9/871: chown d0/db/d89/d32/d58/d5f/fc8 25856 1 2024-03-13T14:49:59.716 INFO:tasks.workunit.client.0.smithi176.stdout:1/749: rmdir d2/de/d3f/d3e/d86/db3/de5 0 2024-03-13T14:49:59.720 INFO:tasks.workunit.client.0.smithi176.stdout:2/793: dread d1/dc/d2d/d38/d5d/f62 [0,4194304] 0 2024-03-13T14:49:59.724 INFO:tasks.workunit.client.1.smithi196.stdout:8/811: dread d0/d10/d12/d17/f28 [0,4194304] 0 2024-03-13T14:49:59.725 INFO:tasks.workunit.client.1.smithi196.stdout:1/925: dwrite d1/d9/d33/d4b/f1b [0,4194304] 0 2024-03-13T14:49:59.725 INFO:tasks.workunit.client.0.smithi176.stdout:7/720: rename d1/d3f/f4d to d1/d1c/d26/d41/db6/fdd 0 2024-03-13T14:49:59.731 INFO:tasks.workunit.client.0.smithi176.stdout:5/729: mknod d5/d31/d86/cf7 0 2024-03-13T14:49:59.733 INFO:tasks.workunit.client.1.smithi196.stdout:4/842: creat d1/d3/d7/f129 x:0 0 0 2024-03-13T14:49:59.735 INFO:tasks.workunit.client.1.smithi196.stdout:5/886: dwrite d4/d1b/dc8/f81 [0,4194304] 0 2024-03-13T14:49:59.736 INFO:tasks.workunit.client.0.smithi176.stdout:3/816: creat d4/d28/d34/d38/f11b x:0 0 0 2024-03-13T14:49:59.738 INFO:tasks.workunit.client.0.smithi176.stdout:9/872: creat d0/d40/d53/d6d/f124 x:0 0 0 2024-03-13T14:49:59.739 INFO:tasks.workunit.client.0.smithi176.stdout:1/750: mknod d2/de/d3f/d6a/ce9 0 2024-03-13T14:49:59.739 INFO:tasks.workunit.client.0.smithi176.stdout:8/691: dwrite d0/db/f19 [0,4194304] 0 2024-03-13T14:49:59.739 INFO:tasks.workunit.client.1.smithi196.stdout:5/887: stat d4/d1b/dc8/d14/d19/d1c/d25/d38/cbe 0 2024-03-13T14:49:59.741 INFO:tasks.workunit.client.0.smithi176.stdout:1/751: chown d2/de/d10/d16/d4a 96686816 1 2024-03-13T14:49:59.743 INFO:tasks.workunit.client.1.smithi196.stdout:8/812: symlink d0/d16/l113 0 2024-03-13T14:49:59.744 INFO:tasks.workunit.client.0.smithi176.stdout:2/794: truncate d1/dc/d61/daa/fb8 826690 0 2024-03-13T14:49:59.744 INFO:tasks.workunit.client.0.smithi176.stdout:8/692: write d0/d24/d29/f67 [786501,80470] 0 2024-03-13T14:49:59.745 INFO:tasks.workunit.client.0.smithi176.stdout:7/721: mkdir d1/d3f/d76/da5/dde 0 2024-03-13T14:49:59.747 INFO:tasks.workunit.client.1.smithi196.stdout:8/813: creat d0/d16/d3d/d7f/f114 x:0 0 0 2024-03-13T14:49:59.753 INFO:tasks.workunit.client.0.smithi176.stdout:0/678: creat d2/d5/dc/fd8 x:0 0 0 2024-03-13T14:49:59.753 INFO:tasks.workunit.client.0.smithi176.stdout:9/873: mkdir d0/d1e/d8d/d125 0 2024-03-13T14:49:59.757 INFO:tasks.workunit.client.1.smithi196.stdout:4/843: link d1/d3/d7/d35/d76/d58/d69/d6c/ld3 d1/d3/d7/d35/d76/d58/d69/de3/l12a 0 2024-03-13T14:49:59.759 INFO:tasks.workunit.client.0.smithi176.stdout:6/779: getdents d0/d10/d19/d27/d2b/de0/dd6 0 2024-03-13T14:49:59.761 INFO:tasks.workunit.client.0.smithi176.stdout:2/795: fdatasync d1/d2a/f75 0 2024-03-13T14:49:59.763 INFO:tasks.workunit.client.1.smithi196.stdout:1/926: write d1/d9/d33/fae [173505,99597] 0 2024-03-13T14:49:59.763 INFO:tasks.workunit.client.0.smithi176.stdout:3/817: dwrite d4/d13/d81/d89/d5b/dc5/fa1 [0,4194304] 0 2024-03-13T14:49:59.766 INFO:tasks.workunit.client.0.smithi176.stdout:6/780: dread - d0/d10/d19/d27/d2b/d2e/f52 zero size 2024-03-13T14:49:59.767 INFO:tasks.workunit.client.1.smithi196.stdout:8/814: dwrite d0/d10/d12/d1e/fe5 [0,4194304] 0 2024-03-13T14:49:59.773 INFO:tasks.workunit.client.1.smithi196.stdout:4/844: fdatasync d1/d1e/d5c/fb5 0 2024-03-13T14:49:59.775 INFO:tasks.workunit.client.0.smithi176.stdout:7/722: mknod d1/d28/d3a/d1b/d8f/cdf 0 2024-03-13T14:49:59.777 INFO:tasks.workunit.client.1.smithi196.stdout:1/927: rename d1/d3f/d5b/d74/f109 to d1/d28/d34/d67/d11f/f141 0 2024-03-13T14:49:59.779 INFO:tasks.workunit.client.0.smithi176.stdout:5/730: rename d5/d22/ff6 to d5/d2c/ff8 0 2024-03-13T14:49:59.780 INFO:tasks.workunit.client.0.smithi176.stdout:5/731: stat d5/f63 0 2024-03-13T14:49:59.781 INFO:tasks.workunit.client.1.smithi196.stdout:1/928: chown d1/d9/d33/c78 192 1 2024-03-13T14:49:59.785 INFO:tasks.workunit.client.0.smithi176.stdout:6/781: creat d0/d10/d19/d27/d2b/d2e/d3e/db4/ffc x:0 0 0 2024-03-13T14:49:59.792 INFO:tasks.workunit.client.0.smithi176.stdout:9/874: rename d0/db/fd9 to d0/db/d89/d4d/f126 0 2024-03-13T14:49:59.795 INFO:tasks.workunit.client.0.smithi176.stdout:2/796: getdents d1/dc/d2d/d90/dcf/dea 0 2024-03-13T14:49:59.797 INFO:tasks.workunit.client.0.smithi176.stdout:1/752: write d2/de/d10/d16/d47/f5c [555687,54585] 0 2024-03-13T14:49:59.797 INFO:tasks.workunit.client.0.smithi176.stdout:8/693: link d0/d24/d29/f72 d0/fdf 0 2024-03-13T14:49:59.802 INFO:tasks.workunit.client.0.smithi176.stdout:1/753: dread d2/de/d10/f6d [0,4194304] 0 2024-03-13T14:49:59.805 INFO:tasks.workunit.client.0.smithi176.stdout:8/694: mkdir d0/d5/d11/d47/db7/de0 0 2024-03-13T14:49:59.805 INFO:tasks.workunit.client.0.smithi176.stdout:1/754: write d2/de/d3f/d3e/d86/fe0 [1886887,130143] 0 2024-03-13T14:49:59.808 INFO:tasks.workunit.client.0.smithi176.stdout:1/755: readlink d2/de/d3f/d50/d83/lc2 0 2024-03-13T14:49:59.809 INFO:tasks.workunit.client.0.smithi176.stdout:9/875: rename d0/db/d89/d7a/d8b/f8f to d0/d1e/f127 0 2024-03-13T14:49:59.810 INFO:tasks.workunit.client.0.smithi176.stdout:8/695: mknod d0/d24/d7b/ce1 0 2024-03-13T14:49:59.812 INFO:tasks.workunit.client.0.smithi176.stdout:5/732: write d5/f63 [2974831,101005] 0 2024-03-13T14:49:59.813 INFO:tasks.workunit.client.0.smithi176.stdout:6/782: rename d0/d10/d19/d27/d2b/d2e/d3e/d9a/df2 to d0/d10/d19/d27/d2b/de0/d76/d7c/dfd 0 2024-03-13T14:49:59.816 INFO:tasks.workunit.client.0.smithi176.stdout:3/818: dwrite d4/f1d [0,4194304] 0 2024-03-13T14:49:59.818 INFO:tasks.workunit.client.0.smithi176.stdout:5/733: mkdir d5/d31/d47/df9 0 2024-03-13T14:49:59.819 INFO:tasks.workunit.client.0.smithi176.stdout:1/756: read d2/d30/f3a [10733125,13587] 0 2024-03-13T14:49:59.819 INFO:tasks.workunit.client.0.smithi176.stdout:7/723: dwrite d1/d3f/d4b/d63/d7a/fa0 [0,4194304] 0 2024-03-13T14:49:59.822 INFO:tasks.workunit.client.0.smithi176.stdout:9/876: creat d0/db/d89/d32/d122/f128 x:0 0 0 2024-03-13T14:49:59.832 INFO:tasks.workunit.client.0.smithi176.stdout:5/734: write d5/d2c/d40/dc7/f6e [399185,81962] 0 2024-03-13T14:49:59.839 INFO:tasks.workunit.client.0.smithi176.stdout:2/797: dwrite d1/d2a/f75 [0,4194304] 0 2024-03-13T14:49:59.841 INFO:tasks.workunit.client.0.smithi176.stdout:3/819: rename d4/d28/d34/d38 to d4/d28/d34/d11c 0 2024-03-13T14:49:59.853 INFO:tasks.workunit.client.0.smithi176.stdout:6/783: dread d0/d10/d19/d27/d2b/de0/dd6/fcd [0,4194304] 0 2024-03-13T14:49:59.853 INFO:tasks.workunit.client.0.smithi176.stdout:3/820: creat d4/d28/d31/d45/d82/dce/f11d x:0 0 0 2024-03-13T14:49:59.859 INFO:tasks.workunit.client.0.smithi176.stdout:1/757: dwrite d2/d30/f3a [0,4194304] 0 2024-03-13T14:49:59.859 INFO:tasks.workunit.client.0.smithi176.stdout:5/735: write d5/d2e/d7e/dad/d7b/fc8 [449438,10507] 0 2024-03-13T14:49:59.859 INFO:tasks.workunit.client.0.smithi176.stdout:7/724: truncate d1/d1c/d26/d41/f5b 3269138 0 2024-03-13T14:49:59.862 INFO:tasks.workunit.client.0.smithi176.stdout:6/784: read - d0/d10/d19/d27/d2b/db1/fd3 zero size 2024-03-13T14:49:59.868 INFO:tasks.workunit.client.0.smithi176.stdout:7/725: write d1/d1c/d4a/f98 [2316564,96498] 0 2024-03-13T14:49:59.872 INFO:tasks.workunit.client.0.smithi176.stdout:9/877: dwrite d0/db/d89/d32/d122/f38 [0,4194304] 0 2024-03-13T14:49:59.880 INFO:tasks.workunit.client.0.smithi176.stdout:8/696: dread d0/db/f3f [0,4194304] 0 2024-03-13T14:49:59.881 INFO:tasks.workunit.client.0.smithi176.stdout:2/798: dwrite d1/dc/fa5 [0,4194304] 0 2024-03-13T14:49:59.881 INFO:tasks.workunit.client.0.smithi176.stdout:1/758: truncate d2/de/d3f/d6a/fda 694196 0 2024-03-13T14:49:59.881 INFO:tasks.workunit.client.0.smithi176.stdout:3/821: dread d4/d28/f29 [0,4194304] 0 2024-03-13T14:49:59.881 INFO:tasks.workunit.client.0.smithi176.stdout:1/759: write d2/d29/f70 [7398643,107458] 0 2024-03-13T14:49:59.881 INFO:tasks.workunit.client.0.smithi176.stdout:3/822: stat d4/d13/cac 0 2024-03-13T14:49:59.884 INFO:tasks.workunit.client.1.smithi196.stdout:4/845: dwrite d1/dfb/d11c/f103 [0,4194304] 0 2024-03-13T14:49:59.888 INFO:tasks.workunit.client.1.smithi196.stdout:5/888: dwrite d4/d1b/dc8/d3c/f84 [0,4194304] 0 2024-03-13T14:49:59.893 INFO:tasks.workunit.client.1.smithi196.stdout:4/846: dread - d1/d3/d7/d35/f10c zero size 2024-03-13T14:49:59.898 INFO:tasks.workunit.client.1.smithi196.stdout:8/815: dwrite d0/d10/da5/d110/fad [0,4194304] 0 2024-03-13T14:49:59.900 INFO:tasks.workunit.client.1.smithi196.stdout:8/816: readlink d0/l35 0 2024-03-13T14:49:59.900 INFO:tasks.workunit.client.0.smithi176.stdout:4/780: sync 2024-03-13T14:49:59.908 INFO:tasks.workunit.client.0.smithi176.stdout:9/878: rmdir d0 39 2024-03-13T14:49:59.911 INFO:tasks.workunit.client.0.smithi176.stdout:8/697: dwrite d0/d24/d29/f67 [0,4194304] 0 2024-03-13T14:49:59.916 INFO:tasks.workunit.client.0.smithi176.stdout:6/785: dwrite d0/d10/d19/d27/d2b/de0/d76/d7c/f9b [0,4194304] 0 2024-03-13T14:49:59.917 INFO:tasks.workunit.client.0.smithi176.stdout:7/726: dwrite d1/d3f/d76/f1e [8388608,4194304] 0 2024-03-13T14:49:59.922 INFO:tasks.workunit.client.0.smithi176.stdout:5/736: dwrite d5/d31/d47/f85 [0,4194304] 0 2024-03-13T14:49:59.923 INFO:tasks.workunit.client.0.smithi176.stdout:2/799: mknod d1/dc/d2d/d90/c10f 0 2024-03-13T14:49:59.924 INFO:tasks.workunit.client.1.smithi196.stdout:1/929: dread d1/d9/d4e/d55/d90/fb1 [0,4194304] 0 2024-03-13T14:49:59.927 INFO:tasks.workunit.client.1.smithi196.stdout:1/930: write d1/d28/da3/fdc [635799,32483] 0 2024-03-13T14:49:59.927 INFO:tasks.workunit.client.0.smithi176.stdout:1/760: unlink d2/de/d3f/d6a/fb5 0 2024-03-13T14:49:59.931 INFO:tasks.workunit.client.0.smithi176.stdout:3/823: truncate d4/d13/d2d/fd7 3037249 0 2024-03-13T14:49:59.938 INFO:tasks.workunit.client.0.smithi176.stdout:4/781: mknod d7/d10/c103 0 2024-03-13T14:49:59.940 INFO:tasks.workunit.client.0.smithi176.stdout:4/782: stat d7/d8/cb0 0 2024-03-13T14:49:59.942 INFO:tasks.workunit.client.0.smithi176.stdout:0/679: sync 2024-03-13T14:49:59.946 INFO:tasks.workunit.client.0.smithi176.stdout:6/786: rmdir d0/de/d38/d7e 39 2024-03-13T14:49:59.948 INFO:tasks.workunit.client.0.smithi176.stdout:0/680: write d2/db/d31/fd7 [970355,37485] 0 2024-03-13T14:49:59.949 INFO:tasks.workunit.client.0.smithi176.stdout:7/727: rename d1/d3f/d4b/d63/fbd to d1/d3f/d81/fe0 0 2024-03-13T14:49:59.950 INFO:tasks.workunit.client.0.smithi176.stdout:8/698: dwrite d0/db/f1a [0,4194304] 0 2024-03-13T14:49:59.951 INFO:tasks.workunit.client.0.smithi176.stdout:5/737: creat d5/d2c/d40/dc7/d89/ffa x:0 0 0 2024-03-13T14:49:59.955 INFO:tasks.workunit.client.0.smithi176.stdout:2/800: symlink d1/dc/d2d/d59/d80/d7e/da3/l110 0 2024-03-13T14:49:59.958 INFO:tasks.workunit.client.0.smithi176.stdout:7/728: dread d1/d1c/d26/d41/d2b/f96 [4194304,4194304] 0 2024-03-13T14:49:59.959 INFO:tasks.workunit.client.0.smithi176.stdout:4/783: creat d7/d10/d2c/df1/f104 x:0 0 0 2024-03-13T14:49:59.960 INFO:tasks.workunit.client.0.smithi176.stdout:1/761: chown d2/de/d3f/d50/c49 0 1 2024-03-13T14:49:59.960 INFO:tasks.workunit.client.0.smithi176.stdout:0/681: truncate d2/d5/d28/d8c/fcc 779597 0 2024-03-13T14:49:59.961 INFO:tasks.workunit.client.0.smithi176.stdout:9/879: truncate d0/d40/d53/d6d/fcc 603820 0 2024-03-13T14:49:59.961 INFO:tasks.workunit.client.0.smithi176.stdout:6/787: mkdir d0/de/d38/d88/d9d/dfe 0 2024-03-13T14:49:59.965 INFO:tasks.workunit.client.0.smithi176.stdout:3/824: rename d4/d28/f67 to d4/d13/d16/d3d/f11e 0 2024-03-13T14:49:59.968 INFO:tasks.workunit.client.0.smithi176.stdout:2/801: mkdir d1/dc/d2d/d59/d80/d7e/da3/d111 0 2024-03-13T14:49:59.969 INFO:tasks.workunit.client.1.smithi196.stdout:5/889: dread d4/d1b/dc8/fd [0,4194304] 0 2024-03-13T14:49:59.970 INFO:tasks.workunit.client.0.smithi176.stdout:9/880: truncate d0/db/d89/d32/d58/d5f/d84/d9a/f9b 5067101 0 2024-03-13T14:49:59.970 INFO:tasks.workunit.client.0.smithi176.stdout:0/682: unlink d2/db/d39/ld4 0 2024-03-13T14:49:59.973 INFO:tasks.workunit.client.0.smithi176.stdout:5/738: read d5/f24 [63369,115203] 0 2024-03-13T14:49:59.974 INFO:tasks.workunit.client.1.smithi196.stdout:4/847: rename d1/d3/d7/d35/d3e/l8e to d1/d3/d7/d35/d76/d58/d69/d9f/dd5/l12b 0 2024-03-13T14:49:59.975 INFO:tasks.workunit.client.1.smithi196.stdout:4/848: write d1/d3/d7/fa4 [1032155,74675] 0 2024-03-13T14:49:59.975 INFO:tasks.workunit.client.1.smithi196.stdout:1/931: mknod d1/d94/c142 0 2024-03-13T14:49:59.977 INFO:tasks.workunit.client.1.smithi196.stdout:8/817: mkdir d0/d16/d71/d68/d115 0 2024-03-13T14:49:59.977 INFO:tasks.workunit.client.1.smithi196.stdout:4/849: chown d1/d3/d7/d35/d76/f38 0 1 2024-03-13T14:49:59.980 INFO:tasks.workunit.client.1.smithi196.stdout:1/932: unlink d1/d9/d33/dd2/f115 0 2024-03-13T14:49:59.981 INFO:tasks.workunit.client.0.smithi176.stdout:6/788: truncate d0/d10/d19/d27/d2b/d2e/d3e/f78 553954 0 2024-03-13T14:49:59.982 INFO:tasks.workunit.client.1.smithi196.stdout:4/850: stat d1/f54 0 2024-03-13T14:49:59.983 INFO:tasks.workunit.client.0.smithi176.stdout:2/802: creat d1/dc/d2d/d59/d80/d76/f112 x:0 0 0 2024-03-13T14:49:59.985 INFO:tasks.workunit.client.0.smithi176.stdout:0/683: fsync d2/f50 0 2024-03-13T14:49:59.985 INFO:tasks.workunit.client.1.smithi196.stdout:3/838: sync 2024-03-13T14:49:59.985 INFO:tasks.workunit.client.1.smithi196.stdout:2/781: sync 2024-03-13T14:49:59.987 INFO:tasks.workunit.client.0.smithi176.stdout:9/881: creat d0/db/d89/d7a/d8b/f129 x:0 0 0 2024-03-13T14:49:59.988 INFO:tasks.workunit.client.0.smithi176.stdout:9/882: readlink d0/d40/d53/la2 0 2024-03-13T14:49:59.989 INFO:tasks.workunit.client.1.smithi196.stdout:1/933: rename d1/d9/d33/d4b/d66/l10d to d1/d3f/d5b/d74/db5/l143 0 2024-03-13T14:49:59.989 INFO:tasks.workunit.client.0.smithi176.stdout:8/699: getdents d0/d5/d11/d47/db7/de0 0 2024-03-13T14:49:59.990 INFO:tasks.workunit.client.1.smithi196.stdout:4/851: rmdir d1/d3/d7/d35/d76/d58/d69/de3/df4 39 2024-03-13T14:49:59.990 INFO:tasks.workunit.client.0.smithi176.stdout:8/700: readlink d0/d5/d11/ld2 0 2024-03-13T14:49:59.992 INFO:tasks.workunit.client.0.smithi176.stdout:8/701: chown d0/d24/f32 623 1 2024-03-13T14:49:59.993 INFO:tasks.workunit.client.1.smithi196.stdout:4/852: chown d1/dfb/d11c/f103 225960 1 2024-03-13T14:49:59.993 INFO:tasks.workunit.client.0.smithi176.stdout:1/762: creat d2/de/d10/d16/d4a/d5f/d66/fea x:0 0 0 2024-03-13T14:49:59.994 INFO:tasks.workunit.client.1.smithi196.stdout:3/839: unlink db/d19/d22/d3a/d6a/d8d/la8 0 2024-03-13T14:49:59.999 INFO:tasks.workunit.client.0.smithi176.stdout:5/739: rename d5/d2c/d40/dc7/l4b to d5/d31/d47/d80/lfb 0 2024-03-13T14:50:00.000 INFO:tasks.workunit.client.1.smithi196.stdout:4/853: creat d1/d3/d7/d35/d76/d58/dc9/f12c x:0 0 0 2024-03-13T14:50:00.001 INFO:tasks.workunit.client.1.smithi196.stdout:2/782: dwrite d0/ffd [0,4194304] 0 2024-03-13T14:50:00.003 INFO:tasks.workunit.client.0.smithi176.stdout:6/789: rmdir d0/de/d54 39 2024-03-13T14:50:00.010 INFO:tasks.workunit.client.0.smithi176.stdout:4/784: dread d7/dab/db5/fc6 [0,4194304] 0 2024-03-13T14:50:00.011 INFO:tasks.workunit.client.1.smithi196.stdout:3/840: mknod db/d15/da1/dc8/c10d 0 2024-03-13T14:50:00.013 INFO:tasks.workunit.client.0.smithi176.stdout:9/883: unlink d0/db/cc 0 2024-03-13T14:50:00.015 INFO:tasks.workunit.client.1.smithi196.stdout:4/854: unlink d1/d3/d7/d35/d3e/l7b 0 2024-03-13T14:50:00.016 INFO:tasks.workunit.client.0.smithi176.stdout:8/702: rmdir d0/d24/d98/dc6 39 2024-03-13T14:50:00.018 INFO:tasks.workunit.client.1.smithi196.stdout:2/783: fdatasync d0/d6/d1d/f30 0 2024-03-13T14:50:00.023 INFO:tasks.workunit.client.0.smithi176.stdout:7/729: link d1/d3f/d4b/d63/d7a/l7f d1/d1c/d4a/dda/le1 0 2024-03-13T14:50:00.023 INFO:tasks.workunit.client.1.smithi196.stdout:4/855: mkdir d1/d3/d4c/d72/daf/d12d 0 2024-03-13T14:50:00.026 INFO:tasks.workunit.client.0.smithi176.stdout:0/684: rename d2/d74/d61/c6c to d2/db/d35/daa/cd9 0 2024-03-13T14:50:00.029 INFO:tasks.workunit.client.0.smithi176.stdout:5/740: mknod d5/d75/cfc 0 2024-03-13T14:50:00.029 INFO:tasks.workunit.client.1.smithi196.stdout:8/818: dread d0/f56 [0,4194304] 0 2024-03-13T14:50:00.032 INFO:tasks.workunit.client.0.smithi176.stdout:9/884: dread d0/db/d89/d32/d58/d67/f106 [0,4194304] 0 2024-03-13T14:50:00.033 INFO:tasks.workunit.client.1.smithi196.stdout:3/841: dread db/d37/d64/f90 [0,4194304] 0 2024-03-13T14:50:00.034 INFO:tasks.workunit.client.1.smithi196.stdout:2/784: creat d0/d7/d9/d5a/dbc/dfe/f100 x:0 0 0 2024-03-13T14:50:00.038 INFO:tasks.workunit.client.1.smithi196.stdout:4/856: mknod d1/d3/d7/d35/d76/d58/d69/c12e 0 2024-03-13T14:50:00.042 INFO:tasks.workunit.client.0.smithi176.stdout:8/703: mknod d0/d5/d3b/ce2 0 2024-03-13T14:50:00.042 INFO:tasks.workunit.client.1.smithi196.stdout:2/785: truncate d0/d7/d9/d9a/daa/fc0 471537 0 2024-03-13T14:50:00.044 INFO:tasks.workunit.client.0.smithi176.stdout:5/741: dwrite d5/d75/fe2 [0,4194304] 0 2024-03-13T14:50:00.048 INFO:tasks.workunit.client.1.smithi196.stdout:2/786: mkdir d0/d7/d9/d61/dd3/d101 0 2024-03-13T14:50:00.049 INFO:tasks.workunit.client.0.smithi176.stdout:9/885: dwrite d0/d40/d53/f114 [0,4194304] 0 2024-03-13T14:50:00.054 INFO:tasks.workunit.client.1.smithi196.stdout:8/819: dread d0/d16/f86 [0,4194304] 0 2024-03-13T14:50:00.059 INFO:tasks.workunit.client.0.smithi176.stdout:1/763: rmdir d2/de/d10/d16/d4a/d5f/d66 39 2024-03-13T14:50:00.061 INFO:tasks.workunit.client.1.smithi196.stdout:1/934: write d1/d38/fc6 [4552597,92339] 0 2024-03-13T14:50:00.067 INFO:tasks.workunit.client.1.smithi196.stdout:2/787: getdents d0/d7/d9/d61 0 2024-03-13T14:50:00.068 INFO:tasks.workunit.client.0.smithi176.stdout:0/685: mknod d2/d74/cda 0 2024-03-13T14:50:00.070 INFO:tasks.workunit.client.0.smithi176.stdout:8/704: rename d0/db/d39/d56/d91/f9c to d0/d24/dc3/fe3 0 2024-03-13T14:50:00.072 INFO:tasks.workunit.client.0.smithi176.stdout:3/825: link d4/d13/c17 d4/d13/d81/d89/c11f 0 2024-03-13T14:50:00.075 INFO:tasks.workunit.client.0.smithi176.stdout:2/803: dwrite d1/f31 [0,4194304] 0 2024-03-13T14:50:00.076 INFO:tasks.workunit.client.0.smithi176.stdout:6/790: link d0/de/d38/la0 d0/de/d8d/lff 0 2024-03-13T14:50:00.086 INFO:tasks.workunit.client.0.smithi176.stdout:8/705: fdatasync d0/f3 0 2024-03-13T14:50:00.087 INFO:tasks.workunit.client.0.smithi176.stdout:3/826: mknod d4/d13/d16/d80/c120 0 2024-03-13T14:50:00.087 INFO:tasks.workunit.client.0.smithi176.stdout:4/785: write d7/d10/d46/d4a/d5d/da1/f9e [1310455,75605] 0 2024-03-13T14:50:00.089 INFO:tasks.workunit.client.0.smithi176.stdout:7/730: truncate d1/d3f/d4b/d63/fb4 1523220 0 2024-03-13T14:50:00.092 INFO:tasks.workunit.client.1.smithi196.stdout:1/935: write d1/d9/d33/d4b/d66/f95 [112065,32005] 0 2024-03-13T14:50:00.094 INFO:tasks.workunit.client.0.smithi176.stdout:5/742: write d5/d2e/d5f/f7c [499174,11191] 0 2024-03-13T14:50:00.096 INFO:tasks.workunit.client.1.smithi196.stdout:1/936: rmdir d1/d28/d34 39 2024-03-13T14:50:00.097 INFO:tasks.workunit.client.1.smithi196.stdout:2/788: write d0/d7/d9/d9a/daa/fc0 [457595,121550] 0 2024-03-13T14:50:00.097 INFO:tasks.workunit.client.0.smithi176.stdout:8/706: rmdir d0/d24/d29/d70 39 2024-03-13T14:50:00.100 INFO:tasks.workunit.client.1.smithi196.stdout:2/789: fdatasync d0/d6/d34/f41 0 2024-03-13T14:50:00.101 INFO:tasks.workunit.client.1.smithi196.stdout:1/937: creat d1/d9/d4e/d55/d90/f144 x:0 0 0 2024-03-13T14:50:00.101 INFO:tasks.workunit.client.0.smithi176.stdout:9/886: getdents d0/db/d89/d4d/dd5 0 2024-03-13T14:50:00.102 INFO:tasks.workunit.client.1.smithi196.stdout:2/790: write d0/d7/f87 [2421606,125908] 0 2024-03-13T14:50:00.105 INFO:tasks.workunit.client.1.smithi196.stdout:2/791: write d0/d7/d40/f46 [2175548,43465] 0 2024-03-13T14:50:00.105 INFO:tasks.workunit.client.0.smithi176.stdout:2/804: write d1/dc/d2d/d38/d5d/f70 [137752,64541] 0 2024-03-13T14:50:00.110 INFO:tasks.workunit.client.0.smithi176.stdout:5/743: creat d5/d2c/d40/dc7/d53/de9/ffd x:0 0 0 2024-03-13T14:50:00.112 INFO:tasks.workunit.client.1.smithi196.stdout:2/792: link d0/d7/d40/l52 d0/d7/d9/d5a/dbc/deb/l102 0 2024-03-13T14:50:00.114 INFO:tasks.workunit.client.0.smithi176.stdout:6/791: dwrite d0/de/d38/d88/d9d/fd7 [4194304,4194304] 0 2024-03-13T14:50:00.116 INFO:tasks.workunit.client.1.smithi196.stdout:2/793: symlink d0/d7/d9/d5a/dbc/dfe/l103 0 2024-03-13T14:50:00.116 INFO:tasks.workunit.client.0.smithi176.stdout:8/707: rename d0/d5/c9b to d0/d5/d11/d47/ce4 0 2024-03-13T14:50:00.119 INFO:tasks.workunit.client.1.smithi196.stdout:1/938: write d1/d9/d4e/d55/d63/da6/fbc [5218049,78066] 0 2024-03-13T14:50:00.121 INFO:tasks.workunit.client.1.smithi196.stdout:2/794: rename d0/d6/d1d to d0/d7/d9/d42/d98/de4/d104 0 2024-03-13T14:50:00.123 INFO:tasks.workunit.client.0.smithi176.stdout:0/686: dread d2/d5/d1e/f92 [0,4194304] 0 2024-03-13T14:50:00.124 INFO:tasks.workunit.client.1.smithi196.stdout:2/795: mknod d0/d6/d4b/d5f/c105 0 2024-03-13T14:50:00.126 INFO:tasks.workunit.client.1.smithi196.stdout:7/965: sync 2024-03-13T14:50:00.129 INFO:tasks.workunit.client.1.smithi196.stdout:1/939: truncate d1/d9/d4e/d55/f113 949799 0 2024-03-13T14:50:00.129 INFO:tasks.workunit.client.0.smithi176.stdout:7/731: dread d1/d1c/f5e [0,4194304] 0 2024-03-13T14:50:00.133 INFO:tasks.workunit.client.1.smithi196.stdout:2/796: read d0/d6/fb3 [60706,11441] 0 2024-03-13T14:50:00.134 INFO:tasks.workunit.client.0.smithi176.stdout:7/732: fsync d1/d3f/d4b/d63/fb3 0 2024-03-13T14:50:00.134 INFO:tasks.workunit.client.0.smithi176.stdout:3/827: dwrite d4/d28/d31/f99 [0,4194304] 0 2024-03-13T14:50:00.135 INFO:tasks.workunit.client.0.smithi176.stdout:6/792: rmdir d0/d10/d19/d27/d2b/ddc 39 2024-03-13T14:50:00.135 INFO:tasks.workunit.client.0.smithi176.stdout:7/733: stat d1/d3f/fab 0 2024-03-13T14:50:00.136 INFO:tasks.workunit.client.1.smithi196.stdout:1/940: write d1/d9/f9a [521099,98016] 0 2024-03-13T14:50:00.139 INFO:tasks.workunit.client.0.smithi176.stdout:9/887: dwrite d0/db/d89/d32/d122/d68/faf [0,4194304] 0 2024-03-13T14:50:00.140 INFO:tasks.workunit.client.0.smithi176.stdout:4/786: dread d7/d10/f27 [0,4194304] 0 2024-03-13T14:50:00.142 INFO:tasks.workunit.client.0.smithi176.stdout:9/888: chown d0/db/d89/d4d/d6e/d100 893428889 1 2024-03-13T14:50:00.142 INFO:tasks.workunit.client.0.smithi176.stdout:5/744: write d5/d31/fb6 [742251,87347] 0 2024-03-13T14:50:00.146 INFO:tasks.workunit.client.1.smithi196.stdout:1/941: creat d1/d28/da3/f145 x:0 0 0 2024-03-13T14:50:00.147 INFO:tasks.workunit.client.1.smithi196.stdout:7/966: dwrite d0/d6/d33/f96 [0,4194304] 0 2024-03-13T14:50:00.152 INFO:tasks.workunit.client.1.smithi196.stdout:2/797: dread d0/ffd [0,4194304] 0 2024-03-13T14:50:00.156 INFO:tasks.workunit.client.0.smithi176.stdout:3/828: mknod d4/d13/d2d/d105/c121 0 2024-03-13T14:50:00.157 INFO:tasks.workunit.client.0.smithi176.stdout:8/708: dwrite d0/db/f25 [0,4194304] 0 2024-03-13T14:50:00.165 INFO:tasks.workunit.client.0.smithi176.stdout:9/889: dwrite d0/db/d89/d4d/d6e/ffb [0,4194304] 0 2024-03-13T14:50:00.170 INFO:tasks.workunit.client.0.smithi176.stdout:4/787: fdatasync d7/d8/d15/f2b 0 2024-03-13T14:50:00.174 INFO:tasks.workunit.client.0.smithi176.stdout:4/788: chown d7/db9/dd8 83445 1 2024-03-13T14:50:00.176 INFO:tasks.workunit.client.1.smithi196.stdout:1/942: link d1/d38/c9b d1/d9/d33/dd2/c146 0 2024-03-13T14:50:00.178 INFO:tasks.workunit.client.0.smithi176.stdout:0/687: link d2/d5/d1e/l24 d2/d5/d28/d8c/ldb 0 2024-03-13T14:50:00.178 INFO:tasks.workunit.client.0.smithi176.stdout:8/709: creat d0/d24/d7b/dab/fe5 x:0 0 0 2024-03-13T14:50:00.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:59 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:50:00.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:59 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:50:00.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:59 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:59 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:59 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:59 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:49:59 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.181 INFO:tasks.workunit.client.1.smithi196.stdout:7/967: link d0/d9/d1e/d3f/d6c/d106/f114 d0/d4d/f14f 0 2024-03-13T14:50:00.181 INFO:tasks.workunit.client.1.smithi196.stdout:7/968: chown d0/d6/f12a 97526 1 2024-03-13T14:50:00.183 INFO:tasks.workunit.client.1.smithi196.stdout:1/943: fsync d1/d3f/d5b/fb8 0 2024-03-13T14:50:00.184 INFO:tasks.workunit.client.1.smithi196.stdout:7/969: creat d0/d6/d140/dcf/de0/f150 x:0 0 0 2024-03-13T14:50:00.185 INFO:tasks.workunit.client.0.smithi176.stdout:6/793: getdents d0/d10/d19 0 2024-03-13T14:50:00.185 INFO:tasks.workunit.client.0.smithi176.stdout:3/829: mkdir d4/d13/d16/d122 0 2024-03-13T14:50:00.186 INFO:tasks.workunit.client.1.smithi196.stdout:7/970: stat d0/d9/d1e/d58/d61/f13c 0 2024-03-13T14:50:00.186 INFO:tasks.workunit.client.0.smithi176.stdout:3/830: dread - d4/f39 zero size 2024-03-13T14:50:00.188 INFO:tasks.workunit.client.0.smithi176.stdout:0/688: chown d2/d5/f95 774 1 2024-03-13T14:50:00.189 INFO:tasks.workunit.client.0.smithi176.stdout:4/789: mknod d7/d8/d15/dd6/dba/c105 0 2024-03-13T14:50:00.192 INFO:tasks.workunit.client.0.smithi176.stdout:6/794: fsync d0/d10/d19/d27/d2b/d2e/dc4/dc5/fde 0 2024-03-13T14:50:00.192 INFO:tasks.workunit.client.0.smithi176.stdout:0/689: write d2/db/d39/d9c/f9e [441293,96918] 0 2024-03-13T14:50:00.193 INFO:tasks.workunit.client.0.smithi176.stdout:3/831: creat d4/d13/d2d/d105/f123 x:0 0 0 2024-03-13T14:50:00.194 INFO:tasks.workunit.client.1.smithi196.stdout:7/971: truncate d0/d9/f5a 2855711 0 2024-03-13T14:50:00.195 INFO:tasks.workunit.client.0.smithi176.stdout:4/790: mknod d7/d2d/c106 0 2024-03-13T14:50:00.195 INFO:tasks.workunit.client.1.smithi196.stdout:1/944: dread d1/d9/f64 [0,4194304] 0 2024-03-13T14:50:00.197 INFO:tasks.workunit.client.0.smithi176.stdout:0/690: mknod d2/d5/dc/d59/cdc 0 2024-03-13T14:50:00.198 INFO:tasks.workunit.client.0.smithi176.stdout:3/832: symlink d4/d13/d81/d89/d5b/l124 0 2024-03-13T14:50:00.199 INFO:tasks.workunit.client.0.smithi176.stdout:4/791: creat d7/d10/d46/d4a/dcc/dff/f107 x:0 0 0 2024-03-13T14:50:00.201 INFO:tasks.workunit.client.0.smithi176.stdout:6/795: truncate d0/d10/d19/d27/fa9 2050179 0 2024-03-13T14:50:00.203 INFO:tasks.workunit.client.1.smithi196.stdout:7/972: rename d0/d9/d1e/d58/d72/f66 to d0/d6/d33/d57/dae/de8/f151 0 2024-03-13T14:50:00.204 INFO:tasks.workunit.client.0.smithi176.stdout:3/833: creat d4/db8/df5/f125 x:0 0 0 2024-03-13T14:50:00.204 INFO:tasks.workunit.client.0.smithi176.stdout:0/691: truncate d2/d5/dc/d20/f37 4404777 0 2024-03-13T14:50:00.207 INFO:tasks.workunit.client.1.smithi196.stdout:7/973: symlink d0/d6/d140/dcf/de0/d104/l152 0 2024-03-13T14:50:00.207 INFO:tasks.workunit.client.0.smithi176.stdout:4/792: creat d7/d10/d46/d4a/dcc/f108 x:0 0 0 2024-03-13T14:50:00.208 INFO:tasks.workunit.client.0.smithi176.stdout:0/692: write d2/d5/d28/f5e [451454,82782] 0 2024-03-13T14:50:00.211 INFO:tasks.workunit.client.1.smithi196.stdout:2/798: write d0/d7/d9/d42/d98/de4/d104/f7f [3460639,69100] 0 2024-03-13T14:50:00.217 INFO:tasks.workunit.client.1.smithi196.stdout:2/799: rmdir d0/d7/d9/d9a 39 2024-03-13T14:50:00.217 INFO:tasks.workunit.client.1.smithi196.stdout:1/945: dread d1/d9/d33/f42 [0,4194304] 0 2024-03-13T14:50:00.219 INFO:tasks.workunit.client.0.smithi176.stdout:2/805: sync 2024-03-13T14:50:00.219 INFO:tasks.workunit.client.0.smithi176.stdout:1/764: sync 2024-03-13T14:50:00.220 INFO:tasks.workunit.client.0.smithi176.stdout:7/734: truncate d1/d1c/d26/d41/d2b/d40/d64/f60 447254 0 2024-03-13T14:50:00.221 INFO:tasks.workunit.client.0.smithi176.stdout:5/745: dwrite d5/d75/f8c [0,4194304] 0 2024-03-13T14:50:00.222 INFO:tasks.workunit.client.1.smithi196.stdout:1/946: symlink d1/d9/d33/d4b/d66/l147 0 2024-03-13T14:50:00.223 INFO:tasks.workunit.client.1.smithi196.stdout:2/800: fsync d0/d7/d9/d9a/daa/fd1 0 2024-03-13T14:50:00.228 INFO:tasks.workunit.client.1.smithi196.stdout:2/801: write d0/d7/d9/d61/d83/ff5 [500992,36573] 0 2024-03-13T14:50:00.228 INFO:tasks.workunit.client.1.smithi196.stdout:1/947: stat d1/d9/d4e/d55/d63/da6/dc8/d11d/d122 0 2024-03-13T14:50:00.230 INFO:tasks.workunit.client.0.smithi176.stdout:8/710: write d0/d24/d29/fbd [1243211,44895] 0 2024-03-13T14:50:00.230 INFO:tasks.workunit.client.0.smithi176.stdout:0/693: getdents d2/d5/dc/d20 0 2024-03-13T14:50:00.232 INFO:tasks.workunit.client.1.smithi196.stdout:1/948: write d1/d9/d4e/d55/d90/f144 [182061,90251] 0 2024-03-13T14:50:00.233 INFO:tasks.workunit.client.0.smithi176.stdout:2/806: rmdir d1/dc/d2d/d4b/ddb 39 2024-03-13T14:50:00.234 INFO:tasks.workunit.client.0.smithi176.stdout:1/765: creat d2/d30/dd3/feb x:0 0 0 2024-03-13T14:50:00.237 INFO:tasks.workunit.client.0.smithi176.stdout:5/746: creat d5/d2c/d40/dc7/d53/de9/ffe x:0 0 0 2024-03-13T14:50:00.240 INFO:tasks.workunit.client.0.smithi176.stdout:1/766: chown d2/de/d3f/d3e 13673 1 2024-03-13T14:50:00.241 INFO:tasks.workunit.client.0.smithi176.stdout:7/735: fsync d1/d1c/d4a/f8d 0 2024-03-13T14:50:00.242 INFO:tasks.workunit.client.1.smithi196.stdout:1/949: creat d1/d28/d34/d67/d11f/f148 x:0 0 0 2024-03-13T14:50:00.243 INFO:tasks.workunit.client.0.smithi176.stdout:3/834: dwrite d4/d28/fc0 [0,4194304] 0 2024-03-13T14:50:00.249 INFO:tasks.workunit.client.1.smithi196.stdout:1/950: mkdir d1/d9/d33/d4b/d66/d149 0 2024-03-13T14:50:00.251 INFO:tasks.workunit.client.1.smithi196.stdout:2/802: dwrite d0/d6/d4b/d5f/f81 [4194304,4194304] 0 2024-03-13T14:50:00.252 INFO:tasks.workunit.client.0.smithi176.stdout:9/890: dwrite d0/db/d89/d32/d122/d68/f85 [0,4194304] 0 2024-03-13T14:50:00.252 INFO:tasks.workunit.client.0.smithi176.stdout:1/767: unlink d2/de/d3f/d19/l96 0 2024-03-13T14:50:00.252 INFO:tasks.workunit.client.0.smithi176.stdout:4/793: write d7/d8/d45/ff3 [4777058,99962] 0 2024-03-13T14:50:00.255 INFO:tasks.workunit.client.0.smithi176.stdout:1/768: chown d2/de/d3f/d3e/la0 135 1 2024-03-13T14:50:00.256 INFO:tasks.workunit.client.0.smithi176.stdout:6/796: dread d0/d10/fc0 [0,4194304] 0 2024-03-13T14:50:00.265 INFO:tasks.workunit.client.0.smithi176.stdout:0/694: creat d2/d74/fdd x:0 0 0 2024-03-13T14:50:00.266 INFO:tasks.workunit.client.0.smithi176.stdout:2/807: dread d1/f31 [0,4194304] 0 2024-03-13T14:50:00.267 INFO:tasks.workunit.client.1.smithi196.stdout:7/974: dread d0/d9/d1e/f4b [0,4194304] 0 2024-03-13T14:50:00.268 INFO:tasks.workunit.client.0.smithi176.stdout:9/891: stat d0/db/d89/d32/d122/c3b 0 2024-03-13T14:50:00.268 INFO:tasks.workunit.client.0.smithi176.stdout:5/747: mkdir d5/d2c/d40/dc7/d89/df2/dff 0 2024-03-13T14:50:00.269 INFO:tasks.workunit.client.1.smithi196.stdout:1/951: creat d1/d9/d4e/d55/f14a x:0 0 0 2024-03-13T14:50:00.270 INFO:tasks.workunit.client.0.smithi176.stdout:4/794: rename d7/d10/c103 to d7/d10/d46/c109 0 2024-03-13T14:50:00.272 INFO:tasks.workunit.client.0.smithi176.stdout:6/797: creat d0/d10/d19/d7d/f100 x:0 0 0 2024-03-13T14:50:00.272 INFO:tasks.workunit.client.0.smithi176.stdout:0/695: creat d2/d74/d9f/fde x:0 0 0 2024-03-13T14:50:00.274 INFO:tasks.workunit.client.0.smithi176.stdout:9/892: symlink d0/db/def/l12a 0 2024-03-13T14:50:00.276 INFO:tasks.workunit.client.1.smithi196.stdout:7/975: mkdir d0/d9/d1e/d58/d72/d153 0 2024-03-13T14:50:00.278 INFO:tasks.workunit.client.1.smithi196.stdout:2/803: dwrite d0/d7/d9/fdf [0,4194304] 0 2024-03-13T14:50:00.279 INFO:tasks.workunit.client.0.smithi176.stdout:1/769: read d2/d29/fce [3628376,42855] 0 2024-03-13T14:50:00.282 INFO:tasks.workunit.client.1.smithi196.stdout:1/952: mkdir d1/d3f/d5b/d74/dd4/d108/d14b 0 2024-03-13T14:50:00.284 INFO:tasks.workunit.client.0.smithi176.stdout:6/798: mknod d0/de/d38/d70/c101 0 2024-03-13T14:50:00.286 INFO:tasks.workunit.client.0.smithi176.stdout:7/736: link d1/d28/d3a/d1b/lb1 d1/d1c/le2 0 2024-03-13T14:50:00.287 INFO:tasks.workunit.client.0.smithi176.stdout:7/737: stat d1/d3f/d81/f8e 0 2024-03-13T14:50:00.288 INFO:tasks.workunit.client.1.smithi196.stdout:7/976: mkdir d0/d6/d140/dcf/de0/d154 0 2024-03-13T14:50:00.289 INFO:tasks.workunit.client.1.smithi196.stdout:7/977: chown d0/d9/d1e/c6d 31149 1 2024-03-13T14:50:00.290 INFO:tasks.workunit.client.0.smithi176.stdout:4/795: dwrite d7/fe9 [0,4194304] 0 2024-03-13T14:50:00.291 INFO:tasks.workunit.client.1.smithi196.stdout:1/953: write d1/d28/d34/f8e [9223755,75127] 0 2024-03-13T14:50:00.294 INFO:tasks.workunit.client.0.smithi176.stdout:1/770: mknod d2/d29/cec 0 2024-03-13T14:50:00.296 INFO:tasks.workunit.client.0.smithi176.stdout:1/771: stat d2/de/dd1/fd4 0 2024-03-13T14:50:00.298 INFO:tasks.workunit.client.0.smithi176.stdout:3/835: dwrite d4/d13/f2a [0,4194304] 0 2024-03-13T14:50:00.301 INFO:tasks.workunit.client.0.smithi176.stdout:5/748: link d5/d31/d47/d80/f4d d5/d31/d86/dc0/f100 0 2024-03-13T14:50:00.301 INFO:tasks.workunit.client.0.smithi176.stdout:8/711: write d0/d24/d98/dc6/db9/fdb [766678,130768] 0 2024-03-13T14:50:00.303 INFO:tasks.workunit.client.1.smithi196.stdout:1/954: rmdir d1/d9/d4e/d55/d63/db9 39 2024-03-13T14:50:00.305 INFO:tasks.workunit.client.0.smithi176.stdout:4/796: dread d7/d10/d46/d4a/d5d/da1/f9e [0,4194304] 0 2024-03-13T14:50:00.309 INFO:tasks.workunit.client.0.smithi176.stdout:6/799: mkdir d0/de/d8d/d102 0 2024-03-13T14:50:00.312 INFO:tasks.workunit.client.0.smithi176.stdout:7/738: symlink d1/d1c/le3 0 2024-03-13T14:50:00.313 INFO:tasks.workunit.client.0.smithi176.stdout:4/797: read d7/d8/d15/f81 [1376390,118901] 0 2024-03-13T14:50:00.313 INFO:tasks.workunit.client.1.smithi196.stdout:1/955: creat d1/d9/d6c/d101/f14c x:0 0 0 2024-03-13T14:50:00.314 INFO:tasks.workunit.client.0.smithi176.stdout:4/798: chown c0 3883409 1 2024-03-13T14:50:00.316 INFO:tasks.workunit.client.1.smithi196.stdout:1/956: rmdir d1/d9/d4e 39 2024-03-13T14:50:00.316 INFO:tasks.workunit.client.0.smithi176.stdout:5/749: symlink d5/d2e/d7e/dad/d7b/l101 0 2024-03-13T14:50:00.318 INFO:tasks.workunit.client.0.smithi176.stdout:0/696: rmdir d2/db/d39/d9c/dd5 0 2024-03-13T14:50:00.323 INFO:tasks.workunit.client.0.smithi176.stdout:3/836: mkdir d4/d13/d16/db6/de8/d126 0 2024-03-13T14:50:00.323 INFO:tasks.workunit.client.1.smithi196.stdout:9/854: sync 2024-03-13T14:50:00.323 INFO:tasks.workunit.client.1.smithi196.stdout:0/884: sync 2024-03-13T14:50:00.324 INFO:tasks.workunit.client.0.smithi176.stdout:3/837: fdatasync d4/d28/d31/f62 0 2024-03-13T14:50:00.326 INFO:tasks.workunit.client.1.smithi196.stdout:2/804: dread d0/d6/d34/fbe [0,4194304] 0 2024-03-13T14:50:00.326 INFO:tasks.workunit.client.0.smithi176.stdout:4/799: mknod d7/d8/d33/d96/c10a 0 2024-03-13T14:50:00.330 INFO:tasks.workunit.client.1.smithi196.stdout:0/885: symlink d8/da/d25/d26/d48/l124 0 2024-03-13T14:50:00.330 INFO:tasks.workunit.client.1.smithi196.stdout:7/978: write d0/d4d/dd1/dd6/fe5 [507080,3427] 0 2024-03-13T14:50:00.333 INFO:tasks.workunit.client.1.smithi196.stdout:7/979: fsync d0/d6/d140/d93/fc6 0 2024-03-13T14:50:00.334 INFO:tasks.workunit.client.0.smithi176.stdout:6/800: mkdir d0/de/d38/d7e/d103 0 2024-03-13T14:50:00.335 INFO:tasks.workunit.client.1.smithi196.stdout:2/805: creat d0/d6/d4b/d5f/f106 x:0 0 0 2024-03-13T14:50:00.335 INFO:tasks.workunit.client.1.smithi196.stdout:7/980: read d0/f76 [1230826,63015] 0 2024-03-13T14:50:00.337 INFO:tasks.workunit.client.1.smithi196.stdout:7/981: chown d0/d6/d140/d93/d94 2 1 2024-03-13T14:50:00.337 INFO:tasks.workunit.client.0.smithi176.stdout:2/808: truncate d1/dc/d2d/d59/d80/fe5 1633984 0 2024-03-13T14:50:00.342 INFO:tasks.workunit.client.1.smithi196.stdout:1/957: write d1/d38/f16 [18557,31086] 0 2024-03-13T14:50:00.345 INFO:tasks.workunit.client.1.smithi196.stdout:2/806: creat d0/d6/d34/f107 x:0 0 0 2024-03-13T14:50:00.347 INFO:tasks.workunit.client.1.smithi196.stdout:0/886: chown d8/da/d25/d26/d54/ff3 11927 1 2024-03-13T14:50:00.351 INFO:tasks.workunit.client.1.smithi196.stdout:7/982: dread d0/d9/d1e/d58/d72/f7c [0,4194304] 0 2024-03-13T14:50:00.352 INFO:tasks.workunit.client.0.smithi176.stdout:6/801: creat d0/de/d38/d88/d9d/f104 x:0 0 0 2024-03-13T14:50:00.352 INFO:tasks.workunit.client.1.smithi196.stdout:2/807: chown d0/d7/d9/c1c 280689487 1 2024-03-13T14:50:00.353 INFO:tasks.workunit.client.0.smithi176.stdout:0/697: dwrite d2/d5/d1e/f3e [4194304,4194304] 0 2024-03-13T14:50:00.354 INFO:tasks.workunit.client.1.smithi196.stdout:2/808: chown d0/d7/d40/c4a 28 1 2024-03-13T14:50:00.357 INFO:tasks.workunit.client.0.smithi176.stdout:8/712: dread d0/d5/d11/d47/f6a [0,4194304] 0 2024-03-13T14:50:00.357 INFO:tasks.workunit.client.0.smithi176.stdout:4/800: write d7/d2d/f6b [1441578,32097] 0 2024-03-13T14:50:00.358 INFO:tasks.workunit.client.1.smithi196.stdout:9/855: getdents d7/de/d4b/d1d/d93/dd7 0 2024-03-13T14:50:00.358 INFO:tasks.workunit.client.1.smithi196.stdout:0/887: creat d8/da/d25/d26/d48/d82/f125 x:0 0 0 2024-03-13T14:50:00.358 INFO:tasks.workunit.client.1.smithi196.stdout:9/856: truncate d7/f101 749442 0 2024-03-13T14:50:00.360 INFO:tasks.workunit.client.1.smithi196.stdout:2/809: fsync d0/d6/f84 0 2024-03-13T14:50:00.361 INFO:tasks.workunit.client.1.smithi196.stdout:7/983: mkdir d0/d9/d155 0 2024-03-13T14:50:00.361 INFO:tasks.workunit.client.0.smithi176.stdout:2/809: creat d1/dc/d2d/d59/d80/d7e/da3/d111/f113 x:0 0 0 2024-03-13T14:50:00.362 INFO:tasks.workunit.client.1.smithi196.stdout:9/857: unlink d7/de/lab 0 2024-03-13T14:50:00.363 INFO:tasks.workunit.client.0.smithi176.stdout:1/772: dread d2/d29/f3c [0,4194304] 0 2024-03-13T14:50:00.364 INFO:tasks.workunit.client.0.smithi176.stdout:5/750: dread d5/d2c/d40/f8f [4194304,4194304] 0 2024-03-13T14:50:00.366 INFO:tasks.workunit.client.1.smithi196.stdout:1/958: getdents d1/d9/d4e/dcc 0 2024-03-13T14:50:00.367 INFO:tasks.workunit.client.0.smithi176.stdout:7/739: dwrite d1/d1c/f5e [4194304,4194304] 0 2024-03-13T14:50:00.367 INFO:tasks.workunit.client.1.smithi196.stdout:0/888: mknod d8/da/d25/d26/d48/d82/da0/df2/d108/c126 0 2024-03-13T14:50:00.368 INFO:tasks.workunit.client.0.smithi176.stdout:8/713: read d0/d5/d11/f1b [1227876,15072] 0 2024-03-13T14:50:00.373 INFO:tasks.workunit.client.0.smithi176.stdout:6/802: truncate d0/d10/d19/d27/d2b/de0/dd6/fd1 991483 0 2024-03-13T14:50:00.373 INFO:tasks.workunit.client.0.smithi176.stdout:8/714: chown d0/db/d39 49577421 1 2024-03-13T14:50:00.374 INFO:tasks.workunit.client.1.smithi196.stdout:7/984: rename d0/d6/d33/f40 to d0/d9/d1e/d58/d72/da9/da3/d142/f156 0 2024-03-13T14:50:00.375 INFO:tasks.workunit.client.0.smithi176.stdout:1/773: write d2/de/d3f/d50/d83/dcb/fb7 [2927127,103236] 0 2024-03-13T14:50:00.375 INFO:tasks.workunit.client.1.smithi196.stdout:1/959: dread - d1/d9/d33/d4b/d66/fc0 zero size 2024-03-13T14:50:00.376 INFO:tasks.workunit.client.0.smithi176.stdout:3/838: dread d4/d28/d31/d45/fbc [4194304,4194304] 0 2024-03-13T14:50:00.377 INFO:tasks.workunit.client.0.smithi176.stdout:1/774: chown d2/de/dd1/fd4 34471 1 2024-03-13T14:50:00.378 INFO:tasks.workunit.client.1.smithi196.stdout:1/960: write d1/d38/f16 [397873,8435] 0 2024-03-13T14:50:00.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:59 smithi176 ceph-mon[29064]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:50:00.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:59 smithi176 ceph-mon[29064]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:50:00.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:59 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:59 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:59 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:59 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:49:59 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:00.379 INFO:tasks.workunit.client.1.smithi196.stdout:1/961: chown d1/d3f/d5b 8981478 1 2024-03-13T14:50:00.379 INFO:tasks.workunit.client.0.smithi176.stdout:6/803: read d0/de/d38/d7e/f84 [4697735,25516] 0 2024-03-13T14:50:00.382 INFO:tasks.workunit.client.0.smithi176.stdout:4/801: creat d7/dab/f10b x:0 0 0 2024-03-13T14:50:00.383 INFO:tasks.workunit.client.0.smithi176.stdout:4/802: readlink d7/d8/d45/l73 0 2024-03-13T14:50:00.384 INFO:tasks.workunit.client.0.smithi176.stdout:7/740: creat d1/d3f/d65/fe4 x:0 0 0 2024-03-13T14:50:00.386 INFO:tasks.workunit.client.0.smithi176.stdout:7/741: chown d1/d1c/d4a/f62 14080 1 2024-03-13T14:50:00.390 INFO:tasks.workunit.client.1.smithi196.stdout:9/858: write d7/de/d4b/d1d/fca [754709,81925] 0 2024-03-13T14:50:00.394 INFO:tasks.workunit.client.0.smithi176.stdout:2/810: creat d1/dc/d2d/f114 x:0 0 0 2024-03-13T14:50:00.394 INFO:tasks.workunit.client.1.smithi196.stdout:9/859: write d7/de/d74/d94/fa8 [530106,86482] 0 2024-03-13T14:50:00.398 INFO:tasks.workunit.client.1.smithi196.stdout:2/810: dwrite d0/d6/d34/f7e [0,4194304] 0 2024-03-13T14:50:00.398 INFO:tasks.workunit.client.1.smithi196.stdout:1/962: mkdir d1/d28/d34/d13c/d14d 0 2024-03-13T14:50:00.398 INFO:tasks.workunit.client.0.smithi176.stdout:6/804: symlink d0/d10/d19/d27/d2b/d2e/dc4/l105 0 2024-03-13T14:50:00.398 INFO:tasks.workunit.client.0.smithi176.stdout:4/803: symlink d7/d10/d46/d4a/dcc/l10c 0 2024-03-13T14:50:00.398 INFO:tasks.workunit.client.0.smithi176.stdout:2/811: write d1/dc/d2d/d38/d5d/fe6 [245585,7266] 0 2024-03-13T14:50:00.399 INFO:tasks.workunit.client.0.smithi176.stdout:2/812: fdatasync d1/d2a/f75 0 2024-03-13T14:50:00.401 INFO:tasks.workunit.client.0.smithi176.stdout:4/804: chown d7/d8/d15/dd6/dba/c105 729 1 2024-03-13T14:50:00.404 INFO:tasks.workunit.client.0.smithi176.stdout:4/805: fsync d7/d10/d46/d4a/dcc/dff/f107 0 2024-03-13T14:50:00.404 INFO:tasks.workunit.client.0.smithi176.stdout:6/805: read d0/d10/d19/d27/d2b/d2e/f7a [284443,104476] 0 2024-03-13T14:50:00.407 INFO:tasks.workunit.client.1.smithi196.stdout:0/889: getdents d8/da/d25/d26/d35/d3b/d5f/d33/d44 0 2024-03-13T14:50:00.411 INFO:tasks.workunit.client.1.smithi196.stdout:1/963: symlink d1/d38/l14e 0 2024-03-13T14:50:00.412 INFO:tasks.workunit.client.1.smithi196.stdout:2/811: rmdir d0/d7/d9/d42/d98/de4/d104 39 2024-03-13T14:50:00.413 INFO:tasks.workunit.client.1.smithi196.stdout:7/985: getdents d0/d6/d140/dcf 0 2024-03-13T14:50:00.415 INFO:tasks.workunit.client.1.smithi196.stdout:0/890: mkdir d8/da/d25/d26/d35/d3b/d5f/d33/d127 0 2024-03-13T14:50:00.417 INFO:tasks.workunit.client.1.smithi196.stdout:1/964: read - d1/d28/d34/f10f zero size 2024-03-13T14:50:00.417 INFO:tasks.workunit.client.0.smithi176.stdout:6/806: mkdir d0/d10/d19/d106 0 2024-03-13T14:50:00.420 INFO:tasks.workunit.client.1.smithi196.stdout:7/986: mkdir d0/d6/db4/d13f/d157 0 2024-03-13T14:50:00.424 INFO:tasks.workunit.client.1.smithi196.stdout:7/987: rmdir d0/d9/d31 39 2024-03-13T14:50:00.426 INFO:tasks.workunit.client.0.smithi176.stdout:1/775: dwrite d2/d29/d73/f94 [0,4194304] 0 2024-03-13T14:50:00.426 INFO:tasks.workunit.client.0.smithi176.stdout:5/751: dwrite d5/d2e/d7e/dad/fe1 [0,4194304] 0 2024-03-13T14:50:00.428 INFO:tasks.workunit.client.0.smithi176.stdout:7/742: write d1/d28/d3a/d1b/d8f/f99 [446702,52034] 0 2024-03-13T14:50:00.434 INFO:tasks.workunit.client.1.smithi196.stdout:7/988: rename d0/d9/d1e/d3f to d0/d6/d33/d110/d158 0 2024-03-13T14:50:00.435 INFO:tasks.workunit.client.0.smithi176.stdout:9/893: sync 2024-03-13T14:50:00.439 INFO:tasks.workunit.client.1.smithi196.stdout:0/891: dwrite d8/da/d25/d26/d35/d3b/d5f/d17/f4d [0,4194304] 0 2024-03-13T14:50:00.440 INFO:tasks.workunit.client.0.smithi176.stdout:4/806: dwrite d7/d8/d45/fdb [0,4194304] 0 2024-03-13T14:50:00.440 INFO:tasks.workunit.client.0.smithi176.stdout:9/894: stat d0/d1e/df7 0 2024-03-13T14:50:00.442 INFO:tasks.workunit.client.0.smithi176.stdout:2/813: dread d1/dc/d2d/d59/d80/d5c/f5f [0,4194304] 0 2024-03-13T14:50:00.444 INFO:tasks.workunit.client.1.smithi196.stdout:9/860: dwrite d7/de/d74/d94/dfb/d28/f82 [0,4194304] 0 2024-03-13T14:50:00.445 INFO:tasks.workunit.client.0.smithi176.stdout:5/752: link d5/d2e/d5f/fdf d5/d2e/d5f/f102 0 2024-03-13T14:50:00.453 INFO:tasks.workunit.client.0.smithi176.stdout:5/753: rename d5/d2c/d40/dc7/d89/df2 to d5/d2c/d40/dc7/d89/df2/d103 22 2024-03-13T14:50:00.455 INFO:tasks.workunit.client.1.smithi196.stdout:0/892: rename d8/da/d25/d26/d35/d3b/f7d to d8/d37/f128 0 2024-03-13T14:50:00.458 INFO:tasks.workunit.client.0.smithi176.stdout:2/814: creat d1/dc/d2d/d59/d80/d7e/f115 x:0 0 0 2024-03-13T14:50:00.458 INFO:tasks.workunit.client.0.smithi176.stdout:5/754: chown d5/d31/d47/f85 59262603 1 2024-03-13T14:50:00.459 INFO:tasks.workunit.client.1.smithi196.stdout:5/890: sync 2024-03-13T14:50:00.459 INFO:tasks.workunit.client.1.smithi196.stdout:0/893: symlink d8/l129 0 2024-03-13T14:50:00.460 INFO:tasks.workunit.client.0.smithi176.stdout:1/776: dwrite d2/de/d3f/d3e/f5b [0,4194304] 0 2024-03-13T14:50:00.464 INFO:tasks.workunit.client.1.smithi196.stdout:1/965: write d1/d28/d34/f9c [370500,80604] 0 2024-03-13T14:50:00.470 INFO:tasks.workunit.client.1.smithi196.stdout:2/812: dread d0/d6/d4b/d5f/fb7 [0,4194304] 0 2024-03-13T14:50:00.474 INFO:tasks.workunit.client.1.smithi196.stdout:0/894: getdents d8/da/d25/d26/d35/d3b/d6c 0 2024-03-13T14:50:00.474 INFO:tasks.workunit.client.0.smithi176.stdout:9/895: dwrite d0/db/d89/d90/fe5 [0,4194304] 0 2024-03-13T14:50:00.474 INFO:tasks.workunit.client.0.smithi176.stdout:2/815: mknod d1/dc/c116 0 2024-03-13T14:50:00.477 INFO:tasks.workunit.client.1.smithi196.stdout:7/989: dwrite d0/d9/d31/f12c [0,4194304] 0 2024-03-13T14:50:00.477 INFO:tasks.workunit.client.0.smithi176.stdout:6/807: write d0/de/d54/fb8 [959637,64655] 0 2024-03-13T14:50:00.479 INFO:tasks.workunit.client.1.smithi196.stdout:1/966: mknod d1/d9/d33/d4b/c14f 0 2024-03-13T14:50:00.479 INFO:tasks.workunit.client.0.smithi176.stdout:1/777: mkdir d2/de/d3f/d50/d83/ded 0 2024-03-13T14:50:00.489 INFO:tasks.workunit.client.0.smithi176.stdout:9/896: symlink d0/db/d89/d90/l12b 0 2024-03-13T14:50:00.491 INFO:tasks.workunit.client.1.smithi196.stdout:0/895: mknod d8/da/d25/d26/d35/c12a 0 2024-03-13T14:50:00.495 INFO:tasks.workunit.client.1.smithi196.stdout:7/990: creat d0/d78/d81/f159 x:0 0 0 2024-03-13T14:50:00.498 INFO:tasks.workunit.client.0.smithi176.stdout:5/755: rename d5/d2c/d40/d76/cdb to d5/d2c/d40/c104 0 2024-03-13T14:50:00.499 INFO:tasks.workunit.client.1.smithi196.stdout:9/861: write d7/de/d4b/d1d/f23 [1491095,19412] 0 2024-03-13T14:50:00.500 INFO:tasks.workunit.client.1.smithi196.stdout:1/967: dwrite d1/d9/f13b [0,4194304] 0 2024-03-13T14:50:00.501 INFO:tasks.workunit.client.0.smithi176.stdout:2/816: symlink d1/dc/d2d/d4b/ddb/l117 0 2024-03-13T14:50:00.504 INFO:tasks.workunit.client.1.smithi196.stdout:5/891: write d4/d1b/fc2 [1941468,7491] 0 2024-03-13T14:50:00.504 INFO:tasks.workunit.client.0.smithi176.stdout:4/807: dwrite d7/d8/f4f [0,4194304] 0 2024-03-13T14:50:00.504 INFO:tasks.workunit.client.0.smithi176.stdout:2/817: truncate d1/dc/d2d/f114 120508 0 2024-03-13T14:50:00.506 INFO:tasks.workunit.client.1.smithi196.stdout:2/813: write d0/d6/d34/d5b/f95 [321035,101528] 0 2024-03-13T14:50:00.507 INFO:tasks.workunit.client.0.smithi176.stdout:2/818: stat d1/dc/d2d/d38/l41 0 2024-03-13T14:50:00.508 INFO:tasks.workunit.client.1.smithi196.stdout:7/991: dread - d0/d78/fd5 zero size 2024-03-13T14:50:00.512 INFO:tasks.workunit.client.0.smithi176.stdout:5/756: creat d5/d2c/d8a/f105 x:0 0 0 2024-03-13T14:50:00.512 INFO:tasks.workunit.client.1.smithi196.stdout:9/862: rename d7/de/d74/d94/dfb/d28/l37 to d7/de/d16/l127 0 2024-03-13T14:50:00.512 INFO:tasks.workunit.client.1.smithi196.stdout:7/992: fsync d0/d9/d1e/d58/d61/f13c 0 2024-03-13T14:50:00.513 INFO:tasks.workunit.client.0.smithi176.stdout:6/808: mknod d0/d10/d19/d27/d2b/c107 0 2024-03-13T14:50:00.515 INFO:tasks.workunit.client.1.smithi196.stdout:1/968: symlink d1/d9/l150 0 2024-03-13T14:50:00.516 INFO:tasks.workunit.client.0.smithi176.stdout:9/897: mkdir d0/db/d12c 0 2024-03-13T14:50:00.518 INFO:tasks.workunit.client.0.smithi176.stdout:1/778: rmdir d2/d30/d63/d22/de7 0 2024-03-13T14:50:00.522 INFO:tasks.workunit.client.1.smithi196.stdout:7/993: rmdir d0/d6/d33/d57 39 2024-03-13T14:50:00.527 INFO:tasks.workunit.client.1.smithi196.stdout:5/892: rename d4/d1b/dc8/d14/d37/fee to d4/d1b/dc8/d14/d19/da7/d105/f119 0 2024-03-13T14:50:00.528 INFO:tasks.workunit.client.0.smithi176.stdout:2/819: mkdir d1/dc/d2d/d4b/d118 0 2024-03-13T14:50:00.532 INFO:tasks.workunit.client.1.smithi196.stdout:3/842: sync 2024-03-13T14:50:00.532 INFO:tasks.workunit.client.1.smithi196.stdout:8/820: sync 2024-03-13T14:50:00.533 INFO:tasks.workunit.client.1.smithi196.stdout:4/857: sync 2024-03-13T14:50:00.535 INFO:tasks.workunit.client.1.smithi196.stdout:0/896: truncate d8/ff9 617499 0 2024-03-13T14:50:00.538 INFO:tasks.workunit.client.1.smithi196.stdout:2/814: dwrite d0/d7/d9/d42/d98/de4/d104/f30 [0,4194304] 0 2024-03-13T14:50:00.540 INFO:tasks.workunit.client.1.smithi196.stdout:9/863: creat d7/de/f128 x:0 0 0 2024-03-13T14:50:00.548 INFO:tasks.workunit.client.1.smithi196.stdout:1/969: rename d1/fcf to d1/d9/d33/f151 0 2024-03-13T14:50:00.550 INFO:tasks.workunit.client.1.smithi196.stdout:0/897: creat d8/da/d25/d26/d48/d82/da0/df2/d108/f12b x:0 0 0 2024-03-13T14:50:00.552 INFO:tasks.workunit.client.1.smithi196.stdout:7/994: creat d0/d6/d140/f15a x:0 0 0 2024-03-13T14:50:00.554 INFO:tasks.workunit.client.1.smithi196.stdout:0/898: chown d8/da/d25/d26/d35/d3b/d5f/d33/dfd 22 1 2024-03-13T14:50:00.555 INFO:tasks.workunit.client.1.smithi196.stdout:1/970: fdatasync d1/d9/d33/d4b/d66/f7c 0 2024-03-13T14:50:00.558 INFO:tasks.workunit.client.1.smithi196.stdout:4/858: mknod d1/d3/d7/d35/d76/d58/d69/de3/c12f 0 2024-03-13T14:50:00.559 INFO:tasks.workunit.client.1.smithi196.stdout:1/971: write d1/d38/f16 [4649591,47374] 0 2024-03-13T14:50:00.559 INFO:tasks.workunit.client.1.smithi196.stdout:0/899: chown d8/da/d25/d26/d35/d3b/d59/d6a 17535232 1 2024-03-13T14:50:00.568 INFO:tasks.workunit.client.1.smithi196.stdout:1/972: dread d1/d38/fef [0,4194304] 0 2024-03-13T14:50:00.569 INFO:tasks.workunit.client.1.smithi196.stdout:3/843: write db/d15/d44/fb0 [1462299,69218] 0 2024-03-13T14:50:00.575 INFO:tasks.workunit.client.1.smithi196.stdout:2/815: creat d0/d6/f108 x:0 0 0 2024-03-13T14:50:00.576 INFO:tasks.workunit.client.1.smithi196.stdout:8/821: dwrite d0/d63/f85 [0,4194304] 0 2024-03-13T14:50:00.581 INFO:tasks.workunit.client.1.smithi196.stdout:5/893: dwrite d4/d1b/dc8/d14/d19/d1c/d25/f8e [0,4194304] 0 2024-03-13T14:50:00.581 INFO:tasks.workunit.client.1.smithi196.stdout:0/900: rename d8/da/d74 to d8/da/d25/d26/d48/d82/db3/d12c 0 2024-03-13T14:50:00.585 INFO:tasks.workunit.client.1.smithi196.stdout:9/864: write d7/de/d4b/d1d/d93/dd7/f118 [264426,98342] 0 2024-03-13T14:50:00.589 INFO:tasks.workunit.client.1.smithi196.stdout:9/865: readlink d7/de/d4b/d1d/d93/la3 0 2024-03-13T14:50:00.589 INFO:tasks.workunit.client.1.smithi196.stdout:5/894: stat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d 0 2024-03-13T14:50:00.591 INFO:tasks.workunit.client.1.smithi196.stdout:9/866: dread - d7/de/d16/d70/feb zero size 2024-03-13T14:50:00.594 INFO:tasks.workunit.client.0.smithi176.stdout:0/698: sync 2024-03-13T14:50:00.594 INFO:tasks.workunit.client.0.smithi176.stdout:8/715: sync 2024-03-13T14:50:00.595 INFO:tasks.workunit.client.0.smithi176.stdout:0/699: stat d2/db 0 2024-03-13T14:50:00.596 INFO:tasks.workunit.client.0.smithi176.stdout:8/716: write d0/d24/f7d [685098,112352] 0 2024-03-13T14:50:00.597 INFO:tasks.workunit.client.1.smithi196.stdout:4/859: rename d1/d3/d7/d35/d76/d58/dc9/f12c to d1/d3/d7/d35/d76/d58/dc9/f130 0 2024-03-13T14:50:00.598 INFO:tasks.workunit.client.1.smithi196.stdout:3/844: mkdir db/d19/d50/db7/da9/dc0/d10e 0 2024-03-13T14:50:00.599 INFO:tasks.workunit.client.1.smithi196.stdout:7/995: write d0/d6/d140/f11a [432598,128613] 0 2024-03-13T14:50:00.602 INFO:tasks.workunit.client.1.smithi196.stdout:3/845: chown db/d15/d44/ff5 4905 1 2024-03-13T14:50:00.605 INFO:tasks.workunit.client.1.smithi196.stdout:3/846: chown db/d15/d44/d51/fc6 77518 1 2024-03-13T14:50:00.606 INFO:tasks.workunit.client.1.smithi196.stdout:8/822: creat d0/d16/d3d/d8b/d48/f116 x:0 0 0 2024-03-13T14:50:00.611 INFO:tasks.workunit.client.1.smithi196.stdout:5/895: mknod d4/d1b/dc8/d4a/c11a 0 2024-03-13T14:50:00.612 INFO:tasks.workunit.client.1.smithi196.stdout:9/867: symlink d7/de/d4b/d1d/d93/dd7/df1/l129 0 2024-03-13T14:50:00.620 INFO:tasks.workunit.client.1.smithi196.stdout:5/896: dread d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/fb8 [0,4194304] 0 2024-03-13T14:50:00.620 INFO:tasks.workunit.client.0.smithi176.stdout:9/898: fdatasync d0/db/f3f 0 2024-03-13T14:50:00.622 INFO:tasks.workunit.client.0.smithi176.stdout:4/808: getdents d7/d8/d15/d23 0 2024-03-13T14:50:00.626 INFO:tasks.workunit.client.1.smithi196.stdout:9/868: rename d7/d71/faf to d7/de/d4b/f12a 0 2024-03-13T14:50:00.629 INFO:tasks.workunit.client.1.smithi196.stdout:9/869: chown d7/cdb 81884 1 2024-03-13T14:50:00.630 INFO:tasks.workunit.client.0.smithi176.stdout:0/700: chown d2/d5/f87 6 1 2024-03-13T14:50:00.630 INFO:tasks.workunit.client.0.smithi176.stdout:5/757: rename d5/d22/fb3 to d5/d2c/f106 0 2024-03-13T14:50:00.631 INFO:tasks.workunit.client.1.smithi196.stdout:2/816: dwrite d0/d7/d9/d61/fb6 [0,4194304] 0 2024-03-13T14:50:00.631 INFO:tasks.workunit.client.0.smithi176.stdout:0/701: fdatasync d2/f6 0 2024-03-13T14:50:00.631 INFO:tasks.workunit.client.1.smithi196.stdout:0/901: dread d8/da/d25/d26/d35/d3b/d6c/fc6 [0,4194304] 0 2024-03-13T14:50:00.632 INFO:tasks.workunit.client.1.smithi196.stdout:2/817: stat d0/d6/d4b/f79 0 2024-03-13T14:50:00.637 INFO:tasks.workunit.client.1.smithi196.stdout:5/897: dread d4/d1b/dc8/d4a/fa3 [0,4194304] 0 2024-03-13T14:50:00.639 INFO:tasks.workunit.client.0.smithi176.stdout:6/809: sync 2024-03-13T14:50:00.639 INFO:tasks.workunit.client.0.smithi176.stdout:9/899: dwrite d0/d1e/d8d/dbe/f10e [0,4194304] 0 2024-03-13T14:50:00.640 INFO:tasks.workunit.client.0.smithi176.stdout:3/839: sync 2024-03-13T14:50:00.640 INFO:tasks.workunit.client.0.smithi176.stdout:1/779: sync 2024-03-13T14:50:00.640 INFO:tasks.workunit.client.0.smithi176.stdout:7/743: sync 2024-03-13T14:50:00.640 INFO:tasks.workunit.client.0.smithi176.stdout:4/809: rmdir d7/db9 39 2024-03-13T14:50:00.640 INFO:tasks.workunit.client.1.smithi196.stdout:5/898: stat d4/d1b/dc8/d14/lf6 0 2024-03-13T14:50:00.640 INFO:tasks.workunit.client.1.smithi196.stdout:9/870: mkdir d7/de/de5/d12b 0 2024-03-13T14:50:00.648 INFO:tasks.workunit.client.0.smithi176.stdout:5/758: fsync d5/d2e/d5f/fd3 0 2024-03-13T14:50:00.651 INFO:tasks.workunit.client.1.smithi196.stdout:4/860: getdents d1/d3/d7/d35/d76/d58/dc9 0 2024-03-13T14:50:00.652 INFO:tasks.workunit.client.0.smithi176.stdout:0/702: mknod d2/db/d35/cdf 0 2024-03-13T14:50:00.657 INFO:tasks.workunit.client.0.smithi176.stdout:3/840: stat d4/d13/d16/f84 0 2024-03-13T14:50:00.658 INFO:tasks.workunit.client.1.smithi196.stdout:5/899: creat d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd/f11b x:0 0 0 2024-03-13T14:50:00.659 INFO:tasks.workunit.client.0.smithi176.stdout:3/841: readlink d4/d28/l63 0 2024-03-13T14:50:00.659 INFO:tasks.workunit.client.0.smithi176.stdout:9/900: symlink d0/d1e/d8d/dbe/de8/l12d 0 2024-03-13T14:50:00.661 INFO:tasks.workunit.client.1.smithi196.stdout:0/902: mknod d8/da/d25/d26/d35/d3b/d50/c12d 0 2024-03-13T14:50:00.661 INFO:tasks.workunit.client.1.smithi196.stdout:7/996: dwrite d0/d4d/fc4 [0,4194304] 0 2024-03-13T14:50:00.663 INFO:tasks.workunit.client.0.smithi176.stdout:4/810: fdatasync d7/d8/f3b 0 2024-03-13T14:50:00.665 INFO:tasks.workunit.client.1.smithi196.stdout:9/871: mkdir d7/de/d4b/d1d/d93/dd7/df1/d115/d12c 0 2024-03-13T14:50:00.666 INFO:tasks.workunit.client.0.smithi176.stdout:7/744: mkdir d1/d28/d3a/d1b/d8f/dc1/dbb/de5 0 2024-03-13T14:50:00.667 INFO:tasks.workunit.client.1.smithi196.stdout:0/903: dread - d8/da/d25/d26/d35/d3b/d5f/d33/d44/fdf zero size 2024-03-13T14:50:00.668 INFO:tasks.workunit.client.1.smithi196.stdout:2/818: dwrite d0/d7/f89 [0,4194304] 0 2024-03-13T14:50:00.669 INFO:tasks.workunit.client.1.smithi196.stdout:9/872: stat d7/de/d4b/d1d/d93/dd7/df1/d115/d12c 0 2024-03-13T14:50:00.672 INFO:tasks.workunit.client.1.smithi196.stdout:9/873: stat d7/de/d74/fad 0 2024-03-13T14:50:00.673 INFO:tasks.workunit.client.0.smithi176.stdout:1/780: mkdir d2/de/d10/d16/d47/db4/dee 0 2024-03-13T14:50:00.674 INFO:tasks.workunit.client.0.smithi176.stdout:2/820: getdents d1/dc/d2d/d59/d80/d7e/da3 0 2024-03-13T14:50:00.676 INFO:tasks.workunit.client.0.smithi176.stdout:1/781: chown d2/de/dd1 952761197 1 2024-03-13T14:50:00.678 INFO:tasks.workunit.client.0.smithi176.stdout:6/810: creat d0/d10/dca/dd2/f108 x:0 0 0 2024-03-13T14:50:00.679 INFO:tasks.workunit.client.0.smithi176.stdout:1/782: readlink d2/de/d3f/d50/d83/dcb/le6 0 2024-03-13T14:50:00.682 INFO:tasks.workunit.client.1.smithi196.stdout:7/997: dwrite d0/fa [0,4194304] 0 2024-03-13T14:50:00.686 INFO:tasks.workunit.client.1.smithi196.stdout:2/819: unlink d0/d7/d40/c43 0 2024-03-13T14:50:00.687 INFO:tasks.workunit.client.0.smithi176.stdout:5/759: dread d5/d2e/fd2 [0,4194304] 0 2024-03-13T14:50:00.688 INFO:tasks.workunit.client.0.smithi176.stdout:8/717: rename d0/d24/d29/c55 to d0/d24/d98/ce6 0 2024-03-13T14:50:00.688 INFO:tasks.workunit.client.1.smithi196.stdout:2/820: fdatasync d0/d7/d8e/fdd 0 2024-03-13T14:50:00.689 INFO:tasks.workunit.client.0.smithi176.stdout:8/718: readlink d0/db/l8f 0 2024-03-13T14:50:00.690 INFO:tasks.workunit.client.1.smithi196.stdout:2/821: stat d0/d7/d9/d5a/df2 0 2024-03-13T14:50:00.696 INFO:tasks.workunit.client.0.smithi176.stdout:4/811: unlink d7/d10/c17 0 2024-03-13T14:50:00.696 INFO:tasks.workunit.client.1.smithi196.stdout:4/861: link d1/d3/d7/d35/d76/d58/dc9/dd7/ce1 d1/d3/d7/d35/d76/d58/dc6/c131 0 2024-03-13T14:50:00.699 INFO:tasks.workunit.client.1.smithi196.stdout:0/904: fsync d8/da/d25/d26/d54/d88/f84 0 2024-03-13T14:50:00.702 INFO:tasks.workunit.client.1.smithi196.stdout:7/998: creat d0/d6/d33/d110/d158/d6c/f15b x:0 0 0 2024-03-13T14:50:00.702 INFO:tasks.workunit.client.1.smithi196.stdout:0/905: write d8/da/d25/d26/f2d [1456190,31109] 0 2024-03-13T14:50:00.705 INFO:tasks.workunit.client.1.smithi196.stdout:2/822: creat d0/d6/d4b/d5f/f109 x:0 0 0 2024-03-13T14:50:00.709 INFO:tasks.workunit.client.0.smithi176.stdout:3/842: creat d4/d28/d31/d52/dff/f127 x:0 0 0 2024-03-13T14:50:00.711 INFO:tasks.workunit.client.0.smithi176.stdout:3/843: dread - d4/d28/d31/d45/d82/dce/f11d zero size 2024-03-13T14:50:00.715 INFO:tasks.workunit.client.1.smithi196.stdout:4/862: truncate d1/d3/d7/d35/d3e/fe4 844641 0 2024-03-13T14:50:00.716 INFO:tasks.workunit.client.1.smithi196.stdout:5/900: write d4/d1b/dc8/d14/d19/d1c/d25/d38/f56 [2471921,110693] 0 2024-03-13T14:50:00.716 INFO:tasks.workunit.client.0.smithi176.stdout:5/760: symlink d5/d2c/d8a/l107 0 2024-03-13T14:50:00.717 INFO:tasks.workunit.client.1.smithi196.stdout:4/863: readlink d1/d1e/d43/l73 0 2024-03-13T14:50:00.718 INFO:tasks.workunit.client.0.smithi176.stdout:5/761: chown d5/f9a 2 1 2024-03-13T14:50:00.719 INFO:tasks.workunit.client.1.smithi196.stdout:7/999: stat d0/d6/d33/d57/dae/de8 0 2024-03-13T14:50:00.719 INFO:tasks.workunit.client.0.smithi176.stdout:4/812: creat d7/d8/d2f/f10d x:0 0 0 2024-03-13T14:50:00.720 INFO:tasks.workunit.client.0.smithi176.stdout:4/813: stat d7/d10/f27 0 2024-03-13T14:50:00.720 INFO:tasks.workunit.client.1.smithi196.stdout:0/906: fsync d8/da/d25/d26/d54/d88/f84 0 2024-03-13T14:50:00.723 INFO:tasks.workunit.client.1.smithi196.stdout:2/823: fsync d0/f37 0 2024-03-13T14:50:00.724 INFO:tasks.workunit.client.1.smithi196.stdout:9/874: link d7/d9/d83/fc0 d7/de/d4b/d1d/d35/dcc/f12d 0 2024-03-13T14:50:00.725 INFO:tasks.workunit.client.0.smithi176.stdout:2/821: write d1/dc/d2d/d4b/ddb/fdc [63730,27583] 0 2024-03-13T14:50:00.726 INFO:tasks.workunit.client.1.smithi196.stdout:4/864: dread - d1/d1e/d5c/ff7 zero size 2024-03-13T14:50:00.729 INFO:tasks.workunit.client.1.smithi196.stdout:5/901: truncate d4/d1b/dc8/d14/d19/d1c/f23 1184965 0 2024-03-13T14:50:00.734 INFO:tasks.workunit.client.1.smithi196.stdout:2/824: truncate d0/d6/d4b/d96/fa7 550673 0 2024-03-13T14:50:00.736 INFO:tasks.workunit.client.0.smithi176.stdout:9/901: rename d0/db/d89/d7a/ff3 to d0/d17/f12e 0 2024-03-13T14:50:00.738 INFO:tasks.workunit.client.1.smithi196.stdout:4/865: write d1/d3/d4c/d72/d7d/d80/fac [4731613,43491] 0 2024-03-13T14:50:00.739 INFO:tasks.workunit.client.0.smithi176.stdout:6/811: truncate d0/de/d38/d70/f9e 149585 0 2024-03-13T14:50:00.740 INFO:tasks.workunit.client.0.smithi176.stdout:5/762: write d5/d2c/d40/dc7/fda [6606107,111294] 0 2024-03-13T14:50:00.740 INFO:tasks.workunit.client.0.smithi176.stdout:3/844: write d4/d28/d34/d11c/f95 [4679008,99914] 0 2024-03-13T14:50:00.742 INFO:tasks.workunit.client.0.smithi176.stdout:7/745: creat d1/d1c/d26/d41/d2b/fe6 x:0 0 0 2024-03-13T14:50:00.743 INFO:tasks.workunit.client.0.smithi176.stdout:3/845: write d4/d28/d31/d52/f6a [4033584,31161] 0 2024-03-13T14:50:00.744 INFO:tasks.workunit.client.0.smithi176.stdout:4/814: mkdir d7/d10/d46/d4a/d5d/da1/de8/d10e 0 2024-03-13T14:50:00.747 INFO:tasks.workunit.client.0.smithi176.stdout:0/703: getdents d2/d5/d28/d8c 0 2024-03-13T14:50:00.747 INFO:tasks.workunit.client.0.smithi176.stdout:2/822: creat d1/dc/d2d/d59/d80/d7e/f119 x:0 0 0 2024-03-13T14:50:00.749 INFO:tasks.workunit.client.0.smithi176.stdout:0/704: fsync d2/d74/f91 0 2024-03-13T14:50:00.750 INFO:tasks.workunit.client.0.smithi176.stdout:1/783: creat d2/de/d3f/fef x:0 0 0 2024-03-13T14:50:00.750 INFO:tasks.workunit.client.1.smithi196.stdout:2/825: symlink d0/d7/d9/d61/l10a 0 2024-03-13T14:50:00.751 INFO:tasks.workunit.client.0.smithi176.stdout:3/846: truncate d4/d28/d31/d52/dff/f127 975240 0 2024-03-13T14:50:00.752 INFO:tasks.workunit.client.0.smithi176.stdout:2/823: fdatasync d1/dc/d61/daa/fe4 0 2024-03-13T14:50:00.753 INFO:tasks.workunit.client.0.smithi176.stdout:1/784: readlink d2/d29/l8d 0 2024-03-13T14:50:00.754 INFO:tasks.workunit.client.1.smithi196.stdout:0/907: truncate d8/da/d25/d26/d35/d3b/d50/d55/fd4 2833550 0 2024-03-13T14:50:00.755 INFO:tasks.workunit.client.1.smithi196.stdout:0/908: stat d8/da/d25/d26/d35/d3b/d50/d5d/f57 0 2024-03-13T14:50:00.755 INFO:tasks.workunit.client.1.smithi196.stdout:4/866: symlink d1/d3/d7/d35/d3e/l132 0 2024-03-13T14:50:00.756 INFO:tasks.workunit.client.0.smithi176.stdout:6/812: mkdir d0/de/d38/d70/d109 0 2024-03-13T14:50:00.758 INFO:tasks.workunit.client.0.smithi176.stdout:9/902: dwrite d0/d40/d53/fba [0,4194304] 0 2024-03-13T14:50:00.760 INFO:tasks.workunit.client.1.smithi196.stdout:9/875: truncate d7/de/d4b/d1d/f66 6620344 0 2024-03-13T14:50:00.762 INFO:tasks.workunit.client.1.smithi196.stdout:5/902: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/d101/f11c x:0 0 0 2024-03-13T14:50:00.763 INFO:tasks.workunit.client.1.smithi196.stdout:2/826: truncate d0/d7/d9/d9a/daa/fd1 244060 0 2024-03-13T14:50:00.765 INFO:tasks.workunit.client.0.smithi176.stdout:7/746: write d1/d28/d3a/d1b/fa6 [1030924,11024] 0 2024-03-13T14:50:00.765 INFO:tasks.workunit.client.0.smithi176.stdout:4/815: symlink d7/d10/d46/d4a/d5d/da1/de8/l10f 0 2024-03-13T14:50:00.769 INFO:tasks.workunit.client.1.smithi196.stdout:2/827: write d0/d7/d9/d42/d98/de4/fe6 [250439,28808] 0 2024-03-13T14:50:00.770 INFO:tasks.workunit.client.1.smithi196.stdout:4/867: creat d1/d3/d7/d35/d76/d58/dc9/f133 x:0 0 0 2024-03-13T14:50:00.771 INFO:tasks.workunit.client.0.smithi176.stdout:1/785: dread d2/d29/d73/f94 [0,4194304] 0 2024-03-13T14:50:00.772 INFO:tasks.workunit.client.1.smithi196.stdout:9/876: mkdir d7/de/d16/d70/d12e 0 2024-03-13T14:50:00.774 INFO:tasks.workunit.client.1.smithi196.stdout:0/909: chown d8/da/d25/d26/d35/d3b/d50/d55/fd4 1145275346 1 2024-03-13T14:50:00.777 INFO:tasks.workunit.client.0.smithi176.stdout:2/824: readlink d1/dc/d2d/d4f/l91 0 2024-03-13T14:50:00.781 INFO:tasks.workunit.client.1.smithi196.stdout:2/828: symlink d0/d7/d9/d5a/dbc/l10b 0 2024-03-13T14:50:00.784 INFO:tasks.workunit.client.0.smithi176.stdout:9/903: dwrite d0/dd8/ff0 [0,4194304] 0 2024-03-13T14:50:00.785 INFO:tasks.workunit.client.0.smithi176.stdout:4/816: fsync d7/dab/db5/fc6 0 2024-03-13T14:50:00.787 INFO:tasks.workunit.client.0.smithi176.stdout:6/813: dwrite d0/d10/d19/d27/d2b/d2e/d3e/db4/fdd [0,4194304] 0 2024-03-13T14:50:00.790 INFO:tasks.workunit.client.0.smithi176.stdout:6/814: fdatasync d0/d9/f55 0 2024-03-13T14:50:00.791 INFO:tasks.workunit.client.1.smithi196.stdout:9/877: rename d7/de/d74/d94/dfb/d28/d6c/fc2 to d7/d9/f12f 0 2024-03-13T14:50:00.794 INFO:tasks.workunit.client.1.smithi196.stdout:5/903: dread d4/d1b/dc8/d3c/f57 [0,4194304] 0 2024-03-13T14:50:00.800 INFO:tasks.workunit.client.1.smithi196.stdout:2/829: rename d0/d7/d9/lc1 to d0/d7/d9/d42/d98/de4/d67/l10c 0 2024-03-13T14:50:00.801 INFO:tasks.workunit.client.0.smithi176.stdout:2/825: write d1/dc/d1c/f37 [4863416,84966] 0 2024-03-13T14:50:00.804 INFO:tasks.workunit.client.1.smithi196.stdout:5/904: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/f11d x:0 0 0 2024-03-13T14:50:00.806 INFO:tasks.workunit.client.0.smithi176.stdout:8/719: rename d0/d24/d29/f72 to d0/d24/d98/fe7 0 2024-03-13T14:50:00.808 INFO:tasks.workunit.client.0.smithi176.stdout:4/817: creat d7/d10/d2c/f110 x:0 0 0 2024-03-13T14:50:00.810 INFO:tasks.workunit.client.1.smithi196.stdout:9/878: dread d7/de/d4b/f59 [0,4194304] 0 2024-03-13T14:50:00.810 INFO:tasks.workunit.client.1.smithi196.stdout:0/910: dread d8/da/d25/f7b [0,4194304] 0 2024-03-13T14:50:00.815 INFO:tasks.workunit.client.0.smithi176.stdout:6/815: unlink d0/d10/d19/d27/d2b/d2e/d3e/cb2 0 2024-03-13T14:50:00.815 INFO:tasks.workunit.client.0.smithi176.stdout:0/705: getdents d2/d5 0 2024-03-13T14:50:00.815 INFO:tasks.workunit.client.0.smithi176.stdout:5/763: write d5/d2b/fcd [186636,6524] 0 2024-03-13T14:50:00.816 INFO:tasks.workunit.client.0.smithi176.stdout:2/826: creat d1/dc/d2d/d59/d80/d7e/da3/f11a x:0 0 0 2024-03-13T14:50:00.819 INFO:tasks.workunit.client.1.smithi196.stdout:9/879: rmdir d7/de/d74/d94/dfb/d28/de4 39 2024-03-13T14:50:00.819 INFO:tasks.workunit.client.0.smithi176.stdout:6/816: chown d0/c2f 264724906 1 2024-03-13T14:50:00.820 INFO:tasks.workunit.client.1.smithi196.stdout:4/868: dread d1/d1e/d5c/f5f [0,4194304] 0 2024-03-13T14:50:00.821 INFO:tasks.workunit.client.0.smithi176.stdout:1/786: link d2/d30/d63/d22/c57 d2/de/d10/d16/d47/db0/cf0 0 2024-03-13T14:50:00.821 INFO:tasks.workunit.client.0.smithi176.stdout:5/764: write f0 [615319,8296] 0 2024-03-13T14:50:00.824 INFO:tasks.workunit.client.1.smithi196.stdout:4/869: stat d1/d3/d4c/d72/d7d/d80/d55 0 2024-03-13T14:50:00.825 INFO:tasks.workunit.client.0.smithi176.stdout:0/706: creat d2/d5/dc/d59/fe0 x:0 0 0 2024-03-13T14:50:00.827 INFO:tasks.workunit.client.0.smithi176.stdout:7/747: dread d1/d28/d3a/f20 [0,4194304] 0 2024-03-13T14:50:00.828 INFO:tasks.workunit.client.1.smithi196.stdout:5/905: dwrite d4/d1b/dc8/d14/d19/d26/f51 [0,4194304] 0 2024-03-13T14:50:00.828 INFO:tasks.workunit.client.0.smithi176.stdout:4/818: dread d7/d8/df6/f67 [0,4194304] 0 2024-03-13T14:50:00.830 INFO:tasks.workunit.client.1.smithi196.stdout:2/830: dwrite d0/d7/d9/d42/d98/de4/d104/f5e [0,4194304] 0 2024-03-13T14:50:00.833 INFO:tasks.workunit.client.0.smithi176.stdout:8/720: dwrite d0/d24/d29/d82/fd0 [0,4194304] 0 2024-03-13T14:50:00.837 INFO:tasks.workunit.client.0.smithi176.stdout:8/721: write d0/d24/f6f [4435552,93639] 0 2024-03-13T14:50:00.838 INFO:tasks.workunit.client.1.smithi196.stdout:5/906: creat d4/d1b/dc8/d14/d19/d26/d95/f11e x:0 0 0 2024-03-13T14:50:00.842 INFO:tasks.workunit.client.1.smithi196.stdout:3/847: sync 2024-03-13T14:50:00.842 INFO:tasks.workunit.client.1.smithi196.stdout:8/823: sync 2024-03-13T14:50:00.842 INFO:tasks.workunit.client.1.smithi196.stdout:1/973: sync 2024-03-13T14:50:00.842 INFO:tasks.workunit.client.1.smithi196.stdout:9/880: link d7/de/d74/d94/dfb/lce d7/de/d4b/d1d/d93/da7/l130 0 2024-03-13T14:50:00.842 INFO:tasks.workunit.client.0.smithi176.stdout:1/787: symlink d2/de/d3f/d19/lf1 0 2024-03-13T14:50:00.842 INFO:tasks.workunit.client.0.smithi176.stdout:7/748: rmdir d1/d1c/d4a 39 2024-03-13T14:50:00.849 INFO:tasks.workunit.client.0.smithi176.stdout:1/788: fsync d2/de/d10/d16/d59/fa1 0 2024-03-13T14:50:00.852 INFO:tasks.workunit.client.0.smithi176.stdout:2/827: link d1/dc/d2d/d59/d80/d7e/da3/d111/f113 d1/dc/d2d/d4f/f11b 0 2024-03-13T14:50:00.857 INFO:tasks.workunit.client.1.smithi196.stdout:8/824: rmdir d0/d10/d12/d17/d42 39 2024-03-13T14:50:00.859 INFO:tasks.workunit.client.0.smithi176.stdout:2/828: chown d1/dc/d2d/d59/d80/d7e/da3/lc5 696528 1 2024-03-13T14:50:00.861 INFO:tasks.workunit.client.0.smithi176.stdout:4/819: dwrite d7/d10/d2c/f77 [0,4194304] 0 2024-03-13T14:50:00.863 INFO:tasks.workunit.client.0.smithi176.stdout:9/904: write d0/db/d89/d32/d122/f41 [1446365,67932] 0 2024-03-13T14:50:00.864 INFO:tasks.workunit.client.1.smithi196.stdout:5/907: creat d4/d1b/dc8/f11f x:0 0 0 2024-03-13T14:50:00.864 INFO:tasks.workunit.client.1.smithi196.stdout:0/911: write d8/da/f32 [1970791,36205] 0 2024-03-13T14:50:00.866 INFO:tasks.workunit.client.1.smithi196.stdout:8/825: mkdir d0/d10/d12/d1e/db1/d117 0 2024-03-13T14:50:00.869 INFO:tasks.workunit.client.0.smithi176.stdout:4/820: unlink d7/l9 0 2024-03-13T14:50:00.871 INFO:tasks.workunit.client.1.smithi196.stdout:3/848: link db/d71/lf7 db/d19/d4a/d9b/de9/df0/l10f 0 2024-03-13T14:50:00.872 INFO:tasks.workunit.client.1.smithi196.stdout:8/826: dread - d0/d10/d12/df5/d47/f98 zero size 2024-03-13T14:50:00.874 INFO:tasks.workunit.client.0.smithi176.stdout:4/821: rmdir d7/d8/d15/dd6/dba 39 2024-03-13T14:50:00.878 INFO:tasks.workunit.client.1.smithi196.stdout:8/827: creat d0/d10/d79/f118 x:0 0 0 2024-03-13T14:50:00.883 INFO:tasks.workunit.client.0.smithi176.stdout:4/822: truncate d7/d10/f83 5783940 0 2024-03-13T14:50:00.885 INFO:tasks.workunit.client.0.smithi176.stdout:1/789: dwrite d2/de/d3f/d6a/fe2 [0,4194304] 0 2024-03-13T14:50:00.887 INFO:tasks.workunit.client.0.smithi176.stdout:6/817: dread d0/de/f12 [0,4194304] 0 2024-03-13T14:50:00.890 INFO:tasks.workunit.client.1.smithi196.stdout:8/828: rename d0/d10/d12/f53 to d0/d16/d3d/d7f/f119 0 2024-03-13T14:50:00.890 INFO:tasks.workunit.client.1.smithi196.stdout:4/870: dread d1/f22 [0,4194304] 0 2024-03-13T14:50:00.891 INFO:tasks.workunit.client.0.smithi176.stdout:4/823: mkdir d7/d8/d33/d96/d111 0 2024-03-13T14:50:00.892 INFO:tasks.workunit.client.1.smithi196.stdout:0/912: dread d8/da/d25/f2f [0,4194304] 0 2024-03-13T14:50:00.894 INFO:tasks.workunit.client.1.smithi196.stdout:2/831: write d0/d7/d9/d42/d98/fa8 [811335,54586] 0 2024-03-13T14:50:00.895 INFO:tasks.workunit.client.1.smithi196.stdout:8/829: write d0/f7e [4221278,108659] 0 2024-03-13T14:50:00.896 INFO:tasks.workunit.client.1.smithi196.stdout:5/908: dwrite d4/d1b/dc8/fd [0,4194304] 0 2024-03-13T14:50:00.898 INFO:tasks.workunit.client.1.smithi196.stdout:2/832: chown d0/d7/d56 3146 1 2024-03-13T14:50:00.899 INFO:tasks.workunit.client.1.smithi196.stdout:9/881: write d7/d9/d17/f51 [2158151,15003] 0 2024-03-13T14:50:00.900 INFO:tasks.workunit.client.0.smithi176.stdout:0/707: dwrite d2/d5/f95 [0,4194304] 0 2024-03-13T14:50:00.900 INFO:tasks.workunit.client.0.smithi176.stdout:5/765: truncate d5/d2c/d40/f8f 1405989 0 2024-03-13T14:50:00.902 INFO:tasks.workunit.client.0.smithi176.stdout:5/766: stat d5/lc5 0 2024-03-13T14:50:00.902 INFO:tasks.workunit.client.0.smithi176.stdout:6/818: fdatasync d0/d10/d19/d27/d2b/f65 0 2024-03-13T14:50:00.902 INFO:tasks.workunit.client.0.smithi176.stdout:0/708: stat d2/db/d39/l63 0 2024-03-13T14:50:00.903 INFO:tasks.workunit.client.1.smithi196.stdout:9/882: chown d7/de/d4b/d1d/f23 1 1 2024-03-13T14:50:00.906 INFO:tasks.workunit.client.1.smithi196.stdout:9/883: dread - d7/de/d74/d94/dda/f113 zero size 2024-03-13T14:50:00.915 INFO:tasks.workunit.client.0.smithi176.stdout:3/847: sync 2024-03-13T14:50:00.915 INFO:tasks.workunit.client.1.smithi196.stdout:1/974: dread d1/d9/d4e/d55/d63/da6/dc8/d11d/f121 [0,4194304] 0 2024-03-13T14:50:00.921 INFO:tasks.workunit.client.0.smithi176.stdout:3/848: truncate d4/d13/d81/f10b 762892 0 2024-03-13T14:50:00.921 INFO:tasks.workunit.client.0.smithi176.stdout:5/767: symlink d5/d2e/d7e/d96/l108 0 2024-03-13T14:50:00.922 INFO:tasks.workunit.client.0.smithi176.stdout:6/819: rmdir d0/d10/d19/d27/d2b/d2e 39 2024-03-13T14:50:00.922 INFO:tasks.workunit.client.0.smithi176.stdout:0/709: truncate d2/d5/d1e/f3d 1360974 0 2024-03-13T14:50:00.923 INFO:tasks.workunit.client.0.smithi176.stdout:8/722: dwrite d0/fdf [0,4194304] 0 2024-03-13T14:50:00.926 INFO:tasks.workunit.client.0.smithi176.stdout:0/710: dread - d2/d5/f96 zero size 2024-03-13T14:50:00.928 INFO:tasks.workunit.client.0.smithi176.stdout:5/768: dread - d5/d2c/d40/dc7/d89/ffa zero size 2024-03-13T14:50:00.930 INFO:tasks.workunit.client.0.smithi176.stdout:5/769: stat d5/d31/f61 0 2024-03-13T14:50:00.934 INFO:tasks.workunit.client.1.smithi196.stdout:3/849: dwrite db/d19/d22/d3a/fad [0,4194304] 0 2024-03-13T14:50:00.936 INFO:tasks.workunit.client.0.smithi176.stdout:3/849: dread d4/d28/f29 [0,4194304] 0 2024-03-13T14:50:00.937 INFO:tasks.workunit.client.1.smithi196.stdout:1/975: dwrite d1/d9/d33/d4b/f1b [4194304,4194304] 0 2024-03-13T14:50:00.939 INFO:tasks.workunit.client.0.smithi176.stdout:2/829: dwrite d1/dc/d2d/fca [0,4194304] 0 2024-03-13T14:50:00.941 INFO:tasks.workunit.client.0.smithi176.stdout:4/824: rename d7/d10/c6a to d7/c112 0 2024-03-13T14:50:00.942 INFO:tasks.workunit.client.1.smithi196.stdout:0/913: dread d8/da/d25/d26/d35/d3b/d50/d55/f72 [0,4194304] 0 2024-03-13T14:50:00.945 INFO:tasks.workunit.client.0.smithi176.stdout:5/770: write d5/d2c/d40/dc7/d53/de9/ffd [602348,126193] 0 2024-03-13T14:50:00.954 INFO:tasks.workunit.client.0.smithi176.stdout:8/723: symlink d0/d5/d11/d47/db7/de0/le8 0 2024-03-13T14:50:00.960 INFO:tasks.workunit.client.0.smithi176.stdout:2/830: rename d1/dc/d2d/d90/c10f to d1/dc/d1c/c11c 0 2024-03-13T14:50:00.962 INFO:tasks.workunit.client.0.smithi176.stdout:3/850: mknod d4/d13/d81/d89/c128 0 2024-03-13T14:50:00.963 INFO:tasks.workunit.client.0.smithi176.stdout:8/724: fsync d0/db/d39/d56/f79 0 2024-03-13T14:50:00.966 INFO:tasks.workunit.client.0.smithi176.stdout:8/725: write d0/fdf [3023407,112828] 0 2024-03-13T14:50:00.967 INFO:tasks.workunit.client.0.smithi176.stdout:4/825: symlink d7/d8/d15/dd6/dba/l113 0 2024-03-13T14:50:00.968 INFO:tasks.workunit.client.0.smithi176.stdout:8/726: fdatasync d0/d24/d29/fbd 0 2024-03-13T14:50:00.973 INFO:tasks.workunit.client.0.smithi176.stdout:1/790: write d2/de/d3f/d50/f7a [3063133,46237] 0 2024-03-13T14:50:00.980 INFO:tasks.workunit.client.0.smithi176.stdout:5/771: dwrite d5/d2e/d7e/dad/d7b/fc8 [0,4194304] 0 2024-03-13T14:50:00.980 INFO:tasks.workunit.client.0.smithi176.stdout:4/826: mkdir d7/d8/d15/d23/d114 0 2024-03-13T14:50:00.981 INFO:tasks.workunit.client.0.smithi176.stdout:8/727: fsync d0/db/d39/d56/f65 0 2024-03-13T14:50:00.985 INFO:tasks.workunit.client.1.smithi196.stdout:5/909: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/f120 x:0 0 0 2024-03-13T14:50:00.985 INFO:tasks.workunit.client.0.smithi176.stdout:6/820: dread d0/d10/d19/d27/f4d [0,4194304] 0 2024-03-13T14:50:00.987 INFO:tasks.workunit.client.0.smithi176.stdout:4/827: chown d7/d2d/c106 0 1 2024-03-13T14:50:00.987 INFO:tasks.workunit.client.0.smithi176.stdout:6/821: stat d0/ff3 0 2024-03-13T14:50:00.990 INFO:tasks.workunit.client.0.smithi176.stdout:5/772: readlink d5/d31/d47/d80/lfb 0 2024-03-13T14:50:00.994 INFO:tasks.workunit.client.1.smithi196.stdout:3/850: fsync db/f13 0 2024-03-13T14:50:00.997 INFO:tasks.workunit.client.0.smithi176.stdout:8/728: chown d0/db/fd 9021661 1 2024-03-13T14:50:00.997 INFO:tasks.workunit.client.1.smithi196.stdout:8/830: dwrite d0/d10/d12/df5/f3c [0,4194304] 0 2024-03-13T14:50:00.999 INFO:tasks.workunit.client.0.smithi176.stdout:1/791: rename d2/d30/d63/l8f to d2/de/d10/d16/lf2 0 2024-03-13T14:50:01.005 INFO:tasks.workunit.client.0.smithi176.stdout:7/749: sync 2024-03-13T14:50:01.006 INFO:tasks.workunit.client.1.smithi196.stdout:4/871: mknod d1/d1e/c134 0 2024-03-13T14:50:01.006 INFO:tasks.workunit.client.1.smithi196.stdout:0/914: creat d8/da/d25/d26/d48/d82/db3/dd5/f12e x:0 0 0 2024-03-13T14:50:01.009 INFO:tasks.workunit.client.1.smithi196.stdout:9/884: fsync d7/de/d74/fd4 0 2024-03-13T14:50:01.010 INFO:tasks.workunit.client.1.smithi196.stdout:9/885: readlink d7/de/d4b/lb3 0 2024-03-13T14:50:01.013 INFO:tasks.workunit.client.0.smithi176.stdout:4/828: dwrite d7/d8/f4f [0,4194304] 0 2024-03-13T14:50:01.017 INFO:tasks.workunit.client.0.smithi176.stdout:2/831: getdents d1/dc/d2d/d90 0 2024-03-13T14:50:01.019 INFO:tasks.workunit.client.0.smithi176.stdout:0/711: dwrite d2/d5/f64 [0,4194304] 0 2024-03-13T14:50:01.019 INFO:tasks.workunit.client.1.smithi196.stdout:3/851: mkdir db/d15/da1/dc8/d110 0 2024-03-13T14:50:01.028 INFO:tasks.workunit.client.0.smithi176.stdout:6/822: rmdir d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1 39 2024-03-13T14:50:01.029 INFO:tasks.workunit.client.1.smithi196.stdout:0/915: dwrite d8/da/d25/d26/d35/d3b/d50/d5d/da5/f112 [0,4194304] 0 2024-03-13T14:50:01.034 INFO:tasks.workunit.client.1.smithi196.stdout:9/886: truncate d7/de/d74/d94/dfb/f38 2474346 0 2024-03-13T14:50:01.035 INFO:tasks.workunit.client.0.smithi176.stdout:2/832: unlink d1/dc/d61/daa/fe4 0 2024-03-13T14:50:01.040 INFO:tasks.workunit.client.0.smithi176.stdout:9/905: sync 2024-03-13T14:50:01.042 INFO:tasks.workunit.client.0.smithi176.stdout:0/712: creat d2/db/d35/daa/fe1 x:0 0 0 2024-03-13T14:50:01.043 INFO:tasks.workunit.client.0.smithi176.stdout:0/713: chown d2/d5/d1e/cbb 19212 1 2024-03-13T14:50:01.045 INFO:tasks.workunit.client.0.smithi176.stdout:8/729: creat d0/d24/d98/fe9 x:0 0 0 2024-03-13T14:50:01.046 INFO:tasks.workunit.client.0.smithi176.stdout:3/851: write d4/d13/d81/d89/f94 [4847046,83624] 0 2024-03-13T14:50:01.046 INFO:tasks.workunit.client.1.smithi196.stdout:3/852: rename db/d71/fa5 to db/d19/dc1/f111 0 2024-03-13T14:50:01.047 INFO:tasks.workunit.client.0.smithi176.stdout:0/714: read d2/db/f1c [2846499,19624] 0 2024-03-13T14:50:01.048 INFO:tasks.workunit.client.1.smithi196.stdout:3/853: chown db/d15/d44/d51/fb4 2 1 2024-03-13T14:50:01.050 INFO:tasks.workunit.client.1.smithi196.stdout:2/833: getdents d0/d7/d56/dc8 0 2024-03-13T14:50:01.050 INFO:tasks.workunit.client.0.smithi176.stdout:5/773: dread d5/d2c/d40/dc7/f6e [0,4194304] 0 2024-03-13T14:50:01.057 INFO:tasks.workunit.client.1.smithi196.stdout:9/887: fsync d7/de/d74/d94/dfb/d28/d6c/f8f 0 2024-03-13T14:50:01.059 INFO:tasks.workunit.client.1.smithi196.stdout:1/976: link d1/d3f/c4a d1/d9/d33/d4b/c152 0 2024-03-13T14:50:01.061 INFO:tasks.workunit.client.1.smithi196.stdout:1/977: chown d1/d9/d4e/d55/d63/da6/fec 4016 1 2024-03-13T14:50:01.063 INFO:tasks.workunit.client.1.smithi196.stdout:4/872: creat d1/d3/d7/f135 x:0 0 0 2024-03-13T14:50:01.065 INFO:tasks.workunit.client.1.smithi196.stdout:3/854: symlink db/d71/l112 0 2024-03-13T14:50:01.066 INFO:tasks.workunit.client.1.smithi196.stdout:3/855: stat db/d19/d22/d3a/d53/c9c 0 2024-03-13T14:50:01.066 INFO:tasks.workunit.client.0.smithi176.stdout:4/829: link d7/d8/d45/c48 d7/d8/d33/d96/c115 0 2024-03-13T14:50:01.068 INFO:tasks.workunit.client.1.smithi196.stdout:0/916: mknod d8/da/d25/d26/d48/d82/db3/dd5/d120/c12f 0 2024-03-13T14:50:01.069 INFO:tasks.workunit.client.0.smithi176.stdout:1/792: write d2/de/d10/d16/d4a/d5f/da8/fb1 [682665,89261] 0 2024-03-13T14:50:01.070 INFO:tasks.workunit.client.1.smithi196.stdout:5/910: link d4/d1b/dc8/d14/d19/da7/fca d4/d60/f121 0 2024-03-13T14:50:01.074 INFO:tasks.workunit.client.0.smithi176.stdout:7/750: link d1/d28/d3a/f14 d1/d1c/d4a/fe7 0 2024-03-13T14:50:01.075 INFO:tasks.workunit.client.0.smithi176.stdout:9/906: truncate d0/d1e/d6f/f7b 124045 0 2024-03-13T14:50:01.077 INFO:tasks.workunit.client.1.smithi196.stdout:9/888: fdatasync d7/de/d4b/d1d/d22/f53 0 2024-03-13T14:50:01.079 INFO:tasks.workunit.client.1.smithi196.stdout:8/831: dwrite d0/d16/d3d/d7f/fa8 [0,4194304] 0 2024-03-13T14:50:01.080 INFO:tasks.workunit.client.1.smithi196.stdout:1/978: unlink d1/d9/d33/d4b/c14f 0 2024-03-13T14:50:01.082 INFO:tasks.workunit.client.0.smithi176.stdout:8/730: mkdir d0/d5/dea 0 2024-03-13T14:50:01.083 INFO:tasks.workunit.client.0.smithi176.stdout:8/731: stat d0/db/cd6 0 2024-03-13T14:50:01.086 INFO:tasks.workunit.client.0.smithi176.stdout:3/852: creat d4/d13/d16/ddd/f129 x:0 0 0 2024-03-13T14:50:01.088 INFO:tasks.workunit.client.0.smithi176.stdout:6/823: write d0/de/f49 [1078283,5923] 0 2024-03-13T14:50:01.090 INFO:tasks.workunit.client.0.smithi176.stdout:5/774: symlink d5/d2e/l109 0 2024-03-13T14:50:01.097 INFO:tasks.workunit.client.0.smithi176.stdout:0/715: write d2/f68 [747776,96227] 0 2024-03-13T14:50:01.097 INFO:tasks.workunit.client.0.smithi176.stdout:1/793: truncate d2/d30/f65 1801765 0 2024-03-13T14:50:01.098 INFO:tasks.workunit.client.1.smithi196.stdout:9/889: rmdir d7/de/d4b/d1d/d93/dd7/df1 39 2024-03-13T14:50:01.101 INFO:tasks.workunit.client.0.smithi176.stdout:7/751: mknod d1/ce8 0 2024-03-13T14:50:01.102 INFO:tasks.workunit.client.1.smithi196.stdout:1/979: mknod d1/d3f/c153 0 2024-03-13T14:50:01.106 INFO:tasks.workunit.client.0.smithi176.stdout:9/907: mkdir d0/d1e/d8d/dbe/d12f 0 2024-03-13T14:50:01.108 INFO:tasks.workunit.client.1.smithi196.stdout:4/873: mkdir d1/d136 0 2024-03-13T14:50:01.112 INFO:tasks.workunit.client.1.smithi196.stdout:2/834: dwrite d0/d7/f21 [0,4194304] 0 2024-03-13T14:50:01.119 INFO:tasks.workunit.client.1.smithi196.stdout:3/856: unlink db/l9e 0 2024-03-13T14:50:01.122 INFO:tasks.workunit.client.0.smithi176.stdout:2/833: getdents d1/dc/d1c/d107 0 2024-03-13T14:50:01.125 INFO:tasks.workunit.client.0.smithi176.stdout:2/834: read - d1/dc/d2d/ff7 zero size 2024-03-13T14:50:01.127 INFO:tasks.workunit.client.1.smithi196.stdout:9/890: unlink d7/de/d74/d94/dfb/l25 0 2024-03-13T14:50:01.128 INFO:tasks.workunit.client.0.smithi176.stdout:6/824: dread d0/d10/f3f [4194304,4194304] 0 2024-03-13T14:50:01.132 INFO:tasks.workunit.client.0.smithi176.stdout:8/732: write d0/db/d53/fa1 [698008,123952] 0 2024-03-13T14:50:01.136 INFO:tasks.workunit.client.1.smithi196.stdout:0/917: truncate d8/d37/f128 2907988 0 2024-03-13T14:50:01.137 INFO:tasks.workunit.client.0.smithi176.stdout:8/733: chown d0/d24/d98/dc6/f13 1940 1 2024-03-13T14:50:01.138 INFO:tasks.workunit.client.1.smithi196.stdout:5/911: write d4/d1b/dc8/d14/d19/da7/fca [772414,103330] 0 2024-03-13T14:50:01.142 INFO:tasks.workunit.client.1.smithi196.stdout:8/832: dwrite d0/d16/d3d/d8b/d48/f5a [4194304,4194304] 0 2024-03-13T14:50:01.143 INFO:tasks.workunit.client.0.smithi176.stdout:4/830: link d7/d8/d33/l71 d7/dab/db5/l116 0 2024-03-13T14:50:01.143 INFO:tasks.workunit.client.1.smithi196.stdout:3/857: fsync db/d19/d4a/f8c 0 2024-03-13T14:50:01.146 INFO:tasks.workunit.client.0.smithi176.stdout:6/825: dwrite d0/d9/f55 [0,4194304] 0 2024-03-13T14:50:01.147 INFO:tasks.workunit.client.0.smithi176.stdout:2/835: creat d1/dc/d61/d9e/f11d x:0 0 0 2024-03-13T14:50:01.150 INFO:tasks.workunit.client.0.smithi176.stdout:0/716: fdatasync d2/db/d39/fa9 0 2024-03-13T14:50:01.155 INFO:tasks.workunit.client.0.smithi176.stdout:7/752: symlink d1/d1c/d26/d41/d2b/d40/le9 0 2024-03-13T14:50:01.158 INFO:tasks.workunit.client.0.smithi176.stdout:3/853: write d4/d13/d16/d3d/f5d [4299807,106376] 0 2024-03-13T14:50:01.159 INFO:tasks.workunit.client.1.smithi196.stdout:5/912: truncate d4/d1b/dc8/d14/d19/d1c/d2d/f4e 213652 0 2024-03-13T14:50:01.159 INFO:tasks.workunit.client.1.smithi196.stdout:4/874: dwrite d1/d3/d4c/d72/daf/f119 [0,4194304] 0 2024-03-13T14:50:01.165 INFO:tasks.workunit.client.1.smithi196.stdout:3/858: fdatasync db/d15/d44/d79/f80 0 2024-03-13T14:50:01.166 INFO:tasks.workunit.client.0.smithi176.stdout:8/734: unlink d0/d24/d98/dc6/d40/f46 0 2024-03-13T14:50:01.167 INFO:tasks.workunit.client.1.smithi196.stdout:1/980: rename d1/d3f/d5b/d74/f65 to d1/d9/d4e/d55/d63/db9/dbb/f154 0 2024-03-13T14:50:01.170 INFO:tasks.workunit.client.0.smithi176.stdout:4/831: unlink d7/d10/d46/c7c 0 2024-03-13T14:50:01.171 INFO:tasks.workunit.client.1.smithi196.stdout:2/835: dwrite d0/d7/d9/d9a/fb4 [0,4194304] 0 2024-03-13T14:50:01.172 INFO:tasks.workunit.client.0.smithi176.stdout:8/735: write d0/d24/d29/fbd [758964,104667] 0 2024-03-13T14:50:01.178 INFO:tasks.workunit.client.0.smithi176.stdout:2/836: creat d1/dc/d2d/d59/d80/d7e/f11e x:0 0 0 2024-03-13T14:50:01.178 INFO:tasks.workunit.client.0.smithi176.stdout:5/775: dwrite d5/d2e/f34 [0,4194304] 0 2024-03-13T14:50:01.179 INFO:tasks.workunit.client.1.smithi196.stdout:0/918: write d8/d37/fdc [355691,19473] 0 2024-03-13T14:50:01.180 INFO:tasks.workunit.client.0.smithi176.stdout:5/776: chown d5/d22 19 1 2024-03-13T14:50:01.188 INFO:tasks.workunit.client.0.smithi176.stdout:3/854: unlink d4/d28/d34/d11c/fd6 0 2024-03-13T14:50:01.193 INFO:tasks.workunit.client.1.smithi196.stdout:5/913: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/df1/f122 x:0 0 0 2024-03-13T14:50:01.196 INFO:tasks.workunit.client.0.smithi176.stdout:6/826: mknod d0/d10/c10a 0 2024-03-13T14:50:01.197 INFO:tasks.workunit.client.0.smithi176.stdout:6/827: fsync d0/d9/f6f 0 2024-03-13T14:50:01.198 INFO:tasks.workunit.client.0.smithi176.stdout:2/837: dread d1/dc/fae [0,4194304] 0 2024-03-13T14:50:01.198 INFO:tasks.workunit.client.1.smithi196.stdout:4/875: dwrite d1/f117 [0,4194304] 0 2024-03-13T14:50:01.206 INFO:tasks.workunit.client.0.smithi176.stdout:9/908: dwrite d0/d40/d53/d6d/fcc [0,4194304] 0 2024-03-13T14:50:01.208 INFO:tasks.workunit.client.1.smithi196.stdout:2/836: dwrite d0/d7/fe3 [4194304,4194304] 0 2024-03-13T14:50:01.208 INFO:tasks.workunit.client.1.smithi196.stdout:0/919: symlink d8/da/d25/d26/d35/d3b/d50/d5d/l130 0 2024-03-13T14:50:01.209 INFO:tasks.workunit.client.0.smithi176.stdout:9/909: chown d0/db/d89/d32/d58/d5f 18281229 1 2024-03-13T14:50:01.213 INFO:tasks.workunit.client.0.smithi176.stdout:1/794: getdents d2/d30/d63/d22 0 2024-03-13T14:50:01.215 INFO:tasks.workunit.client.0.smithi176.stdout:4/832: truncate d7/d10/d46/d4a/d5d/fda 912221 0 2024-03-13T14:50:01.215 INFO:tasks.workunit.client.1.smithi196.stdout:8/833: write d0/d16/d3d/d9c/fc0 [605431,11344] 0 2024-03-13T14:50:01.216 INFO:tasks.workunit.client.0.smithi176.stdout:1/795: chown d2/de/dd1/fd4 54 1 2024-03-13T14:50:01.217 INFO:tasks.workunit.client.1.smithi196.stdout:3/859: truncate db/d19/d22/d3a/d6a/fcc 837321 0 2024-03-13T14:50:01.220 INFO:tasks.workunit.client.0.smithi176.stdout:1/796: write d2/de/d3f/d6a/fe2 [1308960,52328] 0 2024-03-13T14:50:01.220 INFO:tasks.workunit.client.0.smithi176.stdout:7/753: fdatasync d1/d1c/d26/d41/d2b/d40/d64/f60 0 2024-03-13T14:50:01.222 INFO:tasks.workunit.client.1.smithi196.stdout:5/914: mknod d4/d1b/dc8/d14/d37/c123 0 2024-03-13T14:50:01.224 INFO:tasks.workunit.client.0.smithi176.stdout:3/855: readlink d4/d13/d16/d80/la6 0 2024-03-13T14:50:01.224 INFO:tasks.workunit.client.0.smithi176.stdout:1/797: readlink d2/d30/d63/l2d 0 2024-03-13T14:50:01.227 INFO:tasks.workunit.client.1.smithi196.stdout:9/891: dwrite d7/de/d74/d94/fb8 [4194304,4194304] 0 2024-03-13T14:50:01.234 INFO:tasks.workunit.client.1.smithi196.stdout:5/915: write d4/d1b/dc8/d14/d19/d26/dab/f10b [581853,36508] 0 2024-03-13T14:50:01.235 INFO:tasks.workunit.client.1.smithi196.stdout:5/916: readlink d4/d1b/dc8/d3c/dd0/l10f 0 2024-03-13T14:50:01.238 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:00 smithi176 ceph-mon[29064]: pgmap v19: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 107 MiB/s rd, 266 MiB/s wr, 576 op/s 2024-03-13T14:50:01.238 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:00 smithi176 ceph-mon[29064]: Reconfiguring prometheus.smithi176 (dependencies changed)... 2024-03-13T14:50:01.238 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:00 smithi176 ceph-mon[29064]: overall HEALTH_OK 2024-03-13T14:50:01.238 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:00 smithi176 ceph-mon[29064]: Reconfiguring daemon prometheus.smithi176 on smithi176 2024-03-13T14:50:01.240 INFO:tasks.workunit.client.0.smithi176.stdout:0/717: creat d2/d74/fe2 x:0 0 0 2024-03-13T14:50:01.242 INFO:tasks.workunit.client.0.smithi176.stdout:0/718: stat d2/db/d39/f73 0 2024-03-13T14:50:01.243 INFO:tasks.workunit.client.1.smithi196.stdout:8/834: creat d0/d10/da5/d110/f11a x:0 0 0 2024-03-13T14:50:01.248 INFO:tasks.workunit.client.0.smithi176.stdout:9/910: creat d0/db/d89/d4d/f130 x:0 0 0 2024-03-13T14:50:01.255 INFO:tasks.workunit.client.0.smithi176.stdout:7/754: fdatasync d1/d1c/d26/d41/d2b/d40/d64/f60 0 2024-03-13T14:50:01.256 INFO:tasks.workunit.client.1.smithi196.stdout:1/981: write d1/d38/f137 [851632,77188] 0 2024-03-13T14:50:01.258 INFO:tasks.workunit.client.1.smithi196.stdout:4/876: creat d1/d3/f137 x:0 0 0 2024-03-13T14:50:01.260 INFO:tasks.workunit.client.0.smithi176.stdout:1/798: mkdir d2/de/d3f/d50/d83/d97/df3 0 2024-03-13T14:50:01.262 INFO:tasks.workunit.client.0.smithi176.stdout:6/828: symlink d0/de/d38/d70/d109/l10b 0 2024-03-13T14:50:01.263 INFO:tasks.workunit.client.0.smithi176.stdout:2/838: mknod d1/dc/d2d/d90/dcf/dea/c11f 0 2024-03-13T14:50:01.263 INFO:tasks.workunit.client.0.smithi176.stdout:6/829: stat d0/d10/dca 0 2024-03-13T14:50:01.264 INFO:tasks.workunit.client.1.smithi196.stdout:1/982: fsync f0 0 2024-03-13T14:50:01.266 INFO:tasks.workunit.client.1.smithi196.stdout:2/837: rename d0/d7/d9/d9a/fca to d0/d7/d9/d9a/f10d 0 2024-03-13T14:50:01.266 INFO:tasks.workunit.client.1.smithi196.stdout:1/983: chown d1/d38/f137 345479869 1 2024-03-13T14:50:01.267 INFO:tasks.workunit.client.0.smithi176.stdout:8/736: creat d0/feb x:0 0 0 2024-03-13T14:50:01.270 INFO:tasks.workunit.client.0.smithi176.stdout:4/833: symlink d7/db9/dd8/l117 0 2024-03-13T14:50:01.270 INFO:tasks.workunit.client.0.smithi176.stdout:4/834: stat d7/d8/d15/f38 0 2024-03-13T14:50:01.271 INFO:tasks.workunit.client.0.smithi176.stdout:4/835: readlink d7/d10/d46/lb6 0 2024-03-13T14:50:01.271 INFO:tasks.workunit.client.0.smithi176.stdout:9/911: rename d0/d1e/d8d/dbe/fde to d0/db/d89/d32/d58/f131 0 2024-03-13T14:50:01.272 INFO:tasks.workunit.client.0.smithi176.stdout:9/912: stat d0/d40/c4b 0 2024-03-13T14:50:01.275 INFO:tasks.workunit.client.1.smithi196.stdout:2/838: mkdir d0/d7/d9/d5a/dbc/deb/d10e 0 2024-03-13T14:50:01.276 INFO:tasks.workunit.client.1.smithi196.stdout:3/860: write db/d15/d44/d51/fb4 [1480244,37030] 0 2024-03-13T14:50:01.277 INFO:tasks.workunit.client.1.smithi196.stdout:0/920: write d8/da/d25/d26/d35/d3b/d6c/f6e [2917236,11149] 0 2024-03-13T14:50:01.279 INFO:tasks.workunit.client.0.smithi176.stdout:0/719: symlink d2/db/d31/dc4/le3 0 2024-03-13T14:50:01.282 INFO:tasks.workunit.client.1.smithi196.stdout:8/835: dread d0/d10/f19 [0,4194304] 0 2024-03-13T14:50:01.283 INFO:tasks.workunit.client.1.smithi196.stdout:4/877: rename d1/d3/d7/d35/d3e/lca to d1/d3/d7/d35/d76/d58/l138 0 2024-03-13T14:50:01.284 INFO:tasks.workunit.client.1.smithi196.stdout:2/839: dread - d0/d7/d9/d42/d98/de4/d67/fe9 zero size 2024-03-13T14:50:01.284 INFO:tasks.workunit.client.0.smithi176.stdout:2/839: write d1/dc/d2d/d59/d80/d5c/f5f [3972570,30080] 0 2024-03-13T14:50:01.290 INFO:tasks.workunit.client.1.smithi196.stdout:3/861: mknod db/d15/d44/d79/c113 0 2024-03-13T14:50:01.291 INFO:tasks.workunit.client.0.smithi176.stdout:8/737: mkdir d0/d24/d7b/dec 0 2024-03-13T14:50:01.292 INFO:tasks.workunit.client.0.smithi176.stdout:8/738: truncate d0/d24/f7d 1373631 0 2024-03-13T14:50:01.293 INFO:tasks.workunit.client.1.smithi196.stdout:1/984: dwrite d1/d9/f13b [0,4194304] 0 2024-03-13T14:50:01.294 INFO:tasks.workunit.client.1.smithi196.stdout:3/862: stat db/d19/d22/d3a/c6b 0 2024-03-13T14:50:01.297 INFO:tasks.workunit.client.1.smithi196.stdout:8/836: read - d0/d10/da5/d110/fc3 zero size 2024-03-13T14:50:01.297 INFO:tasks.workunit.client.1.smithi196.stdout:5/917: dwrite d4/d1b/dc8/d14/d19/d1c/fa2 [0,4194304] 0 2024-03-13T14:50:01.300 INFO:tasks.workunit.client.1.smithi196.stdout:1/985: chown d1/d9/d33/d4b/d66/l97 623054 1 2024-03-13T14:50:01.302 INFO:tasks.workunit.client.1.smithi196.stdout:0/921: symlink d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe/l131 0 2024-03-13T14:50:01.303 INFO:tasks.workunit.client.0.smithi176.stdout:5/777: getdents d5/d2e/d7e/dad 0 2024-03-13T14:50:01.305 INFO:tasks.workunit.client.1.smithi196.stdout:4/878: stat d1/d3/dbc/ca8 0 2024-03-13T14:50:01.309 INFO:tasks.workunit.client.1.smithi196.stdout:2/840: mknod d0/d7/d9/d5a/dbc/dfe/c10f 0 2024-03-13T14:50:01.311 INFO:tasks.workunit.client.1.smithi196.stdout:1/986: read d1/d9/d33/dd2/dde/ff2 [1111076,7671] 0 2024-03-13T14:50:01.312 INFO:tasks.workunit.client.1.smithi196.stdout:2/841: stat d0/d6/c2c 0 2024-03-13T14:50:01.312 INFO:tasks.workunit.client.1.smithi196.stdout:8/837: rmdir d0/d16/d71 39 2024-03-13T14:50:01.313 INFO:tasks.workunit.client.0.smithi176.stdout:3/856: dread d4/d13/d16/d3d/f93 [0,4194304] 0 2024-03-13T14:50:01.314 INFO:tasks.workunit.client.0.smithi176.stdout:3/857: stat d4/l71 0 2024-03-13T14:50:01.314 INFO:tasks.workunit.client.1.smithi196.stdout:2/842: fsync d0/d7/d40/f46 0 2024-03-13T14:50:01.315 INFO:tasks.workunit.client.0.smithi176.stdout:7/755: symlink d1/d1c/d26/d41/d2b/d40/d64/lea 0 2024-03-13T14:50:01.317 INFO:tasks.workunit.client.0.smithi176.stdout:1/799: truncate d2/de/d3f/f52 2034660 0 2024-03-13T14:50:01.318 INFO:tasks.workunit.client.1.smithi196.stdout:5/918: mkdir d4/d1b/dc8/d14/d124 0 2024-03-13T14:50:01.319 INFO:tasks.workunit.client.0.smithi176.stdout:1/800: write d2/de/dd1/fd4 [1000588,10562] 0 2024-03-13T14:50:01.319 INFO:tasks.workunit.client.1.smithi196.stdout:5/919: chown d4/d1b/dc8/d4a 11568 1 2024-03-13T14:50:01.325 INFO:tasks.workunit.client.1.smithi196.stdout:4/879: rmdir d1/d3 39 2024-03-13T14:50:01.329 INFO:tasks.workunit.client.1.smithi196.stdout:5/920: read d4/d1b/dc8/d14/d19/da7/d105/f119 [4175046,55378] 0 2024-03-13T14:50:01.331 INFO:tasks.workunit.client.0.smithi176.stdout:6/830: truncate d0/d10/d19/d27/d2b/de0/f98 3674956 0 2024-03-13T14:50:01.332 INFO:tasks.workunit.client.0.smithi176.stdout:6/831: fdatasync d0/de/d38/fe9 0 2024-03-13T14:50:01.332 INFO:tasks.workunit.client.1.smithi196.stdout:1/987: readlink d1/d9/d33/d4b/l127 0 2024-03-13T14:50:01.333 INFO:tasks.workunit.client.0.smithi176.stdout:6/832: chown d0/d10/d19 20612 1 2024-03-13T14:50:01.333 INFO:tasks.workunit.client.1.smithi196.stdout:3/863: write db/fbc [1028814,53790] 0 2024-03-13T14:50:01.336 INFO:tasks.workunit.client.0.smithi176.stdout:5/778: mknod d5/d31/c10a 0 2024-03-13T14:50:01.337 INFO:tasks.workunit.client.1.smithi196.stdout:5/921: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/db2/f116 692779 0 2024-03-13T14:50:01.340 INFO:tasks.workunit.client.1.smithi196.stdout:8/838: chown d0/d16/d3d/d8b/l25 1381574614 1 2024-03-13T14:50:01.340 INFO:tasks.workunit.client.0.smithi176.stdout:4/836: creat d7/d8/d33/d96/d111/f118 x:0 0 0 2024-03-13T14:50:01.342 INFO:tasks.workunit.client.0.smithi176.stdout:4/837: stat d7/d10/d2c/f110 0 2024-03-13T14:50:01.342 INFO:tasks.workunit.client.0.smithi176.stdout:9/913: creat d0/d1e/df7/d11f/f132 x:0 0 0 2024-03-13T14:50:01.346 INFO:tasks.workunit.client.0.smithi176.stdout:7/756: mknod d1/d1c/d26/d41/d2b/d40/d64/d94/ceb 0 2024-03-13T14:50:01.348 INFO:tasks.workunit.client.1.smithi196.stdout:4/880: mknod d1/d1e/d5c/d105/c139 0 2024-03-13T14:50:01.350 INFO:tasks.workunit.client.0.smithi176.stdout:1/801: readlink d2/l95 0 2024-03-13T14:50:01.352 INFO:tasks.workunit.client.0.smithi176.stdout:0/720: mknod d2/d5/d28/ce4 0 2024-03-13T14:50:01.360 INFO:tasks.workunit.client.1.smithi196.stdout:0/922: link d8/da/d25/d26/d48/d82/db3/dd5/c11c d8/da/d25/d26/d48/d82/db3/dd5/c132 0 2024-03-13T14:50:01.363 INFO:tasks.workunit.client.1.smithi196.stdout:1/988: creat d1/d9/d33/d4b/d66/d149/f155 x:0 0 0 2024-03-13T14:50:01.365 INFO:tasks.workunit.client.1.smithi196.stdout:1/989: chown d1/d9/d4e/c8a 0 1 2024-03-13T14:50:01.372 INFO:tasks.workunit.client.0.smithi176.stdout:2/840: truncate d1/f3c 3564650 0 2024-03-13T14:50:01.373 INFO:tasks.workunit.client.0.smithi176.stdout:1/802: unlink d2/de/d10/fa9 0 2024-03-13T14:50:01.378 INFO:tasks.workunit.client.1.smithi196.stdout:3/864: truncate db/d15/d44/fb0 114154 0 2024-03-13T14:50:01.381 INFO:tasks.workunit.client.1.smithi196.stdout:0/923: symlink d8/da/d25/d26/d35/d3b/d59/d6a/l133 0 2024-03-13T14:50:01.381 INFO:tasks.workunit.client.1.smithi196.stdout:4/881: dwrite d1/d3/d4c/f108 [0,4194304] 0 2024-03-13T14:50:01.382 INFO:tasks.workunit.client.0.smithi176.stdout:8/739: creat d0/d24/d98/fed x:0 0 0 2024-03-13T14:50:01.382 INFO:tasks.workunit.client.0.smithi176.stdout:8/740: stat d0/f52 0 2024-03-13T14:50:01.382 INFO:tasks.workunit.client.1.smithi196.stdout:5/922: creat d4/d1b/dc8/d14/d19/d26/d33/f125 x:0 0 0 2024-03-13T14:50:01.383 INFO:tasks.workunit.client.1.smithi196.stdout:3/865: chown db/d19/d22/d3a/d6a/d8d 751 1 2024-03-13T14:50:01.383 INFO:tasks.workunit.client.1.smithi196.stdout:1/990: creat d1/d28/d34/d67/f156 x:0 0 0 2024-03-13T14:50:01.384 INFO:tasks.workunit.client.1.smithi196.stdout:8/839: creat d0/d16/f11b x:0 0 0 2024-03-13T14:50:01.384 INFO:tasks.workunit.client.0.smithi176.stdout:5/779: dwrite d5/d2e/d5f/fd3 [0,4194304] 0 2024-03-13T14:50:01.387 INFO:tasks.workunit.client.1.smithi196.stdout:3/866: stat db/d19/d22/d3a/c5e 0 2024-03-13T14:50:01.388 INFO:tasks.workunit.client.0.smithi176.stdout:3/858: creat d4/d28/d31/d45/d82/f12a x:0 0 0 2024-03-13T14:50:01.389 INFO:tasks.workunit.client.0.smithi176.stdout:4/838: dread d7/d8/d33/f7f [0,4194304] 0 2024-03-13T14:50:01.389 INFO:tasks.workunit.client.0.smithi176.stdout:7/757: mknod d1/d1c/cec 0 2024-03-13T14:50:01.390 INFO:tasks.workunit.client.0.smithi176.stdout:9/914: dread d0/dd8/d82/f10f [0,4194304] 0 2024-03-13T14:50:01.393 INFO:tasks.workunit.client.1.smithi196.stdout:2/843: dwrite d0/d7/d9/d9a/daa/fd1 [0,4194304] 0 2024-03-13T14:50:01.394 INFO:tasks.workunit.client.0.smithi176.stdout:3/859: chown d4/d13/d81/d89/l70 987328345 1 2024-03-13T14:50:01.395 INFO:tasks.workunit.client.0.smithi176.stdout:1/803: chown d2/de/d10/d16/d4a/d5f/d66/fe1 0 1 2024-03-13T14:50:01.396 INFO:tasks.workunit.client.1.smithi196.stdout:1/991: symlink d1/d3f/d5b/d74/dd4/d108/l157 0 2024-03-13T14:50:01.399 INFO:tasks.workunit.client.1.smithi196.stdout:5/923: mkdir d4/d1b/dc8/d14/d19/da7/d126 0 2024-03-13T14:50:01.404 INFO:tasks.workunit.client.0.smithi176.stdout:5/780: creat d5/d2c/d40/dc7/d53/de9/f10b x:0 0 0 2024-03-13T14:50:01.406 INFO:tasks.workunit.client.1.smithi196.stdout:3/867: unlink db/d19/dc1/f111 0 2024-03-13T14:50:01.408 INFO:tasks.workunit.client.0.smithi176.stdout:2/841: fdatasync d1/dc/d2d/d38/d5d/f79 0 2024-03-13T14:50:01.411 INFO:tasks.workunit.client.1.smithi196.stdout:4/882: readlink d1/d3/d7/d35/d76/d58/d69/de3/df4/l115 0 2024-03-13T14:50:01.414 INFO:tasks.workunit.client.0.smithi176.stdout:8/741: dread d0/d24/d98/dc6/f2a [0,4194304] 0 2024-03-13T14:50:01.416 INFO:tasks.workunit.client.1.smithi196.stdout:2/844: dread - d0/d7/d9/d42/d98/de4/d104/fb1 zero size 2024-03-13T14:50:01.418 INFO:tasks.workunit.client.1.smithi196.stdout:1/992: symlink d1/d28/d34/d100/l158 0 2024-03-13T14:50:01.419 INFO:tasks.workunit.client.0.smithi176.stdout:9/915: rmdir d0/d1e/d8d/dbe 39 2024-03-13T14:50:01.422 INFO:tasks.workunit.client.1.smithi196.stdout:1/993: truncate d1/d9/d33/dd2/f12d 8303 0 2024-03-13T14:50:01.422 INFO:tasks.workunit.client.0.smithi176.stdout:1/804: creat d2/d30/d63/ff4 x:0 0 0 2024-03-13T14:50:01.423 INFO:tasks.workunit.client.0.smithi176.stdout:0/721: write d2/f69 [1607349,106467] 0 2024-03-13T14:50:01.424 INFO:tasks.workunit.client.1.smithi196.stdout:5/924: fsync d4/d1b/dc8/d14/d19/d1c/d25/d38/ff3 0 2024-03-13T14:50:01.424 INFO:tasks.workunit.client.0.smithi176.stdout:1/805: chown d2/de/dbf/fa5 870 1 2024-03-13T14:50:01.427 INFO:tasks.workunit.client.1.smithi196.stdout:5/925: write d4/d1b/dc8/d14/d19/d26/d33/f125 [1027806,31736] 0 2024-03-13T14:50:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:00 smithi196 ceph-mon[37486]: pgmap v19: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 107 MiB/s rd, 266 MiB/s wr, 576 op/s 2024-03-13T14:50:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:00 smithi196 ceph-mon[37486]: Reconfiguring prometheus.smithi176 (dependencies changed)... 2024-03-13T14:50:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:00 smithi196 ceph-mon[37486]: overall HEALTH_OK 2024-03-13T14:50:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:00 smithi196 ceph-mon[37486]: Reconfiguring daemon prometheus.smithi176 on smithi176 2024-03-13T14:50:01.429 INFO:tasks.workunit.client.1.smithi196.stdout:4/883: mkdir d1/d3/d4c/d13a 0 2024-03-13T14:50:01.437 INFO:tasks.workunit.client.1.smithi196.stdout:8/840: dwrite d0/f56 [0,4194304] 0 2024-03-13T14:50:01.445 INFO:tasks.workunit.client.1.smithi196.stdout:0/924: dwrite d8/da/d25/d26/d54/f9a [0,4194304] 0 2024-03-13T14:50:01.452 INFO:tasks.workunit.client.1.smithi196.stdout:1/994: truncate d1/d9/d4e/d55/f6e 1423485 0 2024-03-13T14:50:01.460 INFO:tasks.workunit.client.1.smithi196.stdout:2/845: rename d0/d7/d9/d61/dc7 to d0/d7/d9/d42/d98/de4/d110 0 2024-03-13T14:50:01.462 INFO:tasks.workunit.client.0.smithi176.stdout:5/781: mknod d5/d2e/d7e/dad/d7b/c10c 0 2024-03-13T14:50:01.464 INFO:tasks.workunit.client.0.smithi176.stdout:0/722: rename d2/d74/d61/ld3 to d2/d5/le5 0 2024-03-13T14:50:01.464 INFO:tasks.workunit.client.1.smithi196.stdout:5/926: dwrite d4/d1b/dc8/d14/d19/d1c/d25/fba [0,4194304] 0 2024-03-13T14:50:01.466 INFO:tasks.workunit.client.1.smithi196.stdout:2/846: readlink d0/d7/d9/d42/d98/de4/d104/l4d 0 2024-03-13T14:50:01.466 INFO:tasks.workunit.client.0.smithi176.stdout:1/806: dread d2/de/d10/d16/d4a/d5f/da8/fb1 [0,4194304] 0 2024-03-13T14:50:01.468 INFO:tasks.workunit.client.0.smithi176.stdout:8/742: dwrite d0/db/d39/d56/d91/fcc [0,4194304] 0 2024-03-13T14:50:01.469 INFO:tasks.workunit.client.0.smithi176.stdout:5/782: dread - d5/d31/d47/d80/fbf zero size 2024-03-13T14:50:01.469 INFO:tasks.workunit.client.0.smithi176.stdout:7/758: link d1/d28/d3a/d1b/l72 d1/d1c/d4a/dda/led 0 2024-03-13T14:50:01.470 INFO:tasks.workunit.client.1.smithi196.stdout:0/925: link d8/da/d25/d26/d35/d3b/d50/c63 d8/da/d25/d26/d54/d88/d113/c134 0 2024-03-13T14:50:01.470 INFO:tasks.workunit.client.0.smithi176.stdout:0/723: mknod d2/db/d31/ce6 0 2024-03-13T14:50:01.471 INFO:tasks.workunit.client.0.smithi176.stdout:5/783: stat d5/d22/fa6 0 2024-03-13T14:50:01.474 INFO:tasks.workunit.client.1.smithi196.stdout:5/927: symlink d4/d1b/dc8/d14/d19/d1c/d2d/dc5/l127 0 2024-03-13T14:50:01.475 INFO:tasks.workunit.client.0.smithi176.stdout:8/743: mknod d0/d24/d29/d82/cee 0 2024-03-13T14:50:01.478 INFO:tasks.workunit.client.1.smithi196.stdout:1/995: link d1/d9/d33/d4b/d66/l6d d1/d9/d4e/d55/d63/da6/dc8/d11d/d124/l159 0 2024-03-13T14:50:01.482 INFO:tasks.workunit.client.1.smithi196.stdout:8/841: dwrite d0/d16/d58/fd0 [0,4194304] 0 2024-03-13T14:50:01.482 INFO:tasks.workunit.client.1.smithi196.stdout:2/847: dread d0/d6/d4b/d96/fa7 [0,4194304] 0 2024-03-13T14:50:01.483 INFO:tasks.workunit.client.0.smithi176.stdout:5/784: creat d5/d2c/d40/f10d x:0 0 0 2024-03-13T14:50:01.485 INFO:tasks.workunit.client.0.smithi176.stdout:5/785: stat d5/d2b/l41 0 2024-03-13T14:50:01.487 INFO:tasks.workunit.client.1.smithi196.stdout:0/926: creat d8/da/d25/d26/d48/d82/db7/f135 x:0 0 0 2024-03-13T14:50:01.489 INFO:tasks.workunit.client.0.smithi176.stdout:7/759: rename d1/d3f/d76/da5/fb8 to d1/d1c/d26/d41/d2b/fee 0 2024-03-13T14:50:01.490 INFO:tasks.workunit.client.1.smithi196.stdout:2/848: creat d0/d6/d34/f111 x:0 0 0 2024-03-13T14:50:01.493 INFO:tasks.workunit.client.1.smithi196.stdout:8/842: truncate d0/d10/d12/d1e/f34 4838148 0 2024-03-13T14:50:01.495 INFO:tasks.workunit.client.1.smithi196.stdout:8/843: stat d0/d10/d12/d1e/cc4 0 2024-03-13T14:50:01.495 INFO:tasks.workunit.client.0.smithi176.stdout:5/786: mkdir d5/d31/def/d10e 0 2024-03-13T14:50:01.496 INFO:tasks.workunit.client.1.smithi196.stdout:0/927: creat d8/da/d25/d26/d48/d82/f136 x:0 0 0 2024-03-13T14:50:01.496 INFO:tasks.workunit.client.0.smithi176.stdout:8/744: read d0/d24/f57 [4817242,125794] 0 2024-03-13T14:50:01.496 INFO:tasks.workunit.client.0.smithi176.stdout:7/760: read d1/d1c/d26/d41/d2b/d40/d64/d94/f7e [3196966,25622] 0 2024-03-13T14:50:01.501 INFO:tasks.workunit.client.1.smithi196.stdout:5/928: link d4/d1b/dc8/d3c/d6a/d106/f113 d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/f128 0 2024-03-13T14:50:01.502 INFO:tasks.workunit.client.0.smithi176.stdout:8/745: chown d0/d24/d98/dc6/f69 4399 1 2024-03-13T14:50:01.507 INFO:tasks.workunit.client.1.smithi196.stdout:1/996: rename d1/d9/d33/d4b/f75 to d1/f15a 0 2024-03-13T14:50:01.510 INFO:tasks.workunit.client.1.smithi196.stdout:0/928: mknod d8/da/d25/d26/d35/d3b/d50/d5d/da5/c137 0 2024-03-13T14:50:01.511 INFO:tasks.workunit.client.1.smithi196.stdout:2/849: mknod d0/d7/d9/d42/c112 0 2024-03-13T14:50:01.512 INFO:tasks.workunit.client.0.smithi176.stdout:5/787: mknod d5/d2c/d40/dc7/d89/df2/dff/c10f 0 2024-03-13T14:50:01.514 INFO:tasks.workunit.client.1.smithi196.stdout:3/868: write db/d15/f88 [440879,119465] 0 2024-03-13T14:50:01.522 INFO:tasks.workunit.client.0.smithi176.stdout:5/788: unlink d5/d2e/d7e/dad/f65 0 2024-03-13T14:50:01.524 INFO:tasks.workunit.client.1.smithi196.stdout:1/997: symlink d1/d3f/d5b/d74/db5/l15b 0 2024-03-13T14:50:01.524 INFO:tasks.workunit.client.0.smithi176.stdout:6/833: dread d0/d10/d19/d27/d2b/de0/f98 [0,4194304] 0 2024-03-13T14:50:01.527 INFO:tasks.workunit.client.0.smithi176.stdout:6/834: dread - d0/d10/d19/d7d/f100 zero size 2024-03-13T14:50:01.527 INFO:tasks.workunit.client.1.smithi196.stdout:0/929: creat d8/da/d25/d26/d48/d82/da0/f138 x:0 0 0 2024-03-13T14:50:01.528 INFO:tasks.workunit.client.0.smithi176.stdout:3/860: dwrite d4/d13/d81/d89/f27 [0,4194304] 0 2024-03-13T14:50:01.530 INFO:tasks.workunit.client.1.smithi196.stdout:2/850: dread d0/d7/d9/d42/d98/de4/d104/f5e [0,4194304] 0 2024-03-13T14:50:01.534 INFO:tasks.workunit.client.1.smithi196.stdout:4/884: write d1/d3/d7/d35/d3e/f89 [20711,118437] 0 2024-03-13T14:50:01.535 INFO:tasks.workunit.client.0.smithi176.stdout:5/789: read - d5/d2e/f58 zero size 2024-03-13T14:50:01.536 INFO:tasks.workunit.client.1.smithi196.stdout:3/869: read db/d37/f97 [3015117,87272] 0 2024-03-13T14:50:01.545 INFO:tasks.workunit.client.0.smithi176.stdout:4/839: dwrite d7/d10/d46/d4a/d5d/f69 [4194304,4194304] 0 2024-03-13T14:50:01.548 INFO:tasks.workunit.client.0.smithi176.stdout:9/916: dwrite d0/db/d89/d4d/d6e/f71 [0,4194304] 0 2024-03-13T14:50:01.551 INFO:tasks.workunit.client.0.smithi176.stdout:9/917: chown d0/db/ff8 2 1 2024-03-13T14:50:01.553 INFO:tasks.workunit.client.1.smithi196.stdout:3/870: truncate db/d19/d50/f58 972496 0 2024-03-13T14:50:01.555 INFO:tasks.workunit.client.0.smithi176.stdout:1/807: dread f1 [0,4194304] 0 2024-03-13T14:50:01.555 INFO:tasks.workunit.client.0.smithi176.stdout:8/746: dread d0/db/d39/d56/f9f [0,4194304] 0 2024-03-13T14:50:01.563 INFO:tasks.workunit.client.1.smithi196.stdout:4/885: mknod d1/c13b 0 2024-03-13T14:50:01.573 INFO:tasks.workunit.client.1.smithi196.stdout:2/851: rename d0/d7/d9/d42/c112 to d0/d7/d9/d9a/c113 0 2024-03-13T14:50:01.576 INFO:tasks.workunit.client.1.smithi196.stdout:4/886: rmdir d1/d3/d4c/d72/d7d/d80 39 2024-03-13T14:50:01.580 INFO:tasks.workunit.client.0.smithi176.stdout:7/761: write d1/d1c/d26/d41/db6/fdd [570706,56819] 0 2024-03-13T14:50:01.581 INFO:tasks.workunit.client.1.smithi196.stdout:8/844: write d0/d10/d12/d1e/f5e [221828,65710] 0 2024-03-13T14:50:01.581 INFO:tasks.workunit.client.1.smithi196.stdout:2/852: creat d0/d7/d9/d5a/dbc/dfe/f114 x:0 0 0 2024-03-13T14:50:01.582 INFO:tasks.workunit.client.0.smithi176.stdout:7/762: chown d1/d7/cd 269 1 2024-03-13T14:50:01.584 INFO:tasks.workunit.client.1.smithi196.stdout:4/887: creat d1/d3/d7/d35/d76/d58/dc9/dd7/deb/f13c x:0 0 0 2024-03-13T14:50:01.584 INFO:tasks.workunit.client.1.smithi196.stdout:8/845: write d0/d10/d12/df5/f3c [712638,101719] 0 2024-03-13T14:50:01.585 INFO:tasks.workunit.client.1.smithi196.stdout:2/853: write d0/d6/d4b/d5f/f81 [4186729,78116] 0 2024-03-13T14:50:01.586 INFO:tasks.workunit.client.1.smithi196.stdout:8/846: chown d0/f51 5 1 2024-03-13T14:50:01.588 INFO:tasks.workunit.client.1.smithi196.stdout:3/871: rmdir db/d19/d22/d3a/d6a/d8d 0 2024-03-13T14:50:01.589 INFO:tasks.workunit.client.1.smithi196.stdout:8/847: chown d0/d16/d3d/d8b/ce8 14790949 1 2024-03-13T14:50:01.597 INFO:tasks.workunit.client.1.smithi196.stdout:4/888: mkdir d1/dfb/d11c/d13d 0 2024-03-13T14:50:01.599 INFO:tasks.workunit.client.1.smithi196.stdout:4/889: dread - d1/d3/d7/d35/d76/d58/dc9/dd7/deb/f13c zero size 2024-03-13T14:50:01.600 INFO:tasks.workunit.client.1.smithi196.stdout:2/854: creat d0/d6/d90/f115 x:0 0 0 2024-03-13T14:50:01.603 INFO:tasks.workunit.client.1.smithi196.stdout:2/855: write d0/d7/d40/f46 [2632332,45256] 0 2024-03-13T14:50:01.603 INFO:tasks.workunit.client.1.smithi196.stdout:4/890: write d1/d3/d7/d35/d76/fed [2396605,67118] 0 2024-03-13T14:50:01.606 INFO:tasks.workunit.client.1.smithi196.stdout:8/848: truncate d0/d10/da5/d110/f76 1170891 0 2024-03-13T14:50:01.607 INFO:tasks.workunit.client.1.smithi196.stdout:1/998: truncate d1/d9/d4e/d55/d90/fb1 1190767 0 2024-03-13T14:50:01.610 INFO:tasks.workunit.client.1.smithi196.stdout:0/930: dwrite d8/da/d25/d26/d54/fa7 [0,4194304] 0 2024-03-13T14:50:01.617 INFO:tasks.workunit.client.1.smithi196.stdout:2/856: dread - d0/d6/d4b/fc2 zero size 2024-03-13T14:50:01.619 INFO:tasks.workunit.client.1.smithi196.stdout:2/857: readlink d0/d7/d9/d42/d98/de4/d104/l72 0 2024-03-13T14:50:01.621 INFO:tasks.workunit.client.1.smithi196.stdout:8/849: mknod d0/d16/d3d/d7f/c11c 0 2024-03-13T14:50:01.623 INFO:tasks.workunit.client.1.smithi196.stdout:8/850: dread - d0/d16/f11b zero size 2024-03-13T14:50:01.628 INFO:tasks.workunit.client.1.smithi196.stdout:0/931: creat d8/da/d25/d26/d35/d3b/d59/f139 x:0 0 0 2024-03-13T14:50:01.629 INFO:tasks.workunit.client.1.smithi196.stdout:4/891: getdents d1/d3/d4c/d13a 0 2024-03-13T14:50:01.637 INFO:tasks.workunit.client.1.smithi196.stdout:0/932: mknod d8/d37/c13a 0 2024-03-13T14:50:01.638 INFO:tasks.workunit.client.1.smithi196.stdout:3/872: write db/d15/d44/d79/fe6 [982706,120852] 0 2024-03-13T14:50:01.639 INFO:tasks.workunit.client.1.smithi196.stdout:4/892: rmdir d1/d3/dbc 39 2024-03-13T14:50:01.641 INFO:tasks.workunit.client.1.smithi196.stdout:0/933: stat d8/da/d25/d26/d48/d82/da0/df2/d108/f12b 0 2024-03-13T14:50:01.650 INFO:tasks.workunit.client.1.smithi196.stdout:1/999: rename d1/d28/d34/d67/d11f to d1/d9/d4e/d55/d63/d15c 0 2024-03-13T14:50:01.658 INFO:tasks.workunit.client.0.smithi176.stdout:6/835: rmdir d0/de/d8d 39 2024-03-13T14:50:01.665 INFO:tasks.workunit.client.1.smithi196.stdout:2/858: rmdir d0/d7/d9/d5a/df2 0 2024-03-13T14:50:01.672 INFO:tasks.workunit.client.0.smithi176.stdout:4/840: read - d7/d10/d46/d4a/dcc/ff7 zero size 2024-03-13T14:50:01.675 INFO:tasks.workunit.client.1.smithi196.stdout:9/892: sync 2024-03-13T14:50:01.675 INFO:tasks.workunit.client.1.smithi196.stdout:8/851: getdents d0/d16/d3d/d9c/d87/d8c/da9 0 2024-03-13T14:50:01.676 INFO:tasks.workunit.client.0.smithi176.stdout:4/841: write d7/d8/d15/d23/fa8 [2529050,30552] 0 2024-03-13T14:50:01.677 INFO:tasks.workunit.client.0.smithi176.stdout:1/808: mknod d2/d30/dd3/cf5 0 2024-03-13T14:50:01.680 INFO:tasks.workunit.client.0.smithi176.stdout:8/747: mkdir d0/d24/d98/dc6/db9/def 0 2024-03-13T14:50:01.681 INFO:tasks.workunit.client.0.smithi176.stdout:6/836: rmdir d0/dc1 39 2024-03-13T14:50:01.682 INFO:tasks.workunit.client.0.smithi176.stdout:6/837: stat d0/de/f2c 0 2024-03-13T14:50:01.684 INFO:tasks.workunit.client.0.smithi176.stdout:0/724: mknod d2/d5/d28/d8c/d9d/ce7 0 2024-03-13T14:50:01.684 INFO:tasks.workunit.client.1.smithi196.stdout:8/852: dread - d0/d16/fd9 zero size 2024-03-13T14:50:01.684 INFO:tasks.workunit.client.1.smithi196.stdout:3/873: write db/d15/d44/ff5 [234957,13312] 0 2024-03-13T14:50:01.692 INFO:tasks.workunit.client.0.smithi176.stdout:4/842: creat d7/d2d/da7/f119 x:0 0 0 2024-03-13T14:50:01.692 INFO:tasks.workunit.client.0.smithi176.stdout:9/918: write d0/db/d89/f16 [1330656,30142] 0 2024-03-13T14:50:01.694 INFO:tasks.workunit.client.0.smithi176.stdout:1/809: mknod d2/de/dd1/cf6 0 2024-03-13T14:50:01.694 INFO:tasks.workunit.client.0.smithi176.stdout:4/843: write d7/d8/d15/d23/fa8 [2427803,32315] 0 2024-03-13T14:50:01.697 INFO:tasks.workunit.client.0.smithi176.stdout:4/844: write d7/d8/d45/ff3 [3130718,106774] 0 2024-03-13T14:50:01.697 INFO:tasks.workunit.client.1.smithi196.stdout:2/859: getdents d0/d7/d9/d42/d98 0 2024-03-13T14:50:01.698 INFO:tasks.workunit.client.1.smithi196.stdout:4/893: dwrite d1/d3/d7/d35/f52 [0,4194304] 0 2024-03-13T14:50:01.701 INFO:tasks.workunit.client.0.smithi176.stdout:8/748: rename d0/d5/dea to d0/d5/d11/d47/db7/de0/df0 0 2024-03-13T14:50:01.702 INFO:tasks.workunit.client.1.smithi196.stdout:0/934: dwrite d8/da/fc [4194304,4194304] 0 2024-03-13T14:50:01.702 INFO:tasks.workunit.client.1.smithi196.stdout:9/893: write d7/de/d16/f60 [4846725,55903] 0 2024-03-13T14:50:01.707 INFO:tasks.workunit.client.1.smithi196.stdout:2/860: chown d0/d7/d8e/fdd 941 1 2024-03-13T14:50:01.711 INFO:tasks.workunit.client.0.smithi176.stdout:9/919: mknod d0/d40/d53/d6d/de1/c133 0 2024-03-13T14:50:01.711 INFO:tasks.workunit.client.0.smithi176.stdout:1/810: mkdir d2/de/d10/d16/d4a/d5f/d66/df7 0 2024-03-13T14:50:01.712 INFO:tasks.workunit.client.0.smithi176.stdout:7/763: link d1/d7/l24 d1/d3f/d4b/d63/lef 0 2024-03-13T14:50:01.714 INFO:tasks.workunit.client.1.smithi196.stdout:8/853: getdents d0/d10/da5/db2/dc8 0 2024-03-13T14:50:01.714 INFO:tasks.workunit.client.0.smithi176.stdout:7/764: fdatasync d1/d3f/d76/fb9 0 2024-03-13T14:50:01.716 INFO:tasks.workunit.client.1.smithi196.stdout:3/874: dwrite db/d19/f103 [0,4194304] 0 2024-03-13T14:50:01.718 INFO:tasks.workunit.client.0.smithi176.stdout:4/845: unlink c4 0 2024-03-13T14:50:01.721 INFO:tasks.workunit.client.1.smithi196.stdout:0/935: rmdir d8/da/d25/d26/d48/d82/da0/df2 39 2024-03-13T14:50:01.723 INFO:tasks.workunit.client.0.smithi176.stdout:8/749: mknod d0/db/cf1 0 2024-03-13T14:50:01.727 INFO:tasks.workunit.client.0.smithi176.stdout:5/790: write d5/d2e/d7e/dad/f88 [690243,22839] 0 2024-03-13T14:50:01.734 INFO:tasks.workunit.client.1.smithi196.stdout:4/894: dwrite d1/d3/d4c/d72/d7d/d80/fac [0,4194304] 0 2024-03-13T14:50:01.735 INFO:tasks.workunit.client.0.smithi176.stdout:1/811: dwrite d2/de/d3f/d50/f7a [0,4194304] 0 2024-03-13T14:50:01.739 INFO:tasks.workunit.client.1.smithi196.stdout:5/929: sync 2024-03-13T14:50:01.752 INFO:tasks.workunit.client.0.smithi176.stdout:2/842: sync 2024-03-13T14:50:01.752 INFO:tasks.workunit.client.0.smithi176.stdout:3/861: sync 2024-03-13T14:50:01.758 INFO:tasks.workunit.client.1.smithi196.stdout:2/861: dread d0/f1a [0,4194304] 0 2024-03-13T14:50:01.767 INFO:tasks.workunit.client.1.smithi196.stdout:9/894: dread d7/de/d4b/f14 [0,4194304] 0 2024-03-13T14:50:01.832 INFO:tasks.workunit.client.1.smithi196.stdout:0/936: truncate d8/d37/f73 1083941 0 2024-03-13T14:50:01.835 INFO:tasks.workunit.client.0.smithi176.stdout:4/846: mkdir d7/d8/d45/d11a 0 2024-03-13T14:50:01.837 INFO:tasks.workunit.client.0.smithi176.stdout:4/847: stat d7/d2d/fa4 0 2024-03-13T14:50:01.840 INFO:tasks.workunit.client.1.smithi196.stdout:2/862: rmdir d0/d7/d9/d42/d98/de4/d67 39 2024-03-13T14:50:01.844 INFO:tasks.workunit.client.1.smithi196.stdout:9/895: truncate f5 2586637 0 2024-03-13T14:50:01.847 INFO:tasks.workunit.client.0.smithi176.stdout:8/750: creat d0/d24/d98/dc6/ff2 x:0 0 0 2024-03-13T14:50:01.847 INFO:tasks.workunit.client.1.smithi196.stdout:3/875: getdents db/d19/d50/db7/dff 0 2024-03-13T14:50:01.849 INFO:tasks.workunit.client.0.smithi176.stdout:6/838: mkdir d0/de/d8d/d102/d10c 0 2024-03-13T14:50:01.851 INFO:tasks.workunit.client.0.smithi176.stdout:6/839: readlink d0/d10/led 0 2024-03-13T14:50:01.852 INFO:tasks.workunit.client.1.smithi196.stdout:8/854: creat d0/f11d x:0 0 0 2024-03-13T14:50:01.854 INFO:tasks.workunit.client.1.smithi196.stdout:0/937: symlink d8/da/d25/d26/d48/d82/l13b 0 2024-03-13T14:50:01.863 INFO:tasks.workunit.client.0.smithi176.stdout:1/812: read d2/f21 [642965,73018] 0 2024-03-13T14:50:01.865 INFO:tasks.workunit.client.1.smithi196.stdout:9/896: creat d7/de/d4b/d1d/d93/da7/f131 x:0 0 0 2024-03-13T14:50:01.868 INFO:tasks.workunit.client.1.smithi196.stdout:9/897: truncate d7/de/f128 875731 0 2024-03-13T14:50:01.870 INFO:tasks.workunit.client.1.smithi196.stdout:3/876: fsync db/d19/fa2 0 2024-03-13T14:50:01.871 INFO:tasks.workunit.client.1.smithi196.stdout:9/898: chown d7/de/d74/d94/dfb/dbf/f90 0 1 2024-03-13T14:50:01.872 INFO:tasks.workunit.client.1.smithi196.stdout:8/855: symlink d0/d10/d79/l11e 0 2024-03-13T14:50:01.878 INFO:tasks.workunit.client.1.smithi196.stdout:0/938: mknod d8/c13c 0 2024-03-13T14:50:01.878 INFO:tasks.workunit.client.0.smithi176.stdout:6/840: write d0/d10/d19/d27/d2b/f39 [210157,69342] 0 2024-03-13T14:50:01.884 INFO:tasks.workunit.client.1.smithi196.stdout:4/895: sync 2024-03-13T14:50:01.886 INFO:tasks.workunit.client.1.smithi196.stdout:3/877: readlink db/d19/l56 0 2024-03-13T14:50:01.888 INFO:tasks.workunit.client.0.smithi176.stdout:6/841: mknod d0/d10/d19/d27/d2b/de0/d76/d7c/c10d 0 2024-03-13T14:50:01.893 INFO:tasks.workunit.client.1.smithi196.stdout:9/899: truncate d7/de/d74/d94/dfb/dbf/f5c 3600450 0 2024-03-13T14:50:01.894 INFO:tasks.workunit.client.1.smithi196.stdout:8/856: dwrite d0/d63/f85 [0,4194304] 0 2024-03-13T14:50:01.896 INFO:tasks.workunit.client.1.smithi196.stdout:9/900: fsync d7/f101 0 2024-03-13T14:50:01.897 INFO:tasks.workunit.client.0.smithi176.stdout:6/842: write d0/d10/d19/d27/d2b/ddc/fef [533912,118674] 0 2024-03-13T14:50:01.905 INFO:tasks.workunit.client.1.smithi196.stdout:0/939: dread d8/da/d25/d26/d35/d3b/d59/d6a/f8d [0,4194304] 0 2024-03-13T14:50:01.940 INFO:tasks.workunit.client.1.smithi196.stdout:8/857: chown d0/d1c/c111 2 1 2024-03-13T14:50:01.940 INFO:tasks.workunit.client.1.smithi196.stdout:4/896: fdatasync d1/fd0 0 2024-03-13T14:50:01.944 INFO:tasks.workunit.client.1.smithi196.stdout:3/878: mknod db/d19/c114 0 2024-03-13T14:50:01.945 INFO:tasks.workunit.client.1.smithi196.stdout:0/940: creat d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe/f13d x:0 0 0 2024-03-13T14:50:01.948 INFO:tasks.workunit.client.1.smithi196.stdout:8/858: creat d0/d16/d3d/d9c/d87/f11f x:0 0 0 2024-03-13T14:50:01.952 INFO:tasks.workunit.client.1.smithi196.stdout:9/901: fsync d7/de/d4b/d1d/d35/dcc/f107 0 2024-03-13T14:50:01.958 INFO:tasks.workunit.client.1.smithi196.stdout:8/859: truncate d0/d16/d3d/d8b/f38 3857006 0 2024-03-13T14:50:01.967 INFO:tasks.workunit.client.1.smithi196.stdout:0/941: dread d8/da/d25/d26/d35/d3b/d6c/fc6 [0,4194304] 0 2024-03-13T14:50:01.969 INFO:tasks.workunit.client.1.smithi196.stdout:9/902: dread d7/de/f87 [4194304,4194304] 0 2024-03-13T14:50:01.973 INFO:tasks.workunit.client.1.smithi196.stdout:0/942: symlink d8/da/d25/d26/d48/d82/db3/d12c/l13e 0 2024-03-13T14:50:01.975 INFO:tasks.workunit.client.1.smithi196.stdout:9/903: dread d7/de/d74/d94/dfb/d28/d6c/d7b/fb4 [0,4194304] 0 2024-03-13T14:50:01.975 INFO:tasks.workunit.client.1.smithi196.stdout:8/860: link d0/d10/d12/d17/d42/fd2 d0/d10/d70/dc9/f120 0 2024-03-13T14:50:01.977 INFO:tasks.workunit.client.1.smithi196.stdout:0/943: truncate d8/f9 2425643 0 2024-03-13T14:50:01.978 INFO:tasks.workunit.client.1.smithi196.stdout:3/879: dread db/d15/d44/d79/fba [0,4194304] 0 2024-03-13T14:50:01.979 INFO:tasks.workunit.client.1.smithi196.stdout:8/861: symlink d0/d16/dd7/l121 0 2024-03-13T14:50:01.980 INFO:tasks.workunit.client.1.smithi196.stdout:9/904: creat d7/de/d74/d94/dfb/dbf/f132 x:0 0 0 2024-03-13T14:50:01.982 INFO:tasks.workunit.client.1.smithi196.stdout:0/944: creat d8/f13f x:0 0 0 2024-03-13T14:50:01.983 INFO:tasks.workunit.client.1.smithi196.stdout:3/880: fsync db/f3d 0 2024-03-13T14:50:01.986 INFO:tasks.workunit.client.1.smithi196.stdout:9/905: rmdir d7/de/d4b/d1d/d35 39 2024-03-13T14:50:01.993 INFO:tasks.workunit.client.1.smithi196.stdout:3/881: getdents db/d15/da1/dc8 0 2024-03-13T14:50:01.995 INFO:tasks.workunit.client.1.smithi196.stdout:3/882: read db/d15/d44/d79/fba [1188497,12917] 0 2024-03-13T14:50:01.997 INFO:tasks.workunit.client.1.smithi196.stdout:8/862: rename d0/d16/d3d/d7f/dda/c107 to d0/d16/d71/d9a/dfc/c122 0 2024-03-13T14:50:02.000 INFO:tasks.workunit.client.0.smithi176.stdout:0/725: rename d2/db/d35/daa/cd9 to d2/db/d31/dc4/ce8 0 2024-03-13T14:50:02.006 INFO:tasks.workunit.client.0.smithi176.stdout:6/843: rename d0/de/d54/c91 to d0/d10/d19/d27/d2b/de0/d76/d7c/dfd/c10e 0 2024-03-13T14:50:02.007 INFO:tasks.workunit.client.0.smithi176.stdout:6/844: chown d0/d10/dca/de1 18446 1 2024-03-13T14:50:02.008 INFO:tasks.workunit.client.1.smithi196.stdout:8/863: truncate d0/d10/da5/d110/f76 1519003 0 2024-03-13T14:50:02.008 INFO:tasks.workunit.client.0.smithi176.stdout:7/765: write d1/d28/d3a/d1b/f89 [808654,104378] 0 2024-03-13T14:50:02.010 INFO:tasks.workunit.client.0.smithi176.stdout:7/766: readlink d1/d1c/d26/l6f 0 2024-03-13T14:50:02.012 INFO:tasks.workunit.client.0.smithi176.stdout:7/767: fsync d1/d1c/d4a/f98 0 2024-03-13T14:50:02.012 INFO:tasks.workunit.client.0.smithi176.stdout:6/845: creat d0/d10/d19/d27/d2b/de0/d76/d7c/f10f x:0 0 0 2024-03-13T14:50:02.013 INFO:tasks.workunit.client.1.smithi196.stdout:3/883: dwrite db/f36 [0,4194304] 0 2024-03-13T14:50:02.014 INFO:tasks.workunit.client.0.smithi176.stdout:0/726: rename d2/d5/d28/d8c/ldb to d2/db/d39/d9c/le9 0 2024-03-13T14:50:02.015 INFO:tasks.workunit.client.1.smithi196.stdout:3/884: readlink db/d15/d44/d51/la6 0 2024-03-13T14:50:02.017 INFO:tasks.workunit.client.1.smithi196.stdout:5/930: write d4/d1b/dc8/d14/d37/f78 [2089915,84831] 0 2024-03-13T14:50:02.017 INFO:tasks.workunit.client.1.smithi196.stdout:3/885: readlink db/d19/d22/d3a/d53/laa 0 2024-03-13T14:50:02.020 INFO:tasks.workunit.client.0.smithi176.stdout:0/727: creat d2/db/d39/fea x:0 0 0 2024-03-13T14:50:02.021 INFO:tasks.workunit.client.1.smithi196.stdout:3/886: chown db/d37/cab 2 1 2024-03-13T14:50:02.027 INFO:tasks.workunit.client.0.smithi176.stdout:7/768: symlink d1/d1c/d26/d41/d2b/d40/lf0 0 2024-03-13T14:50:02.028 INFO:tasks.workunit.client.0.smithi176.stdout:5/791: rmdir d5/d2c 39 2024-03-13T14:50:02.031 INFO:tasks.workunit.client.0.smithi176.stdout:0/728: fsync d2/f9b 0 2024-03-13T14:50:02.032 INFO:tasks.workunit.client.1.smithi196.stdout:3/887: fdatasync db/d15/d44/d79/fba 0 2024-03-13T14:50:02.033 INFO:tasks.workunit.client.1.smithi196.stdout:8/864: mknod d0/d16/d3d/d9c/d87/d8c/c123 0 2024-03-13T14:50:02.034 INFO:tasks.workunit.client.1.smithi196.stdout:8/865: readlink d0/d10/d79/l11e 0 2024-03-13T14:50:02.041 INFO:tasks.workunit.client.0.smithi176.stdout:5/792: dread d5/d2e/f34 [0,4194304] 0 2024-03-13T14:50:02.042 INFO:tasks.workunit.client.1.smithi196.stdout:8/866: fdatasync d0/d10/f77 0 2024-03-13T14:50:02.043 INFO:tasks.workunit.client.0.smithi176.stdout:5/793: chown d5/d2e/d7e 31383 1 2024-03-13T14:50:02.046 INFO:tasks.workunit.client.0.smithi176.stdout:3/862: write d4/d13/d81/d89/d5b/dc5/fa5 [1033407,116787] 0 2024-03-13T14:50:02.047 INFO:tasks.workunit.client.0.smithi176.stdout:5/794: truncate d5/d31/f3e 68851 0 2024-03-13T14:50:02.052 INFO:tasks.workunit.client.0.smithi176.stdout:9/920: truncate d0/db/d89/d4d/d6e/f71 2213310 0 2024-03-13T14:50:02.055 INFO:tasks.workunit.client.0.smithi176.stdout:3/863: rename d4/db8 to d4/d28/d31/d45/d82/d12b 0 2024-03-13T14:50:02.058 INFO:tasks.workunit.client.1.smithi196.stdout:2/863: truncate d0/d7/d9/d9a/fb4 57041 0 2024-03-13T14:50:02.060 INFO:tasks.workunit.client.0.smithi176.stdout:0/729: dread d2/db/d39/fa2 [0,4194304] 0 2024-03-13T14:50:02.060 INFO:tasks.workunit.client.0.smithi176.stdout:9/921: rename d0/d40/d53/la2 to d0/db/d89/d90/df5/l134 0 2024-03-13T14:50:02.062 INFO:tasks.workunit.client.0.smithi176.stdout:8/751: truncate d0/d24/f32 7577732 0 2024-03-13T14:50:02.069 INFO:tasks.workunit.client.0.smithi176.stdout:0/730: dread d2/d74/d9f/fbf [0,4194304] 0 2024-03-13T14:50:02.070 INFO:tasks.workunit.client.0.smithi176.stdout:3/864: mkdir d4/d28/d31/d45/d82/d87/d102/d12c 0 2024-03-13T14:50:02.071 INFO:tasks.workunit.client.0.smithi176.stdout:9/922: symlink d0/db/d89/d32/d122/l135 0 2024-03-13T14:50:02.073 INFO:tasks.workunit.client.0.smithi176.stdout:8/752: rmdir d0/db/d39/d56/d93 39 2024-03-13T14:50:02.074 INFO:tasks.workunit.client.0.smithi176.stdout:8/753: chown d0/db/d39/d56/d91 3 1 2024-03-13T14:50:02.075 INFO:tasks.workunit.client.0.smithi176.stdout:0/731: creat d2/d5/dc/d20/feb x:0 0 0 2024-03-13T14:50:02.076 INFO:tasks.workunit.client.0.smithi176.stdout:0/732: chown d2/d74/cda 13195 1 2024-03-13T14:50:02.077 INFO:tasks.workunit.client.0.smithi176.stdout:9/923: rename d0/db/d89/d4d/d6e/d100/d110 to d0/d40/d53/d6d/d97/d136 0 2024-03-13T14:50:02.078 INFO:tasks.workunit.client.0.smithi176.stdout:9/924: chown d0/d40/d53/d6d/d97/fbc 79904 1 2024-03-13T14:50:02.082 INFO:tasks.workunit.client.0.smithi176.stdout:8/754: getdents d0/d5 0 2024-03-13T14:50:02.083 INFO:tasks.workunit.client.0.smithi176.stdout:0/733: rename d2/db/f1c to d2/d5/d1e/d97/dcf/fec 0 2024-03-13T14:50:02.086 INFO:tasks.workunit.client.0.smithi176.stdout:9/925: rename d0/d40/d53/d6d/d97/d136/l121 to d0/d1e/d8d/d9f/l137 0 2024-03-13T14:50:02.087 INFO:tasks.workunit.client.0.smithi176.stdout:9/926: fsync d0/d40/d53/d6d/f124 0 2024-03-13T14:50:02.088 INFO:tasks.workunit.client.0.smithi176.stdout:8/755: rename d0/d24/d98/ld1 to d0/d24/d7b/lf3 0 2024-03-13T14:50:02.090 INFO:tasks.workunit.client.0.smithi176.stdout:9/927: fsync d0/db/d89/d90/fcd 0 2024-03-13T14:50:02.092 INFO:tasks.workunit.client.0.smithi176.stdout:8/756: symlink d0/d5/d11/d47/db7/de0/df0/lf4 0 2024-03-13T14:50:02.093 INFO:tasks.workunit.client.0.smithi176.stdout:9/928: chown d0/db/d89/d32/d58/d5f/f75 0 1 2024-03-13T14:50:02.093 INFO:tasks.workunit.client.0.smithi176.stdout:8/757: dread - d0/d24/d98/dc6/d40/fd3 zero size 2024-03-13T14:50:02.096 INFO:tasks.workunit.client.0.smithi176.stdout:8/758: truncate d0/d24/d29/f36 376923 0 2024-03-13T14:50:02.105 INFO:tasks.workunit.client.1.smithi196.stdout:9/906: rename d7/de/d4b/d1d/d22/f126 to d7/de/d4b/dd1/f133 0 2024-03-13T14:50:02.110 INFO:tasks.workunit.client.0.smithi176.stdout:8/759: dwrite d0/d24/d98/fe9 [0,4194304] 0 2024-03-13T14:50:02.111 INFO:tasks.workunit.client.1.smithi196.stdout:4/897: sync 2024-03-13T14:50:02.113 INFO:tasks.workunit.client.1.smithi196.stdout:4/898: chown d1/d1e/c11a 60145 1 2024-03-13T14:50:02.121 INFO:tasks.workunit.client.1.smithi196.stdout:4/899: mknod d1/dfb/d11c/d13d/c13e 0 2024-03-13T14:50:02.122 INFO:tasks.workunit.client.1.smithi196.stdout:4/900: stat d1/fd0 0 2024-03-13T14:50:02.125 INFO:tasks.workunit.client.1.smithi196.stdout:0/945: dwrite d8/da/d25/d26/d35/d3b/d59/f9f [0,4194304] 0 2024-03-13T14:50:02.134 INFO:tasks.workunit.client.1.smithi196.stdout:4/901: link d1/d3/d7/d35/d76/fed d1/d3/d7/d35/d76/d58/f13f 0 2024-03-13T14:50:02.139 INFO:tasks.workunit.client.1.smithi196.stdout:5/931: write d4/d1b/dc8/d14/d19/d26/dab/fb9 [850625,54924] 0 2024-03-13T14:50:02.144 INFO:tasks.workunit.client.1.smithi196.stdout:0/946: dwrite d8/da/f32 [0,4194304] 0 2024-03-13T14:50:02.148 INFO:tasks.workunit.client.1.smithi196.stdout:4/902: rename d1/d3/d7/f1d to d1/d3/d7/d35/d76/d58/dc9/dd7/f140 0 2024-03-13T14:50:02.148 INFO:tasks.workunit.client.1.smithi196.stdout:3/888: write db/d19/d50/db7/f42 [3726065,105740] 0 2024-03-13T14:50:02.152 INFO:tasks.workunit.client.1.smithi196.stdout:8/867: write d0/f51 [3163812,5828] 0 2024-03-13T14:50:02.159 INFO:tasks.workunit.client.0.smithi176.stdout:4/848: sync 2024-03-13T14:50:02.159 INFO:tasks.workunit.client.0.smithi176.stdout:1/813: sync 2024-03-13T14:50:02.160 INFO:tasks.workunit.client.0.smithi176.stdout:2/843: sync 2024-03-13T14:50:02.161 INFO:tasks.workunit.client.0.smithi176.stdout:7/769: dwrite d1/d1c/d26/d41/d2b/d40/d64/d94/f83 [0,4194304] 0 2024-03-13T14:50:02.163 INFO:tasks.workunit.client.1.smithi196.stdout:4/903: link d1/d3/d7/d35/d76/d58/dc6/f11f d1/d3/d4c/f141 0 2024-03-13T14:50:02.165 INFO:tasks.workunit.client.0.smithi176.stdout:5/795: dwrite d5/d2e/d7e/dad/f92 [0,4194304] 0 2024-03-13T14:50:02.165 INFO:tasks.workunit.client.1.smithi196.stdout:8/868: getdents d0/d16/d3d/d9c/d87 0 2024-03-13T14:50:02.167 INFO:tasks.workunit.client.0.smithi176.stdout:4/849: symlink d7/d10/d2c/df1/l11b 0 2024-03-13T14:50:02.172 INFO:tasks.workunit.client.0.smithi176.stdout:4/850: read d7/d8/d15/f7b [8850,22061] 0 2024-03-13T14:50:02.173 INFO:tasks.workunit.client.0.smithi176.stdout:7/770: mkdir d1/d1c/d26/d41/db6/df1 0 2024-03-13T14:50:02.179 INFO:tasks.workunit.client.1.smithi196.stdout:2/864: dwrite d0/d7/d9/d61/d83/fad [0,4194304] 0 2024-03-13T14:50:02.181 INFO:tasks.workunit.client.0.smithi176.stdout:2/844: fdatasync d1/dc/d2d/d90/dcf/da0/fa6 0 2024-03-13T14:50:02.182 INFO:tasks.workunit.client.1.smithi196.stdout:0/947: dwrite d8/da/d25/ff0 [0,4194304] 0 2024-03-13T14:50:02.183 INFO:tasks.workunit.client.0.smithi176.stdout:2/845: fdatasync d1/dc/d1c/f37 0 2024-03-13T14:50:02.193 INFO:tasks.workunit.client.1.smithi196.stdout:0/948: write d8/da/d25/d26/d35/d3b/d50/d5d/da5/f112 [1264590,83092] 0 2024-03-13T14:50:02.198 INFO:tasks.workunit.client.0.smithi176.stdout:3/865: truncate d4/d28/d31/d52/f6a 2907329 0 2024-03-13T14:50:02.198 INFO:tasks.workunit.client.0.smithi176.stdout:5/796: truncate d5/d22/d15/f84 978342 0 2024-03-13T14:50:02.198 INFO:tasks.workunit.client.0.smithi176.stdout:5/797: chown d5/d31 87 1 2024-03-13T14:50:02.205 INFO:tasks.workunit.client.0.smithi176.stdout:0/734: write d2/d5/dc/d20/f37 [1172383,70966] 0 2024-03-13T14:50:02.206 INFO:tasks.workunit.client.0.smithi176.stdout:7/771: creat d1/d3f/d76/da5/ff2 x:0 0 0 2024-03-13T14:50:02.216 INFO:tasks.workunit.client.0.smithi176.stdout:9/929: write d0/db/d89/d32/d58/d5f/fc8 [1125090,13288] 0 2024-03-13T14:50:02.219 INFO:tasks.workunit.client.0.smithi176.stdout:1/814: dread d2/de/d3f/d50/d83/fae [8388608,4194304] 0 2024-03-13T14:50:02.220 INFO:tasks.workunit.client.0.smithi176.stdout:2/846: symlink d1/dc/d2d/d59/d80/d77/l120 0 2024-03-13T14:50:02.222 INFO:tasks.workunit.client.0.smithi176.stdout:2/847: truncate d1/dc/d2d/d4b/ddb/fdc 600439 0 2024-03-13T14:50:02.223 INFO:tasks.workunit.client.0.smithi176.stdout:8/760: write d0/db/d39/d56/f79 [2318775,95747] 0 2024-03-13T14:50:02.225 INFO:tasks.workunit.client.0.smithi176.stdout:6/846: sync 2024-03-13T14:50:02.226 INFO:tasks.workunit.client.0.smithi176.stdout:5/798: truncate d5/d2b/f5b 3421121 0 2024-03-13T14:50:02.228 INFO:tasks.workunit.client.0.smithi176.stdout:6/847: chown d0/de/d38/d6b/l71 108865808 1 2024-03-13T14:50:02.230 INFO:tasks.workunit.client.1.smithi196.stdout:9/907: dwrite d7/de/d16/fe1 [0,4194304] 0 2024-03-13T14:50:02.231 INFO:tasks.workunit.client.0.smithi176.stdout:0/735: mkdir d2/db/d35/ded 0 2024-03-13T14:50:02.234 INFO:tasks.workunit.client.0.smithi176.stdout:7/772: chown d1/d1c/d26/d41/fcc 3 1 2024-03-13T14:50:02.237 INFO:tasks.workunit.client.0.smithi176.stdout:9/930: creat d0/d1e/d8d/d9f/f138 x:0 0 0 2024-03-13T14:50:02.240 INFO:tasks.workunit.client.0.smithi176.stdout:3/866: mknod d4/d13/d16/c12d 0 2024-03-13T14:50:02.240 INFO:tasks.workunit.client.0.smithi176.stdout:1/815: dread d2/de/d3f/d50/d83/f98 [0,4194304] 0 2024-03-13T14:50:02.241 INFO:tasks.workunit.client.0.smithi176.stdout:2/848: fdatasync d1/dc/d2d/f7a 0 2024-03-13T14:50:02.244 INFO:tasks.workunit.client.1.smithi196.stdout:9/908: read d7/de/d74/d94/dfb/f3a [3645846,14659] 0 2024-03-13T14:50:02.244 INFO:tasks.workunit.client.0.smithi176.stdout:5/799: creat d5/d2b/f110 x:0 0 0 2024-03-13T14:50:02.245 INFO:tasks.workunit.client.1.smithi196.stdout:3/889: write db/d19/d22/f3c [4144950,1878] 0 2024-03-13T14:50:02.245 INFO:tasks.workunit.client.0.smithi176.stdout:2/849: dread - d1/dc/d2d/d59/d80/d7e/da3/f11a zero size 2024-03-13T14:50:02.247 INFO:tasks.workunit.client.0.smithi176.stdout:2/850: chown d1/dc/d1c/f37 46 1 2024-03-13T14:50:02.249 INFO:tasks.workunit.client.0.smithi176.stdout:2/851: dread - d1/dc/d2d/d4f/f11b zero size 2024-03-13T14:50:02.250 INFO:tasks.workunit.client.0.smithi176.stdout:0/736: dread d2/db/d39/f47 [0,4194304] 0 2024-03-13T14:50:02.250 INFO:tasks.workunit.client.0.smithi176.stdout:9/931: rmdir d0/db/d89/d32 39 2024-03-13T14:50:02.251 INFO:tasks.workunit.client.1.smithi196.stdout:4/904: write d1/dfb/d11c/fb6 [739580,31277] 0 2024-03-13T14:50:02.252 INFO:tasks.workunit.client.1.smithi196.stdout:3/890: mkdir db/d19/d50/db7/da9/d115 0 2024-03-13T14:50:02.255 INFO:tasks.workunit.client.0.smithi176.stdout:2/852: write d1/dc/d2d/d59/d80/d5c/f5f [2702060,47270] 0 2024-03-13T14:50:02.258 INFO:tasks.workunit.client.1.smithi196.stdout:4/905: dread d1/d3/d7/d35/f63 [0,4194304] 0 2024-03-13T14:50:02.259 INFO:tasks.workunit.client.0.smithi176.stdout:3/867: creat d4/d28/d4f/f12e x:0 0 0 2024-03-13T14:50:02.261 INFO:tasks.workunit.client.1.smithi196.stdout:3/891: creat db/d19/d22/d3a/d68/d8e/f116 x:0 0 0 2024-03-13T14:50:02.261 INFO:tasks.workunit.client.0.smithi176.stdout:4/851: write d7/d10/f85 [184993,3149] 0 2024-03-13T14:50:02.263 INFO:tasks.workunit.client.1.smithi196.stdout:2/865: write d0/d7/d9/d42/d98/de4/d104/f23 [3221756,43042] 0 2024-03-13T14:50:02.264 INFO:tasks.workunit.client.1.smithi196.stdout:4/906: mknod d1/dfb/d11c/c142 0 2024-03-13T14:50:02.266 INFO:tasks.workunit.client.1.smithi196.stdout:3/892: mknod db/d19/d4a/d9b/de9/c117 0 2024-03-13T14:50:02.267 INFO:tasks.workunit.client.1.smithi196.stdout:4/907: fsync d1/d3/d7/f65 0 2024-03-13T14:50:02.270 INFO:tasks.workunit.client.1.smithi196.stdout:4/908: stat d1/d3/d7/d35/d76/d58/d69/de3/df4 0 2024-03-13T14:50:02.272 INFO:tasks.workunit.client.0.smithi176.stdout:7/773: symlink d1/d3f/d76/da5/dde/lf3 0 2024-03-13T14:50:02.273 INFO:tasks.workunit.client.0.smithi176.stdout:7/774: chown d1/d28/d3a/f20 11 1 2024-03-13T14:50:02.274 INFO:tasks.workunit.client.1.smithi196.stdout:0/949: write d8/ff9 [268750,118662] 0 2024-03-13T14:50:02.274 INFO:tasks.workunit.client.0.smithi176.stdout:7/775: chown d1/d3f/d76/da5/dde 12573716 1 2024-03-13T14:50:02.278 INFO:tasks.workunit.client.1.smithi196.stdout:3/893: creat db/d19/d22/d3a/d53/f118 x:0 0 0 2024-03-13T14:50:02.282 INFO:tasks.workunit.client.1.smithi196.stdout:8/869: truncate d0/d16/d3d/d8b/f38 2730046 0 2024-03-13T14:50:02.286 INFO:tasks.workunit.client.1.smithi196.stdout:2/866: dwrite d0/d7/d9/d42/d98/de4/d104/f7f [8388608,4194304] 0 2024-03-13T14:50:02.290 INFO:tasks.workunit.client.1.smithi196.stdout:9/909: write d7/d71/f9e [653849,105616] 0 2024-03-13T14:50:02.293 INFO:tasks.workunit.client.0.smithi176.stdout:3/868: rmdir d4/d13/d81 39 2024-03-13T14:50:02.293 INFO:tasks.workunit.client.1.smithi196.stdout:5/932: sync 2024-03-13T14:50:02.294 INFO:tasks.workunit.client.1.smithi196.stdout:9/910: read d7/de/f29 [273049,89466] 0 2024-03-13T14:50:02.305 INFO:tasks.workunit.client.0.smithi176.stdout:6/848: write d0/de/d38/d70/f9e [1025660,126677] 0 2024-03-13T14:50:02.306 INFO:tasks.workunit.client.1.smithi196.stdout:5/933: fsync d4/d1b/dc8/d14/d19/d26/d33/f8f 0 2024-03-13T14:50:02.309 INFO:tasks.workunit.client.0.smithi176.stdout:5/800: rename d5/d22 to d5/d2e/d7e/dad/d7b/da7/d111 0 2024-03-13T14:50:02.310 INFO:tasks.workunit.client.1.smithi196.stdout:2/867: mkdir d0/d7/d9/d42/d98/de4/d67/d116 0 2024-03-13T14:50:02.316 INFO:tasks.workunit.client.1.smithi196.stdout:9/911: fsync d7/f69 0 2024-03-13T14:50:02.318 INFO:tasks.workunit.client.0.smithi176.stdout:9/932: creat d0/db/d89/d32/d58/d67/f139 x:0 0 0 2024-03-13T14:50:02.322 INFO:tasks.workunit.client.1.smithi196.stdout:8/870: dread d0/d10/d12/d1e/f34 [0,4194304] 0 2024-03-13T14:50:02.322 INFO:tasks.workunit.client.1.smithi196.stdout:0/950: write d8/da/d25/d26/d35/d3b/d6c/f9d [975078,39454] 0 2024-03-13T14:50:02.327 INFO:tasks.workunit.client.0.smithi176.stdout:4/852: dread d7/f50 [0,4194304] 0 2024-03-13T14:50:02.328 INFO:tasks.workunit.client.1.smithi196.stdout:0/951: write d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f87 [1093491,58291] 0 2024-03-13T14:50:02.329 INFO:tasks.workunit.client.0.smithi176.stdout:6/849: mkdir d0/d10/d19/d27/d2b/db1/d110 0 2024-03-13T14:50:02.329 INFO:tasks.workunit.client.1.smithi196.stdout:4/909: dwrite d1/d3/d4c/d72/d7d/d80/f86 [0,4194304] 0 2024-03-13T14:50:02.334 INFO:tasks.workunit.client.0.smithi176.stdout:1/816: dwrite d2/de/d10/f72 [0,4194304] 0 2024-03-13T14:50:02.337 INFO:tasks.workunit.client.1.smithi196.stdout:8/871: fsync d0/d10/d12/df5/fa6 0 2024-03-13T14:50:02.337 INFO:tasks.workunit.client.0.smithi176.stdout:5/801: unlink d5/d2b/c6c 0 2024-03-13T14:50:02.338 INFO:tasks.workunit.client.0.smithi176.stdout:7/776: write d1/f1f [5090308,118009] 0 2024-03-13T14:50:02.340 INFO:tasks.workunit.client.1.smithi196.stdout:3/894: dwrite db/f2d [0,4194304] 0 2024-03-13T14:50:02.341 INFO:tasks.workunit.client.0.smithi176.stdout:0/737: write d2/d5/fb2 [443070,33774] 0 2024-03-13T14:50:02.343 INFO:tasks.workunit.client.0.smithi176.stdout:5/802: read - d5/d31/d47/d80/fbf zero size 2024-03-13T14:50:02.345 INFO:tasks.workunit.client.1.smithi196.stdout:9/912: mknod d7/de/d74/c134 0 2024-03-13T14:50:02.349 INFO:tasks.workunit.client.0.smithi176.stdout:2/853: write d1/dc/d2d/d4b/d69/fb4 [1770192,7081] 0 2024-03-13T14:50:02.350 INFO:tasks.workunit.client.0.smithi176.stdout:3/869: mkdir d4/d28/d34/d12f 0 2024-03-13T14:50:02.352 INFO:tasks.workunit.client.1.smithi196.stdout:4/910: symlink d1/d3/d7/d35/d76/d58/dc9/l143 0 2024-03-13T14:50:02.354 INFO:tasks.workunit.client.0.smithi176.stdout:8/761: truncate d0/d24/f32 529845 0 2024-03-13T14:50:02.356 INFO:tasks.workunit.client.1.smithi196.stdout:8/872: dread d0/d10/d12/df5/f3c [0,4194304] 0 2024-03-13T14:50:02.357 INFO:tasks.workunit.client.1.smithi196.stdout:2/868: creat d0/d7/d9/d42/d98/de4/f117 x:0 0 0 2024-03-13T14:50:02.360 INFO:tasks.workunit.client.1.smithi196.stdout:3/895: rmdir db/d37 39 2024-03-13T14:50:02.362 INFO:tasks.workunit.client.0.smithi176.stdout:7/777: mkdir d1/d28/d3a/d1b/d8f/dd1/df4 0 2024-03-13T14:50:02.368 INFO:tasks.workunit.client.1.smithi196.stdout:5/934: dread d4/d1b/dc8/d14/f5e [4194304,4194304] 0 2024-03-13T14:50:02.368 INFO:tasks.workunit.client.0.smithi176.stdout:9/933: rename d0/db/d89/d32/d58/d5f/d84/d9a/cfa to d0/d1e/d8d/dbe/c13a 0 2024-03-13T14:50:02.371 INFO:tasks.workunit.client.1.smithi196.stdout:5/935: rename d4 to d4/d1b/dc8/d14/d124/d129 22 2024-03-13T14:50:02.372 INFO:tasks.workunit.client.0.smithi176.stdout:0/738: dwrite d2/db/d35/daa/fe1 [0,4194304] 0 2024-03-13T14:50:02.375 INFO:tasks.workunit.client.0.smithi176.stdout:6/850: creat d0/de/d38/d88/d9d/dfe/f111 x:0 0 0 2024-03-13T14:50:02.378 INFO:tasks.workunit.client.1.smithi196.stdout:8/873: creat d0/d10/d12/d1e/dc5/f124 x:0 0 0 2024-03-13T14:50:02.381 INFO:tasks.workunit.client.0.smithi176.stdout:8/762: mknod d0/d5/d11/d47/db7/cf5 0 2024-03-13T14:50:02.382 INFO:tasks.workunit.client.0.smithi176.stdout:6/851: dread - d0/d10/d19/d27/d2b/d2e/d3e/db4/ffc zero size 2024-03-13T14:50:02.383 INFO:tasks.workunit.client.1.smithi196.stdout:9/913: link d7/de/d4b/d1d/d93/da7/fb6 d7/de/d16/f135 0 2024-03-13T14:50:02.385 INFO:tasks.workunit.client.0.smithi176.stdout:6/852: write d0/de/d38/fe9 [97873,83947] 0 2024-03-13T14:50:02.386 INFO:tasks.workunit.client.1.smithi196.stdout:5/936: unlink d4/d1b/dc8/d14/d19/d1c/d25/d38/cbe 0 2024-03-13T14:50:02.391 INFO:tasks.workunit.client.0.smithi176.stdout:5/803: creat d5/d2c/d40/dc7/d89/df2/dff/f112 x:0 0 0 2024-03-13T14:50:02.391 INFO:tasks.workunit.client.0.smithi176.stdout:6/853: dread - d0/d10/dca/dd2/f108 zero size 2024-03-13T14:50:02.395 INFO:tasks.workunit.client.0.smithi176.stdout:9/934: dread d0/dd8/d82/fb7 [0,4194304] 0 2024-03-13T14:50:02.399 INFO:tasks.workunit.client.0.smithi176.stdout:0/739: rename d2/db/d31/c6b to d2/db/d35/daa/cee 0 2024-03-13T14:50:02.402 INFO:tasks.workunit.client.1.smithi196.stdout:0/952: dread d8/da/d25/d26/f60 [0,4194304] 0 2024-03-13T14:50:02.403 INFO:tasks.workunit.client.1.smithi196.stdout:5/937: fdatasync d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/fdd 0 2024-03-13T14:50:02.406 INFO:tasks.workunit.client.0.smithi176.stdout:5/804: chown d5/d2c/d40/f8f 1485 1 2024-03-13T14:50:02.407 INFO:tasks.workunit.client.0.smithi176.stdout:3/870: write d4/d28/d31/fc3 [2720462,33996] 0 2024-03-13T14:50:02.409 INFO:tasks.workunit.client.1.smithi196.stdout:8/874: getdents d0/d16/d71/d68/d115 0 2024-03-13T14:50:02.410 INFO:tasks.workunit.client.1.smithi196.stdout:8/875: stat d0/d16/c32 0 2024-03-13T14:50:02.412 INFO:tasks.workunit.client.0.smithi176.stdout:4/853: dwrite d7/d8/d15/d23/f35 [0,4194304] 0 2024-03-13T14:50:02.419 INFO:tasks.workunit.client.0.smithi176.stdout:9/935: dread d0/db/d89/d4d/fc3 [0,4194304] 0 2024-03-13T14:50:02.419 INFO:tasks.workunit.client.0.smithi176.stdout:3/871: fdatasync d4/d28/f49 0 2024-03-13T14:50:02.419 INFO:tasks.workunit.client.1.smithi196.stdout:4/911: dwrite d1/d3/d7/d35/d76/fba [0,4194304] 0 2024-03-13T14:50:02.423 INFO:tasks.workunit.client.1.smithi196.stdout:2/869: getdents d0/d50 0 2024-03-13T14:50:02.423 INFO:tasks.workunit.client.1.smithi196.stdout:4/912: dread - d1/d3/f137 zero size 2024-03-13T14:50:02.426 INFO:tasks.workunit.client.1.smithi196.stdout:2/870: write d0/d7/d8e/fdd [3872927,77576] 0 2024-03-13T14:50:02.427 INFO:tasks.workunit.client.0.smithi176.stdout:8/763: dwrite d0/d24/d98/dc6/d40/fd3 [0,4194304] 0 2024-03-13T14:50:02.429 INFO:tasks.workunit.client.1.smithi196.stdout:3/896: dread db/d19/d22/d3a/d53/f10c [4194304,4194304] 0 2024-03-13T14:50:02.429 INFO:tasks.workunit.client.0.smithi176.stdout:1/817: dwrite d2/d30/f65 [0,4194304] 0 2024-03-13T14:50:02.431 INFO:tasks.workunit.client.1.smithi196.stdout:5/938: truncate d4/d1b/dc8/d14/d19/d1c/f23 1720079 0 2024-03-13T14:50:02.432 INFO:tasks.workunit.client.0.smithi176.stdout:2/854: dread d1/dc/d2d/d90/dcf/f44 [0,4194304] 0 2024-03-13T14:50:02.433 INFO:tasks.workunit.client.0.smithi176.stdout:8/764: stat d0/d24/d7b 0 2024-03-13T14:50:02.433 INFO:tasks.workunit.client.1.smithi196.stdout:9/914: getdents d7/d9 0 2024-03-13T14:50:02.434 INFO:tasks.workunit.client.1.smithi196.stdout:3/897: chown db/d15/d44/d79/fe6 26947926 1 2024-03-13T14:50:02.435 INFO:tasks.workunit.client.0.smithi176.stdout:6/854: write d0/de/d38/f80 [1676514,92291] 0 2024-03-13T14:50:02.436 INFO:tasks.workunit.client.0.smithi176.stdout:3/872: creat d4/d13/d16/d3d/f130 x:0 0 0 2024-03-13T14:50:02.442 INFO:tasks.workunit.client.1.smithi196.stdout:4/913: mkdir d1/d3/d4c/d72/d106/d144 0 2024-03-13T14:50:02.442 INFO:tasks.workunit.client.0.smithi176.stdout:1/818: readlink d2/de/d10/d16/d47/lc8 0 2024-03-13T14:50:02.443 INFO:tasks.workunit.client.0.smithi176.stdout:5/805: getdents d5/d31/d47/d80 0 2024-03-13T14:50:02.444 INFO:tasks.workunit.client.0.smithi176.stdout:9/936: mkdir d0/db/d89/d32/d122/d13b 0 2024-03-13T14:50:02.444 INFO:tasks.workunit.client.0.smithi176.stdout:2/855: write d1/dc/d2d/d59/d80/d7e/f119 [984852,56445] 0 2024-03-13T14:50:02.447 INFO:tasks.workunit.client.1.smithi196.stdout:2/871: creat d0/d7/d9/d61/f118 x:0 0 0 2024-03-13T14:50:02.447 INFO:tasks.workunit.client.0.smithi176.stdout:9/937: chown d0/dd8 10308303 1 2024-03-13T14:50:02.448 INFO:tasks.workunit.client.0.smithi176.stdout:0/740: dwrite d2/d5/d1e/f92 [4194304,4194304] 0 2024-03-13T14:50:02.454 INFO:tasks.workunit.client.1.smithi196.stdout:0/953: write d8/da/d25/d26/d35/d3b/d59/d6a/fdd [666357,7398] 0 2024-03-13T14:50:02.454 INFO:tasks.workunit.client.0.smithi176.stdout:2/856: chown d1/dc/d2d/d59/d80/d5c/d78/fed 3694187 1 2024-03-13T14:50:02.456 INFO:tasks.workunit.client.1.smithi196.stdout:2/872: truncate d0/f77 2677849 0 2024-03-13T14:50:02.458 INFO:tasks.workunit.client.0.smithi176.stdout:3/873: mkdir d4/d28/d31/d52/de9/d131 0 2024-03-13T14:50:02.459 INFO:tasks.workunit.client.1.smithi196.stdout:0/954: readlink d8/da/d25/d26/d48/d82/l11e 0 2024-03-13T14:50:02.460 INFO:tasks.workunit.client.0.smithi176.stdout:3/874: stat d4/d28/d34/da4 0 2024-03-13T14:50:02.463 INFO:tasks.workunit.client.0.smithi176.stdout:3/875: read d4/d28/d31/f103 [166294,73540] 0 2024-03-13T14:50:02.465 INFO:tasks.workunit.client.1.smithi196.stdout:5/939: dwrite d4/d1b/dc8/d14/d19/d1c/d25/fba [0,4194304] 0 2024-03-13T14:50:02.466 INFO:tasks.workunit.client.0.smithi176.stdout:5/806: mknod d5/d2c/d40/dc7/d53/de9/c113 0 2024-03-13T14:50:02.469 INFO:tasks.workunit.client.0.smithi176.stdout:9/938: dwrite d0/db/d89/d32/d58/d5f/d84/d9a/f9b [4194304,4194304] 0 2024-03-13T14:50:02.469 INFO:tasks.workunit.client.1.smithi196.stdout:3/898: creat db/d37/f119 x:0 0 0 2024-03-13T14:50:02.471 INFO:tasks.workunit.client.1.smithi196.stdout:2/873: rmdir d0/d7/d40 39 2024-03-13T14:50:02.473 INFO:tasks.workunit.client.1.smithi196.stdout:3/899: dread - db/d19/d22/d3a/d68/d8e/f116 zero size 2024-03-13T14:50:02.474 INFO:tasks.workunit.client.1.smithi196.stdout:9/915: dwrite d7/de/d4b/d1d/f54 [0,4194304] 0 2024-03-13T14:50:02.481 INFO:tasks.workunit.client.0.smithi176.stdout:8/765: dread d0/d5/f4d [0,4194304] 0 2024-03-13T14:50:02.482 INFO:tasks.workunit.client.1.smithi196.stdout:4/914: rename d1/d3/d4c/l75 to d1/d3/d7/d35/d76/d58/dc9/dd7/l145 0 2024-03-13T14:50:02.491 INFO:tasks.workunit.client.1.smithi196.stdout:8/876: dread d0/d16/f86 [0,4194304] 0 2024-03-13T14:50:02.491 INFO:tasks.workunit.client.0.smithi176.stdout:1/819: write d2/f21 [846101,92993] 0 2024-03-13T14:50:02.493 INFO:tasks.workunit.client.1.smithi196.stdout:9/916: creat d7/d9/f136 x:0 0 0 2024-03-13T14:50:02.495 INFO:tasks.workunit.client.1.smithi196.stdout:0/955: dwrite d8/f1e [0,4194304] 0 2024-03-13T14:50:02.496 INFO:tasks.workunit.client.1.smithi196.stdout:9/917: chown d7/d71 11 1 2024-03-13T14:50:02.499 INFO:tasks.workunit.client.1.smithi196.stdout:9/918: fsync d7/de/d16/fe1 0 2024-03-13T14:50:02.502 INFO:tasks.workunit.client.1.smithi196.stdout:3/900: creat db/d19/d50/db7/da9/f11a x:0 0 0 2024-03-13T14:50:02.508 INFO:tasks.workunit.client.1.smithi196.stdout:4/915: unlink d1/d3/f3a 0 2024-03-13T14:50:02.521 INFO:tasks.workunit.client.0.smithi176.stdout:2/857: write d1/dc/d2d/d38/fe8 [820569,127229] 0 2024-03-13T14:50:02.522 INFO:tasks.workunit.client.1.smithi196.stdout:2/874: truncate d0/d7/d9/fc9 985547 0 2024-03-13T14:50:02.524 INFO:tasks.workunit.client.1.smithi196.stdout:8/877: fdatasync d0/d10/d12/df5/d47/f98 0 2024-03-13T14:50:02.527 INFO:tasks.workunit.client.0.smithi176.stdout:6/855: link d0/de/f49 d0/de/d54/f112 0 2024-03-13T14:50:02.529 INFO:tasks.workunit.client.0.smithi176.stdout:7/778: sync 2024-03-13T14:50:02.531 INFO:tasks.workunit.client.0.smithi176.stdout:5/807: dwrite d5/d31/d47/d80/fd6 [0,4194304] 0 2024-03-13T14:50:02.532 INFO:tasks.workunit.client.0.smithi176.stdout:3/876: mknod d4/d13/d16/db6/de8/d126/c132 0 2024-03-13T14:50:02.541 INFO:tasks.workunit.client.0.smithi176.stdout:2/858: truncate d1/dc/d2d/d90/fc9 2881668 0 2024-03-13T14:50:02.541 INFO:tasks.workunit.client.0.smithi176.stdout:6/856: fdatasync d0/de/f12 0 2024-03-13T14:50:02.543 INFO:tasks.workunit.client.1.smithi196.stdout:5/940: dwrite d4/d1b/dc8/d14/d19/f58 [0,4194304] 0 2024-03-13T14:50:02.544 INFO:tasks.workunit.client.0.smithi176.stdout:6/857: chown d0/de/d38/d70 383 1 2024-03-13T14:50:02.547 INFO:tasks.workunit.client.1.smithi196.stdout:5/941: chown d4/d1b/dc8/d14/f5e 7182 1 2024-03-13T14:50:02.548 INFO:tasks.workunit.client.0.smithi176.stdout:0/741: dwrite d2/d5/f87 [0,4194304] 0 2024-03-13T14:50:02.549 INFO:tasks.workunit.client.0.smithi176.stdout:9/939: write d0/d1e/d6f/f7b [91454,88148] 0 2024-03-13T14:50:02.554 INFO:tasks.workunit.client.0.smithi176.stdout:8/766: dwrite d0/f52 [0,4194304] 0 2024-03-13T14:50:02.554 INFO:tasks.workunit.client.0.smithi176.stdout:1/820: dwrite d2/d30/f4c [4194304,4194304] 0 2024-03-13T14:50:02.554 INFO:tasks.workunit.client.1.smithi196.stdout:3/901: link db/f2c db/d19/d4a/d9b/f11b 0 2024-03-13T14:50:02.566 INFO:tasks.workunit.client.0.smithi176.stdout:6/858: dread d0/de/d54/f112 [0,4194304] 0 2024-03-13T14:50:02.567 INFO:tasks.workunit.client.1.smithi196.stdout:2/875: dread d0/d7/d9/d5a/f97 [0,4194304] 0 2024-03-13T14:50:02.567 INFO:tasks.workunit.client.1.smithi196.stdout:5/942: dread - d4/d1b/dc8/d14/d19/d26/d33/dde/ff8 zero size 2024-03-13T14:50:02.567 INFO:tasks.workunit.client.1.smithi196.stdout:3/902: mknod db/d19/d50/db7/c11c 0 2024-03-13T14:50:02.569 INFO:tasks.workunit.client.1.smithi196.stdout:9/919: creat d7/de/d4b/d1d/d35/dcc/f137 x:0 0 0 2024-03-13T14:50:02.571 INFO:tasks.workunit.client.1.smithi196.stdout:0/956: getdents d8/da/d25/d26/d35/d3b/d50/d5d/d7e/de8 0 2024-03-13T14:50:02.576 INFO:tasks.workunit.client.1.smithi196.stdout:3/903: creat db/d19/d4a/d9b/f11d x:0 0 0 2024-03-13T14:50:02.586 INFO:tasks.workunit.client.1.smithi196.stdout:2/876: creat d0/d7/d9/d42/d98/de4/d104/f119 x:0 0 0 2024-03-13T14:50:02.589 INFO:tasks.workunit.client.1.smithi196.stdout:5/943: rename d4/d1b/dc8/d3c/d6a/d91 to d4/d1b/dc8/d14/d37/d12a 0 2024-03-13T14:50:02.599 INFO:tasks.workunit.client.1.smithi196.stdout:0/957: mknod d8/da/d25/d26/d48/d82/db7/c140 0 2024-03-13T14:50:02.607 INFO:tasks.workunit.client.1.smithi196.stdout:8/878: dwrite d0/d63/dcb/ff6 [0,4194304] 0 2024-03-13T14:50:02.607 INFO:tasks.workunit.client.1.smithi196.stdout:2/877: mkdir d0/d7/d9/d42/d98/d11a 0 2024-03-13T14:50:02.610 INFO:tasks.workunit.client.1.smithi196.stdout:3/904: dwrite db/d19/d22/f3c [0,4194304] 0 2024-03-13T14:50:02.614 INFO:tasks.workunit.client.1.smithi196.stdout:5/944: mkdir d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/d12b 0 2024-03-13T14:50:02.618 INFO:tasks.workunit.client.1.smithi196.stdout:9/920: dread d7/de/d74/d94/dfb/f38 [0,4194304] 0 2024-03-13T14:50:02.619 INFO:tasks.workunit.client.1.smithi196.stdout:4/916: dread d1/d3/d7/d35/d76/d58/dc9/dd7/f140 [0,4194304] 0 2024-03-13T14:50:02.622 INFO:tasks.workunit.client.1.smithi196.stdout:0/958: unlink d8/da/d25/d26/d35/d3b/d6c/fc6 0 2024-03-13T14:50:02.625 INFO:tasks.workunit.client.1.smithi196.stdout:0/959: chown d8/da/d25/d26/d35/d3b/d5f/d33/d44 1321544585 1 2024-03-13T14:50:02.629 INFO:tasks.workunit.client.1.smithi196.stdout:2/878: creat d0/d7/d9/d5a/dbc/deb/f11b x:0 0 0 2024-03-13T14:50:02.632 INFO:tasks.workunit.client.1.smithi196.stdout:2/879: readlink d0/d6/d4b/d5f/lf6 0 2024-03-13T14:50:02.635 INFO:tasks.workunit.client.0.smithi176.stdout:4/854: sync 2024-03-13T14:50:02.640 INFO:tasks.workunit.client.1.smithi196.stdout:4/917: dread d1/d3/d7/d35/f52 [0,4194304] 0 2024-03-13T14:50:02.642 INFO:tasks.workunit.client.1.smithi196.stdout:5/945: dwrite d4/d1b/dc8/d14/d19/d26/d33/f125 [0,4194304] 0 2024-03-13T14:50:02.643 INFO:tasks.workunit.client.1.smithi196.stdout:5/946: chown d4/d60/c74 1022581091 1 2024-03-13T14:50:02.666 INFO:tasks.workunit.client.1.smithi196.stdout:3/905: write db/d19/d50/db7/f9d [707008,16325] 0 2024-03-13T14:50:02.670 INFO:tasks.workunit.client.1.smithi196.stdout:3/906: write db/d15/d44/d79/fe6 [16382,16708] 0 2024-03-13T14:50:02.671 INFO:tasks.workunit.client.1.smithi196.stdout:0/960: dread d8/da/d25/d26/d35/d3b/d5f/d33/f34 [0,4194304] 0 2024-03-13T14:50:02.681 INFO:tasks.workunit.client.1.smithi196.stdout:9/921: truncate d7/de/f87 4389662 0 2024-03-13T14:50:02.683 INFO:tasks.workunit.client.1.smithi196.stdout:4/918: truncate d1/d3/d7/d35/d76/fb9 217963 0 2024-03-13T14:50:02.684 INFO:tasks.workunit.client.0.smithi176.stdout:5/808: mknod d5/d31/def/c114 0 2024-03-13T14:50:02.684 INFO:tasks.workunit.client.0.smithi176.stdout:4/855: dread d7/dab/fc1 [0,4194304] 0 2024-03-13T14:50:02.684 INFO:tasks.workunit.client.1.smithi196.stdout:9/922: read d7/d71/f9e [513295,4569] 0 2024-03-13T14:50:02.685 INFO:tasks.workunit.client.0.smithi176.stdout:5/809: write d5/d31/d47/f85 [274078,115299] 0 2024-03-13T14:50:02.686 INFO:tasks.workunit.client.0.smithi176.stdout:4/856: write d7/d10/d46/d4a/dcc/dff/f107 [211203,28043] 0 2024-03-13T14:50:02.687 INFO:tasks.workunit.client.1.smithi196.stdout:3/907: mknod db/d19/d4a/d9b/c11e 0 2024-03-13T14:50:02.688 INFO:tasks.workunit.client.1.smithi196.stdout:5/947: mkdir d4/d1b/dc8/d14/d19/d26/d33/db3/d12c 0 2024-03-13T14:50:02.690 INFO:tasks.workunit.client.0.smithi176.stdout:2/859: creat d1/dc/d2d/d59/d80/d5c/f121 x:0 0 0 2024-03-13T14:50:02.692 INFO:tasks.workunit.client.1.smithi196.stdout:8/879: getdents d0/d16/d3d/d9c/d87/d8c/da9/db8 0 2024-03-13T14:50:02.696 INFO:tasks.workunit.client.0.smithi176.stdout:1/821: truncate d2/de/f53 1194743 0 2024-03-13T14:50:02.698 INFO:tasks.workunit.client.1.smithi196.stdout:9/923: mkdir d7/de/de5/d138 0 2024-03-13T14:50:02.700 INFO:tasks.workunit.client.0.smithi176.stdout:8/767: rename d0/d5/d11/lc9 to d0/d5/d3b/lf6 0 2024-03-13T14:50:02.703 INFO:tasks.workunit.client.0.smithi176.stdout:6/859: creat d0/d10/d19/d27/d2b/de0/d76/d7c/dfd/f113 x:0 0 0 2024-03-13T14:50:02.706 INFO:tasks.workunit.client.1.smithi196.stdout:2/880: getdents d0/d7/d56 0 2024-03-13T14:50:02.706 INFO:tasks.workunit.client.0.smithi176.stdout:7/779: symlink d1/d28/d3a/d1b/d8f/dd1/df4/lf5 0 2024-03-13T14:50:02.709 INFO:tasks.workunit.client.1.smithi196.stdout:2/881: write d0/d6/d4b/d5f/dce/ff9 [548523,66351] 0 2024-03-13T14:50:02.709 INFO:tasks.workunit.client.1.smithi196.stdout:8/880: mknod d0/d16/d71/d68/c125 0 2024-03-13T14:50:02.713 INFO:tasks.workunit.client.0.smithi176.stdout:4/857: creat d7/d10/d46/d4a/dcc/dff/f11c x:0 0 0 2024-03-13T14:50:02.715 INFO:tasks.workunit.client.0.smithi176.stdout:8/768: dwrite d0/f42 [0,4194304] 0 2024-03-13T14:50:02.715 INFO:tasks.workunit.client.1.smithi196.stdout:9/924: mknod d7/de/d74/d94/dfb/dbf/c139 0 2024-03-13T14:50:02.716 INFO:tasks.workunit.client.0.smithi176.stdout:8/769: chown d0/ccf 2847 1 2024-03-13T14:50:02.718 INFO:tasks.workunit.client.0.smithi176.stdout:2/860: creat d1/d2a/f122 x:0 0 0 2024-03-13T14:50:02.721 INFO:tasks.workunit.client.1.smithi196.stdout:3/908: dread db/d19/f62 [0,4194304] 0 2024-03-13T14:50:02.722 INFO:tasks.workunit.client.0.smithi176.stdout:0/742: mknod d2/cef 0 2024-03-13T14:50:02.723 INFO:tasks.workunit.client.0.smithi176.stdout:0/743: fdatasync d2/d5/dc/fca 0 2024-03-13T14:50:02.724 INFO:tasks.workunit.client.0.smithi176.stdout:1/822: symlink d2/de/d10/d16/d4a/d5f/da4/lf8 0 2024-03-13T14:50:02.726 INFO:tasks.workunit.client.0.smithi176.stdout:5/810: write d5/d2c/fe7 [522499,128252] 0 2024-03-13T14:50:02.726 INFO:tasks.workunit.client.0.smithi176.stdout:0/744: write d2/db/d31/fd7 [770576,126167] 0 2024-03-13T14:50:02.728 INFO:tasks.workunit.client.0.smithi176.stdout:0/745: fdatasync d2/d74/fdd 0 2024-03-13T14:50:02.728 INFO:tasks.workunit.client.1.smithi196.stdout:2/882: fdatasync d0/d7/d40/f46 0 2024-03-13T14:50:02.733 INFO:tasks.workunit.client.1.smithi196.stdout:4/919: link d1/d3/d7/d35/d76/f38 d1/d3/d7/d35/d76/d58/d69/de3/df4/f146 0 2024-03-13T14:50:02.734 INFO:tasks.workunit.client.1.smithi196.stdout:0/961: getdents d8/da/d25/d26/d35/d3b/d59/d6a 0 2024-03-13T14:50:02.736 INFO:tasks.workunit.client.0.smithi176.stdout:2/861: unlink d1/d2a/c9c 0 2024-03-13T14:50:02.736 INFO:tasks.workunit.client.1.smithi196.stdout:0/962: dread - d8/da/d25/d26/d35/d3b/d59/f9b zero size 2024-03-13T14:50:02.736 INFO:tasks.workunit.client.1.smithi196.stdout:3/909: symlink db/d19/d22/d3a/d68/d8e/l11f 0 2024-03-13T14:50:02.737 INFO:tasks.workunit.client.1.smithi196.stdout:5/948: link d4/d1b/dc8/d4a/l85 d4/d1b/dc8/d14/d19/d26/d33/d61/l12d 0 2024-03-13T14:50:02.738 INFO:tasks.workunit.client.0.smithi176.stdout:6/860: mkdir d0/de/d54/d114 0 2024-03-13T14:50:02.740 INFO:tasks.workunit.client.1.smithi196.stdout:0/963: readlink d8/da/d25/d26/d48/d82/db3/d12c/l13e 0 2024-03-13T14:50:02.741 INFO:tasks.workunit.client.1.smithi196.stdout:5/949: write d4/d1b/dc8/d14/d19/d26/dab/fb9 [106175,122537] 0 2024-03-13T14:50:02.743 INFO:tasks.workunit.client.0.smithi176.stdout:5/811: unlink d5/d2c/d40/c97 0 2024-03-13T14:50:02.746 INFO:tasks.workunit.client.1.smithi196.stdout:2/883: dread - d0/d7/d9/d42/d98/de4/d67/fcc zero size 2024-03-13T14:50:02.748 INFO:tasks.workunit.client.1.smithi196.stdout:9/925: chown d7/d9/d17/fff 30303761 1 2024-03-13T14:50:02.749 INFO:tasks.workunit.client.1.smithi196.stdout:4/920: mkdir d1/d3/d4c/d72/d7d/d80/d147 0 2024-03-13T14:50:02.751 INFO:tasks.workunit.client.0.smithi176.stdout:2/862: mknod d1/dc/d2d/d59/d8b/daf/c123 0 2024-03-13T14:50:02.752 INFO:tasks.workunit.client.1.smithi196.stdout:3/910: creat db/d19/d22/d3a/d68/d8e/f120 x:0 0 0 2024-03-13T14:50:02.753 INFO:tasks.workunit.client.1.smithi196.stdout:8/881: write d0/d10/d12/d17/f57 [4221191,15446] 0 2024-03-13T14:50:02.753 INFO:tasks.workunit.client.1.smithi196.stdout:4/921: dread - d1/d3/d7/d35/d76/d58/dc9/dd7/deb/f13c zero size 2024-03-13T14:50:02.757 INFO:tasks.workunit.client.1.smithi196.stdout:0/964: mkdir d8/da/d25/d26/d35/d3b/d5f/d33/d44/d141 0 2024-03-13T14:50:02.757 INFO:tasks.workunit.client.0.smithi176.stdout:2/863: write d1/dc/d2d/d4b/d69/fb4 [1588755,8779] 0 2024-03-13T14:50:02.762 INFO:tasks.workunit.client.0.smithi176.stdout:1/823: write d2/de/d10/d16/d47/db4/fc1 [176951,53657] 0 2024-03-13T14:50:02.763 INFO:tasks.workunit.client.1.smithi196.stdout:3/911: creat db/d19/dc1/f121 x:0 0 0 2024-03-13T14:50:02.763 INFO:tasks.workunit.client.0.smithi176.stdout:8/770: dread d0/d24/d98/dc6/d40/f48 [0,4194304] 0 2024-03-13T14:50:02.764 INFO:tasks.workunit.client.0.smithi176.stdout:7/780: dwrite d1/d3f/fab [0,4194304] 0 2024-03-13T14:50:02.766 INFO:tasks.workunit.client.0.smithi176.stdout:2/864: write d1/d2a/d33/ff9 [2298191,74222] 0 2024-03-13T14:50:02.766 INFO:tasks.workunit.client.0.smithi176.stdout:4/858: dwrite d7/d8/d45/f5c [0,4194304] 0 2024-03-13T14:50:02.768 INFO:tasks.workunit.client.1.smithi196.stdout:4/922: mknod d1/d1e/d5c/c148 0 2024-03-13T14:50:02.769 INFO:tasks.workunit.client.0.smithi176.stdout:6/861: write d0/d10/d19/fd5 [64181,83583] 0 2024-03-13T14:50:02.773 INFO:tasks.workunit.client.0.smithi176.stdout:6/862: chown d0/d10/d19/d27/d2b/d2e/db7/de4 55 1 2024-03-13T14:50:02.774 INFO:tasks.workunit.client.0.smithi176.stdout:0/746: write d2/db/d35/f3f [300958,77339] 0 2024-03-13T14:50:02.774 INFO:tasks.workunit.client.0.smithi176.stdout:1/824: fdatasync d2/de/d3f/f52 0 2024-03-13T14:50:02.775 INFO:tasks.workunit.client.0.smithi176.stdout:6/863: fdatasync d0/d10/dca/dd2/fe3 0 2024-03-13T14:50:02.775 INFO:tasks.workunit.client.1.smithi196.stdout:0/965: mkdir d8/da/d25/d26/d35/d3b/d6c/d142 0 2024-03-13T14:50:02.777 INFO:tasks.workunit.client.0.smithi176.stdout:8/771: read d0/d24/d98/dc6/f13 [3168054,103333] 0 2024-03-13T14:50:02.780 INFO:tasks.workunit.client.0.smithi176.stdout:1/825: readlink d2/de/d10/d16/d4a/d5f/da4/lf8 0 2024-03-13T14:50:02.782 INFO:tasks.workunit.client.0.smithi176.stdout:4/859: rename d7/d8/d15/dd6 to d7/d10/d46/d4a/dcc/dd0/d11d 0 2024-03-13T14:50:02.783 INFO:tasks.workunit.client.1.smithi196.stdout:5/950: truncate d4/d1b/dc8/d14/d19/f58 493887 0 2024-03-13T14:50:02.783 INFO:tasks.workunit.client.0.smithi176.stdout:2/865: write d1/dc/d2d/d59/d8b/f109 [4896548,39253] 0 2024-03-13T14:50:02.784 INFO:tasks.workunit.client.0.smithi176.stdout:7/781: dwrite d1/d3f/d76/fb9 [8388608,4194304] 0 2024-03-13T14:50:02.787 INFO:tasks.workunit.client.1.smithi196.stdout:9/926: creat d7/de/de5/d138/f13a x:0 0 0 2024-03-13T14:50:02.787 INFO:tasks.workunit.client.0.smithi176.stdout:2/866: chown d1/dc/d2d/d90/dcf/l92 13 1 2024-03-13T14:50:02.789 INFO:tasks.workunit.client.1.smithi196.stdout:5/951: stat d4/d1b/dc8/d14/d19/d1c/d2d/d82 0 2024-03-13T14:50:02.793 INFO:tasks.workunit.client.0.smithi176.stdout:8/772: chown d0/d24/d29/d70 203867 1 2024-03-13T14:50:02.794 INFO:tasks.workunit.client.0.smithi176.stdout:0/747: rmdir d2/d5/d28 39 2024-03-13T14:50:02.796 INFO:tasks.workunit.client.1.smithi196.stdout:2/884: creat d0/d7/d9/d61/dd3/d101/f11c x:0 0 0 2024-03-13T14:50:02.797 INFO:tasks.workunit.client.0.smithi176.stdout:4/860: rmdir d7/d10/d46/d4a/d5d/da1 39 2024-03-13T14:50:02.797 INFO:tasks.workunit.client.1.smithi196.stdout:3/912: mkdir db/d122 0 2024-03-13T14:50:02.802 INFO:tasks.workunit.client.0.smithi176.stdout:7/782: mkdir d1/d7/df6 0 2024-03-13T14:50:02.802 INFO:tasks.workunit.client.1.smithi196.stdout:0/966: dwrite d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f119 [0,4194304] 0 2024-03-13T14:50:02.811 INFO:tasks.workunit.client.1.smithi196.stdout:4/923: write d1/d3/d7/f49 [3630272,92072] 0 2024-03-13T14:50:02.814 INFO:tasks.workunit.client.1.smithi196.stdout:5/952: mknod d4/d1b/dc8/d14/d19/d1c/d25/c12e 0 2024-03-13T14:50:02.817 INFO:tasks.workunit.client.0.smithi176.stdout:9/940: sync 2024-03-13T14:50:02.817 INFO:tasks.workunit.client.0.smithi176.stdout:3/877: sync 2024-03-13T14:50:02.821 INFO:tasks.workunit.client.0.smithi176.stdout:9/941: readlink d0/db/d89/d32/d58/d5f/d84/d9a/l10c 0 2024-03-13T14:50:02.822 INFO:tasks.workunit.client.0.smithi176.stdout:6/864: dwrite d0/d10/d19/d27/fac [0,4194304] 0 2024-03-13T14:50:02.826 INFO:tasks.workunit.client.1.smithi196.stdout:8/882: dwrite d0/d16/d3d/d7f/f119 [0,4194304] 0 2024-03-13T14:50:02.826 INFO:tasks.workunit.client.0.smithi176.stdout:7/783: fdatasync d1/d3f/d4b/fc2 0 2024-03-13T14:50:02.828 INFO:tasks.workunit.client.0.smithi176.stdout:8/773: mkdir d0/db/d53/df7 0 2024-03-13T14:50:02.832 INFO:tasks.workunit.client.1.smithi196.stdout:0/967: mknod d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe/c143 0 2024-03-13T14:50:02.836 INFO:tasks.workunit.client.0.smithi176.stdout:0/748: chown d2/d5/dc/l67 246 1 2024-03-13T14:50:02.840 INFO:tasks.workunit.client.0.smithi176.stdout:7/784: dwrite d1/d3f/d65/fe4 [0,4194304] 0 2024-03-13T14:50:02.842 INFO:tasks.workunit.client.1.smithi196.stdout:9/927: dread d7/de/d16/d70/f7a [0,4194304] 0 2024-03-13T14:50:02.842 INFO:tasks.workunit.client.1.smithi196.stdout:0/968: dread d8/da/d25/d26/d48/dba/ff6 [0,4194304] 0 2024-03-13T14:50:02.842 INFO:tasks.workunit.client.1.smithi196.stdout:3/913: dwrite db/f29 [0,4194304] 0 2024-03-13T14:50:02.843 INFO:tasks.workunit.client.0.smithi176.stdout:7/785: readlink d1/d28/d3a/d1b/d8f/dd1/df4/lf5 0 2024-03-13T14:50:02.844 INFO:tasks.workunit.client.0.smithi176.stdout:9/942: creat d0/d40/d53/d6d/d97/f13c x:0 0 0 2024-03-13T14:50:02.849 INFO:tasks.workunit.client.1.smithi196.stdout:8/883: symlink d0/d10/l126 0 2024-03-13T14:50:02.849 INFO:tasks.workunit.client.0.smithi176.stdout:2/867: link d1/dc/d2d/d4b/cda d1/dc/d61/d9e/c124 0 2024-03-13T14:50:02.850 INFO:tasks.workunit.client.0.smithi176.stdout:8/774: mkdir d0/d5/d11/d47/db7/de0/df0/df8 0 2024-03-13T14:50:02.859 INFO:tasks.workunit.client.1.smithi196.stdout:4/924: dwrite d1/d1e/d5c/f5f [0,4194304] 0 2024-03-13T14:50:02.861 INFO:tasks.workunit.client.0.smithi176.stdout:3/878: creat d4/d28/d31/d45/d82/d87/d102/d12c/f133 x:0 0 0 2024-03-13T14:50:02.873 INFO:tasks.workunit.client.0.smithi176.stdout:7/786: creat d1/d3f/d65/ff7 x:0 0 0 2024-03-13T14:50:02.876 INFO:tasks.workunit.client.0.smithi176.stdout:6/865: getdents d0/d10/d19/d106 0 2024-03-13T14:50:02.877 INFO:tasks.workunit.client.1.smithi196.stdout:2/885: dread d0/d7/d9/d61/fb6 [0,4194304] 0 2024-03-13T14:50:02.878 INFO:tasks.workunit.client.0.smithi176.stdout:6/866: chown d0/d10/d19/d27 0 1 2024-03-13T14:50:02.883 INFO:tasks.workunit.client.0.smithi176.stdout:9/943: creat d0/db/d89/d32/d58/f13d x:0 0 0 2024-03-13T14:50:02.886 INFO:tasks.workunit.client.1.smithi196.stdout:0/969: link d8/d37/l91 d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe/l144 0 2024-03-13T14:50:02.891 INFO:tasks.workunit.client.0.smithi176.stdout:3/879: rename d4/d28/d34/c35 to d4/d13/d16/ddd/c134 0 2024-03-13T14:50:02.891 INFO:tasks.workunit.client.1.smithi196.stdout:0/970: fsync d8/da/d25/d26/d54/d88/f45 0 2024-03-13T14:50:02.895 INFO:tasks.workunit.client.0.smithi176.stdout:0/749: write d2/d5/dc/d20/f48 [1516241,14612] 0 2024-03-13T14:50:02.897 INFO:tasks.workunit.client.0.smithi176.stdout:2/868: write d1/dc/d61/feb [669225,57875] 0 2024-03-13T14:50:02.897 INFO:tasks.workunit.client.0.smithi176.stdout:7/787: truncate d1/d3f/d81/fca 538791 0 2024-03-13T14:50:02.899 INFO:tasks.workunit.client.0.smithi176.stdout:6/867: fdatasync d0/de/d54/f75 0 2024-03-13T14:50:02.901 INFO:tasks.workunit.client.0.smithi176.stdout:9/944: mknod d0/db/d89/d32/d58/d5f/d84/c13e 0 2024-03-13T14:50:02.902 INFO:tasks.workunit.client.0.smithi176.stdout:8/775: mknod d0/db/d53/df7/cf9 0 2024-03-13T14:50:02.905 INFO:tasks.workunit.client.1.smithi196.stdout:0/971: chown d8/da/d25/d26/d35/d3b/d50/d55/f64 14743 1 2024-03-13T14:50:02.905 INFO:tasks.workunit.client.1.smithi196.stdout:3/914: dwrite db/f13 [0,4194304] 0 2024-03-13T14:50:02.909 INFO:tasks.workunit.client.1.smithi196.stdout:9/928: dwrite d7/d9/f77 [0,4194304] 0 2024-03-13T14:50:02.913 INFO:tasks.workunit.client.0.smithi176.stdout:7/788: creat d1/d7/ff8 x:0 0 0 2024-03-13T14:50:02.914 INFO:tasks.workunit.client.0.smithi176.stdout:7/789: stat d1/d1c/d26/l6f 0 2024-03-13T14:50:02.916 INFO:tasks.workunit.client.0.smithi176.stdout:5/812: sync 2024-03-13T14:50:02.921 INFO:tasks.workunit.client.0.smithi176.stdout:2/869: rename d1/dc/d61/d9e/le2 to d1/dc/d2d/d4b/ddb/l125 0 2024-03-13T14:50:02.925 INFO:tasks.workunit.client.1.smithi196.stdout:8/884: dread d0/d16/d58/fd0 [4194304,4194304] 0 2024-03-13T14:50:02.928 INFO:tasks.workunit.client.0.smithi176.stdout:3/880: creat d4/d13/d81/d89/d5b/f135 x:0 0 0 2024-03-13T14:50:02.930 INFO:tasks.workunit.client.0.smithi176.stdout:5/813: creat d5/d31/d47/f115 x:0 0 0 2024-03-13T14:50:02.933 INFO:tasks.workunit.client.0.smithi176.stdout:7/790: mkdir d1/d28/d3a/d1b/d8f/dc1/db7/df9 0 2024-03-13T14:50:02.933 INFO:tasks.workunit.client.0.smithi176.stdout:6/868: rename d0/de/c28 to d0/dc1/c115 0 2024-03-13T14:50:02.936 INFO:tasks.workunit.client.0.smithi176.stdout:0/750: dwrite d2/d5/d1e/f92 [0,4194304] 0 2024-03-13T14:50:02.936 INFO:tasks.workunit.client.0.smithi176.stdout:2/870: unlink d1/dc/d61/d9e/l10b 0 2024-03-13T14:50:02.942 INFO:tasks.workunit.client.0.smithi176.stdout:3/881: symlink d4/d28/d31/d52/l136 0 2024-03-13T14:50:02.950 INFO:tasks.workunit.client.0.smithi176.stdout:8/776: creat d0/db/d39/d56/ffa x:0 0 0 2024-03-13T14:50:02.954 INFO:tasks.workunit.client.0.smithi176.stdout:5/814: dread d5/d31/d47/f59 [0,4194304] 0 2024-03-13T14:50:02.954 INFO:tasks.workunit.client.0.smithi176.stdout:6/869: mkdir d0/de/d38/d88/d116 0 2024-03-13T14:50:02.954 INFO:tasks.workunit.client.1.smithi196.stdout:9/929: dread d7/de/d74/d94/fb8 [4194304,4194304] 0 2024-03-13T14:50:02.957 INFO:tasks.workunit.client.0.smithi176.stdout:0/751: truncate d2/f50 863401 0 2024-03-13T14:50:02.957 INFO:tasks.workunit.client.0.smithi176.stdout:9/945: write d0/db/d89/d4d/d5d/f7d [5202262,126412] 0 2024-03-13T14:50:02.957 INFO:tasks.workunit.client.0.smithi176.stdout:5/815: dread - d5/d2c/d40/d76/ff4 zero size 2024-03-13T14:50:02.961 INFO:tasks.workunit.client.0.smithi176.stdout:3/882: truncate d4/d13/d81/d89/f2b 1238989 0 2024-03-13T14:50:02.964 INFO:tasks.workunit.client.0.smithi176.stdout:9/946: stat d0/db/d89/d32/d58/d5f/d84/d9a/lbf 0 2024-03-13T14:50:02.965 INFO:tasks.workunit.client.1.smithi196.stdout:3/915: dread db/d19/f2b [0,4194304] 0 2024-03-13T14:50:02.967 INFO:tasks.workunit.client.1.smithi196.stdout:3/916: fdatasync db/d19/d4a/d9b/f11d 0 2024-03-13T14:50:02.970 INFO:tasks.workunit.client.1.smithi196.stdout:3/917: chown db/d15/l2a 1 1 2024-03-13T14:50:02.971 INFO:tasks.workunit.client.1.smithi196.stdout:3/918: write db/d15/d44/d79/ff6 [341579,48069] 0 2024-03-13T14:50:02.972 INFO:tasks.workunit.client.0.smithi176.stdout:5/816: readlink d5/d2c/d40/dc7/d89/le0 0 2024-03-13T14:50:02.972 INFO:tasks.workunit.client.0.smithi176.stdout:5/817: dread - d5/d2c/ff8 zero size 2024-03-13T14:50:02.974 INFO:tasks.workunit.client.0.smithi176.stdout:5/818: fdatasync d5/d2e/d43/fea 0 2024-03-13T14:50:02.988 INFO:tasks.workunit.client.0.smithi176.stdout:9/947: creat d0/db/d89/d32/d122/d56/f13f x:0 0 0 2024-03-13T14:50:02.991 INFO:tasks.workunit.client.0.smithi176.stdout:7/791: dwrite d1/d3f/d4b/f6b [0,4194304] 0 2024-03-13T14:50:02.994 INFO:tasks.workunit.client.0.smithi176.stdout:8/777: write d0/f30 [92134,75607] 0 2024-03-13T14:50:02.996 INFO:tasks.workunit.client.1.smithi196.stdout:9/930: dread d7/d9/d17/dbc/fa9 [0,4194304] 0 2024-03-13T14:50:03.001 INFO:tasks.workunit.client.1.smithi196.stdout:4/925: link d1/d3/d7/d35/d76/d58/d69/de3/df4/l115 d1/d1e/d43/l149 0 2024-03-13T14:50:03.002 INFO:tasks.workunit.client.0.smithi176.stdout:9/948: mknod d0/d1e/d6f/c140 0 2024-03-13T14:50:03.003 INFO:tasks.workunit.client.0.smithi176.stdout:7/792: readlink d1/d1c/d4a/l6c 0 2024-03-13T14:50:03.004 INFO:tasks.workunit.client.0.smithi176.stdout:9/949: dread - d0/db/d89/d32/d122/d56/f13f zero size 2024-03-13T14:50:03.006 INFO:tasks.workunit.client.0.smithi176.stdout:6/870: write d0/d10/d19/d27/d2b/d2e/fad [1432560,72587] 0 2024-03-13T14:50:03.007 INFO:tasks.workunit.client.1.smithi196.stdout:0/972: symlink d8/da/d25/dd7/l145 0 2024-03-13T14:50:03.008 INFO:tasks.workunit.client.0.smithi176.stdout:6/871: fsync d0/d10/fc0 0 2024-03-13T14:50:03.008 INFO:tasks.workunit.client.0.smithi176.stdout:5/819: link d5/d31/d47/d80/lfb d5/d2e/d7e/dad/d7b/da7/d111/d15/l116 0 2024-03-13T14:50:03.011 INFO:tasks.workunit.client.0.smithi176.stdout:7/793: mkdir d1/d3f/d65/dfa 0 2024-03-13T14:50:03.019 INFO:tasks.workunit.client.1.smithi196.stdout:3/919: unlink db/f29 0 2024-03-13T14:50:03.020 INFO:tasks.workunit.client.1.smithi196.stdout:9/931: truncate d7/de/d4b/d1d/d22/f108 630711 0 2024-03-13T14:50:03.022 INFO:tasks.workunit.client.0.smithi176.stdout:3/883: dwrite d4/d28/d34/f6e [0,4194304] 0 2024-03-13T14:50:03.025 INFO:tasks.workunit.client.1.smithi196.stdout:2/886: creat d0/d7/f11d x:0 0 0 2024-03-13T14:50:03.027 INFO:tasks.workunit.client.0.smithi176.stdout:7/794: creat d1/d1c/d26/d41/d2b/d40/d64/d94/ffb x:0 0 0 2024-03-13T14:50:03.029 INFO:tasks.workunit.client.0.smithi176.stdout:0/752: dwrite d2/db/f2c [0,4194304] 0 2024-03-13T14:50:03.029 INFO:tasks.workunit.client.0.smithi176.stdout:9/950: rename d0/db/d89/d32/d58/d5f/fc8 to d0/db/d89/f141 0 2024-03-13T14:50:03.030 INFO:tasks.workunit.client.0.smithi176.stdout:7/795: read - d1/d7/ff8 zero size 2024-03-13T14:50:03.030 INFO:tasks.workunit.client.1.smithi196.stdout:0/973: creat d8/da/dd8/f146 x:0 0 0 2024-03-13T14:50:03.031 INFO:tasks.workunit.client.0.smithi176.stdout:6/872: dwrite d0/d9/f55 [0,4194304] 0 2024-03-13T14:50:03.034 INFO:tasks.workunit.client.0.smithi176.stdout:5/820: dwrite d5/d2c/d40/dc7/d89/ffa [0,4194304] 0 2024-03-13T14:50:03.036 INFO:tasks.workunit.client.0.smithi176.stdout:7/796: chown d1/d1c/d26/d41/d2b/d40/d64/d94/c9c 21964100 1 2024-03-13T14:50:03.039 INFO:tasks.workunit.client.0.smithi176.stdout:1/826: sync 2024-03-13T14:50:03.039 INFO:tasks.workunit.client.0.smithi176.stdout:4/861: sync 2024-03-13T14:50:03.041 INFO:tasks.workunit.client.0.smithi176.stdout:1/827: chown d2/de/dd1/fdf 35926811 1 2024-03-13T14:50:03.046 INFO:tasks.workunit.client.0.smithi176.stdout:3/884: symlink d4/d13/d81/d89/d5b/dc5/df0/l137 0 2024-03-13T14:50:03.055 INFO:tasks.workunit.client.0.smithi176.stdout:8/778: dwrite d0/d24/f32 [0,4194304] 0 2024-03-13T14:50:03.055 INFO:tasks.workunit.client.1.smithi196.stdout:3/920: creat db/d19/d50/db7/f123 x:0 0 0 2024-03-13T14:50:03.057 INFO:tasks.workunit.client.0.smithi176.stdout:9/951: mknod d0/d1e/d8d/dbe/c142 0 2024-03-13T14:50:03.060 INFO:tasks.workunit.client.0.smithi176.stdout:6/873: mkdir d0/d10/dca/dd2/d117 0 2024-03-13T14:50:03.060 INFO:tasks.workunit.client.1.smithi196.stdout:8/885: dwrite d0/d10/d12/fcf [0,4194304] 0 2024-03-13T14:50:03.065 INFO:tasks.workunit.client.1.smithi196.stdout:9/932: truncate d7/de/d74/f85 2430189 0 2024-03-13T14:50:03.066 INFO:tasks.workunit.client.1.smithi196.stdout:2/887: chown d0/f77 204 1 2024-03-13T14:50:03.069 INFO:tasks.workunit.client.1.smithi196.stdout:0/974: mknod d8/da/d25/d26/d35/c147 0 2024-03-13T14:50:03.075 INFO:tasks.workunit.client.0.smithi176.stdout:7/797: rename d1/d3f to d1/d1c/d26/d41/db6/df1/dfc 0 2024-03-13T14:50:03.076 INFO:tasks.workunit.client.0.smithi176.stdout:3/885: chown d4/d13/d16/d3d/f11e 4393113 1 2024-03-13T14:50:03.077 INFO:tasks.workunit.client.1.smithi196.stdout:3/921: symlink db/d19/d50/db7/l124 0 2024-03-13T14:50:03.078 INFO:tasks.workunit.client.0.smithi176.stdout:3/886: chown d4/d13/d81/d89/cda 70993 1 2024-03-13T14:50:03.079 INFO:tasks.workunit.client.0.smithi176.stdout:2/871: sync 2024-03-13T14:50:03.080 INFO:tasks.workunit.client.0.smithi176.stdout:3/887: readlink d4/d13/d5e/ld1 0 2024-03-13T14:50:03.084 INFO:tasks.workunit.client.1.smithi196.stdout:2/888: dwrite d0/d7/d9/d5a/dbc/dfe/f100 [0,4194304] 0 2024-03-13T14:50:03.090 INFO:tasks.workunit.client.1.smithi196.stdout:9/933: readlink d7/de/d4b/d1d/d93/da7/l130 0 2024-03-13T14:50:03.091 INFO:tasks.workunit.client.1.smithi196.stdout:0/975: dwrite d8/da/d25/d26/d54/f9a [4194304,4194304] 0 2024-03-13T14:50:03.093 INFO:tasks.workunit.client.0.smithi176.stdout:5/821: symlink d5/d2c/l117 0 2024-03-13T14:50:03.095 INFO:tasks.workunit.client.0.smithi176.stdout:4/862: readlink d7/d8/df6/lb7 0 2024-03-13T14:50:03.096 INFO:tasks.workunit.client.1.smithi196.stdout:4/926: getdents d1/d3/d7/d35/d76/d58/dc6 0 2024-03-13T14:50:03.097 INFO:tasks.workunit.client.0.smithi176.stdout:6/874: truncate d0/d10/d19/d27/d2b/d2e/dc4/dc5/fde 1913079 0 2024-03-13T14:50:03.102 INFO:tasks.workunit.client.1.smithi196.stdout:9/934: unlink d7/f15 0 2024-03-13T14:50:03.102 INFO:tasks.workunit.client.0.smithi176.stdout:7/798: truncate d1/d1c/d26/d41/db6/df1/dfc/d4b/fc2 244293 0 2024-03-13T14:50:03.104 INFO:tasks.workunit.client.1.smithi196.stdout:4/927: dread - d1/d3/f102 zero size 2024-03-13T14:50:03.105 INFO:tasks.workunit.client.0.smithi176.stdout:2/872: symlink d1/dc/d2d/d59/d80/d5c/d78/l126 0 2024-03-13T14:50:03.107 INFO:tasks.workunit.client.1.smithi196.stdout:4/928: write d1/d3/d7/f135 [146587,112381] 0 2024-03-13T14:50:03.112 INFO:tasks.workunit.client.1.smithi196.stdout:8/886: link d0/d16/d71/d9a/dfc/c122 d0/d16/d3d/d9c/d87/d8c/da9/c127 0 2024-03-13T14:50:03.114 INFO:tasks.workunit.client.1.smithi196.stdout:8/887: write d0/d16/f11b [489432,41710] 0 2024-03-13T14:50:03.115 INFO:tasks.workunit.client.0.smithi176.stdout:1/828: creat d2/de/d10/d16/d4a/d5f/ff9 x:0 0 0 2024-03-13T14:50:03.117 INFO:tasks.workunit.client.0.smithi176.stdout:4/863: rename d7/d10/d2c/f32 to d7/d10/d46/d4a/dcc/dd0/f11e 0 2024-03-13T14:50:03.118 INFO:tasks.workunit.client.0.smithi176.stdout:0/753: dwrite d2/d5/d28/f54 [0,4194304] 0 2024-03-13T14:50:03.119 INFO:tasks.workunit.client.1.smithi196.stdout:2/889: write d0/d6/f84 [5493427,127974] 0 2024-03-13T14:50:03.120 INFO:tasks.workunit.client.1.smithi196.stdout:0/976: getdents d8/da/d25/d26 0 2024-03-13T14:50:03.123 INFO:tasks.workunit.client.0.smithi176.stdout:0/754: chown d2/d5/d1e/d97/cbe 279972 1 2024-03-13T14:50:03.123 INFO:tasks.workunit.client.1.smithi196.stdout:0/977: write d8/da/f32 [1360845,12200] 0 2024-03-13T14:50:03.125 INFO:tasks.workunit.client.0.smithi176.stdout:3/888: write d4/d13/d81/d89/d5b/ffb [1262757,19813] 0 2024-03-13T14:50:03.126 INFO:tasks.workunit.client.1.smithi196.stdout:2/890: unlink d0/ffd 0 2024-03-13T14:50:03.127 INFO:tasks.workunit.client.0.smithi176.stdout:3/889: readlink d4/d28/lf1 0 2024-03-13T14:50:03.127 INFO:tasks.workunit.client.0.smithi176.stdout:9/952: creat d0/f143 x:0 0 0 2024-03-13T14:50:03.130 INFO:tasks.workunit.client.0.smithi176.stdout:8/779: link d0/d24/d98/dc6/f8a d0/d5/d11/d47/db7/de0/df0/df8/ffb 0 2024-03-13T14:50:03.131 INFO:tasks.workunit.client.0.smithi176.stdout:5/822: write d5/d31/d86/f9d [637744,83796] 0 2024-03-13T14:50:03.133 INFO:tasks.workunit.client.0.smithi176.stdout:8/780: chown d0/d24/d98/dc6/f2a 27903541 1 2024-03-13T14:50:03.134 INFO:tasks.workunit.client.1.smithi196.stdout:2/891: creat d0/d7/d9/d9a/daa/f11e x:0 0 0 2024-03-13T14:50:03.134 INFO:tasks.workunit.client.0.smithi176.stdout:6/875: write d0/de/f2c [7987665,122527] 0 2024-03-13T14:50:03.136 INFO:tasks.workunit.client.0.smithi176.stdout:6/876: chown d0/d10/d19/c90 658622185 1 2024-03-13T14:50:03.138 INFO:tasks.workunit.client.0.smithi176.stdout:6/877: chown d0/d10/d19/d27/d2b/de0/d76/ccf 0 1 2024-03-13T14:50:03.139 INFO:tasks.workunit.client.1.smithi196.stdout:0/978: link d8/c13c d8/da/d25/d26/d35/d3b/d5f/d33/c148 0 2024-03-13T14:50:03.139 INFO:tasks.workunit.client.0.smithi176.stdout:1/829: chown d2/de/d10/d16/d4a/d5f/cd6 4770236 1 2024-03-13T14:50:03.141 INFO:tasks.workunit.client.0.smithi176.stdout:7/799: fdatasync d1/d1c/d4a/fa7 0 2024-03-13T14:50:03.142 INFO:tasks.workunit.client.1.smithi196.stdout:2/892: mknod d0/d7/d9/c11f 0 2024-03-13T14:50:03.143 INFO:tasks.workunit.client.0.smithi176.stdout:0/755: creat d2/d5/d28/d8c/d9d/ff0 x:0 0 0 2024-03-13T14:50:03.144 INFO:tasks.workunit.client.0.smithi176.stdout:9/953: mkdir d0/d1e/df7/d11f/d144 0 2024-03-13T14:50:03.144 INFO:tasks.workunit.client.1.smithi196.stdout:4/929: dwrite d1/d3/d4c/d72/ff3 [0,4194304] 0 2024-03-13T14:50:03.145 INFO:tasks.workunit.client.0.smithi176.stdout:3/890: rename d4/d28/d4f/cbb to d4/d28/d34/d11c/c138 0 2024-03-13T14:50:03.146 INFO:tasks.workunit.client.0.smithi176.stdout:3/891: readlink d4/l71 0 2024-03-13T14:50:03.157 INFO:tasks.workunit.client.1.smithi196.stdout:2/893: symlink d0/d7/d9/d42/d98/d11a/l120 0 2024-03-13T14:50:03.158 INFO:tasks.workunit.client.1.smithi196.stdout:0/979: write d8/da/d25/d26/d35/d3b/d59/d6a/f8d [506047,5317] 0 2024-03-13T14:50:03.160 INFO:tasks.workunit.client.1.smithi196.stdout:2/894: fsync d0/d6/d34/f41 0 2024-03-13T14:50:03.161 INFO:tasks.workunit.client.1.smithi196.stdout:4/930: symlink d1/d3/d7/ddf/l14a 0 2024-03-13T14:50:03.161 INFO:tasks.workunit.client.0.smithi176.stdout:8/781: read d0/d5/d11/f68 [2250509,30262] 0 2024-03-13T14:50:03.163 INFO:tasks.workunit.client.0.smithi176.stdout:3/892: stat d4/d13/d81/d89/d5b/dc5/fc7 0 2024-03-13T14:50:03.164 INFO:tasks.workunit.client.0.smithi176.stdout:5/823: mkdir d5/d31/def/d10e/d118 0 2024-03-13T14:50:03.170 INFO:tasks.workunit.client.0.smithi176.stdout:4/864: link d7/d8/d45/fac d7/d8/d15/d23/f11f 0 2024-03-13T14:50:03.171 INFO:tasks.workunit.client.0.smithi176.stdout:8/782: symlink d0/d5/d11/d47/lfc 0 2024-03-13T14:50:03.171 INFO:tasks.workunit.client.0.smithi176.stdout:9/954: link d0/db/d89/d7a/fff d0/db/d12c/f145 0 2024-03-13T14:50:03.172 INFO:tasks.workunit.client.0.smithi176.stdout:2/873: getdents d1/dc/d61/d104 0 2024-03-13T14:50:03.172 INFO:tasks.workunit.client.1.smithi196.stdout:0/980: dread d8/da/d25/d26/d35/d3b/d5f/d33/f34 [0,4194304] 0 2024-03-13T14:50:03.173 INFO:tasks.workunit.client.0.smithi176.stdout:3/893: fdatasync d4/d28/d31/d45/d82/d12b/df5/ff8 0 2024-03-13T14:50:03.175 INFO:tasks.workunit.client.0.smithi176.stdout:6/878: creat d0/d10/d19/d27/d2b/f118 x:0 0 0 2024-03-13T14:50:03.176 INFO:tasks.workunit.client.0.smithi176.stdout:5/824: write d5/d2e/d7e/dad/fe1 [545394,107267] 0 2024-03-13T14:50:03.177 INFO:tasks.workunit.client.0.smithi176.stdout:9/955: mknod d0/d1e/d8d/d9f/c146 0 2024-03-13T14:50:03.180 INFO:tasks.workunit.client.0.smithi176.stdout:8/783: chown d0/d5/d11/d47/ce4 0 1 2024-03-13T14:50:03.184 INFO:tasks.workunit.client.1.smithi196.stdout:2/895: write d0/d7/d9/fc9 [673509,105287] 0 2024-03-13T14:50:03.186 INFO:tasks.workunit.client.1.smithi196.stdout:4/931: write d1/d3/d7/d35/d76/fb9 [375588,74764] 0 2024-03-13T14:50:03.188 INFO:tasks.workunit.client.1.smithi196.stdout:2/896: chown d0/d6/d4b/d5f/dce/ff9 341305958 1 2024-03-13T14:50:03.193 INFO:tasks.workunit.client.1.smithi196.stdout:2/897: mknod d0/d6/d4b/d5f/c121 0 2024-03-13T14:50:03.199 INFO:tasks.workunit.client.0.smithi176.stdout:7/800: getdents d1/d1c/d26/d41/d2b/d40 0 2024-03-13T14:50:03.200 INFO:tasks.workunit.client.0.smithi176.stdout:3/894: unlink d4/d13/d81/f10b 0 2024-03-13T14:50:03.200 INFO:tasks.workunit.client.0.smithi176.stdout:5/825: symlink d5/d31/def/l119 0 2024-03-13T14:50:03.201 INFO:tasks.workunit.client.0.smithi176.stdout:2/874: dread - d1/dc/d2d/d59/d80/d5c/d78/fed zero size 2024-03-13T14:50:03.201 INFO:tasks.workunit.client.1.smithi196.stdout:2/898: rename d0/d6/d34/f62 to d0/d6/d4b/d96/f122 0 2024-03-13T14:50:03.203 INFO:tasks.workunit.client.0.smithi176.stdout:3/895: chown d4/d13/d16/ddd 57 1 2024-03-13T14:50:03.205 INFO:tasks.workunit.client.1.smithi196.stdout:4/932: dwrite d1/d3/d4c/d72/ff3 [0,4194304] 0 2024-03-13T14:50:03.205 INFO:tasks.workunit.client.1.smithi196.stdout:0/981: dwrite d8/da/d25/f10f [0,4194304] 0 2024-03-13T14:50:03.206 INFO:tasks.workunit.client.0.smithi176.stdout:5/826: dread - d5/d2e/d5f/fdc zero size 2024-03-13T14:50:03.207 INFO:tasks.workunit.client.1.smithi196.stdout:2/899: creat d0/d6/d4b/f123 x:0 0 0 2024-03-13T14:50:03.207 INFO:tasks.workunit.client.0.smithi176.stdout:0/756: truncate d2/d5/f87 3931262 0 2024-03-13T14:50:03.210 INFO:tasks.workunit.client.0.smithi176.stdout:9/956: unlink d0/db/def/db8/f116 0 2024-03-13T14:50:03.211 INFO:tasks.workunit.client.1.smithi196.stdout:2/900: fsync d0/f37 0 2024-03-13T14:50:03.212 INFO:tasks.workunit.client.1.smithi196.stdout:0/982: write d8/da/d25/d26/d48/d82/da0/f138 [139200,118061] 0 2024-03-13T14:50:03.213 INFO:tasks.workunit.client.0.smithi176.stdout:2/875: fsync d1/dc/d2d/d59/d8b/daf/db2/f10e 0 2024-03-13T14:50:03.213 INFO:tasks.workunit.client.0.smithi176.stdout:3/896: dread - d4/d28/d31/d45/d82/d87/d102/d12c/f133 zero size 2024-03-13T14:50:03.217 INFO:tasks.workunit.client.1.smithi196.stdout:4/933: mkdir d1/d3/d7/d35/d76/d58/d69/d6c/d14b 0 2024-03-13T14:50:03.217 INFO:tasks.workunit.client.0.smithi176.stdout:1/830: dwrite d2/f76 [0,4194304] 0 2024-03-13T14:50:03.219 INFO:tasks.workunit.client.0.smithi176.stdout:3/897: stat d4/d13/d16/d80/c120 0 2024-03-13T14:50:03.223 INFO:tasks.workunit.client.0.smithi176.stdout:7/801: rename d1/d1c/d26/d41/d2b/d40/f8a to d1/d28/d3a/d1b/d8f/dd1/df4/ffd 0 2024-03-13T14:50:03.227 INFO:tasks.workunit.client.0.smithi176.stdout:0/757: mkdir d2/db/d39/d9c/df1 0 2024-03-13T14:50:03.227 INFO:tasks.workunit.client.0.smithi176.stdout:9/957: dread - d0/db/d89/d4d/d6e/fea zero size 2024-03-13T14:50:03.228 INFO:tasks.workunit.client.1.smithi196.stdout:4/934: creat d1/d3/d4c/d72/daf/f14c x:0 0 0 2024-03-13T14:50:03.230 INFO:tasks.workunit.client.0.smithi176.stdout:6/879: dwrite d0/d10/d19/d27/d2b/de0/d76/fba [0,4194304] 0 2024-03-13T14:50:03.234 INFO:tasks.workunit.client.0.smithi176.stdout:2/876: mkdir d1/dc/d2d/d59/d80/d5c/d127 0 2024-03-13T14:50:03.235 INFO:tasks.workunit.client.1.smithi196.stdout:0/983: link d8/da/d25/d26/d54/ff3 d8/da/d25/d26/d35/d3b/d50/d5d/f149 0 2024-03-13T14:50:03.236 INFO:tasks.workunit.client.0.smithi176.stdout:3/898: rmdir d4/d13/d81/d89/d5b/dfc 39 2024-03-13T14:50:03.236 INFO:tasks.workunit.client.0.smithi176.stdout:8/784: dread d0/d5/d11/d47/f6a [0,4194304] 0 2024-03-13T14:50:03.237 INFO:tasks.workunit.client.1.smithi196.stdout:4/935: mknod d1/d3/d7/d35/d76/d58/d69/d6c/d14b/c14d 0 2024-03-13T14:50:03.239 INFO:tasks.workunit.client.0.smithi176.stdout:1/831: write d2/de/d3f/d3e/d86/fe0 [137465,117827] 0 2024-03-13T14:50:03.240 INFO:tasks.workunit.client.0.smithi176.stdout:1/832: readlink d2/de/d10/l24 0 2024-03-13T14:50:03.241 INFO:tasks.workunit.client.0.smithi176.stdout:6/880: fdatasync d0/f8 0 2024-03-13T14:50:03.244 INFO:tasks.workunit.client.0.smithi176.stdout:9/958: write d0/d1e/f4a [3294315,31803] 0 2024-03-13T14:50:03.247 INFO:tasks.workunit.client.0.smithi176.stdout:5/827: rmdir d5/d2b/dce 0 2024-03-13T14:50:03.250 INFO:tasks.workunit.client.0.smithi176.stdout:6/881: chown d0/d10/d19/d27/d2b/de0/d76/d7c/l96 1892 1 2024-03-13T14:50:03.250 INFO:tasks.workunit.client.1.smithi196.stdout:4/936: readlink d1/l24 0 2024-03-13T14:50:03.251 INFO:tasks.workunit.client.0.smithi176.stdout:0/758: symlink d2/lf2 0 2024-03-13T14:50:03.252 INFO:tasks.workunit.client.0.smithi176.stdout:6/882: write d0/de/d38/d70/faf [9421098,22608] 0 2024-03-13T14:50:03.252 INFO:tasks.workunit.client.1.smithi196.stdout:4/937: dread - d1/d3/d4c/f57 zero size 2024-03-13T14:50:03.252 INFO:tasks.workunit.client.0.smithi176.stdout:8/785: rename d0/db/f3f to d0/d5/ffd 0 2024-03-13T14:50:03.255 INFO:tasks.workunit.client.0.smithi176.stdout:1/833: mknod d2/de/d3f/d6a/cfa 0 2024-03-13T14:50:03.259 INFO:tasks.workunit.client.1.smithi196.stdout:4/938: mknod d1/d3/d7/d35/d76/d58/d69/d9f/dd5/c14e 0 2024-03-13T14:50:03.260 INFO:tasks.workunit.client.1.smithi196.stdout:4/939: stat d1/d1e/d5c 0 2024-03-13T14:50:03.263 INFO:tasks.workunit.client.0.smithi176.stdout:8/786: rename d0/db/l8f to d0/db/d53/df7/lfe 0 2024-03-13T14:50:03.265 INFO:tasks.workunit.client.0.smithi176.stdout:5/828: mkdir d5/d31/d86/dc0/d11a 0 2024-03-13T14:50:03.265 INFO:tasks.workunit.client.0.smithi176.stdout:9/959: dwrite d0/db/d89/d4d/d6e/ffb [0,4194304] 0 2024-03-13T14:50:03.265 INFO:tasks.workunit.client.0.smithi176.stdout:7/802: write d1/d1c/d26/d41/fcc [1071223,30629] 0 2024-03-13T14:50:03.266 INFO:tasks.workunit.client.1.smithi196.stdout:4/940: mkdir d1/d3/d7/d35/d76/d58/d69/d6c/d14f 0 2024-03-13T14:50:03.268 INFO:tasks.workunit.client.0.smithi176.stdout:5/829: write d5/d31/d47/d80/fd6 [1373748,5977] 0 2024-03-13T14:50:03.269 INFO:tasks.workunit.client.0.smithi176.stdout:6/883: symlink d0/de/d38/l119 0 2024-03-13T14:50:03.271 INFO:tasks.workunit.client.0.smithi176.stdout:2/877: write d1/dc/d1c/fa4 [654881,6470] 0 2024-03-13T14:50:03.272 INFO:tasks.workunit.client.1.smithi196.stdout:4/941: fdatasync d1/d3/d7/d35/d3e/f64 0 2024-03-13T14:50:03.275 INFO:tasks.workunit.client.1.smithi196.stdout:4/942: write d1/dfb/d11c/fb6 [191678,73411] 0 2024-03-13T14:50:03.275 INFO:tasks.workunit.client.1.smithi196.stdout:0/984: dwrite d8/da/d25/f5a [0,4194304] 0 2024-03-13T14:50:03.279 INFO:tasks.workunit.client.0.smithi176.stdout:9/960: dread d0/d17/fa3 [0,4194304] 0 2024-03-13T14:50:03.279 INFO:tasks.workunit.client.1.smithi196.stdout:0/985: readlink d8/da/d25/d26/d35/d3b/d5f/d33/l116 0 2024-03-13T14:50:03.284 INFO:tasks.workunit.client.1.smithi196.stdout:4/943: fsync d1/d3/d7/f5d 0 2024-03-13T14:50:03.286 INFO:tasks.workunit.client.0.smithi176.stdout:8/787: mknod d0/db/d53/cff 0 2024-03-13T14:50:03.287 INFO:tasks.workunit.client.1.smithi196.stdout:0/986: fsync d8/da/d25/d26/d35/d3b/d59/f9b 0 2024-03-13T14:50:03.287 INFO:tasks.workunit.client.0.smithi176.stdout:5/830: truncate d5/d2c/d40/dc7/d89/fc4 300384 0 2024-03-13T14:50:03.290 INFO:tasks.workunit.client.0.smithi176.stdout:8/788: read d0/d24/f32 [3326653,86282] 0 2024-03-13T14:50:03.291 INFO:tasks.workunit.client.1.smithi196.stdout:4/944: mknod d1/d3/d7/d35/d76/d58/dc9/c150 0 2024-03-13T14:50:03.291 INFO:tasks.workunit.client.0.smithi176.stdout:6/884: rename d0/d10/d19/d27/d2b/de0/d76/ccf to d0/d10/d19/d27/d2b/d2e/dc4/c11a 0 2024-03-13T14:50:03.293 INFO:tasks.workunit.client.0.smithi176.stdout:2/878: mknod d1/dc/d2d/d59/d80/d7e/da3/d111/c128 0 2024-03-13T14:50:03.294 INFO:tasks.workunit.client.0.smithi176.stdout:8/789: truncate d0/db/d39/d56/ffa 167400 0 2024-03-13T14:50:03.294 INFO:tasks.workunit.client.1.smithi196.stdout:0/987: mknod d8/da/d25/d26/d48/d82/c14a 0 2024-03-13T14:50:03.294 INFO:tasks.workunit.client.0.smithi176.stdout:1/834: link d2/de/d10/d16/d47/c8b d2/de/d10/d16/d4a/d5f/da4/cfb 0 2024-03-13T14:50:03.295 INFO:tasks.workunit.client.0.smithi176.stdout:6/885: read d0/d10/d19/d27/d2b/de0/dd6/fd1 [51998,22098] 0 2024-03-13T14:50:03.295 INFO:tasks.workunit.client.0.smithi176.stdout:2/879: chown d1/dc/d1c/l22 7270728 1 2024-03-13T14:50:03.297 INFO:tasks.workunit.client.0.smithi176.stdout:7/803: symlink d1/d28/d3a/d1b/d8f/dc1/db7/df9/lfe 0 2024-03-13T14:50:03.300 INFO:tasks.workunit.client.1.smithi196.stdout:4/945: getdents d1/d136 0 2024-03-13T14:50:03.304 INFO:tasks.workunit.client.1.smithi196.stdout:5/953: sync 2024-03-13T14:50:03.305 INFO:tasks.workunit.client.0.smithi176.stdout:8/790: dread d0/d24/d98/dc6/d40/f77 [0,4194304] 0 2024-03-13T14:50:03.305 INFO:tasks.workunit.client.0.smithi176.stdout:5/831: symlink d5/d2c/l11b 0 2024-03-13T14:50:03.307 INFO:tasks.workunit.client.1.smithi196.stdout:4/946: readlink d1/d3/d7/d35/lfc 0 2024-03-13T14:50:03.310 INFO:tasks.workunit.client.0.smithi176.stdout:2/880: dwrite d1/dc/d2d/d38/d5d/f70 [0,4194304] 0 2024-03-13T14:50:03.313 INFO:tasks.workunit.client.1.smithi196.stdout:0/988: dwrite d8/da/d25/d26/d35/d3b/d5f/d33/d44/dbe/f13d [0,4194304] 0 2024-03-13T14:50:03.313 INFO:tasks.workunit.client.0.smithi176.stdout:2/881: chown d1/dc/d2d/d59/d80/d7e/da3/f11a 0 1 2024-03-13T14:50:03.316 INFO:tasks.workunit.client.0.smithi176.stdout:1/835: creat d2/de/d10/de8/ffc x:0 0 0 2024-03-13T14:50:03.317 INFO:tasks.workunit.client.1.smithi196.stdout:5/954: rename d4/d1b/dc8/d14/d19/d1c/d2d/ceb to d4/d1b/dc8/d14/d37/c12f 0 2024-03-13T14:50:03.319 INFO:tasks.workunit.client.0.smithi176.stdout:5/832: creat d5/d2c/d40/dc7/d89/df2/dff/f11c x:0 0 0 2024-03-13T14:50:03.322 INFO:tasks.workunit.client.0.smithi176.stdout:6/886: link d0/de/d38/d88/d9d/fd4 d0/d10/dca/de1/f11b 0 2024-03-13T14:50:03.326 INFO:tasks.workunit.client.0.smithi176.stdout:9/961: rename d0/db/d89/d4d/f130 to d0/d1e/f147 0 2024-03-13T14:50:03.327 INFO:tasks.workunit.client.1.smithi196.stdout:0/989: mkdir d8/da/d25/d26/d48/d82/da0/df2/d14b 0 2024-03-13T14:50:03.328 INFO:tasks.workunit.client.0.smithi176.stdout:8/791: rmdir d0/d5 39 2024-03-13T14:50:03.330 INFO:tasks.workunit.client.0.smithi176.stdout:1/836: symlink d2/de/d10/d16/d40/lfd 0 2024-03-13T14:50:03.332 INFO:tasks.workunit.client.1.smithi196.stdout:5/955: truncate d4/d1b/dc8/d14/d19/d26/d33/fdb 1783974 0 2024-03-13T14:50:03.335 INFO:tasks.workunit.client.1.smithi196.stdout:4/947: getdents d1/d1e/d43 0 2024-03-13T14:50:03.350 INFO:tasks.workunit.client.0.smithi176.stdout:6/887: write d0/d10/d19/d27/d2b/de0/dd6/fd8 [928613,43674] 0 2024-03-13T14:50:03.350 INFO:tasks.workunit.client.0.smithi176.stdout:1/837: write d2/d29/f3c [3658941,16627] 0 2024-03-13T14:50:03.350 INFO:tasks.workunit.client.1.smithi196.stdout:0/990: rename d8/da/d25/d26/d35/l79 to d8/da/d25/d26/d35/d3b/d50/l14c 0 2024-03-13T14:50:03.351 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:02 smithi196 ceph-mon[37486]: pgmap v20: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 73 MiB/s rd, 173 MiB/s wr, 386 op/s 2024-03-13T14:50:03.351 INFO:tasks.workunit.client.0.smithi176.stdout:9/962: mknod d0/d1e/d8d/d125/c148 0 2024-03-13T14:50:03.352 INFO:tasks.workunit.client.0.smithi176.stdout:9/963: fsync d0/f143 0 2024-03-13T14:50:03.353 INFO:tasks.workunit.client.0.smithi176.stdout:8/792: mknod d0/d24/d7b/dec/c100 0 2024-03-13T14:50:03.358 INFO:tasks.workunit.client.1.smithi196.stdout:0/991: symlink d8/d37/l14d 0 2024-03-13T14:50:03.359 INFO:tasks.workunit.client.0.smithi176.stdout:7/804: rename d1/cc to d1/d28/d3a/d1b/d8f/cff 0 2024-03-13T14:50:03.362 INFO:tasks.workunit.client.1.smithi196.stdout:5/956: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/db2/d115/f130 x:0 0 0 2024-03-13T14:50:03.365 INFO:tasks.workunit.client.1.smithi196.stdout:4/948: dwrite d1/d3/d7/d35/d76/d58/dc6/f11f [0,4194304] 0 2024-03-13T14:50:03.366 INFO:tasks.workunit.client.0.smithi176.stdout:9/964: unlink d0/d40/d53/lc9 0 2024-03-13T14:50:03.367 INFO:tasks.workunit.client.0.smithi176.stdout:8/793: mkdir d0/d24/d7b/dab/d101 0 2024-03-13T14:50:03.367 INFO:tasks.workunit.client.0.smithi176.stdout:9/965: stat d0/db/d89/f16 0 2024-03-13T14:50:03.368 INFO:tasks.workunit.client.0.smithi176.stdout:5/833: link d5/d2e/ca5 d5/d31/d47/c11d 0 2024-03-13T14:50:03.369 INFO:tasks.workunit.client.0.smithi176.stdout:9/966: write d0/f143 [470738,17984] 0 2024-03-13T14:50:03.369 INFO:tasks.workunit.client.1.smithi196.stdout:5/957: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/df5/f131 x:0 0 0 2024-03-13T14:50:03.370 INFO:tasks.workunit.client.1.smithi196.stdout:0/992: dread d8/da/d25/d26/d48/d82/db3/dd5/fe6 [0,4194304] 0 2024-03-13T14:50:03.372 INFO:tasks.workunit.client.0.smithi176.stdout:6/888: mkdir d0/d10/d19/d106/d11c 0 2024-03-13T14:50:03.372 INFO:tasks.workunit.client.0.smithi176.stdout:2/882: rename d1/dc/d2d/d59/lf6 to d1/dc/d2d/d90/l129 0 2024-03-13T14:50:03.376 INFO:tasks.workunit.client.1.smithi196.stdout:5/958: fdatasync d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/fd5 0 2024-03-13T14:50:03.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:02 smithi176 ceph-mon[29064]: pgmap v20: 65 pgs: 65 active+clean; 3.3 GiB data, 9.7 GiB used, 527 GiB / 536 GiB avail; 73 MiB/s rd, 173 MiB/s wr, 386 op/s 2024-03-13T14:50:03.381 INFO:tasks.workunit.client.1.smithi196.stdout:0/993: rename d8/da/d25/d26/d48/d82/db7/f135 to d8/f14e 0 2024-03-13T14:50:03.384 INFO:tasks.workunit.client.0.smithi176.stdout:5/834: creat d5/d2e/d7e/dad/d7b/da7/d111/d15/f11e x:0 0 0 2024-03-13T14:50:03.387 INFO:tasks.workunit.client.0.smithi176.stdout:2/883: creat d1/dc/d2d/d59/f12a x:0 0 0 2024-03-13T14:50:03.388 INFO:tasks.workunit.client.0.smithi176.stdout:9/967: creat d0/db/d12c/f149 x:0 0 0 2024-03-13T14:50:03.389 INFO:tasks.workunit.client.1.smithi196.stdout:4/949: write d1/d3/d4c/d72/daf/fd6 [116714,46449] 0 2024-03-13T14:50:03.391 INFO:tasks.workunit.client.0.smithi176.stdout:1/838: dwrite d2/de/d10/d16/d4a/d5f/d66/f84 [0,4194304] 0 2024-03-13T14:50:03.393 INFO:tasks.workunit.client.1.smithi196.stdout:4/950: chown d1/d3/d4c/d72/d7d/d80/d55/c11e 706 1 2024-03-13T14:50:03.393 INFO:tasks.workunit.client.0.smithi176.stdout:2/884: write d1/dc/d2d/d38/d5d/f70 [1852518,108723] 0 2024-03-13T14:50:03.394 INFO:tasks.workunit.client.1.smithi196.stdout:0/994: mknod d8/da/d25/d26/d35/d3b/d6c/d142/c14f 0 2024-03-13T14:50:03.396 INFO:tasks.workunit.client.0.smithi176.stdout:5/835: unlink d5/d2e/d7e/dad/d7b/da7/d111/l8d 0 2024-03-13T14:50:03.399 INFO:tasks.workunit.client.0.smithi176.stdout:5/836: write d5/d2e/d7e/dad/d7b/fc8 [939231,79418] 0 2024-03-13T14:50:03.403 INFO:tasks.workunit.client.0.smithi176.stdout:9/968: mkdir d0/d40/d14a 0 2024-03-13T14:50:03.406 INFO:tasks.workunit.client.0.smithi176.stdout:9/969: rename d0/d1e/d8d to d0/d1e/d8d/da7/d14b 22 2024-03-13T14:50:03.406 INFO:tasks.workunit.client.0.smithi176.stdout:6/889: dread d0/d10/d19/d27/f61 [0,4194304] 0 2024-03-13T14:50:03.406 INFO:tasks.workunit.client.0.smithi176.stdout:1/839: symlink d2/de/d3f/d50/d83/lfe 0 2024-03-13T14:50:03.409 INFO:tasks.workunit.client.1.smithi196.stdout:5/959: getdents d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0 0 2024-03-13T14:50:03.409 INFO:tasks.workunit.client.0.smithi176.stdout:6/890: write d0/ff3 [985696,103042] 0 2024-03-13T14:50:03.412 INFO:tasks.workunit.client.1.smithi196.stdout:5/960: read d4/d1b/dc8/d14/d37/f78 [791382,122898] 0 2024-03-13T14:50:03.413 INFO:tasks.workunit.client.0.smithi176.stdout:7/805: dread d1/d1c/d26/d41/d2b/f85 [0,4194304] 0 2024-03-13T14:50:03.414 INFO:tasks.workunit.client.0.smithi176.stdout:2/885: mkdir d1/dc/d2d/d59/d80/d12b 0 2024-03-13T14:50:03.415 INFO:tasks.workunit.client.0.smithi176.stdout:8/794: truncate d0/d24/d98/dc6/db9/fdb 730332 0 2024-03-13T14:50:03.415 INFO:tasks.workunit.client.0.smithi176.stdout:7/806: chown d1/d1c/d26/d41/d2b/f85 237 1 2024-03-13T14:50:03.415 INFO:tasks.workunit.client.1.smithi196.stdout:5/961: chown d4/d1b/dc8/d3c/d6a/d106/fe4 839 1 2024-03-13T14:50:03.417 INFO:tasks.workunit.client.0.smithi176.stdout:1/840: write d2/de/d3f/d3e/d86/fbc [1027786,1148] 0 2024-03-13T14:50:03.418 INFO:tasks.workunit.client.0.smithi176.stdout:7/807: stat d1/d1c/d26/d41/db6/df1/dfc/d76/da5/ff2 0 2024-03-13T14:50:03.418 INFO:tasks.workunit.client.0.smithi176.stdout:9/970: dread d0/db/d89/f141 [0,4194304] 0 2024-03-13T14:50:03.418 INFO:tasks.workunit.client.1.smithi196.stdout:5/962: chown d4/d1b/dc8/d14/d19/d26/fec 0 1 2024-03-13T14:50:03.420 INFO:tasks.workunit.client.0.smithi176.stdout:6/891: stat d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1 0 2024-03-13T14:50:03.424 INFO:tasks.workunit.client.1.smithi196.stdout:4/951: write d1/d3/d7/ddf/fe2 [4690238,118138] 0 2024-03-13T14:50:03.424 INFO:tasks.workunit.client.1.smithi196.stdout:5/963: write d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/df1/f122 [71477,53952] 0 2024-03-13T14:50:03.424 INFO:tasks.workunit.client.1.smithi196.stdout:0/995: write d8/da/d25/d26/d54/ff3 [1083698,51701] 0 2024-03-13T14:50:03.426 INFO:tasks.workunit.client.0.smithi176.stdout:1/841: rename d2/d30/d63/l2d to d2/de/dd1/lff 0 2024-03-13T14:50:03.429 INFO:tasks.workunit.client.1.smithi196.stdout:0/996: read d8/da/d25/d26/d35/d3b/d50/d5d/d7e/f119 [2426892,105052] 0 2024-03-13T14:50:03.429 INFO:tasks.workunit.client.0.smithi176.stdout:9/971: unlink d0/db/d89/d32/d122/f41 0 2024-03-13T14:50:03.430 INFO:tasks.workunit.client.0.smithi176.stdout:9/972: chown d0/d17 1274 1 2024-03-13T14:50:03.433 INFO:tasks.workunit.client.1.smithi196.stdout:5/964: unlink d4/d1b/dc8/d3c/d6a/d106/fe4 0 2024-03-13T14:50:03.435 INFO:tasks.workunit.client.0.smithi176.stdout:9/973: truncate d0/db/d89/d4d/f126 657478 0 2024-03-13T14:50:03.436 INFO:tasks.workunit.client.0.smithi176.stdout:7/808: creat d1/d28/d3a/d1b/d8f/dc1/dbb/de5/f100 x:0 0 0 2024-03-13T14:50:03.441 INFO:tasks.workunit.client.0.smithi176.stdout:2/886: dread d1/d2a/f75 [0,4194304] 0 2024-03-13T14:50:03.441 INFO:tasks.workunit.client.1.smithi196.stdout:4/952: mknod d1/d3/d7/c151 0 2024-03-13T14:50:03.443 INFO:tasks.workunit.client.0.smithi176.stdout:6/892: dwrite d0/f4e [0,4194304] 0 2024-03-13T14:50:03.443 INFO:tasks.workunit.client.0.smithi176.stdout:8/795: mknod d0/d5/c102 0 2024-03-13T14:50:03.446 INFO:tasks.workunit.client.1.smithi196.stdout:5/965: truncate d4/d1b/dc8/d14/d19/f96 1461181 0 2024-03-13T14:50:03.448 INFO:tasks.workunit.client.0.smithi176.stdout:9/974: mkdir d0/d40/d14c 0 2024-03-13T14:50:03.450 INFO:tasks.workunit.client.0.smithi176.stdout:7/809: fsync d1/d28/d3a/d1b/d8f/fac 0 2024-03-13T14:50:03.451 INFO:tasks.workunit.client.1.smithi196.stdout:5/966: fsync d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/db2/f116 0 2024-03-13T14:50:03.452 INFO:tasks.workunit.client.0.smithi176.stdout:1/842: write d2/de/d10/d16/d59/f6e [1018595,32581] 0 2024-03-13T14:50:03.456 INFO:tasks.workunit.client.0.smithi176.stdout:9/975: mkdir d0/db/d89/d32/d58/d67/d14d 0 2024-03-13T14:50:03.457 INFO:tasks.workunit.client.0.smithi176.stdout:1/843: creat d2/de/d3f/d50/d83/d97/f100 x:0 0 0 2024-03-13T14:50:03.458 INFO:tasks.workunit.client.0.smithi176.stdout:7/810: truncate d1/d1c/d26/d41/db6/df1/dfc/d76/da5/ff2 550420 0 2024-03-13T14:50:03.461 INFO:tasks.workunit.client.0.smithi176.stdout:9/976: write d0/db/d89/d32/d58/d5f/d84/d9a/f9b [9053162,113021] 0 2024-03-13T14:50:03.462 INFO:tasks.workunit.client.0.smithi176.stdout:6/893: dwrite d0/d10/d19/d27/d2b/d2e/dc4/dc5/fe7 [0,4194304] 0 2024-03-13T14:50:03.462 INFO:tasks.workunit.client.0.smithi176.stdout:2/887: truncate d1/dc/d1c/f37 2721760 0 2024-03-13T14:50:03.467 INFO:tasks.workunit.client.1.smithi196.stdout:5/967: rename d4/d1b/dc8/d14/d37/c12f to d4/d1b/dc8/d14/d19/da7/d126/c132 0 2024-03-13T14:50:03.469 INFO:tasks.workunit.client.1.smithi196.stdout:0/997: dread d8/da/fb0 [0,4194304] 0 2024-03-13T14:50:03.471 INFO:tasks.workunit.client.0.smithi176.stdout:9/977: read d0/db/d89/d32/f95 [648451,6825] 0 2024-03-13T14:50:03.474 INFO:tasks.workunit.client.1.smithi196.stdout:4/953: dwrite d1/d1e/d5c/ff1 [0,4194304] 0 2024-03-13T14:50:03.475 INFO:tasks.workunit.client.0.smithi176.stdout:2/888: creat d1/dc/d2d/d4b/f12c x:0 0 0 2024-03-13T14:50:03.476 INFO:tasks.workunit.client.0.smithi176.stdout:6/894: dread d0/d10/d19/d27/d2b/de0/dd6/fd1 [0,4194304] 0 2024-03-13T14:50:03.476 INFO:tasks.workunit.client.0.smithi176.stdout:9/978: mknod d0/dd8/c14e 0 2024-03-13T14:50:03.478 INFO:tasks.workunit.client.1.smithi196.stdout:0/998: creat d8/da/d25/d26/d35/d3b/d50/d55/f150 x:0 0 0 2024-03-13T14:50:03.479 INFO:tasks.workunit.client.1.smithi196.stdout:5/968: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/f56 4163472 0 2024-03-13T14:50:03.479 INFO:tasks.workunit.client.0.smithi176.stdout:7/811: dwrite d1/d28/f2a [0,4194304] 0 2024-03-13T14:50:03.479 INFO:tasks.workunit.client.0.smithi176.stdout:2/889: dread - d1/dc/d2d/d59/d8b/daf/db2/f10e zero size 2024-03-13T14:50:03.481 INFO:tasks.workunit.client.1.smithi196.stdout:5/969: readlink d4/d1b/dc8/d14/l45 0 2024-03-13T14:50:03.481 INFO:tasks.workunit.client.0.smithi176.stdout:8/796: truncate d0/db/d39/d56/f5e 4186788 0 2024-03-13T14:50:03.486 INFO:tasks.workunit.client.0.smithi176.stdout:9/979: fsync d0/db/d89/d32/f62 0 2024-03-13T14:50:03.493 INFO:tasks.workunit.client.1.smithi196.stdout:5/970: fsync d4/d1b/dc8/d14/d19/d26/d68/fd3 0 2024-03-13T14:50:03.496 INFO:tasks.workunit.client.0.smithi176.stdout:2/890: unlink d1/dc/d2d/d90/dcf/f44 0 2024-03-13T14:50:03.496 INFO:tasks.workunit.client.0.smithi176.stdout:1/844: dwrite d2/de/d10/f6d [0,4194304] 0 2024-03-13T14:50:03.497 INFO:tasks.workunit.client.1.smithi196.stdout:4/954: getdents d1/d3/d7/d35/d76/d58/d69/d6c 0 2024-03-13T14:50:03.502 INFO:tasks.workunit.client.0.smithi176.stdout:8/797: write d0/d5/d3b/f41 [5777682,128807] 0 2024-03-13T14:50:03.503 INFO:tasks.workunit.client.1.smithi196.stdout:4/955: mknod d1/dfb/d11c/c152 0 2024-03-13T14:50:03.508 INFO:tasks.workunit.client.1.smithi196.stdout:5/971: mkdir d4/d1b/dc8/d14/d19/d26/d95/d118/d133 0 2024-03-13T14:50:03.509 INFO:tasks.workunit.client.1.smithi196.stdout:0/999: write d8/da/d25/d26/d48/d82/db3/d12c/fae [267236,55868] 0 2024-03-13T14:50:03.513 INFO:tasks.workunit.client.1.smithi196.stdout:4/956: creat d1/d136/f153 x:0 0 0 2024-03-13T14:50:03.514 INFO:tasks.workunit.client.0.smithi176.stdout:9/980: link d0/d40/d53/fb5 d0/d40/d53/d6d/f14f 0 2024-03-13T14:50:03.515 INFO:tasks.workunit.client.0.smithi176.stdout:2/891: dwrite d1/d2a/d33/ff9 [0,4194304] 0 2024-03-13T14:50:03.517 INFO:tasks.workunit.client.1.smithi196.stdout:5/972: mknod d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/c134 0 2024-03-13T14:50:03.521 INFO:tasks.workunit.client.0.smithi176.stdout:2/892: write d1/dc/d2d/d59/d80/d7e/f11e [501272,49836] 0 2024-03-13T14:50:03.521 INFO:tasks.workunit.client.1.smithi196.stdout:5/973: write d4/d1b/dc8/d14/d19/d1c/fa2 [4115916,18391] 0 2024-03-13T14:50:03.521 INFO:tasks.workunit.client.0.smithi176.stdout:7/812: dwrite d1/d1c/d26/d41/d2b/fbc [0,4194304] 0 2024-03-13T14:50:03.524 INFO:tasks.workunit.client.0.smithi176.stdout:2/893: chown d1/dc/d61/f6d 1809 1 2024-03-13T14:50:03.525 INFO:tasks.workunit.client.0.smithi176.stdout:8/798: creat d0/d24/d29/f103 x:0 0 0 2024-03-13T14:50:03.525 INFO:tasks.workunit.client.1.smithi196.stdout:4/957: symlink d1/l154 0 2024-03-13T14:50:03.526 INFO:tasks.workunit.client.0.smithi176.stdout:7/813: chown d1/d1c/d26/d41/d2b/ddc/fc0 2956760 1 2024-03-13T14:50:03.529 INFO:tasks.workunit.client.0.smithi176.stdout:8/799: read d0/f30 [597491,35455] 0 2024-03-13T14:50:03.531 INFO:tasks.workunit.client.0.smithi176.stdout:8/800: truncate d0/feb 809645 0 2024-03-13T14:50:03.535 INFO:tasks.workunit.client.1.smithi196.stdout:4/958: rmdir d1/d3/d7/d35/d76/d58/d69/de3/df4 39 2024-03-13T14:50:03.538 INFO:tasks.workunit.client.0.smithi176.stdout:4/865: sync 2024-03-13T14:50:03.540 INFO:tasks.workunit.client.1.smithi196.stdout:5/974: link d4/d1b/dc8/d14/d19/da7/f111 d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd/f135 0 2024-03-13T14:50:03.542 INFO:tasks.workunit.client.1.smithi196.stdout:4/959: mkdir d1/dfb/d155 0 2024-03-13T14:50:03.546 INFO:tasks.workunit.client.1.smithi196.stdout:3/922: sync 2024-03-13T14:50:03.548 INFO:tasks.workunit.client.0.smithi176.stdout:1/845: write d2/de/f53 [829890,92735] 0 2024-03-13T14:50:03.550 INFO:tasks.workunit.client.1.smithi196.stdout:4/960: creat d1/d1e/d5c/f156 x:0 0 0 2024-03-13T14:50:03.553 INFO:tasks.workunit.client.0.smithi176.stdout:7/814: dwrite d1/d1c/d26/d41/db6/df1/dfc/d76/da5/ff2 [0,4194304] 0 2024-03-13T14:50:03.555 INFO:tasks.workunit.client.0.smithi176.stdout:2/894: write d1/dc/d2d/d59/f8a [2353687,85224] 0 2024-03-13T14:50:03.558 INFO:tasks.workunit.client.1.smithi196.stdout:3/923: creat db/d15/f125 x:0 0 0 2024-03-13T14:50:03.559 INFO:tasks.workunit.client.0.smithi176.stdout:9/981: dwrite d0/d1e/d8d/ff4 [0,4194304] 0 2024-03-13T14:50:03.563 INFO:tasks.workunit.client.1.smithi196.stdout:4/961: creat d1/d3/d7/f157 x:0 0 0 2024-03-13T14:50:03.564 INFO:tasks.workunit.client.1.smithi196.stdout:3/924: rename db/d15/da1/dc8/c10d to db/d71/c126 0 2024-03-13T14:50:03.568 INFO:tasks.workunit.client.1.smithi196.stdout:4/962: mkdir d1/d3/d4c/d72/daf/d158 0 2024-03-13T14:50:03.570 INFO:tasks.workunit.client.1.smithi196.stdout:4/963: dread - d1/dfb/d11c/f112 zero size 2024-03-13T14:50:03.573 INFO:tasks.workunit.client.1.smithi196.stdout:5/975: dwrite d4/d1b/dc8/d14/d19/d1c/d25/d38/fe3 [0,4194304] 0 2024-03-13T14:50:03.574 INFO:tasks.workunit.client.0.smithi176.stdout:8/801: dread d0/fdf [0,4194304] 0 2024-03-13T14:50:03.575 INFO:tasks.workunit.client.1.smithi196.stdout:5/976: stat d4/d1b/dc8/d14/d19/da7/d105 0 2024-03-13T14:50:03.575 INFO:tasks.workunit.client.1.smithi196.stdout:3/925: link db/d19/d22/d3a/d68/d8e/f116 db/d71/f127 0 2024-03-13T14:50:03.577 INFO:tasks.workunit.client.0.smithi176.stdout:8/802: truncate d0/db/fbc 103762 0 2024-03-13T14:50:03.585 INFO:tasks.workunit.client.1.smithi196.stdout:5/977: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d80/d101/f136 x:0 0 0 2024-03-13T14:50:03.588 INFO:tasks.workunit.client.1.smithi196.stdout:4/964: rmdir d1/d1e/d5c 39 2024-03-13T14:50:03.592 INFO:tasks.workunit.client.1.smithi196.stdout:5/978: mknod d4/d1b/dc8/c137 0 2024-03-13T14:50:03.599 INFO:tasks.workunit.client.1.smithi196.stdout:5/979: dread d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/fdd [0,4194304] 0 2024-03-13T14:50:03.607 INFO:tasks.workunit.client.0.smithi176.stdout:2/895: mknod d1/d2a/d33/c12d 0 2024-03-13T14:50:03.609 INFO:tasks.workunit.client.1.smithi196.stdout:9/935: sync 2024-03-13T14:50:03.609 INFO:tasks.workunit.client.1.smithi196.stdout:8/888: sync 2024-03-13T14:50:03.609 INFO:tasks.workunit.client.1.smithi196.stdout:2/901: sync 2024-03-13T14:50:03.609 INFO:tasks.workunit.client.0.smithi176.stdout:7/815: rename d1/d1c/d26/d41/db2 to d1/d1c/d26/d41/db6/df1/dfc/d76/da5/d101 0 2024-03-13T14:50:03.614 INFO:tasks.workunit.client.1.smithi196.stdout:4/965: dread d1/d1e/f7f [0,4194304] 0 2024-03-13T14:50:03.616 INFO:tasks.workunit.client.1.smithi196.stdout:2/902: mkdir d0/d7/d9/d61/d83/d124 0 2024-03-13T14:50:03.619 INFO:tasks.workunit.client.1.smithi196.stdout:5/980: dread d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/ff2 [0,4194304] 0 2024-03-13T14:50:03.621 INFO:tasks.workunit.client.0.smithi176.stdout:9/982: rename d0/db/d89/d7a/fff to d0/db/d89/d32/d58/d67/d119/f150 0 2024-03-13T14:50:03.622 INFO:tasks.workunit.client.0.smithi176.stdout:0/759: sync 2024-03-13T14:50:03.622 INFO:tasks.workunit.client.0.smithi176.stdout:3/899: sync 2024-03-13T14:50:03.622 INFO:tasks.workunit.client.0.smithi176.stdout:5/837: sync 2024-03-13T14:50:03.623 INFO:tasks.workunit.client.1.smithi196.stdout:4/966: creat d1/d3/d4c/d72/d7d/d80/d55/f159 x:0 0 0 2024-03-13T14:50:03.624 INFO:tasks.workunit.client.0.smithi176.stdout:7/816: rmdir d1/d1c/d26/d41/db6/df1/dfc/d4b/d63 39 2024-03-13T14:50:03.626 INFO:tasks.workunit.client.1.smithi196.stdout:3/926: dwrite db/d19/f2b [4194304,4194304] 0 2024-03-13T14:50:03.626 INFO:tasks.workunit.client.0.smithi176.stdout:2/896: mkdir d1/dc/d2d/d38/d12e 0 2024-03-13T14:50:03.627 INFO:tasks.workunit.client.1.smithi196.stdout:8/889: dwrite d0/d16/d3d/d7f/fa8 [0,4194304] 0 2024-03-13T14:50:03.634 INFO:tasks.workunit.client.1.smithi196.stdout:5/981: unlink d4/d1b/dc8/d3c/d6a/lef 0 2024-03-13T14:50:03.634 INFO:tasks.workunit.client.0.smithi176.stdout:5/838: write d5/d2c/d40/dc7/fb9 [1384781,115042] 0 2024-03-13T14:50:03.637 INFO:tasks.workunit.client.1.smithi196.stdout:8/890: truncate d0/d10/d12/df5/f59 5209602 0 2024-03-13T14:50:03.640 INFO:tasks.workunit.client.0.smithi176.stdout:5/839: write d5/d2c/d40/dc7/d89/df2/dff/f11c [331564,112383] 0 2024-03-13T14:50:03.641 INFO:tasks.workunit.client.1.smithi196.stdout:2/903: rename d0/d6/d4b/d5f/fae to d0/d7/d9/d9a/f125 0 2024-03-13T14:50:03.643 INFO:tasks.workunit.client.0.smithi176.stdout:2/897: mkdir d1/dc/de0/d12f 0 2024-03-13T14:50:03.643 INFO:tasks.workunit.client.0.smithi176.stdout:5/840: write d5/d2e/d7e/dad/f92 [320909,14038] 0 2024-03-13T14:50:03.645 INFO:tasks.workunit.client.0.smithi176.stdout:0/760: symlink d2/d5/dc/lf3 0 2024-03-13T14:50:03.646 INFO:tasks.workunit.client.0.smithi176.stdout:5/841: write d5/d2e/d7e/dad/f88 [458029,88062] 0 2024-03-13T14:50:03.655 INFO:tasks.workunit.client.0.smithi176.stdout:4/866: dwrite d7/d8/d15/fbc [0,4194304] 0 2024-03-13T14:50:03.658 INFO:tasks.workunit.client.0.smithi176.stdout:1/846: truncate d2/de/d10/f72 1273554 0 2024-03-13T14:50:03.659 INFO:tasks.workunit.client.1.smithi196.stdout:3/927: creat db/d15/d44/f128 x:0 0 0 2024-03-13T14:50:03.661 INFO:tasks.workunit.client.1.smithi196.stdout:9/936: write d7/de/d4b/d1d/d93/da7/fb6 [112507,17604] 0 2024-03-13T14:50:03.663 INFO:tasks.workunit.client.1.smithi196.stdout:9/937: readlink d7/d9/d17/l92 0 2024-03-13T14:50:03.665 INFO:tasks.workunit.client.0.smithi176.stdout:2/898: chown d1/dc/d2d/d59/d80/d77/cd0 463237187 1 2024-03-13T14:50:03.665 INFO:tasks.workunit.client.0.smithi176.stdout:6/895: sync 2024-03-13T14:50:03.669 INFO:tasks.workunit.client.0.smithi176.stdout:5/842: mkdir d5/d2e/d43/d11f 0 2024-03-13T14:50:03.670 INFO:tasks.workunit.client.0.smithi176.stdout:6/896: stat d0/d10/d19/d27/d2b/de0/dd6/fd8 0 2024-03-13T14:50:03.671 INFO:tasks.workunit.client.0.smithi176.stdout:9/983: link d0/db/d89/d7a/d8b/cc1 d0/db/d89/d4d/d5d/c151 0 2024-03-13T14:50:03.673 INFO:tasks.workunit.client.0.smithi176.stdout:8/803: dwrite d0/f5c [0,4194304] 0 2024-03-13T14:50:03.675 INFO:tasks.workunit.client.1.smithi196.stdout:3/928: symlink db/d19/d22/l129 0 2024-03-13T14:50:03.675 INFO:tasks.workunit.client.1.smithi196.stdout:8/891: write d0/d10/d12/d1e/f6a [436965,33474] 0 2024-03-13T14:50:03.678 INFO:tasks.workunit.client.0.smithi176.stdout:9/984: truncate d0/d40/d53/d6d/f14f 5201234 0 2024-03-13T14:50:03.678 INFO:tasks.workunit.client.0.smithi176.stdout:7/817: write d1/d1c/d26/d41/f5b [2550639,49592] 0 2024-03-13T14:50:03.684 INFO:tasks.workunit.client.0.smithi176.stdout:7/818: truncate d1/d1c/d26/d41/fcc 1271957 0 2024-03-13T14:50:03.686 INFO:tasks.workunit.client.0.smithi176.stdout:7/819: dread - d1/d1c/d26/d41/d2b/fee zero size 2024-03-13T14:50:03.686 INFO:tasks.workunit.client.0.smithi176.stdout:3/900: dwrite d4/d28/d31/d52/f6a [0,4194304] 0 2024-03-13T14:50:03.688 INFO:tasks.workunit.client.1.smithi196.stdout:2/904: link d0/d6/d34/f41 d0/d7/d9/d61/f126 0 2024-03-13T14:50:03.689 INFO:tasks.workunit.client.1.smithi196.stdout:3/929: symlink db/d19/d4a/l12a 0 2024-03-13T14:50:03.691 INFO:tasks.workunit.client.0.smithi176.stdout:2/899: mkdir d1/dc/d1c/d107/d130 0 2024-03-13T14:50:03.691 INFO:tasks.workunit.client.0.smithi176.stdout:7/820: truncate d1/d1c/d26/d41/d2b/fee 885134 0 2024-03-13T14:50:03.693 INFO:tasks.workunit.client.1.smithi196.stdout:4/967: truncate d1/d1e/d5c/f5f 3720437 0 2024-03-13T14:50:03.693 INFO:tasks.workunit.client.1.smithi196.stdout:3/930: write db/d19/d22/f3c [4650110,103894] 0 2024-03-13T14:50:03.696 INFO:tasks.workunit.client.1.smithi196.stdout:3/931: write db/d19/d50/db7/f123 [793794,108868] 0 2024-03-13T14:50:03.696 INFO:tasks.workunit.client.1.smithi196.stdout:8/892: symlink d0/d10/d12/d1e/d2d/l128 0 2024-03-13T14:50:03.697 INFO:tasks.workunit.client.0.smithi176.stdout:5/843: truncate d5/f24 586412 0 2024-03-13T14:50:03.699 INFO:tasks.workunit.client.1.smithi196.stdout:5/982: getdents d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63 0 2024-03-13T14:50:03.700 INFO:tasks.workunit.client.1.smithi196.stdout:8/893: write d0/d10/d12/d17/f10f [4678815,38861] 0 2024-03-13T14:50:03.701 INFO:tasks.workunit.client.0.smithi176.stdout:8/804: rmdir d0/d24/dc3 39 2024-03-13T14:50:03.703 INFO:tasks.workunit.client.1.smithi196.stdout:2/905: fsync d0/d7/fd4 0 2024-03-13T14:50:03.705 INFO:tasks.workunit.client.0.smithi176.stdout:0/761: dwrite d2/db/d35/f8d [0,4194304] 0 2024-03-13T14:50:03.706 INFO:tasks.workunit.client.1.smithi196.stdout:2/906: dread - d0/d7/d9/d42/d98/de4/d104/f119 zero size 2024-03-13T14:50:03.707 INFO:tasks.workunit.client.1.smithi196.stdout:9/938: link d7/d71/cc1 d7/d71/c13b 0 2024-03-13T14:50:03.707 INFO:tasks.workunit.client.0.smithi176.stdout:1/847: write d2/de/d10/d16/f27 [9235728,69740] 0 2024-03-13T14:50:03.711 INFO:tasks.workunit.client.1.smithi196.stdout:4/968: creat d1/d3/d7/d35/d76/d58/d69/d9f/f15a x:0 0 0 2024-03-13T14:50:03.712 INFO:tasks.workunit.client.0.smithi176.stdout:3/901: creat d4/d13/d81/f139 x:0 0 0 2024-03-13T14:50:03.716 INFO:tasks.workunit.client.0.smithi176.stdout:6/897: write d0/de/d38/d88/fc3 [596282,113324] 0 2024-03-13T14:50:03.718 INFO:tasks.workunit.client.0.smithi176.stdout:6/898: stat d0/d10/d19/d27/d2b/d2e/db7/de4 0 2024-03-13T14:50:03.719 INFO:tasks.workunit.client.0.smithi176.stdout:7/821: creat d1/d1c/d26/d41/db6/df1/dfc/d81/f102 x:0 0 0 2024-03-13T14:50:03.726 INFO:tasks.workunit.client.1.smithi196.stdout:3/932: mkdir db/d19/d50/db7/dff/d12b 0 2024-03-13T14:50:03.727 INFO:tasks.workunit.client.0.smithi176.stdout:8/805: creat d0/d24/d7b/dab/d86/f104 x:0 0 0 2024-03-13T14:50:03.728 INFO:tasks.workunit.client.0.smithi176.stdout:8/806: stat d0/d24/f57 0 2024-03-13T14:50:03.729 INFO:tasks.workunit.client.1.smithi196.stdout:5/983: truncate d4/d1b/dc8/d3c/dd0/ff4 1554789 0 2024-03-13T14:50:03.730 INFO:tasks.workunit.client.0.smithi176.stdout:4/867: creat d7/f120 x:0 0 0 2024-03-13T14:50:03.732 INFO:tasks.workunit.client.1.smithi196.stdout:9/939: symlink d7/de/d74/l13c 0 2024-03-13T14:50:03.733 INFO:tasks.workunit.client.0.smithi176.stdout:5/844: write d5/d2e/d5f/ff1 [193927,75284] 0 2024-03-13T14:50:03.734 INFO:tasks.workunit.client.1.smithi196.stdout:2/907: write d0/d6/d90/fb0 [627758,62663] 0 2024-03-13T14:50:03.738 INFO:tasks.workunit.client.1.smithi196.stdout:4/969: write d1/d3/f102 [844678,60112] 0 2024-03-13T14:50:03.741 INFO:tasks.workunit.client.1.smithi196.stdout:5/984: write d4/d1b/dc8/d14/d19/d1c/d25/f8e [4075428,109956] 0 2024-03-13T14:50:03.742 INFO:tasks.workunit.client.1.smithi196.stdout:8/894: dread d0/d16/d58/fd0 [0,4194304] 0 2024-03-13T14:50:03.744 INFO:tasks.workunit.client.0.smithi176.stdout:3/902: dread d4/d28/d31/d45/d82/d12b/df5/f3c [0,4194304] 0 2024-03-13T14:50:03.747 INFO:tasks.workunit.client.1.smithi196.stdout:4/970: creat d1/d1e/d5c/d105/f15b x:0 0 0 2024-03-13T14:50:03.749 INFO:tasks.workunit.client.0.smithi176.stdout:1/848: rename d2/faa to d2/de/d3f/d3e/f101 0 2024-03-13T14:50:03.751 INFO:tasks.workunit.client.1.smithi196.stdout:8/895: fdatasync d0/d16/d71/f89 0 2024-03-13T14:50:03.752 INFO:tasks.workunit.client.1.smithi196.stdout:5/985: rmdir d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b 39 2024-03-13T14:50:03.754 INFO:tasks.workunit.client.0.smithi176.stdout:9/985: creat d0/d1e/f152 x:0 0 0 2024-03-13T14:50:03.758 INFO:tasks.workunit.client.0.smithi176.stdout:6/899: dwrite d0/d10/d19/d27/f61 [4194304,4194304] 0 2024-03-13T14:50:03.761 INFO:tasks.workunit.client.0.smithi176.stdout:6/900: write d0/de/d38/fe9 [536552,49732] 0 2024-03-13T14:50:03.764 INFO:tasks.workunit.client.0.smithi176.stdout:0/762: creat d2/d74/ff4 x:0 0 0 2024-03-13T14:50:03.765 INFO:tasks.workunit.client.0.smithi176.stdout:6/901: chown d0/de/d8d/fbd 32161941 1 2024-03-13T14:50:03.765 INFO:tasks.workunit.client.1.smithi196.stdout:3/933: dread db/f2c [0,4194304] 0 2024-03-13T14:50:03.766 INFO:tasks.workunit.client.1.smithi196.stdout:2/908: dwrite d0/d7/d9/d5a/f7c [0,4194304] 0 2024-03-13T14:50:03.767 INFO:tasks.workunit.client.0.smithi176.stdout:5/845: mkdir d5/d31/d86/dc0/d11a/d120 0 2024-03-13T14:50:03.767 INFO:tasks.workunit.client.0.smithi176.stdout:2/900: getdents d1/dc/d2d/d59/d80/d5c/d78 0 2024-03-13T14:50:03.771 INFO:tasks.workunit.client.0.smithi176.stdout:1/849: dwrite d2/f21 [4194304,4194304] 0 2024-03-13T14:50:03.771 INFO:tasks.workunit.client.0.smithi176.stdout:4/868: creat d7/dab/f121 x:0 0 0 2024-03-13T14:50:03.771 INFO:tasks.workunit.client.0.smithi176.stdout:5/846: stat d5/d2b/fcd 0 2024-03-13T14:50:03.772 INFO:tasks.workunit.client.1.smithi196.stdout:3/934: write db/d19/dc1/f121 [462320,100966] 0 2024-03-13T14:50:03.773 INFO:tasks.workunit.client.1.smithi196.stdout:9/940: dwrite d7/de/d4b/f12a [0,4194304] 0 2024-03-13T14:50:03.774 INFO:tasks.workunit.client.0.smithi176.stdout:9/986: write d0/db/d89/d4d/f126 [358898,20808] 0 2024-03-13T14:50:03.775 INFO:tasks.workunit.client.0.smithi176.stdout:2/901: chown d1/dc/d2d/d4f/l6a 11 1 2024-03-13T14:50:03.776 INFO:tasks.workunit.client.0.smithi176.stdout:6/902: mkdir d0/d10/d19/d27/d11d 0 2024-03-13T14:50:03.781 INFO:tasks.workunit.client.0.smithi176.stdout:3/903: dwrite d4/d13/d81/fb5 [0,4194304] 0 2024-03-13T14:50:03.781 INFO:tasks.workunit.client.0.smithi176.stdout:9/987: readlink d0/db/d89/d32/d58/d67/d119/l11b 0 2024-03-13T14:50:03.784 INFO:tasks.workunit.client.1.smithi196.stdout:4/971: dwrite d1/d3/d4c/d72/daf/f14c [0,4194304] 0 2024-03-13T14:50:03.786 INFO:tasks.workunit.client.0.smithi176.stdout:2/902: fdatasync d1/d2a/f82 0 2024-03-13T14:50:03.786 INFO:tasks.workunit.client.0.smithi176.stdout:4/869: write d7/d8/f16 [1693095,54468] 0 2024-03-13T14:50:03.787 INFO:tasks.workunit.client.1.smithi196.stdout:3/935: mknod db/d15/dfb/c12c 0 2024-03-13T14:50:03.787 INFO:tasks.workunit.client.0.smithi176.stdout:0/763: getdents d2/db/d35/daa/daf 0 2024-03-13T14:50:03.791 INFO:tasks.workunit.client.1.smithi196.stdout:8/896: link d0/d10/d12/df5/d47/ca3 d0/d10/da5/de4/c129 0 2024-03-13T14:50:03.794 INFO:tasks.workunit.client.0.smithi176.stdout:2/903: read d1/d2a/d33/f5b [2913149,69224] 0 2024-03-13T14:50:03.794 INFO:tasks.workunit.client.0.smithi176.stdout:1/850: rename d2/d30/d63/ff4 to d2/de/d3f/d19/f102 0 2024-03-13T14:50:03.794 INFO:tasks.workunit.client.0.smithi176.stdout:5/847: unlink d5/d2e/d7e/dad/d7b/c91 0 2024-03-13T14:50:03.798 INFO:tasks.workunit.client.1.smithi196.stdout:4/972: stat d1/d3/d7/d35/d76/d58/d69/de3/l12a 0 2024-03-13T14:50:03.802 INFO:tasks.workunit.client.1.smithi196.stdout:5/986: dwrite d4/d1b/dc8/d14/d19/d1c/d2d/d82/d8b/dbd/f11b [0,4194304] 0 2024-03-13T14:50:03.805 INFO:tasks.workunit.client.1.smithi196.stdout:2/909: write d0/f37 [3429003,112108] 0 2024-03-13T14:50:03.813 INFO:tasks.workunit.client.0.smithi176.stdout:0/764: dread d2/db/d39/d9c/fb0 [0,4194304] 0 2024-03-13T14:50:03.814 INFO:tasks.workunit.client.1.smithi196.stdout:4/973: symlink d1/d3/d7/d35/d76/d58/d69/d9f/dd5/l15c 0 2024-03-13T14:50:03.816 INFO:tasks.workunit.client.0.smithi176.stdout:0/765: dread - d2/d5/dc/fd8 zero size 2024-03-13T14:50:03.816 INFO:tasks.workunit.client.0.smithi176.stdout:6/903: dwrite d0/d10/d19/d27/d2b/d2e/f62 [0,4194304] 0 2024-03-13T14:50:03.817 INFO:tasks.workunit.client.1.smithi196.stdout:3/936: dread db/d19/d22/d3a/f8f [0,4194304] 0 2024-03-13T14:50:03.818 INFO:tasks.workunit.client.0.smithi176.stdout:4/870: symlink d7/d10/d46/d4a/dcc/dd0/d11d/l122 0 2024-03-13T14:50:03.819 INFO:tasks.workunit.client.0.smithi176.stdout:1/851: read d2/de/d3f/d50/f78 [1489666,58661] 0 2024-03-13T14:50:03.823 INFO:tasks.workunit.client.0.smithi176.stdout:9/988: write d0/db/d89/d4d/d5d/fce [997023,128736] 0 2024-03-13T14:50:03.824 INFO:tasks.workunit.client.1.smithi196.stdout:8/897: dwrite d0/f56 [4194304,4194304] 0 2024-03-13T14:50:03.827 INFO:tasks.workunit.client.1.smithi196.stdout:9/941: dwrite d7/de/d74/d94/dfb/dbf/f100 [0,4194304] 0 2024-03-13T14:50:03.827 INFO:tasks.workunit.client.1.smithi196.stdout:5/987: fsync d4/d1b/f29 0 2024-03-13T14:50:03.830 INFO:tasks.workunit.client.1.smithi196.stdout:8/898: write d0/d1c/f7c [1381301,28135] 0 2024-03-13T14:50:03.831 INFO:tasks.workunit.client.1.smithi196.stdout:4/974: fdatasync d1/d3/d7/d35/f40 0 2024-03-13T14:50:03.832 INFO:tasks.workunit.client.0.smithi176.stdout:6/904: mknod d0/d10/d19/d7d/c11e 0 2024-03-13T14:50:03.835 INFO:tasks.workunit.client.1.smithi196.stdout:9/942: chown d7/de/d74/fad 2 1 2024-03-13T14:50:03.838 INFO:tasks.workunit.client.1.smithi196.stdout:8/899: chown d0/d10/d12/d17/d42/dbe/d106 7 1 2024-03-13T14:50:03.840 INFO:tasks.workunit.client.1.smithi196.stdout:2/910: dread d0/d7/d9/d42/d98/de4/d104/f7a [0,4194304] 0 2024-03-13T14:50:03.843 INFO:tasks.workunit.client.1.smithi196.stdout:8/900: dread - d0/d10/d12/d1e/dc5/f124 zero size 2024-03-13T14:50:03.843 INFO:tasks.workunit.client.0.smithi176.stdout:4/871: dread - d7/d10/d46/d4a/d5d/da1/f89 zero size 2024-03-13T14:50:03.848 INFO:tasks.workunit.client.0.smithi176.stdout:5/848: creat d5/d2c/d40/f121 x:0 0 0 2024-03-13T14:50:03.849 INFO:tasks.workunit.client.0.smithi176.stdout:8/807: dread d0/d5/d11/f7f [0,4194304] 0 2024-03-13T14:50:03.851 INFO:tasks.workunit.client.0.smithi176.stdout:7/822: sync 2024-03-13T14:50:03.852 INFO:tasks.workunit.client.0.smithi176.stdout:0/766: write d2/d5/dc/d59/f80 [1443410,67285] 0 2024-03-13T14:50:03.855 INFO:tasks.workunit.client.0.smithi176.stdout:0/767: fsync d2/d74/fe2 0 2024-03-13T14:50:03.855 INFO:tasks.workunit.client.0.smithi176.stdout:5/849: write d5/d2c/d40/dc7/d89/fa1 [4717873,129801] 0 2024-03-13T14:50:03.855 INFO:tasks.workunit.client.0.smithi176.stdout:3/904: getdents d4/d13/d16/dad 0 2024-03-13T14:50:03.856 INFO:tasks.workunit.client.1.smithi196.stdout:9/943: truncate d7/de/d4b/d1d/d93/da7/fd0 130212 0 2024-03-13T14:50:03.857 INFO:tasks.workunit.client.1.smithi196.stdout:9/944: chown d7/de/d16/f60 32567943 1 2024-03-13T14:50:03.858 INFO:tasks.workunit.client.0.smithi176.stdout:1/852: link d2/de/d10/d16/d4a/d5f/d66/fe1 d2/d29/f103 0 2024-03-13T14:50:03.859 INFO:tasks.workunit.client.0.smithi176.stdout:0/768: stat d2/d5/d28/ce4 0 2024-03-13T14:50:03.859 INFO:tasks.workunit.client.1.smithi196.stdout:2/911: mknod d0/d7/d56/c127 0 2024-03-13T14:50:03.859 INFO:tasks.workunit.client.1.smithi196.stdout:2/912: fdatasync d0/d6/d34/f111 0 2024-03-13T14:50:03.861 INFO:tasks.workunit.client.0.smithi176.stdout:8/808: creat d0/d24/d29/d82/f105 x:0 0 0 2024-03-13T14:50:03.866 INFO:tasks.workunit.client.0.smithi176.stdout:2/904: dread d1/dc/d2d/d59/d80/d77/f84 [4194304,4194304] 0 2024-03-13T14:50:03.867 INFO:tasks.workunit.client.0.smithi176.stdout:7/823: creat d1/f103 x:0 0 0 2024-03-13T14:50:03.868 INFO:tasks.workunit.client.0.smithi176.stdout:5/850: dread d5/d2c/d40/dc7/d53/de9/ffd [0,4194304] 0 2024-03-13T14:50:03.869 INFO:tasks.workunit.client.1.smithi196.stdout:2/913: rename d0/d6/d90/c92 to d0/d7/d40/c128 0 2024-03-13T14:50:03.872 INFO:tasks.workunit.client.0.smithi176.stdout:1/853: creat d2/de/d3f/d50/d83/dcb/f104 x:0 0 0 2024-03-13T14:50:03.872 INFO:tasks.workunit.client.1.smithi196.stdout:3/937: truncate db/d19/f2b 4464487 0 2024-03-13T14:50:03.874 INFO:tasks.workunit.client.1.smithi196.stdout:9/945: unlink d7/d9/f12f 0 2024-03-13T14:50:03.875 INFO:tasks.workunit.client.1.smithi196.stdout:8/901: getdents d0/d16/d3d/d9c/d87 0 2024-03-13T14:50:03.876 INFO:tasks.workunit.client.0.smithi176.stdout:6/905: link d0/d10/d19/c90 d0/d10/d19/c11f 0 2024-03-13T14:50:03.877 INFO:tasks.workunit.client.1.smithi196.stdout:2/914: creat d0/d6/d4b/d96/d9e/f129 x:0 0 0 2024-03-13T14:50:03.879 INFO:tasks.workunit.client.0.smithi176.stdout:8/809: rename d0/d24/d98/dc6/f8a to d0/db/f106 0 2024-03-13T14:50:03.881 INFO:tasks.workunit.client.0.smithi176.stdout:9/989: getdents d0/d1e/d8d/dbe 0 2024-03-13T14:50:03.883 INFO:tasks.workunit.client.0.smithi176.stdout:6/906: dread d0/d10/d19/d27/d2b/d2e/f40 [0,4194304] 0 2024-03-13T14:50:03.885 INFO:tasks.workunit.client.0.smithi176.stdout:4/872: dread d7/d8/d15/d23/f35 [0,4194304] 0 2024-03-13T14:50:03.885 INFO:tasks.workunit.client.0.smithi176.stdout:5/851: chown d5/d2c/d40/dc7/d89/fc4 1094002936 1 2024-03-13T14:50:03.886 INFO:tasks.workunit.client.0.smithi176.stdout:1/854: chown d2/de/d10/d16/lf2 3154 1 2024-03-13T14:50:03.887 INFO:tasks.workunit.client.0.smithi176.stdout:5/852: dread - d5/d2e/d7e/ff5 zero size 2024-03-13T14:50:03.889 INFO:tasks.workunit.client.1.smithi196.stdout:5/988: dwrite d4/d1b/dc8/f24 [4194304,4194304] 0 2024-03-13T14:50:03.889 INFO:tasks.workunit.client.1.smithi196.stdout:3/938: mknod db/d37/c12d 0 2024-03-13T14:50:03.892 INFO:tasks.workunit.client.0.smithi176.stdout:8/810: read d0/d5/f20 [925857,95907] 0 2024-03-13T14:50:03.892 INFO:tasks.workunit.client.1.smithi196.stdout:3/939: chown db/d15/d44/d51/fc6 84 1 2024-03-13T14:50:03.892 INFO:tasks.workunit.client.1.smithi196.stdout:4/975: dwrite d1/d3/d7/ddf/f127 [0,4194304] 0 2024-03-13T14:50:03.893 INFO:tasks.workunit.client.0.smithi176.stdout:7/824: rename d1/d28/d3a/f5c to d1/d28/d3a/d1b/d8f/dc1/ddb/f104 0 2024-03-13T14:50:03.893 INFO:tasks.workunit.client.1.smithi196.stdout:3/940: stat db/d15/l2a 0 2024-03-13T14:50:03.894 INFO:tasks.workunit.client.1.smithi196.stdout:9/946: mkdir d7/de/d74/d94/dfb/d28/d13d 0 2024-03-13T14:50:03.896 INFO:tasks.workunit.client.0.smithi176.stdout:9/990: unlink d0/d1e/f147 0 2024-03-13T14:50:03.898 INFO:tasks.workunit.client.0.smithi176.stdout:9/991: chown d0/d1e/d6f/lec 0 1 2024-03-13T14:50:03.899 INFO:tasks.workunit.client.1.smithi196.stdout:8/902: mknod d0/c12a 0 2024-03-13T14:50:03.900 INFO:tasks.workunit.client.0.smithi176.stdout:2/905: creat d1/dc/d2d/d38/d12e/f131 x:0 0 0 2024-03-13T14:50:03.905 INFO:tasks.workunit.client.0.smithi176.stdout:0/769: link d2/d5/dc/lf3 d2/d74/d61/lf5 0 2024-03-13T14:50:03.906 INFO:tasks.workunit.client.1.smithi196.stdout:2/915: mknod d0/d7/d9/d42/c12a 0 2024-03-13T14:50:03.908 INFO:tasks.workunit.client.0.smithi176.stdout:0/770: write d2/d5/d28/d8c/fcc [46068,38709] 0 2024-03-13T14:50:03.908 INFO:tasks.workunit.client.0.smithi176.stdout:6/907: dread d0/de/d38/d88/d9d/fd7 [0,4194304] 0 2024-03-13T14:50:03.908 INFO:tasks.workunit.client.1.smithi196.stdout:5/989: fsync d4/d1b/dc8/d3c/fc3 0 2024-03-13T14:50:03.910 INFO:tasks.workunit.client.1.smithi196.stdout:5/990: fdatasync d4/d1b/dc8/f81 0 2024-03-13T14:50:03.911 INFO:tasks.workunit.client.0.smithi176.stdout:1/855: creat d2/de/d3f/d3e/f105 x:0 0 0 2024-03-13T14:50:03.911 INFO:tasks.workunit.client.0.smithi176.stdout:6/908: chown d0/d10/d19/d27/d2b/de0/d76/fba 19543965 1 2024-03-13T14:50:03.911 INFO:tasks.workunit.client.1.smithi196.stdout:8/903: read d0/d10/d12/df5/fa6 [30843,33902] 0 2024-03-13T14:50:03.913 INFO:tasks.workunit.client.0.smithi176.stdout:5/853: mkdir d5/d2c/d74/d122 0 2024-03-13T14:50:03.914 INFO:tasks.workunit.client.1.smithi196.stdout:3/941: read - db/d37/ffa zero size 2024-03-13T14:50:03.920 INFO:tasks.workunit.client.0.smithi176.stdout:9/992: fsync d0/d17/fa3 0 2024-03-13T14:50:03.923 INFO:tasks.workunit.client.0.smithi176.stdout:2/906: truncate d1/dc/d2d/d59/d80/d76/fd9 534042 0 2024-03-13T14:50:03.924 INFO:tasks.workunit.client.0.smithi176.stdout:2/907: write d1/dc/d1c/f102 [651014,11543] 0 2024-03-13T14:50:03.925 INFO:tasks.workunit.client.0.smithi176.stdout:2/908: readlink d1/d2a/l65 0 2024-03-13T14:50:03.931 INFO:tasks.workunit.client.0.smithi176.stdout:0/771: dread d2/d5/dc/d20/f34 [0,4194304] 0 2024-03-13T14:50:03.933 INFO:tasks.workunit.client.0.smithi176.stdout:0/772: write d2/d5/d28/d8c/d9d/ff0 [673739,25996] 0 2024-03-13T14:50:03.939 INFO:tasks.workunit.client.0.smithi176.stdout:8/811: dread d0/d5/d11/f5f [0,4194304] 0 2024-03-13T14:50:03.941 INFO:tasks.workunit.client.0.smithi176.stdout:9/993: unlink d0/db/d89/d32/d58/d5f/ccf 0 2024-03-13T14:50:03.941 INFO:tasks.workunit.client.1.smithi196.stdout:9/947: dwrite d7/de/d74/ffe [0,4194304] 0 2024-03-13T14:50:03.943 INFO:tasks.workunit.client.0.smithi176.stdout:5/854: dwrite d5/d2c/d40/d76/f95 [0,4194304] 0 2024-03-13T14:50:03.943 INFO:tasks.workunit.client.0.smithi176.stdout:4/873: creat d7/d10/d46/d4a/dcc/dd0/d11d/f123 x:0 0 0 2024-03-13T14:50:03.945 INFO:tasks.workunit.client.0.smithi176.stdout:2/909: mkdir d1/dc/d1c/d107/d132 0 2024-03-13T14:50:03.949 INFO:tasks.workunit.client.0.smithi176.stdout:0/773: stat d2/d5/f46 0 2024-03-13T14:50:03.955 INFO:tasks.workunit.client.0.smithi176.stdout:7/825: creat d1/d1c/d4a/f105 x:0 0 0 2024-03-13T14:50:03.955 INFO:tasks.workunit.client.0.smithi176.stdout:1/856: write d2/de/d10/d16/d47/fbb [232080,1136] 0 2024-03-13T14:50:03.958 INFO:tasks.workunit.client.0.smithi176.stdout:9/994: mkdir d0/d1e/df7/d11f/d153 0 2024-03-13T14:50:03.959 INFO:tasks.workunit.client.0.smithi176.stdout:4/874: mkdir d7/d10/d2c/d124 0 2024-03-13T14:50:03.960 INFO:tasks.workunit.client.0.smithi176.stdout:4/875: stat d7/d8/l13 0 2024-03-13T14:50:03.961 INFO:tasks.workunit.client.0.smithi176.stdout:6/909: link d0/d10/dca/de1/f11b d0/d10/d19/d27/d2b/de0/d76/f120 0 2024-03-13T14:50:03.964 INFO:tasks.workunit.client.0.smithi176.stdout:2/910: mknod d1/dc/de0/d12f/c133 0 2024-03-13T14:50:03.965 INFO:tasks.workunit.client.0.smithi176.stdout:7/826: creat d1/d28/d3a/d1b/d8f/dd1/f106 x:0 0 0 2024-03-13T14:50:03.968 INFO:tasks.workunit.client.0.smithi176.stdout:4/876: creat d7/db9/dd8/f125 x:0 0 0 2024-03-13T14:50:03.970 INFO:tasks.workunit.client.0.smithi176.stdout:6/910: mkdir d0/de/d38/d88/d9d/d121 0 2024-03-13T14:50:03.970 INFO:tasks.workunit.client.0.smithi176.stdout:4/877: dread - d7/dab/f121 zero size 2024-03-13T14:50:03.976 INFO:tasks.workunit.client.0.smithi176.stdout:1/857: dwrite d2/de/d10/d16/f27 [0,4194304] 0 2024-03-13T14:50:03.980 INFO:tasks.workunit.client.0.smithi176.stdout:8/812: getdents d0/db/d39/d56/d93/da7 0 2024-03-13T14:50:03.980 INFO:tasks.workunit.client.0.smithi176.stdout:1/858: write d2/de/d3f/d3e/d86/db3/fc9 [1801036,106928] 0 2024-03-13T14:50:03.981 INFO:tasks.workunit.client.0.smithi176.stdout:8/813: chown d0/d24/d29/d70 215724 1 2024-03-13T14:50:03.983 INFO:tasks.workunit.client.0.smithi176.stdout:2/911: creat d1/dc/d2d/d59/d8b/f134 x:0 0 0 2024-03-13T14:50:03.985 INFO:tasks.workunit.client.0.smithi176.stdout:2/912: chown d1/c24 2914 1 2024-03-13T14:50:03.989 INFO:tasks.workunit.client.0.smithi176.stdout:4/878: dread d7/d10/d46/f4c [0,4194304] 0 2024-03-13T14:50:03.994 INFO:tasks.workunit.client.0.smithi176.stdout:7/827: link d1/d1c/d4a/l6c d1/d28/d3a/d1b/d8f/dc1/ddb/l107 0 2024-03-13T14:50:03.995 INFO:tasks.workunit.client.0.smithi176.stdout:0/774: dread d2/d5/f87 [0,4194304] 0 2024-03-13T14:50:03.996 INFO:tasks.workunit.client.0.smithi176.stdout:7/828: fsync d1/f103 0 2024-03-13T14:50:03.998 INFO:tasks.workunit.client.1.smithi196.stdout:2/916: rmdir d0/d7/d56/dc8 39 2024-03-13T14:50:04.002 INFO:tasks.workunit.client.0.smithi176.stdout:8/814: dwrite d0/d24/d98/fed [0,4194304] 0 2024-03-13T14:50:04.003 INFO:tasks.workunit.client.1.smithi196.stdout:4/976: mkdir d1/d3/d7/d35/d76/d15d 0 2024-03-13T14:50:04.003 INFO:tasks.workunit.client.0.smithi176.stdout:4/879: dread d7/d2d/f6b [0,4194304] 0 2024-03-13T14:50:04.004 INFO:tasks.workunit.client.1.smithi196.stdout:4/977: chown d1/dfb/d11c/d13d/c13e 124133285 1 2024-03-13T14:50:04.009 INFO:tasks.workunit.client.1.smithi196.stdout:5/991: creat d4/d1b/dc8/d14/d19/da7/d126/f138 x:0 0 0 2024-03-13T14:50:04.011 INFO:tasks.workunit.client.0.smithi176.stdout:8/815: symlink d0/d5/d11/d47/l107 0 2024-03-13T14:50:04.013 INFO:tasks.workunit.client.1.smithi196.stdout:3/942: fsync db/d19/d22/d3a/d53/f48 0 2024-03-13T14:50:04.022 INFO:tasks.workunit.client.0.smithi176.stdout:9/995: dwrite d0/d40/f113 [0,4194304] 0 2024-03-13T14:50:04.022 INFO:tasks.workunit.client.0.smithi176.stdout:4/880: rename d7/d8/d15/d23/f11f to d7/d10/d2c/f126 0 2024-03-13T14:50:04.023 INFO:tasks.workunit.client.1.smithi196.stdout:9/948: creat d7/de/d4b/f13e x:0 0 0 2024-03-13T14:50:04.025 INFO:tasks.workunit.client.0.smithi176.stdout:9/996: write d0/db/d89/d32/d58/d67/f139 [1026328,29394] 0 2024-03-13T14:50:04.028 INFO:tasks.workunit.client.1.smithi196.stdout:2/917: mkdir d0/d6/d4b/d5f/d12b 0 2024-03-13T14:50:04.034 INFO:tasks.workunit.client.0.smithi176.stdout:4/881: creat d7/d8/d33/d96/d111/f127 x:0 0 0 2024-03-13T14:50:04.035 INFO:tasks.workunit.client.1.smithi196.stdout:8/904: mkdir d0/d10/d70/d12b 0 2024-03-13T14:50:04.035 INFO:tasks.workunit.client.0.smithi176.stdout:6/911: dwrite d0/de/fcc [0,4194304] 0 2024-03-13T14:50:04.036 INFO:tasks.workunit.client.0.smithi176.stdout:2/913: dread d1/dc/d2d/d59/d80/d77/f84 [0,4194304] 0 2024-03-13T14:50:04.037 INFO:tasks.workunit.client.1.smithi196.stdout:5/992: chown d4/d1b/dc8/d14/d19/d26/dab/fb7 3 1 2024-03-13T14:50:04.042 INFO:tasks.workunit.client.1.smithi196.stdout:3/943: fdatasync db/d19/d22/f31 0 2024-03-13T14:50:04.042 INFO:tasks.workunit.client.1.smithi196.stdout:5/993: write d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d63/db2/f116 [1278191,39219] 0 2024-03-13T14:50:04.043 INFO:tasks.workunit.client.0.smithi176.stdout:6/912: fsync d0/d10/dca/dd2/fe3 0 2024-03-13T14:50:04.043 INFO:tasks.workunit.client.0.smithi176.stdout:7/829: rename d1/d1c/d26/d41/d2b/d40/lf0 to d1/d28/d3a/d1b/d8f/dc1/db7/df9/l108 0 2024-03-13T14:50:04.044 INFO:tasks.workunit.client.0.smithi176.stdout:1/859: write d2/de/d3f/fcc [410787,58880] 0 2024-03-13T14:50:04.045 INFO:tasks.workunit.client.0.smithi176.stdout:9/997: getdents d0/db/d89/d4d/dd5 0 2024-03-13T14:50:04.046 INFO:tasks.workunit.client.1.smithi196.stdout:2/918: read d0/d7/d9/d9a/daa/fb8 [1194238,16518] 0 2024-03-13T14:50:04.048 INFO:tasks.workunit.client.1.smithi196.stdout:2/919: chown d0/d6/d34/d5b/f95 1 1 2024-03-13T14:50:04.049 INFO:tasks.workunit.client.0.smithi176.stdout:9/998: fsync d0/db/d89/d32/d58/d67/f139 0 2024-03-13T14:50:04.050 INFO:tasks.workunit.client.0.smithi176.stdout:0/775: write d2/fa6 [162735,21791] 0 2024-03-13T14:50:04.051 INFO:tasks.workunit.client.1.smithi196.stdout:2/920: chown d0/d7/fd4 11188557 1 2024-03-13T14:50:04.052 INFO:tasks.workunit.client.1.smithi196.stdout:8/905: creat d0/d10/d12/d1e/d2d/f12c x:0 0 0 2024-03-13T14:50:04.055 INFO:tasks.workunit.client.1.smithi196.stdout:8/906: write d0/d10/d79/f118 [61877,61711] 0 2024-03-13T14:50:04.056 INFO:tasks.workunit.client.0.smithi176.stdout:4/882: dwrite d7/d8/f4f [4194304,4194304] 0 2024-03-13T14:50:04.057 INFO:tasks.workunit.client.1.smithi196.stdout:3/944: stat db/d19/f93 0 2024-03-13T14:50:04.059 INFO:tasks.workunit.client.0.smithi176.stdout:8/816: write d0/d5/d11/d47/f7a [1033087,64827] 0 2024-03-13T14:50:04.063 INFO:tasks.workunit.client.1.smithi196.stdout:4/978: creat d1/d3/d7/d35/d76/d58/f15e x:0 0 0 2024-03-13T14:50:04.063 INFO:tasks.workunit.client.0.smithi176.stdout:8/817: dread d0/db/fbc [0,4194304] 0 2024-03-13T14:50:04.065 INFO:tasks.workunit.client.1.smithi196.stdout:8/907: fsync d0/d16/d3d/d8b/f4b 0 2024-03-13T14:50:04.066 INFO:tasks.workunit.client.1.smithi196.stdout:5/994: mkdir d4/d1b/dc8/d14/d19/d1c/d94/d139 0 2024-03-13T14:50:04.069 INFO:tasks.workunit.client.0.smithi176.stdout:6/913: dread d0/de/d38/f80 [0,4194304] 0 2024-03-13T14:50:04.069 INFO:tasks.workunit.client.1.smithi196.stdout:8/908: chown d0/d10/d12/d17/d42/dbe/l102 51099631 1 2024-03-13T14:50:04.071 INFO:tasks.workunit.client.0.smithi176.stdout:4/883: rmdir d7/d10/d46 39 2024-03-13T14:50:04.074 INFO:tasks.workunit.client.0.smithi176.stdout:2/914: dwrite d1/dc/d2d/d4b/d69/fb4 [0,4194304] 0 2024-03-13T14:50:04.074 INFO:tasks.workunit.client.0.smithi176.stdout:6/914: stat d0/de/d38/d6b/l71 0 2024-03-13T14:50:04.074 INFO:tasks.workunit.client.1.smithi196.stdout:2/921: truncate d0/d7/d9/d42/d98/de4/d67/fcc 525295 0 2024-03-13T14:50:04.077 INFO:tasks.workunit.client.0.smithi176.stdout:2/915: chown d1/dc/l1f 1820184327 1 2024-03-13T14:50:04.077 INFO:tasks.workunit.client.1.smithi196.stdout:9/949: dread d7/de/f29 [0,4194304] 0 2024-03-13T14:50:04.077 INFO:tasks.workunit.client.1.smithi196.stdout:2/922: truncate d0/d6/d4b/f123 12494 0 2024-03-13T14:50:04.080 INFO:tasks.workunit.client.0.smithi176.stdout:0/776: mknod d2/d5/d1e/d97/cf6 0 2024-03-13T14:50:04.080 INFO:tasks.workunit.client.1.smithi196.stdout:5/995: fdatasync d4/d1b/dc8/d14/d19/f98 0 2024-03-13T14:50:04.085 INFO:tasks.workunit.client.1.smithi196.stdout:8/909: rename d0/d10/d12/d1e/f6a to d0/d10/da5/de4/f12d 0 2024-03-13T14:50:04.089 INFO:tasks.workunit.client.1.smithi196.stdout:4/979: mkdir d1/d15f 0 2024-03-13T14:50:04.089 INFO:tasks.workunit.client.1.smithi196.stdout:3/945: write db/d19/d22/d3a/d68/d8e/f116 [475460,57737] 0 2024-03-13T14:50:04.090 INFO:tasks.workunit.client.0.smithi176.stdout:2/916: truncate d1/dc/f34 3895929 0 2024-03-13T14:50:04.091 INFO:tasks.workunit.client.1.smithi196.stdout:3/946: chown db/fbc 23 1 2024-03-13T14:50:04.093 INFO:tasks.workunit.client.0.smithi176.stdout:1/860: dwrite d2/de/f26 [0,4194304] 0 2024-03-13T14:50:04.094 INFO:tasks.workunit.client.0.smithi176.stdout:7/830: dread d1/d1c/d4a/fa7 [0,4194304] 0 2024-03-13T14:50:04.095 INFO:tasks.workunit.client.1.smithi196.stdout:9/950: creat d7/de/d74/d94/dfb/d28/d6c/d7b/f13f x:0 0 0 2024-03-13T14:50:04.098 INFO:tasks.workunit.client.0.smithi176.stdout:4/884: chown d7/d10/d46/d4a/d5d/fda 0 1 2024-03-13T14:50:04.099 INFO:tasks.workunit.client.1.smithi196.stdout:8/910: rmdir d0/d63 39 2024-03-13T14:50:04.101 INFO:tasks.workunit.client.1.smithi196.stdout:5/996: creat d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/f13a x:0 0 0 2024-03-13T14:50:04.102 INFO:tasks.workunit.client.0.smithi176.stdout:8/818: dwrite d0/d24/d98/dc6/f6b [4194304,4194304] 0 2024-03-13T14:50:04.102 INFO:tasks.workunit.client.0.smithi176.stdout:6/915: dread d0/d10/d19/d27/d2b/de0/d76/fab [0,4194304] 0 2024-03-13T14:50:04.102 INFO:tasks.workunit.client.0.smithi176.stdout:9/999: link d0/d1e/la9 d0/d1e/d6f/l154 0 2024-03-13T14:50:04.103 INFO:tasks.workunit.client.0.smithi176.stdout:8/819: chown d0/fdf 3583427 1 2024-03-13T14:50:04.105 INFO:tasks.workunit.client.1.smithi196.stdout:2/923: creat d0/d6/d4b/d5f/d12b/f12c x:0 0 0 2024-03-13T14:50:04.105 INFO:tasks.workunit.client.1.smithi196.stdout:5/997: chown d4/d1b/dc8/d14/d19/d1c/d2d/dc5/lc9 163260 1 2024-03-13T14:50:04.110 INFO:tasks.workunit.client.0.smithi176.stdout:0/777: write d2/d5/d28/fb5 [1018150,81500] 0 2024-03-13T14:50:04.114 INFO:tasks.workunit.client.1.smithi196.stdout:3/947: truncate db/d19/d22/d3a/d53/f59 4872517 0 2024-03-13T14:50:04.115 INFO:tasks.workunit.client.0.smithi176.stdout:7/831: mknod d1/d1c/d26/d41/d2b/ddc/c109 0 2024-03-13T14:50:04.116 INFO:tasks.workunit.client.0.smithi176.stdout:3/905: sync 2024-03-13T14:50:04.117 INFO:tasks.workunit.client.0.smithi176.stdout:4/885: write d7/d10/fcb [121079,57116] 0 2024-03-13T14:50:04.120 INFO:tasks.workunit.client.1.smithi196.stdout:2/924: chown d0/d7/d9/d61/lbf 21 1 2024-03-13T14:50:04.123 INFO:tasks.workunit.client.1.smithi196.stdout:2/925: write d0/d6/d4b/d5f/f106 [858277,72702] 0 2024-03-13T14:50:04.124 INFO:tasks.workunit.client.1.smithi196.stdout:4/980: rename d1/d3/d7/d35/d76/d58/d69/de3/df4/fcc to d1/d3/d7/d35/d76/d58/dc6/f160 0 2024-03-13T14:50:04.127 INFO:tasks.workunit.client.0.smithi176.stdout:6/916: dread - d0/d10/dca/de1/f11b zero size 2024-03-13T14:50:04.128 INFO:tasks.workunit.client.1.smithi196.stdout:2/926: write d0/d7/d9/d61/f118 [242437,20240] 0 2024-03-13T14:50:04.129 INFO:tasks.workunit.client.1.smithi196.stdout:5/998: truncate d4/d1b/dc8/d14/d19/d1c/d25/d38/d47/da0/d7d/f112 321087 0 2024-03-13T14:50:04.129 INFO:tasks.workunit.client.0.smithi176.stdout:8/820: unlink d0/d24/l5a 0 2024-03-13T14:50:04.132 INFO:tasks.workunit.client.0.smithi176.stdout:1/861: truncate d2/de/d3f/d50/d83/dcb/f9d 257676 0 2024-03-13T14:50:04.134 INFO:tasks.workunit.client.1.smithi196.stdout:2/927: write d0/d6/f108 [1027325,1540] 0 2024-03-13T14:50:04.136 INFO:tasks.workunit.client.1.smithi196.stdout:3/948: dwrite db/f36 [0,4194304] 0 2024-03-13T14:50:04.136 INFO:tasks.workunit.client.1.smithi196.stdout:5/999: chown d4/d1b/dc8/d14/d19/d26/d33/f8f 94 1 2024-03-13T14:50:04.137 INFO:tasks.workunit.client.0.smithi176.stdout:4/886: symlink d7/d10/d46/d4a/l128 0 2024-03-13T14:50:04.138 INFO:tasks.workunit.client.0.smithi176.stdout:6/917: dread d0/de/d38/d70/f9e [0,4194304] 0 2024-03-13T14:50:04.139 INFO:tasks.workunit.client.1.smithi196.stdout:4/981: creat d1/d3/d7/d35/d76/d58/d69/d6c/d14f/f161 x:0 0 0 2024-03-13T14:50:04.147 INFO:tasks.workunit.client.1.smithi196.stdout:3/949: rename db/d19/d50/db7/dff/d12b to db/d37/d64/d12e 0 2024-03-13T14:50:04.149 INFO:tasks.workunit.client.1.smithi196.stdout:2/928: mkdir d0/d7/d9/d42/d12d 0 2024-03-13T14:50:04.150 INFO:tasks.workunit.client.1.smithi196.stdout:9/951: dread d7/de/f40 [0,4194304] 0 2024-03-13T14:50:04.151 INFO:tasks.workunit.client.1.smithi196.stdout:4/982: mkdir d1/d3/d4c/d72/d7d/d80/d162 0 2024-03-13T14:50:04.153 INFO:tasks.workunit.client.0.smithi176.stdout:0/778: dwrite d2/db/d39/f7a [0,4194304] 0 2024-03-13T14:50:04.153 INFO:tasks.workunit.client.0.smithi176.stdout:7/832: creat d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/f10a x:0 0 0 2024-03-13T14:50:04.155 INFO:tasks.workunit.client.0.smithi176.stdout:6/918: mkdir d0/d10/d19/d27/d2b/de0/d76/d7c/dfd/d122 0 2024-03-13T14:50:04.156 INFO:tasks.workunit.client.1.smithi196.stdout:9/952: unlink d7/de/d4b/f13e 0 2024-03-13T14:50:04.157 INFO:tasks.workunit.client.0.smithi176.stdout:4/887: dwrite d7/d8/d45/ff3 [0,4194304] 0 2024-03-13T14:50:04.158 INFO:tasks.workunit.client.0.smithi176.stdout:7/833: truncate d1/d1c/d26/d41/d2b/fe6 440405 0 2024-03-13T14:50:04.158 INFO:tasks.workunit.client.1.smithi196.stdout:4/983: chown d1/d3/d7/d35/d76/d58/c128 961475037 1 2024-03-13T14:50:04.160 INFO:tasks.workunit.client.1.smithi196.stdout:2/929: fdatasync d0/d7/d9/d42/d98/de4/d104/f7a 0 2024-03-13T14:50:04.161 INFO:tasks.workunit.client.0.smithi176.stdout:2/917: getdents d1/dc/d2d/d90/dcf/da0 0 2024-03-13T14:50:04.163 INFO:tasks.workunit.client.0.smithi176.stdout:3/906: dread d4/d13/d5e/f9a [0,4194304] 0 2024-03-13T14:50:04.165 INFO:tasks.workunit.client.0.smithi176.stdout:2/918: write d1/dc/d2d/d38/d5d/f70 [142048,115018] 0 2024-03-13T14:50:04.166 INFO:tasks.workunit.client.1.smithi196.stdout:3/950: fdatasync db/d19/d4a/f94 0 2024-03-13T14:50:04.168 INFO:tasks.workunit.client.0.smithi176.stdout:7/834: dread d1/d28/d3a/d1b/d8f/f99 [0,4194304] 0 2024-03-13T14:50:04.169 INFO:tasks.workunit.client.0.smithi176.stdout:8/821: rename d0/d5/f4d to d0/d24/d7b/dec/f108 0 2024-03-13T14:50:04.169 INFO:tasks.workunit.client.0.smithi176.stdout:5/855: sync 2024-03-13T14:50:04.170 INFO:tasks.workunit.client.0.smithi176.stdout:7/835: chown d1/d28/d3a/d1b/d8f/dd1/df4 920863076 1 2024-03-13T14:50:04.171 INFO:tasks.workunit.client.0.smithi176.stdout:1/862: write f1 [31077,68295] 0 2024-03-13T14:50:04.172 INFO:tasks.workunit.client.0.smithi176.stdout:0/779: creat d2/d74/d9f/ff7 x:0 0 0 2024-03-13T14:50:04.174 INFO:tasks.workunit.client.0.smithi176.stdout:0/780: chown d2/d74/d61/cbd 214 1 2024-03-13T14:50:04.174 INFO:tasks.workunit.client.0.smithi176.stdout:6/919: creat d0/d10/d19/d27/d2b/d2e/f123 x:0 0 0 2024-03-13T14:50:04.176 INFO:tasks.workunit.client.1.smithi196.stdout:2/930: symlink d0/d6/d90/l12e 0 2024-03-13T14:50:04.176 INFO:tasks.workunit.client.0.smithi176.stdout:6/920: dread - d0/d10/d19/d27/d2b/d2e/f52 zero size 2024-03-13T14:50:04.177 INFO:tasks.workunit.client.1.smithi196.stdout:2/931: stat d0/d7/d9/d42/f80 0 2024-03-13T14:50:04.180 INFO:tasks.workunit.client.1.smithi196.stdout:3/951: unlink db/d19/d22/cee 0 2024-03-13T14:50:04.183 INFO:tasks.workunit.client.1.smithi196.stdout:9/953: mknod d7/de/d16/d70/d12e/c140 0 2024-03-13T14:50:04.185 INFO:tasks.workunit.client.1.smithi196.stdout:9/954: chown d7/de/d4b/f6a 0 1 2024-03-13T14:50:04.186 INFO:tasks.workunit.client.0.smithi176.stdout:3/907: symlink d4/d28/d31/d45/d82/d87/d102/l13a 0 2024-03-13T14:50:04.191 INFO:tasks.workunit.client.0.smithi176.stdout:5/856: creat d5/d2c/d40/dc7/d89/df2/f123 x:0 0 0 2024-03-13T14:50:04.191 INFO:tasks.workunit.client.0.smithi176.stdout:4/888: write d7/d8/d2f/f37 [632102,16754] 0 2024-03-13T14:50:04.192 INFO:tasks.workunit.client.1.smithi196.stdout:9/955: truncate d7/de/f40 1578095 0 2024-03-13T14:50:04.192 INFO:tasks.workunit.client.0.smithi176.stdout:7/836: dread - d1/d1c/d26/fa9 zero size 2024-03-13T14:50:04.192 INFO:tasks.workunit.client.0.smithi176.stdout:2/919: dread d1/dc/fae [0,4194304] 0 2024-03-13T14:50:04.194 INFO:tasks.workunit.client.1.smithi196.stdout:2/932: rename d0/d7/d9/d42/d98/de4/d104/lf0 to d0/d7/d9/d42/l12f 0 2024-03-13T14:50:04.195 INFO:tasks.workunit.client.0.smithi176.stdout:1/863: creat d2/de/d10/d16/d59/f106 x:0 0 0 2024-03-13T14:50:04.195 INFO:tasks.workunit.client.1.smithi196.stdout:2/933: stat d0/d7/d40/c4a 0 2024-03-13T14:50:04.199 INFO:tasks.workunit.client.0.smithi176.stdout:3/908: mknod d4/d13/d16/db6/c13b 0 2024-03-13T14:50:04.200 INFO:tasks.workunit.client.0.smithi176.stdout:3/909: stat d4/d28/c58 0 2024-03-13T14:50:04.202 INFO:tasks.workunit.client.0.smithi176.stdout:4/889: creat d7/d10/d46/d4a/dcc/f129 x:0 0 0 2024-03-13T14:50:04.203 INFO:tasks.workunit.client.1.smithi196.stdout:3/952: dread db/d19/d50/db7/f42 [0,4194304] 0 2024-03-13T14:50:04.204 INFO:tasks.workunit.client.0.smithi176.stdout:4/890: chown d7/d8/d33/d96/c10a 905594698 1 2024-03-13T14:50:04.204 INFO:tasks.workunit.client.0.smithi176.stdout:2/920: unlink d1/dc/l32 0 2024-03-13T14:50:04.206 INFO:tasks.workunit.client.1.smithi196.stdout:2/934: creat d0/d7/d9/f130 x:0 0 0 2024-03-13T14:50:04.212 INFO:tasks.workunit.client.0.smithi176.stdout:6/921: mkdir d0/d124 0 2024-03-13T14:50:04.213 INFO:tasks.workunit.client.0.smithi176.stdout:8/822: creat d0/d5/f109 x:0 0 0 2024-03-13T14:50:04.214 INFO:tasks.workunit.client.1.smithi196.stdout:9/956: dread d7/f20 [0,4194304] 0 2024-03-13T14:50:04.214 INFO:tasks.workunit.client.0.smithi176.stdout:3/910: creat d4/d13/d16/db6/de8/f13c x:0 0 0 2024-03-13T14:50:04.217 INFO:tasks.workunit.client.0.smithi176.stdout:2/921: fdatasync d1/dc/d2d/d4b/d69/fd7 0 2024-03-13T14:50:04.220 INFO:tasks.workunit.client.0.smithi176.stdout:0/781: dwrite d2/d5/d1e/f4e [0,4194304] 0 2024-03-13T14:50:04.223 INFO:tasks.workunit.client.0.smithi176.stdout:1/864: write d2/d29/f103 [722991,118224] 0 2024-03-13T14:50:04.224 INFO:tasks.workunit.client.1.smithi196.stdout:3/953: dwrite db/d19/d50/db7/da9/fb9 [0,4194304] 0 2024-03-13T14:50:04.225 INFO:tasks.workunit.client.0.smithi176.stdout:8/823: readlink d0/la 0 2024-03-13T14:50:04.228 INFO:tasks.workunit.client.0.smithi176.stdout:6/922: symlink d0/d10/d19/d27/d2b/d2e/d3e/db4/l125 0 2024-03-13T14:50:04.228 INFO:tasks.workunit.client.0.smithi176.stdout:8/824: chown d0/d24/d98/dc6/d40/f62 4115949 1 2024-03-13T14:50:04.230 INFO:tasks.workunit.client.0.smithi176.stdout:4/891: symlink d7/d8/d15/l12a 0 2024-03-13T14:50:04.234 INFO:tasks.workunit.client.0.smithi176.stdout:1/865: dread d2/de/d10/d16/d47/db4/fc1 [0,4194304] 0 2024-03-13T14:50:04.235 INFO:tasks.workunit.client.0.smithi176.stdout:5/857: link d5/d2e/d5f/fdf d5/d2c/d40/dc7/d53/de9/f124 0 2024-03-13T14:50:04.236 INFO:tasks.workunit.client.0.smithi176.stdout:1/866: dread - d2/de/d3f/d50/d83/d97/fde zero size 2024-03-13T14:50:04.238 INFO:tasks.workunit.client.0.smithi176.stdout:1/867: truncate d2/d30/dd3/feb 665603 0 2024-03-13T14:50:04.239 INFO:tasks.workunit.client.0.smithi176.stdout:7/837: rename d1/d28/d3a/d1b/l66 to d1/d1c/d26/d41/d2b/d40/l10b 0 2024-03-13T14:50:04.243 INFO:tasks.workunit.client.1.smithi196.stdout:9/957: dwrite d7/de/d4b/d1d/d22/f53 [0,4194304] 0 2024-03-13T14:50:04.246 INFO:tasks.workunit.client.0.smithi176.stdout:4/892: truncate d7/d10/d46/f4c 800157 0 2024-03-13T14:50:04.247 INFO:tasks.workunit.client.1.smithi196.stdout:3/954: rename db/d19/d50/db7/f123 to db/d19/d22/d3a/f12f 0 2024-03-13T14:50:04.250 INFO:tasks.workunit.client.1.smithi196.stdout:3/955: fdatasync db/d19/d4a/d9b/f11d 0 2024-03-13T14:50:04.250 INFO:tasks.workunit.client.0.smithi176.stdout:2/922: mkdir d1/dc/d2d/d135 0 2024-03-13T14:50:04.250 INFO:tasks.workunit.client.0.smithi176.stdout:4/893: write d7/ffb [351040,62583] 0 2024-03-13T14:50:04.250 INFO:tasks.workunit.client.0.smithi176.stdout:4/894: write d7/dab/f10b [1024024,19498] 0 2024-03-13T14:50:04.251 INFO:tasks.workunit.client.0.smithi176.stdout:0/782: mkdir d2/d74/df8 0 2024-03-13T14:50:04.251 INFO:tasks.workunit.client.1.smithi196.stdout:9/958: read d7/de/d4b/d1d/d93/da7/fb6 [55183,128493] 0 2024-03-13T14:50:04.258 INFO:tasks.workunit.client.0.smithi176.stdout:7/838: dread - d1/d28/d3a/d1b/fd9 zero size 2024-03-13T14:50:04.259 INFO:tasks.workunit.client.1.smithi196.stdout:9/959: creat d7/d71/f141 x:0 0 0 2024-03-13T14:50:04.259 INFO:tasks.workunit.client.0.smithi176.stdout:6/923: symlink d0/d10/d19/d27/d2b/de0/d76/d7c/dfd/d122/l126 0 2024-03-13T14:50:04.262 INFO:tasks.workunit.client.1.smithi196.stdout:9/960: write d7/de/d16/f99 [3681999,45739] 0 2024-03-13T14:50:04.264 INFO:tasks.workunit.client.0.smithi176.stdout:3/911: dwrite d4/d28/d34/da4/fa9 [0,4194304] 0 2024-03-13T14:50:04.264 INFO:tasks.workunit.client.0.smithi176.stdout:2/923: truncate d1/f96 1156557 0 2024-03-13T14:50:04.270 INFO:tasks.workunit.client.1.smithi196.stdout:9/961: unlink d7/de/d74/c7d 0 2024-03-13T14:50:04.271 INFO:tasks.workunit.client.1.smithi196.stdout:3/956: write db/d19/d22/d3a/d68/d8e/fd0 [1311033,3840] 0 2024-03-13T14:50:04.271 INFO:tasks.workunit.client.0.smithi176.stdout:0/783: unlink d2/db/d31/fd7 0 2024-03-13T14:50:04.271 INFO:tasks.workunit.client.0.smithi176.stdout:1/868: write d2/de/dbf/fa5 [2226941,25280] 0 2024-03-13T14:50:04.278 INFO:tasks.workunit.client.1.smithi196.stdout:9/962: rmdir d7/de/d74/d94/dfb/d28/d6c/d7b 39 2024-03-13T14:50:04.278 INFO:tasks.workunit.client.0.smithi176.stdout:8/825: truncate d0/d24/d29/f67 1497800 0 2024-03-13T14:50:04.280 INFO:tasks.workunit.client.1.smithi196.stdout:3/957: mkdir db/d19/d22/d3a/d68/d130 0 2024-03-13T14:50:04.283 INFO:tasks.workunit.client.0.smithi176.stdout:8/826: read d0/db/d39/d56/f9f [4088536,79227] 0 2024-03-13T14:50:04.286 INFO:tasks.workunit.client.0.smithi176.stdout:3/912: unlink d4/d28/d31/d52/f6a 0 2024-03-13T14:50:04.289 INFO:tasks.workunit.client.0.smithi176.stdout:5/858: dwrite d5/d2c/d40/dc7/fda [0,4194304] 0 2024-03-13T14:50:04.290 INFO:tasks.workunit.client.0.smithi176.stdout:7/839: write d1/faf [1309584,15998] 0 2024-03-13T14:50:04.292 INFO:tasks.workunit.client.1.smithi196.stdout:3/958: dread db/d19/d50/db7/da9/fb9 [0,4194304] 0 2024-03-13T14:50:04.293 INFO:tasks.workunit.client.0.smithi176.stdout:0/784: dwrite d2/db/d39/d9c/f9e [0,4194304] 0 2024-03-13T14:50:04.297 INFO:tasks.workunit.client.0.smithi176.stdout:4/895: dwrite d7/d10/d46/d4a/d5d/ff5 [0,4194304] 0 2024-03-13T14:50:04.299 INFO:tasks.workunit.client.0.smithi176.stdout:1/869: rename d2/de/d10/de8 to d2/d30/d63/d107 0 2024-03-13T14:50:04.303 INFO:tasks.workunit.client.0.smithi176.stdout:8/827: mkdir d0/d5/d11/d47/db7/d10a 0 2024-03-13T14:50:04.304 INFO:tasks.workunit.client.1.smithi196.stdout:3/959: symlink db/d101/l131 0 2024-03-13T14:50:04.305 INFO:tasks.workunit.client.1.smithi196.stdout:9/963: write f5 [654953,111202] 0 2024-03-13T14:50:04.308 INFO:tasks.workunit.client.1.smithi196.stdout:3/960: readlink db/d101/l131 0 2024-03-13T14:50:04.309 INFO:tasks.workunit.client.0.smithi176.stdout:5/859: creat d5/d2c/d40/dc7/d89/f125 x:0 0 0 2024-03-13T14:50:04.311 INFO:tasks.workunit.client.0.smithi176.stdout:6/924: dwrite d0/d10/d19/d27/d2b/d2e/f7a [0,4194304] 0 2024-03-13T14:50:04.312 INFO:tasks.workunit.client.0.smithi176.stdout:5/860: write d5/d2e/d7e/dad/d7b/fc8 [4536434,60949] 0 2024-03-13T14:50:04.318 INFO:tasks.workunit.client.0.smithi176.stdout:5/861: stat d5/d2c/d40/dc7/d53/de9 0 2024-03-13T14:50:04.318 INFO:tasks.workunit.client.0.smithi176.stdout:6/925: write d0/d10/dca/dd2/fe3 [641624,13545] 0 2024-03-13T14:50:04.319 INFO:tasks.workunit.client.0.smithi176.stdout:4/896: mknod d7/d10/d2c/c12b 0 2024-03-13T14:50:04.321 INFO:tasks.workunit.client.1.smithi196.stdout:9/964: unlink d7/de/d4b/d1d/d22/l34 0 2024-03-13T14:50:04.323 INFO:tasks.workunit.client.0.smithi176.stdout:8/828: creat d0/d24/d98/dc5/f10b x:0 0 0 2024-03-13T14:50:04.327 INFO:tasks.workunit.client.0.smithi176.stdout:3/913: write d4/d28/d31/d45/d82/d12b/df5/f3c [1947975,70241] 0 2024-03-13T14:50:04.333 INFO:tasks.workunit.client.0.smithi176.stdout:7/840: symlink d1/d1c/d26/d41/d2b/d40/d64/l10c 0 2024-03-13T14:50:04.334 INFO:tasks.workunit.client.0.smithi176.stdout:5/862: unlink d5/d2c/f106 0 2024-03-13T14:50:04.335 INFO:tasks.workunit.client.0.smithi176.stdout:3/914: fdatasync d4/d28/fa2 0 2024-03-13T14:50:04.338 INFO:tasks.workunit.client.0.smithi176.stdout:2/924: dread d1/dc/fa5 [0,4194304] 0 2024-03-13T14:50:04.338 INFO:tasks.workunit.client.0.smithi176.stdout:5/863: dread - d5/d31/d47/d80/fbf zero size 2024-03-13T14:50:04.338 INFO:tasks.workunit.client.0.smithi176.stdout:6/926: rmdir d0/d10/d19/d27/d2b/db1/d110 0 2024-03-13T14:50:04.339 INFO:tasks.workunit.client.0.smithi176.stdout:7/841: write d1/d1c/d26/d41/db6/df1/dfc/d65/fe4 [3554044,75289] 0 2024-03-13T14:50:04.341 INFO:tasks.workunit.client.0.smithi176.stdout:3/915: rmdir d4/d13/d16/d80 39 2024-03-13T14:50:04.343 INFO:tasks.workunit.client.1.smithi196.stdout:3/961: dwrite db/d19/d4a/d9b/fb5 [0,4194304] 0 2024-03-13T14:50:04.346 INFO:tasks.workunit.client.0.smithi176.stdout:0/785: truncate d2/d5/d1e/f92 1915397 0 2024-03-13T14:50:04.346 INFO:tasks.workunit.client.0.smithi176.stdout:5/864: fsync d5/d31/d47/f85 0 2024-03-13T14:50:04.349 INFO:tasks.workunit.client.1.smithi196.stdout:3/962: mknod db/d19/d50/db7/da9/c132 0 2024-03-13T14:50:04.350 INFO:tasks.workunit.client.0.smithi176.stdout:4/897: dwrite d7/d8/d45/f5c [0,4194304] 0 2024-03-13T14:50:04.351 INFO:tasks.workunit.client.1.smithi196.stdout:9/965: dwrite d7/de/d74/d94/dfb/f98 [4194304,4194304] 0 2024-03-13T14:50:04.354 INFO:tasks.workunit.client.1.smithi196.stdout:9/966: dread - d7/de/d16/d70/feb zero size 2024-03-13T14:50:04.355 INFO:tasks.workunit.client.1.smithi196.stdout:3/963: symlink db/d19/d22/l133 0 2024-03-13T14:50:04.357 INFO:tasks.workunit.client.1.smithi196.stdout:9/967: write d7/de/d74/d94/dfb/dbf/f132 [173636,39794] 0 2024-03-13T14:50:04.357 INFO:tasks.workunit.client.0.smithi176.stdout:1/870: dwrite d2/fb8 [0,4194304] 0 2024-03-13T14:50:04.359 INFO:tasks.workunit.client.0.smithi176.stdout:2/925: symlink d1/dc/d2d/d59/d80/d5c/d127/l136 0 2024-03-13T14:50:04.364 INFO:tasks.workunit.client.0.smithi176.stdout:1/871: dread - d2/de/d10/d16/d47/f54 zero size 2024-03-13T14:50:04.365 INFO:tasks.workunit.client.0.smithi176.stdout:0/786: fdatasync d2/d74/d61/f66 0 2024-03-13T14:50:04.366 INFO:tasks.workunit.client.0.smithi176.stdout:4/898: unlink d7/d8/d2f/fcd 0 2024-03-13T14:50:04.369 INFO:tasks.workunit.client.1.smithi196.stdout:9/968: creat d7/de/d4b/d1d/d35/dcc/f142 x:0 0 0 2024-03-13T14:50:04.369 INFO:tasks.workunit.client.0.smithi176.stdout:7/842: rename d1/d1c/d4a/fa7 to d1/d28/d3a/d1b/d8f/dc1/dbb/f10d 0 2024-03-13T14:50:04.375 INFO:tasks.workunit.client.0.smithi176.stdout:8/829: dread d0/db/d39/d56/d93/da7/faa [0,4194304] 0 2024-03-13T14:50:04.380 INFO:tasks.workunit.client.0.smithi176.stdout:2/926: unlink d1/dc/d2d/d59/d80/d5c/f5f 0 2024-03-13T14:50:04.381 INFO:tasks.workunit.client.1.smithi196.stdout:9/969: fsync d7/d9/f11 0 2024-03-13T14:50:04.384 INFO:tasks.workunit.client.1.smithi196.stdout:9/970: write d7/de/d4b/d1d/d22/f5a [1803618,50405] 0 2024-03-13T14:50:04.385 INFO:tasks.workunit.client.0.smithi176.stdout:1/872: unlink d2/de/d3f/d50/d83/dcb/le6 0 2024-03-13T14:50:04.387 INFO:tasks.workunit.client.0.smithi176.stdout:1/873: write d2/fb8 [556158,92020] 0 2024-03-13T14:50:04.388 INFO:tasks.workunit.client.1.smithi196.stdout:9/971: dread - d7/de/d4b/dd1/f11d zero size 2024-03-13T14:50:04.391 INFO:tasks.workunit.client.0.smithi176.stdout:7/843: creat d1/d1c/d26/d41/db6/df1/dfc/d65/f10e x:0 0 0 2024-03-13T14:50:04.392 INFO:tasks.workunit.client.0.smithi176.stdout:4/899: dwrite d7/d10/d46/d4a/d5d/f69 [4194304,4194304] 0 2024-03-13T14:50:04.393 INFO:tasks.workunit.client.1.smithi196.stdout:3/964: write db/d15/d44/fb0 [279244,105768] 0 2024-03-13T14:50:04.395 INFO:tasks.workunit.client.1.smithi196.stdout:9/972: creat d7/de/d4b/d1d/d35/f143 x:0 0 0 2024-03-13T14:50:04.396 INFO:tasks.workunit.client.0.smithi176.stdout:6/927: dwrite d0/d10/d19/d27/d2b/de0/d76/fab [0,4194304] 0 2024-03-13T14:50:04.399 INFO:tasks.workunit.client.0.smithi176.stdout:5/865: getdents d5/d2e/d7e/dad 0 2024-03-13T14:50:04.400 INFO:tasks.workunit.client.0.smithi176.stdout:2/927: unlink d1/dc/d2d/d59/d8b/f109 0 2024-03-13T14:50:04.402 INFO:tasks.workunit.client.0.smithi176.stdout:3/916: dwrite d4/d13/f19 [0,4194304] 0 2024-03-13T14:50:04.403 INFO:tasks.workunit.client.1.smithi196.stdout:9/973: chown d7/de/d4b/d1d/d35/lf4 32280308 1 2024-03-13T14:50:04.404 INFO:tasks.workunit.client.0.smithi176.stdout:2/928: fsync d1/dc/d2d/d59/d8b/f134 0 2024-03-13T14:50:04.406 INFO:tasks.workunit.client.0.smithi176.stdout:5/866: chown d5/d2c/d40/dc7/d89/ld7 2847898 1 2024-03-13T14:50:04.408 INFO:tasks.workunit.client.0.smithi176.stdout:5/867: chown d5/d31/def/c114 4725385 1 2024-03-13T14:50:04.409 INFO:tasks.workunit.client.0.smithi176.stdout:2/929: stat d1/dc/de0/fee 0 2024-03-13T14:50:04.410 INFO:tasks.workunit.client.0.smithi176.stdout:1/874: mknod d2/de/dd1/c108 0 2024-03-13T14:50:04.411 INFO:tasks.workunit.client.1.smithi196.stdout:3/965: dwrite db/f2d [0,4194304] 0 2024-03-13T14:50:04.414 INFO:tasks.workunit.client.0.smithi176.stdout:1/875: write d2/d30/d63/d107/ffc [541868,49662] 0 2024-03-13T14:50:04.417 INFO:tasks.workunit.client.1.smithi196.stdout:9/974: rename d7/de/d74/d94/dfb/dbf/f6b to d7/de/d74/d94/dfb/d28/de4/d10a/f144 0 2024-03-13T14:50:04.419 INFO:tasks.workunit.client.0.smithi176.stdout:3/917: creat d4/d28/d31/d52/f13d x:0 0 0 2024-03-13T14:50:04.427 INFO:tasks.workunit.client.1.smithi196.stdout:9/975: creat d7/de/d4b/d1d/d93/df8/f145 x:0 0 0 2024-03-13T14:50:04.429 INFO:tasks.workunit.client.1.smithi196.stdout:3/966: getdents db/d4c 0 2024-03-13T14:50:04.429 INFO:tasks.workunit.client.0.smithi176.stdout:4/900: creat d7/d10/d2c/d124/f12c x:0 0 0 2024-03-13T14:50:04.432 INFO:tasks.workunit.client.1.smithi196.stdout:9/976: mkdir d7/de/d146 0 2024-03-13T14:50:04.433 INFO:tasks.workunit.client.0.smithi176.stdout:7/844: truncate d1/d28/d3a/d1b/d8f/dc1/ddb/f104 216638 0 2024-03-13T14:50:04.433 INFO:tasks.workunit.client.1.smithi196.stdout:9/977: stat d7/de/d4b/d1d/d35/f143 0 2024-03-13T14:50:04.435 INFO:tasks.workunit.client.0.smithi176.stdout:5/868: creat d5/d2e/d43/d11f/f126 x:0 0 0 2024-03-13T14:50:04.435 INFO:tasks.workunit.client.1.smithi196.stdout:3/967: mkdir db/d19/d4a/d9b/de9/df0/d134 0 2024-03-13T14:50:04.436 INFO:tasks.workunit.client.0.smithi176.stdout:3/918: truncate d4/d13/d16/d3d/fb2 1470864 0 2024-03-13T14:50:04.437 INFO:tasks.workunit.client.1.smithi196.stdout:3/968: chown db/d15/d44/d51/l7a 4396411 1 2024-03-13T14:50:04.438 INFO:tasks.workunit.client.0.smithi176.stdout:8/830: getdents d0/d24/d29 0 2024-03-13T14:50:04.439 INFO:tasks.workunit.client.1.smithi196.stdout:9/978: unlink d7/de/d4b/d1d/d35/dcc/f142 0 2024-03-13T14:50:04.441 INFO:tasks.workunit.client.0.smithi176.stdout:4/901: mkdir d7/d8/d15/d94/d12d 0 2024-03-13T14:50:04.442 INFO:tasks.workunit.client.0.smithi176.stdout:0/787: dwrite d2/db/f93 [4194304,4194304] 0 2024-03-13T14:50:04.443 INFO:tasks.workunit.client.0.smithi176.stdout:1/876: fdatasync d2/de/d3f/f6f 0 2024-03-13T14:50:04.443 INFO:tasks.workunit.client.0.smithi176.stdout:4/902: read d7/d8/d45/f5c [1878575,48875] 0 2024-03-13T14:50:04.445 INFO:tasks.workunit.client.0.smithi176.stdout:6/928: getdents d0/de/d8d 0 2024-03-13T14:50:04.447 INFO:tasks.workunit.client.1.smithi196.stdout:9/979: mkdir d7/de/d4b/d1d/d22/d147 0 2024-03-13T14:50:04.454 INFO:tasks.workunit.client.1.smithi196.stdout:3/969: getdents db/d19/d4a/dc5 0 2024-03-13T14:50:04.454 INFO:tasks.workunit.client.0.smithi176.stdout:1/877: chown d2/de/d10/d16/d4a 3769161 1 2024-03-13T14:50:04.454 INFO:tasks.workunit.client.0.smithi176.stdout:8/831: mknod d0/d24/d29/c10c 0 2024-03-13T14:50:04.455 INFO:tasks.workunit.client.0.smithi176.stdout:5/869: mkdir d5/d2e/d7e/dad/d127 0 2024-03-13T14:50:04.456 INFO:tasks.workunit.client.0.smithi176.stdout:0/788: fsync d2/db/d39/fc8 0 2024-03-13T14:50:04.459 INFO:tasks.workunit.client.0.smithi176.stdout:8/832: rename d0/db/d53/lc1 to d0/db/l10d 0 2024-03-13T14:50:04.460 INFO:tasks.workunit.client.1.smithi196.stdout:9/980: dread d7/de/d74/d94/dfb/d28/d6c/f8f [0,4194304] 0 2024-03-13T14:50:04.461 INFO:tasks.workunit.client.1.smithi196.stdout:9/981: dread - d7/d71/f141 zero size 2024-03-13T14:50:04.461 INFO:tasks.workunit.client.0.smithi176.stdout:7/845: link d1/d1c/f5e d1/d1c/d26/d41/f10f 0 2024-03-13T14:50:04.463 INFO:tasks.workunit.client.0.smithi176.stdout:8/833: write d0/d24/d7b/f9e [2232041,55070] 0 2024-03-13T14:50:04.465 INFO:tasks.workunit.client.1.smithi196.stdout:9/982: mknod d7/de/d4b/d1d/d35/c148 0 2024-03-13T14:50:04.466 INFO:tasks.workunit.client.0.smithi176.stdout:2/930: dwrite d1/dc/d2d/d59/d8c/fad [0,4194304] 0 2024-03-13T14:50:04.472 INFO:tasks.workunit.client.1.smithi196.stdout:9/983: creat d7/de/de5/d12b/f149 x:0 0 0 2024-03-13T14:50:04.474 INFO:tasks.workunit.client.0.smithi176.stdout:4/903: dwrite d7/d10/d46/d4a/dcc/dff/f107 [0,4194304] 0 2024-03-13T14:50:04.475 INFO:tasks.workunit.client.0.smithi176.stdout:7/846: fdatasync d1/d1c/d26/d41/d2b/ddc/fc0 0 2024-03-13T14:50:04.477 INFO:tasks.workunit.client.0.smithi176.stdout:7/847: fdatasync d1/d1c/d4a/f98 0 2024-03-13T14:50:04.477 INFO:tasks.workunit.client.0.smithi176.stdout:3/919: write d4/d13/d16/db6/fc4 [1612386,35360] 0 2024-03-13T14:50:04.479 INFO:tasks.workunit.client.0.smithi176.stdout:8/834: creat d0/d24/d7b/dec/f10e x:0 0 0 2024-03-13T14:50:04.479 INFO:tasks.workunit.client.0.smithi176.stdout:2/931: rmdir d1/dc/d61 39 2024-03-13T14:50:04.483 INFO:tasks.workunit.client.0.smithi176.stdout:3/920: mknod d4/c13e 0 2024-03-13T14:50:04.485 INFO:tasks.workunit.client.0.smithi176.stdout:3/921: read - d4/d28/d34/dde/f117 zero size 2024-03-13T14:50:04.485 INFO:tasks.workunit.client.0.smithi176.stdout:2/932: mknod d1/dc/de0/c137 0 2024-03-13T14:50:04.486 INFO:tasks.workunit.client.0.smithi176.stdout:7/848: mknod d1/d1c/d26/d41/db6/df1/dfc/d65/c110 0 2024-03-13T14:50:04.491 INFO:tasks.workunit.client.1.smithi196.stdout:9/984: dwrite d7/de/d4b/dd1/f11d [0,4194304] 0 2024-03-13T14:50:04.492 INFO:tasks.workunit.client.0.smithi176.stdout:1/878: link d2/de/dbf/fc0 d2/d29/d73/dd0/f109 0 2024-03-13T14:50:04.493 INFO:tasks.workunit.client.0.smithi176.stdout:1/879: readlink d2/d30/d63/l68 0 2024-03-13T14:50:04.494 INFO:tasks.workunit.client.0.smithi176.stdout:4/904: mknod d7/d10/c12e 0 2024-03-13T14:50:04.500 INFO:tasks.workunit.client.1.smithi196.stdout:3/970: dwrite db/d19/d4a/f8c [0,4194304] 0 2024-03-13T14:50:04.501 INFO:tasks.workunit.client.0.smithi176.stdout:0/789: write d2/f9b [924578,125015] 0 2024-03-13T14:50:04.509 INFO:tasks.workunit.client.1.smithi196.stdout:9/985: creat d7/d9/f14a x:0 0 0 2024-03-13T14:50:04.513 INFO:tasks.workunit.client.0.smithi176.stdout:1/880: dread d2/de/d10/d16/d59/fa1 [0,4194304] 0 2024-03-13T14:50:04.514 INFO:tasks.workunit.client.1.smithi196.stdout:3/971: creat db/d15/dfb/f135 x:0 0 0 2024-03-13T14:50:04.514 INFO:tasks.workunit.client.0.smithi176.stdout:8/835: rename d0/d5/d11/d47/lb2 to d0/d24/d29/d70/l10f 0 2024-03-13T14:50:04.516 INFO:tasks.workunit.client.0.smithi176.stdout:0/790: mknod d2/d5/dc/d20/da8/cf9 0 2024-03-13T14:50:04.521 INFO:tasks.workunit.client.0.smithi176.stdout:1/881: unlink d2/de/d3f/d50/fa3 0 2024-03-13T14:50:04.523 INFO:tasks.workunit.client.0.smithi176.stdout:5/870: link d5/d2e/d7e/dad/d7b/lc6 d5/d2c/d74/l128 0 2024-03-13T14:50:04.523 INFO:tasks.workunit.client.0.smithi176.stdout:4/905: link d7/fe d7/d8/d15/d94/f12f 0 2024-03-13T14:50:04.526 INFO:tasks.workunit.client.1.smithi196.stdout:3/972: link db/d19/d22/d3a/d53/cf3 db/d37/c136 0 2024-03-13T14:50:04.526 INFO:tasks.workunit.client.0.smithi176.stdout:3/922: link d4/d28/d31/d52/dff/c10d d4/d28/d34/da4/c13f 0 2024-03-13T14:50:04.527 INFO:tasks.workunit.client.0.smithi176.stdout:2/933: link d1/dc/d2d/d59/d80/d7e/da3/d111/c128 d1/c138 0 2024-03-13T14:50:04.528 INFO:tasks.workunit.client.1.smithi196.stdout:9/986: dwrite d7/d9/f14a [0,4194304] 0 2024-03-13T14:50:04.530 INFO:tasks.workunit.client.0.smithi176.stdout:7/849: getdents d1/d28/d3a/d1b/d8f/dc1/dbb 0 2024-03-13T14:50:04.532 INFO:tasks.workunit.client.1.smithi196.stdout:3/973: fsync db/d37/d64/f57 0 2024-03-13T14:50:04.533 INFO:tasks.workunit.client.0.smithi176.stdout:7/850: write d1/d1c/d26/d41/d2b/d40/d64/d94/f83 [2472334,129744] 0 2024-03-13T14:50:04.542 INFO:tasks.workunit.client.0.smithi176.stdout:5/871: rename d5/d31/lcc to d5/d2e/d43/l129 0 2024-03-13T14:50:04.543 INFO:tasks.workunit.client.0.smithi176.stdout:5/872: write d5/d31/d47/f85 [3991679,107555] 0 2024-03-13T14:50:04.544 INFO:tasks.workunit.client.1.smithi196.stdout:3/974: creat db/d19/dc1/f137 x:0 0 0 2024-03-13T14:50:04.546 INFO:tasks.workunit.client.0.smithi176.stdout:4/906: mkdir d7/d10/d46/d4a/d5d/da1/d130 0 2024-03-13T14:50:04.547 INFO:tasks.workunit.client.1.smithi196.stdout:3/975: read db/d15/d44/d79/ff6 [583430,51322] 0 2024-03-13T14:50:04.553 INFO:tasks.workunit.client.0.smithi176.stdout:8/836: write d0/db/f106 [4811066,118801] 0 2024-03-13T14:50:04.553 INFO:tasks.workunit.client.0.smithi176.stdout:2/934: mknod d1/dc/d2d/d90/dcf/da0/c139 0 2024-03-13T14:50:04.555 INFO:tasks.workunit.client.1.smithi196.stdout:3/976: symlink db/d19/d22/d3a/d68/d8e/l138 0 2024-03-13T14:50:04.556 INFO:tasks.workunit.client.1.smithi196.stdout:9/987: dwrite d7/de/d74/d94/dfb/d28/f82 [0,4194304] 0 2024-03-13T14:50:04.590 INFO:tasks.workunit.client.0.smithi176.stdout:0/791: rmdir d2/db/d39/d9c/df1 0 2024-03-13T14:50:04.591 INFO:tasks.workunit.client.0.smithi176.stdout:5/873: dread d5/d31/d47/f59 [0,4194304] 0 2024-03-13T14:50:04.591 INFO:tasks.workunit.client.0.smithi176.stdout:1/882: dread d2/de/dbf/fc0 [0,4194304] 0 2024-03-13T14:50:04.592 INFO:tasks.workunit.client.1.smithi196.stdout:3/977: write db/d15/d44/d79/fe4 [18281,9730] 0 2024-03-13T14:50:04.593 INFO:tasks.workunit.client.0.smithi176.stdout:7/851: creat d1/d28/f111 x:0 0 0 2024-03-13T14:50:04.596 INFO:tasks.workunit.client.0.smithi176.stdout:0/792: mkdir d2/d74/d61/dfa 0 2024-03-13T14:50:04.597 INFO:tasks.workunit.client.1.smithi196.stdout:3/978: symlink db/d19/d4a/dc5/l139 0 2024-03-13T14:50:04.597 INFO:tasks.workunit.client.0.smithi176.stdout:8/837: chown d0/d24/d98/dc6 748006 1 2024-03-13T14:50:04.599 INFO:tasks.workunit.client.0.smithi176.stdout:7/852: write d1/d1c/d4a/f105 [706493,67895] 0 2024-03-13T14:50:04.600 INFO:tasks.workunit.client.0.smithi176.stdout:4/907: truncate d7/db9/dd8/fdd 1331748 0 2024-03-13T14:50:04.600 INFO:tasks.workunit.client.1.smithi196.stdout:3/979: stat db/d19/c24 0 2024-03-13T14:50:04.602 INFO:tasks.workunit.client.0.smithi176.stdout:5/874: creat d5/d2e/d7e/dad/d127/f12a x:0 0 0 2024-03-13T14:50:04.602 INFO:tasks.workunit.client.1.smithi196.stdout:9/988: dwrite d7/de/d74/d94/fd5 [0,4194304] 0 2024-03-13T14:50:04.604 INFO:tasks.workunit.client.0.smithi176.stdout:3/923: dwrite d4/d28/f5c [0,4194304] 0 2024-03-13T14:50:04.604 INFO:tasks.workunit.client.0.smithi176.stdout:7/853: readlink d1/d7/l24 0 2024-03-13T14:50:04.604 INFO:tasks.workunit.client.0.smithi176.stdout:4/908: write d7/dab/f121 [628445,40951] 0 2024-03-13T14:50:04.605 INFO:tasks.workunit.client.0.smithi176.stdout:8/838: unlink d0/d24/d98/dc6/d40/f77 0 2024-03-13T14:50:04.606 INFO:tasks.workunit.client.1.smithi196.stdout:9/989: symlink d7/de/d4b/d1d/d22/d147/l14b 0 2024-03-13T14:50:04.607 INFO:tasks.workunit.client.1.smithi196.stdout:3/980: creat db/d15/d44/f13a x:0 0 0 2024-03-13T14:50:04.611 INFO:tasks.workunit.client.0.smithi176.stdout:7/854: write d1/d1c/d26/d41/db6/df1/dfc/d65/f10e [322436,84306] 0 2024-03-13T14:50:04.616 INFO:tasks.workunit.client.0.smithi176.stdout:3/924: symlink d4/d28/d31/d52/de9/d131/l140 0 2024-03-13T14:50:04.617 INFO:tasks.workunit.client.1.smithi196.stdout:3/981: dread db/d15/d44/d79/fe6 [0,4194304] 0 2024-03-13T14:50:04.617 INFO:tasks.workunit.client.0.smithi176.stdout:1/883: dwrite d2/d29/f3c [4194304,4194304] 0 2024-03-13T14:50:04.622 INFO:tasks.workunit.client.0.smithi176.stdout:7/855: symlink d1/d28/d3a/d1b/d8f/dd1/df4/l112 0 2024-03-13T14:50:04.630 INFO:tasks.workunit.client.0.smithi176.stdout:3/925: truncate d4/d13/d16/dad/fe6 984256 0 2024-03-13T14:50:04.636 INFO:tasks.workunit.client.0.smithi176.stdout:3/926: truncate d4/d13/d81/d89/f6d 909908 0 2024-03-13T14:50:04.637 INFO:tasks.workunit.client.0.smithi176.stdout:4/909: write d7/f50 [1928914,21543] 0 2024-03-13T14:50:04.637 INFO:tasks.workunit.client.1.smithi196.stdout:9/990: write d7/de/d74/d94/dfb/f38 [1414400,80479] 0 2024-03-13T14:50:04.640 INFO:tasks.workunit.client.0.smithi176.stdout:8/839: truncate d0/d24/dc3/fe3 1858457 0 2024-03-13T14:50:04.641 INFO:tasks.workunit.client.1.smithi196.stdout:9/991: creat d7/de/d4b/f14c x:0 0 0 2024-03-13T14:50:04.641 INFO:tasks.workunit.client.0.smithi176.stdout:1/884: truncate d2/de/d10/f6d 871754 0 2024-03-13T14:50:04.646 INFO:tasks.workunit.client.0.smithi176.stdout:3/927: symlink d4/d13/d81/d89/de2/l141 0 2024-03-13T14:50:04.646 INFO:tasks.workunit.client.1.smithi196.stdout:3/982: dwrite db/d15/d44/d79/fd2 [0,4194304] 0 2024-03-13T14:50:04.647 INFO:tasks.workunit.client.0.smithi176.stdout:4/910: symlink d7/dab/l131 0 2024-03-13T14:50:04.649 INFO:tasks.workunit.client.0.smithi176.stdout:1/885: chown d2/de/dbf/fc0 261647619 1 2024-03-13T14:50:04.651 INFO:tasks.workunit.client.0.smithi176.stdout:0/793: dwrite d2/d5/d28/f56 [0,4194304] 0 2024-03-13T14:50:04.653 INFO:tasks.workunit.client.0.smithi176.stdout:1/886: stat d2/d29/l8d 0 2024-03-13T14:50:04.654 INFO:tasks.workunit.client.1.smithi196.stdout:8/911: sync 2024-03-13T14:50:04.654 INFO:tasks.workunit.client.1.smithi196.stdout:4/984: sync 2024-03-13T14:50:04.654 INFO:tasks.workunit.client.0.smithi176.stdout:2/935: dwrite d1/dc/d2d/d59/d80/d5c/d78/ff2 [0,4194304] 0 2024-03-13T14:50:04.654 INFO:tasks.workunit.client.0.smithi176.stdout:4/911: stat d7/db9/fe7 0 2024-03-13T14:50:04.657 INFO:tasks.workunit.client.1.smithi196.stdout:9/992: rename d7/de/d74/d94/dfb/d28/d6c/d7b/l119 to d7/de/d4b/d1d/d93/dd7/df1/d115/d12c/l14d 0 2024-03-13T14:50:04.663 INFO:tasks.workunit.client.1.smithi196.stdout:9/993: creat d7/de/d74/f14e x:0 0 0 2024-03-13T14:50:04.663 INFO:tasks.workunit.client.0.smithi176.stdout:8/840: creat d0/db/d39/d56/d93/f110 x:0 0 0 2024-03-13T14:50:04.664 INFO:tasks.workunit.client.1.smithi196.stdout:9/994: stat d7/de/de5/d138 0 2024-03-13T14:50:04.669 INFO:tasks.workunit.client.0.smithi176.stdout:7/856: dwrite d1/d1c/d26/d41/d2b/d40/d64/f60 [0,4194304] 0 2024-03-13T14:50:04.669 INFO:tasks.workunit.client.0.smithi176.stdout:0/794: mknod d2/d74/df8/cfb 0 2024-03-13T14:50:04.669 INFO:tasks.workunit.client.1.smithi196.stdout:9/995: symlink d7/de/de5/d138/l14f 0 2024-03-13T14:50:04.673 INFO:tasks.workunit.client.1.smithi196.stdout:9/996: fdatasync d7/d71/ff2 0 2024-03-13T14:50:04.675 INFO:tasks.workunit.client.0.smithi176.stdout:5/875: dread d5/d2b/f46 [0,4194304] 0 2024-03-13T14:50:04.677 INFO:tasks.workunit.client.0.smithi176.stdout:1/887: read d2/de/d10/d16/d59/f6e [563357,113540] 0 2024-03-13T14:50:04.678 INFO:tasks.workunit.client.0.smithi176.stdout:8/841: dread d0/d5/d11/f76 [0,4194304] 0 2024-03-13T14:50:04.683 INFO:tasks.workunit.client.0.smithi176.stdout:8/842: chown d0/d24/d98/dc6/d40/l61 32033388 1 2024-03-13T14:50:04.683 INFO:tasks.workunit.client.1.smithi196.stdout:9/997: dread - d7/de/d74/fd3 zero size 2024-03-13T14:50:04.686 INFO:tasks.workunit.client.0.smithi176.stdout:2/936: creat d1/dc/d2d/d90/dcf/dea/f13a x:0 0 0 2024-03-13T14:50:04.688 INFO:tasks.workunit.client.0.smithi176.stdout:0/795: creat d2/db/d39/ffc x:0 0 0 2024-03-13T14:50:04.688 INFO:tasks.workunit.client.1.smithi196.stdout:9/998: dread - d7/de/d74/d94/dfb/d28/de4/d10a/f124 zero size 2024-03-13T14:50:04.688 INFO:tasks.workunit.client.0.smithi176.stdout:3/928: getdents d4/d13/d16/ddd 0 2024-03-13T14:50:04.691 INFO:tasks.workunit.client.1.smithi196.stdout:4/985: write d1/d3/fcf [195318,115997] 0 2024-03-13T14:50:04.693 INFO:tasks.workunit.client.1.smithi196.stdout:3/983: dwrite db/ff [0,4194304] 0 2024-03-13T14:50:04.694 INFO:tasks.workunit.client.1.smithi196.stdout:9/999: chown d7/de/d4b/d1d/d35/dcc/f107 2714428 1 2024-03-13T14:50:04.699 INFO:tasks.workunit.client.0.smithi176.stdout:2/937: readlink d1/dc/d2d/d38/d5d/ld1 0 2024-03-13T14:50:04.704 INFO:tasks.workunit.client.0.smithi176.stdout:6/929: sync 2024-03-13T14:50:04.705 INFO:tasks.workunit.client.0.smithi176.stdout:5/876: dread d5/d2e/d7e/d9f/f5a [0,4194304] 0 2024-03-13T14:50:04.705 INFO:tasks.workunit.client.0.smithi176.stdout:0/796: truncate d2/d74/f6e 2423763 0 2024-03-13T14:50:04.706 INFO:tasks.workunit.client.0.smithi176.stdout:5/877: readlink d5/d2b/l69 0 2024-03-13T14:50:04.708 INFO:tasks.workunit.client.1.smithi196.stdout:4/986: mkdir d1/d3/d4c/d72/d7d/d80/d163 0 2024-03-13T14:50:04.709 INFO:tasks.workunit.client.1.smithi196.stdout:2/935: sync 2024-03-13T14:50:04.710 INFO:tasks.workunit.client.1.smithi196.stdout:8/912: dread d0/d10/d12/df5/fba [0,4194304] 0 2024-03-13T14:50:04.711 INFO:tasks.workunit.client.1.smithi196.stdout:2/936: chown d0/d50/fd5 23736 1 2024-03-13T14:50:04.712 INFO:tasks.workunit.client.0.smithi176.stdout:4/912: rename d7/d8/d33/l63 to d7/dab/l132 0 2024-03-13T14:50:04.713 INFO:tasks.workunit.client.0.smithi176.stdout:4/913: chown d7/d10/d2c/l7a 63326 1 2024-03-13T14:50:04.714 INFO:tasks.workunit.client.0.smithi176.stdout:8/843: symlink d0/d24/d29/l111 0 2024-03-13T14:50:04.716 INFO:tasks.workunit.client.0.smithi176.stdout:6/930: dwrite d0/f4e [0,4194304] 0 2024-03-13T14:50:04.717 INFO:tasks.workunit.client.1.smithi196.stdout:8/913: creat d0/d16/d3d/d8b/d48/f12e x:0 0 0 2024-03-13T14:50:04.719 INFO:tasks.workunit.client.0.smithi176.stdout:6/931: write d0/d10/d19/d27/d2b/de0/dd6/fd8 [513963,45587] 0 2024-03-13T14:50:04.719 INFO:tasks.workunit.client.1.smithi196.stdout:4/987: creat d1/d3/d7/d35/d76/d58/dc6/f164 x:0 0 0 2024-03-13T14:50:04.720 INFO:tasks.workunit.client.1.smithi196.stdout:4/988: dread - d1/dfb/d11c/f8a zero size 2024-03-13T14:50:04.720 INFO:tasks.workunit.client.0.smithi176.stdout:2/938: mknod d1/dc/d2d/d59/d80/d7e/c13b 0 2024-03-13T14:50:04.721 INFO:tasks.workunit.client.1.smithi196.stdout:3/984: creat db/d37/f13b x:0 0 0 2024-03-13T14:50:04.724 INFO:tasks.workunit.client.1.smithi196.stdout:2/937: mkdir d0/d7/d9/d42/d98/de4/d104/d131 0 2024-03-13T14:50:04.726 INFO:tasks.workunit.client.0.smithi176.stdout:1/888: write d2/d29/d73/f94 [749443,52111] 0 2024-03-13T14:50:04.726 INFO:tasks.workunit.client.0.smithi176.stdout:0/797: creat d2/db/d35/ffd x:0 0 0 2024-03-13T14:50:04.728 INFO:tasks.workunit.client.0.smithi176.stdout:5/878: mkdir d5/d2e/d43/d12b 0 2024-03-13T14:50:04.729 INFO:tasks.workunit.client.0.smithi176.stdout:1/889: chown d2/de/dd1/fdf 176068593 1 2024-03-13T14:50:04.730 INFO:tasks.workunit.client.1.smithi196.stdout:3/985: dread - db/d15/d44/d79/fdc zero size 2024-03-13T14:50:04.731 INFO:tasks.workunit.client.0.smithi176.stdout:1/890: fsync d2/de/f26 0 2024-03-13T14:50:04.732 INFO:tasks.workunit.client.1.smithi196.stdout:8/914: mknod d0/d10/d70/d12b/c12f 0 2024-03-13T14:50:04.732 INFO:tasks.workunit.client.0.smithi176.stdout:4/914: write d7/d8/d15/f38 [5003638,20897] 0 2024-03-13T14:50:04.733 INFO:tasks.workunit.client.1.smithi196.stdout:2/938: unlink d0/d6/d4b/f8a 0 2024-03-13T14:50:04.736 INFO:tasks.workunit.client.0.smithi176.stdout:2/939: dwrite d1/dc/d2d/fca [4194304,4194304] 0 2024-03-13T14:50:04.739 INFO:tasks.workunit.client.0.smithi176.stdout:6/932: mkdir d0/d10/d19/d27/d2b/de0/d76/d7c/d127 0 2024-03-13T14:50:04.739 INFO:tasks.workunit.client.1.smithi196.stdout:8/915: unlink d0/d16/d3d/d8b/d48/cd5 0 2024-03-13T14:50:04.740 INFO:tasks.workunit.client.1.smithi196.stdout:8/916: chown d0/d10/d70/d12b 1 1 2024-03-13T14:50:04.741 INFO:tasks.workunit.client.0.smithi176.stdout:6/933: write d0/d10/d19/d27/d2b/d2e/d3e/db4/fdd [2765530,106990] 0 2024-03-13T14:50:04.745 INFO:tasks.workunit.client.1.smithi196.stdout:2/939: mknod d0/d6/d4b/d5f/c132 0 2024-03-13T14:50:04.749 INFO:tasks.workunit.client.0.smithi176.stdout:7/857: rename d1/d1c/d26/d41/d2b/d40/l46 to d1/d28/d3a/d1b/d8f/dc1/db7/df9/l113 0 2024-03-13T14:50:04.752 INFO:tasks.workunit.client.1.smithi196.stdout:2/940: fsync d0/d7/d9/d5a/f6f 0 2024-03-13T14:50:04.754 INFO:tasks.workunit.client.0.smithi176.stdout:8/844: symlink d0/d5/d11/d47/db7/d10a/l112 0 2024-03-13T14:50:04.755 INFO:tasks.workunit.client.0.smithi176.stdout:6/934: dwrite d0/de/d38/d88/d9d/f104 [0,4194304] 0 2024-03-13T14:50:04.758 INFO:tasks.workunit.client.0.smithi176.stdout:6/935: chown d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/fb9 0 1 2024-03-13T14:50:04.759 INFO:tasks.workunit.client.1.smithi196.stdout:2/941: fdatasync d0/d7/d9/d9a/ff4 0 2024-03-13T14:50:04.761 INFO:tasks.workunit.client.1.smithi196.stdout:8/917: dwrite d0/d16/d3d/d8b/d48/f116 [0,4194304] 0 2024-03-13T14:50:04.764 INFO:tasks.workunit.client.1.smithi196.stdout:8/918: chown d0/f56 2359326 1 2024-03-13T14:50:04.766 INFO:tasks.workunit.client.0.smithi176.stdout:2/940: mkdir d1/d2a/d13c 0 2024-03-13T14:50:04.766 INFO:tasks.workunit.client.1.smithi196.stdout:4/989: dwrite d1/d3/d4c/f57 [0,4194304] 0 2024-03-13T14:50:04.769 INFO:tasks.workunit.client.1.smithi196.stdout:4/990: read d1/d3/d4c/d72/daf/f14c [385242,96694] 0 2024-03-13T14:50:04.773 INFO:tasks.workunit.client.0.smithi176.stdout:0/798: mkdir d2/d5/dc/dfe 0 2024-03-13T14:50:04.776 INFO:tasks.workunit.client.0.smithi176.stdout:0/799: write d2/d74/d9f/fde [558022,92749] 0 2024-03-13T14:50:04.777 INFO:tasks.workunit.client.0.smithi176.stdout:0/800: chown d2/d74/fdd 327373475 1 2024-03-13T14:50:04.779 INFO:tasks.workunit.client.1.smithi196.stdout:8/919: rmdir d0/d16/d3d/d7f 39 2024-03-13T14:50:04.781 INFO:tasks.workunit.client.1.smithi196.stdout:8/920: read - d0/d10/da5/d110/fc3 zero size 2024-03-13T14:50:04.783 INFO:tasks.workunit.client.0.smithi176.stdout:7/858: unlink d1/d1c/d26/d41/f10f 0 2024-03-13T14:50:04.784 INFO:tasks.workunit.client.1.smithi196.stdout:3/986: dwrite db/d37/d64/f57 [0,4194304] 0 2024-03-13T14:50:04.791 INFO:tasks.workunit.client.1.smithi196.stdout:2/942: mkdir d0/d7/d9/d61/d83/d124/d133 0 2024-03-13T14:50:04.791 INFO:tasks.workunit.client.0.smithi176.stdout:5/879: mkdir d5/d31/d86/dc0/d11a/d120/d12c 0 2024-03-13T14:50:04.796 INFO:tasks.workunit.client.0.smithi176.stdout:1/891: write d2/de/d3f/f52 [886985,81243] 0 2024-03-13T14:50:04.797 INFO:tasks.workunit.client.0.smithi176.stdout:1/892: stat f1 0 2024-03-13T14:50:04.797 INFO:tasks.workunit.client.1.smithi196.stdout:8/921: stat d0/d10/d12/d17/c97 0 2024-03-13T14:50:04.799 INFO:tasks.workunit.client.0.smithi176.stdout:1/893: write f1 [815533,113867] 0 2024-03-13T14:50:04.802 INFO:tasks.workunit.client.1.smithi196.stdout:3/987: mkdir db/d19/d22/d3a/d6a/d13c 0 2024-03-13T14:50:04.803 INFO:tasks.workunit.client.0.smithi176.stdout:4/915: mkdir d7/d10/d46/d4a/d5d/da1/d130/d133 0 2024-03-13T14:50:04.806 INFO:tasks.workunit.client.1.smithi196.stdout:3/988: truncate db/d19/f103 5086311 0 2024-03-13T14:50:04.809 INFO:tasks.workunit.client.0.smithi176.stdout:5/880: creat d5/d31/f12d x:0 0 0 2024-03-13T14:50:04.810 INFO:tasks.workunit.client.0.smithi176.stdout:5/881: write d5/f14 [660330,6520] 0 2024-03-13T14:50:04.811 INFO:tasks.workunit.client.0.smithi176.stdout:8/845: fsync d0/d24/d29/f9d 0 2024-03-13T14:50:04.812 INFO:tasks.workunit.client.1.smithi196.stdout:3/989: dread - db/d19/d4a/f108 zero size 2024-03-13T14:50:04.813 INFO:tasks.workunit.client.0.smithi176.stdout:1/894: mknod d2/de/d10/d16/d47/db4/c10a 0 2024-03-13T14:50:04.815 INFO:tasks.workunit.client.1.smithi196.stdout:8/922: link d0/d16/d3d/d7f/fa8 d0/d10/d12/d17/d42/f130 0 2024-03-13T14:50:04.817 INFO:tasks.workunit.client.0.smithi176.stdout:6/936: dread d0/d10/d19/d27/d2b/f39 [0,4194304] 0 2024-03-13T14:50:04.819 INFO:tasks.workunit.client.1.smithi196.stdout:4/991: dwrite d1/d3/d7/d35/d76/d58/d69/d9f/fe0 [0,4194304] 0 2024-03-13T14:50:04.820 INFO:tasks.workunit.client.0.smithi176.stdout:6/937: write d0/d10/d19/d27/fac [3816795,33664] 0 2024-03-13T14:50:04.822 INFO:tasks.workunit.client.1.smithi196.stdout:4/992: write d1/d1e/d5c/ff1 [2864275,23719] 0 2024-03-13T14:50:04.823 INFO:tasks.workunit.client.0.smithi176.stdout:4/916: dread d7/d10/d2c/fb2 [0,4194304] 0 2024-03-13T14:50:04.823 INFO:tasks.workunit.client.1.smithi196.stdout:8/923: creat d0/d16/d3d/d9c/d87/d8c/da9/f131 x:0 0 0 2024-03-13T14:50:04.824 INFO:tasks.workunit.client.1.smithi196.stdout:4/993: chown d1 2675231 1 2024-03-13T14:50:04.826 INFO:tasks.workunit.client.0.smithi176.stdout:8/846: unlink d0/db/cf1 0 2024-03-13T14:50:04.827 INFO:tasks.workunit.client.1.smithi196.stdout:3/990: dread db/d19/d4a/d9b/f11b [0,4194304] 0 2024-03-13T14:50:04.829 INFO:tasks.workunit.client.1.smithi196.stdout:8/924: fdatasync d0/d16/d71/f43 0 2024-03-13T14:50:04.830 INFO:tasks.workunit.client.0.smithi176.stdout:1/895: mkdir d2/de/d10/d16/d40/d10b 0 2024-03-13T14:50:04.830 INFO:tasks.workunit.client.1.smithi196.stdout:3/991: chown db/d19/d22/d3a/d68/d8e/f120 234858 1 2024-03-13T14:50:04.832 INFO:tasks.workunit.client.0.smithi176.stdout:3/929: rename d4/d13/c47 to d4/d28/d31/d45/c142 0 2024-03-13T14:50:04.834 INFO:tasks.workunit.client.1.smithi196.stdout:3/992: readlink db/d19/d22/d3a/d68/d8e/l11f 0 2024-03-13T14:50:04.837 INFO:tasks.workunit.client.1.smithi196.stdout:4/994: rename d1/d3/d7/d35/d76/d58/dc9/l101 to d1/d3/d4c/d72/d7d/d80/l165 0 2024-03-13T14:50:04.838 INFO:tasks.workunit.client.1.smithi196.stdout:3/993: chown db/d19/d50/f63 13 1 2024-03-13T14:50:04.839 INFO:tasks.workunit.client.0.smithi176.stdout:4/917: symlink d7/d10/d46/d4a/dcc/dd0/l134 0 2024-03-13T14:50:04.839 INFO:tasks.workunit.client.0.smithi176.stdout:5/882: dread d5/d2e/f3f [0,4194304] 0 2024-03-13T14:50:04.842 INFO:tasks.workunit.client.0.smithi176.stdout:2/941: rename d1/dc/fa5 to d1/dc/d1c/d107/d130/f13d 0 2024-03-13T14:50:04.847 INFO:tasks.workunit.client.1.smithi196.stdout:4/995: mkdir d1/d3/d7/d35/d76/d15d/d166 0 2024-03-13T14:50:04.852 INFO:tasks.workunit.client.1.smithi196.stdout:4/996: creat d1/d1e/d5c/d105/f167 x:0 0 0 2024-03-13T14:50:04.856 INFO:tasks.workunit.client.1.smithi196.stdout:3/994: truncate db/d19/f1d 469539 0 2024-03-13T14:50:04.857 INFO:tasks.workunit.client.1.smithi196.stdout:3/995: readlink db/d37/l7e 0 2024-03-13T14:50:04.860 INFO:tasks.workunit.client.1.smithi196.stdout:4/997: creat d1/f168 x:0 0 0 2024-03-13T14:50:04.860 INFO:tasks.workunit.client.0.smithi176.stdout:3/930: dread d4/f25 [0,4194304] 0 2024-03-13T14:50:04.862 INFO:tasks.workunit.client.0.smithi176.stdout:8/847: dwrite d0/db/fd [0,4194304] 0 2024-03-13T14:50:04.862 INFO:tasks.workunit.client.1.smithi196.stdout:3/996: dread db/d15/d44/fb0 [0,4194304] 0 2024-03-13T14:50:04.865 INFO:tasks.workunit.client.1.smithi196.stdout:3/997: fdatasync db/d19/f109 0 2024-03-13T14:50:04.866 INFO:tasks.workunit.client.1.smithi196.stdout:4/998: getdents d1/d136 0 2024-03-13T14:50:04.868 INFO:tasks.workunit.client.1.smithi196.stdout:4/999: fsync d1/dfb/d11c/f9c 0 2024-03-13T14:50:04.873 INFO:tasks.workunit.client.1.smithi196.stdout:3/998: rename db/d19/d22/d3a/d53/c5d to db/d37/d64/c13d 0 2024-03-13T14:50:04.876 INFO:tasks.workunit.client.0.smithi176.stdout:0/801: rename d2/db/d31/f57 to d2/db/d31/dc4/fff 0 2024-03-13T14:50:04.877 INFO:tasks.workunit.client.1.smithi196.stdout:3/999: read - db/d15/dfb/f135 zero size 2024-03-13T14:50:04.877 INFO:tasks.workunit.client.0.smithi176.stdout:5/883: symlink d5/d2c/l12e 0 2024-03-13T14:50:04.877 INFO:tasks.workunit.client.0.smithi176.stdout:2/942: truncate d1/dc/d2d/f7a 4527968 0 2024-03-13T14:50:04.878 INFO:tasks.workunit.client.0.smithi176.stdout:1/896: creat d2/de/d10/d16/f10c x:0 0 0 2024-03-13T14:50:04.879 INFO:tasks.workunit.client.0.smithi176.stdout:5/884: chown d5/d2c/d74/d122 4 1 2024-03-13T14:50:04.881 INFO:tasks.workunit.client.0.smithi176.stdout:5/885: fdatasync d5/d2c/d40/dc7/d89/fa1 0 2024-03-13T14:50:04.883 INFO:tasks.workunit.client.0.smithi176.stdout:5/886: chown d5/d2c/d74/d122 43641 1 2024-03-13T14:50:04.916 INFO:tasks.workunit.client.1.smithi196.stdout:8/925: dread d0/d16/d3d/d8b/f4b [0,4194304] 0 2024-03-13T14:50:04.926 INFO:tasks.workunit.client.0.smithi176.stdout:8/848: write d0/db/d53/d80/f97 [362547,98995] 0 2024-03-13T14:50:04.926 INFO:tasks.workunit.client.0.smithi176.stdout:4/918: link d7/d8/d15/f2b d7/db9/dd8/f135 0 2024-03-13T14:50:04.927 INFO:tasks.workunit.client.0.smithi176.stdout:3/931: symlink d4/d28/d34/l143 0 2024-03-13T14:50:04.929 INFO:tasks.workunit.client.0.smithi176.stdout:4/919: write d7/d8/f16 [3484991,32871] 0 2024-03-13T14:50:04.929 INFO:tasks.workunit.client.0.smithi176.stdout:0/802: unlink d2/d5/dc/d20/da8/fc5 0 2024-03-13T14:50:04.930 INFO:tasks.workunit.client.0.smithi176.stdout:1/897: write d2/de/d10/d16/d47/db4/fc1 [217705,44019] 0 2024-03-13T14:50:04.932 INFO:tasks.workunit.client.0.smithi176.stdout:0/803: chown d2/d5/d1e/l55 12040 1 2024-03-13T14:50:04.937 INFO:tasks.workunit.client.0.smithi176.stdout:7/859: rename d1/d1c/d26/d41/d2b/f96 to d1/d28/d3a/d1b/d8f/dd1/df4/f114 0 2024-03-13T14:50:04.939 INFO:tasks.workunit.client.0.smithi176.stdout:2/943: creat d1/dc/d61/d9e/f13e x:0 0 0 2024-03-13T14:50:04.940 INFO:tasks.workunit.client.0.smithi176.stdout:4/920: symlink d7/db9/dd8/l136 0 2024-03-13T14:50:04.945 INFO:tasks.workunit.client.0.smithi176.stdout:6/938: rename d0/d10/dca/de1 to d0/de/d38/d6b/da3/d128 0 2024-03-13T14:50:04.947 INFO:tasks.workunit.client.1.smithi196.stdout:8/926: truncate d0/d10/d70/fcc 1981805 0 2024-03-13T14:50:04.949 INFO:tasks.workunit.client.1.smithi196.stdout:8/927: chown d0/d10/d12/d17/l6e 15 1 2024-03-13T14:50:04.949 INFO:tasks.workunit.client.0.smithi176.stdout:0/804: mknod d2/d5/dc/dfe/c100 0 2024-03-13T14:50:04.951 INFO:tasks.workunit.client.0.smithi176.stdout:0/805: write d2/db/d35/f3f [1279463,61333] 0 2024-03-13T14:50:04.952 INFO:tasks.workunit.client.0.smithi176.stdout:6/939: mkdir d0/d10/d19/d27/d2b/de0/dd6/d129 0 2024-03-13T14:50:04.954 INFO:tasks.workunit.client.0.smithi176.stdout:1/898: creat d2/de/d3f/f10d x:0 0 0 2024-03-13T14:50:04.955 INFO:tasks.workunit.client.0.smithi176.stdout:6/940: chown d0/d10/d19/d27/d2b/d2e/d3e/db4 5365487 1 2024-03-13T14:50:04.955 INFO:tasks.workunit.client.1.smithi196.stdout:8/928: mkdir d0/d10/d12/d17/d42/dbe/d132 0 2024-03-13T14:50:04.955 INFO:tasks.workunit.client.0.smithi176.stdout:6/941: readlink d0/dc1/lf4 0 2024-03-13T14:50:04.957 INFO:tasks.workunit.client.1.smithi196.stdout:8/929: read d0/d16/d3d/d8b/f4b [1069901,59777] 0 2024-03-13T14:50:04.960 INFO:tasks.workunit.client.0.smithi176.stdout:0/806: read - d2/d5/d28/dac/fae zero size 2024-03-13T14:50:04.961 INFO:tasks.workunit.client.0.smithi176.stdout:5/887: truncate d5/d2e/d7e/dad/fe1 2975515 0 2024-03-13T14:50:04.962 INFO:tasks.workunit.client.0.smithi176.stdout:8/849: rename d0/d24/f57 to d0/d5/f113 0 2024-03-13T14:50:04.964 INFO:tasks.workunit.client.1.smithi196.stdout:8/930: rename d0/d10/da5/db2/dc8 to d0/d10/d12/d1e/db1/d117/d133 0 2024-03-13T14:50:04.964 INFO:tasks.workunit.client.0.smithi176.stdout:7/860: getdents d1/d28/d3a/d1b/d8f/dd1/df4 0 2024-03-13T14:50:04.965 INFO:tasks.workunit.client.0.smithi176.stdout:6/942: fsync d0/d10/d19/d27/d2b/d2e/d3e/db4/ff7 0 2024-03-13T14:50:04.967 INFO:tasks.workunit.client.0.smithi176.stdout:5/888: rmdir d5/d2e/d43 39 2024-03-13T14:50:04.967 INFO:tasks.workunit.client.0.smithi176.stdout:4/921: dwrite d7/d8/d15/f2b [0,4194304] 0 2024-03-13T14:50:04.968 INFO:tasks.workunit.client.0.smithi176.stdout:0/807: read d2/db/d31/f7f [1533585,85135] 0 2024-03-13T14:50:04.968 INFO:tasks.workunit.client.1.smithi196.stdout:8/931: mkdir d0/d16/d3d/d9c/d87/d8c/da9/db8/d134 0 2024-03-13T14:50:04.970 INFO:tasks.workunit.client.0.smithi176.stdout:4/922: dread - d7/d10/d46/d4a/dcc/f108 zero size 2024-03-13T14:50:04.970 INFO:tasks.workunit.client.0.smithi176.stdout:1/899: link d2/de/l44 d2/de/dd1/l10e 0 2024-03-13T14:50:04.971 INFO:tasks.workunit.client.0.smithi176.stdout:5/889: symlink d5/d2e/d7e/dad/d7b/da7/l12f 0 2024-03-13T14:50:04.972 INFO:tasks.workunit.client.0.smithi176.stdout:1/900: chown d2/de/d3f/d3e/d86 25574058 1 2024-03-13T14:50:04.973 INFO:tasks.workunit.client.0.smithi176.stdout:6/943: creat d0/d10/d19/d27/d2b/de0/d76/d7c/dfd/d122/f12a x:0 0 0 2024-03-13T14:50:04.973 INFO:tasks.workunit.client.0.smithi176.stdout:5/890: readlink d5/d2c/l117 0 2024-03-13T14:50:04.973 INFO:tasks.workunit.client.0.smithi176.stdout:1/901: fdatasync d2/de/f26 0 2024-03-13T14:50:04.973 INFO:tasks.workunit.client.0.smithi176.stdout:0/808: truncate d2/db/f3b 2655678 0 2024-03-13T14:50:04.976 INFO:tasks.workunit.client.0.smithi176.stdout:4/923: mknod d7/d10/d46/d4a/dcc/dd0/d11d/c137 0 2024-03-13T14:50:04.977 INFO:tasks.workunit.client.0.smithi176.stdout:0/809: mkdir d2/db/d35/d101 0 2024-03-13T14:50:04.979 INFO:tasks.workunit.client.0.smithi176.stdout:4/924: mkdir d7/d8/d45/d138 0 2024-03-13T14:50:04.980 INFO:tasks.workunit.client.0.smithi176.stdout:0/810: fsync d2/db/d39/fc8 0 2024-03-13T14:50:04.985 INFO:tasks.workunit.client.0.smithi176.stdout:4/925: fdatasync d7/d8/d15/d94/fa6 0 2024-03-13T14:50:04.987 INFO:tasks.workunit.client.0.smithi176.stdout:7/861: dread d1/faf [0,4194304] 0 2024-03-13T14:50:04.990 INFO:tasks.workunit.client.0.smithi176.stdout:4/926: stat d7/c19 0 2024-03-13T14:50:04.992 INFO:tasks.workunit.client.0.smithi176.stdout:0/811: dread d2/d5/d28/fb5 [0,4194304] 0 2024-03-13T14:50:04.993 INFO:tasks.workunit.client.0.smithi176.stdout:7/862: creat d1/d1c/d26/d41/db6/df1/dfc/d76/f115 x:0 0 0 2024-03-13T14:50:04.995 INFO:tasks.workunit.client.0.smithi176.stdout:4/927: mkdir d7/d10/d139 0 2024-03-13T14:50:04.998 INFO:tasks.workunit.client.0.smithi176.stdout:7/863: symlink d1/d1c/d26/d41/db6/df1/dfc/l116 0 2024-03-13T14:50:05.002 INFO:tasks.workunit.client.1.smithi196.stdout:8/932: dwrite d0/d10/f26 [0,4194304] 0 2024-03-13T14:50:05.002 INFO:tasks.workunit.client.0.smithi176.stdout:4/928: mknod d7/d8/d15/d23/c13a 0 2024-03-13T14:50:05.003 INFO:tasks.workunit.client.0.smithi176.stdout:7/864: chown d1/d1c/d26/d41/db6/df1/dfc/d81/fc5 4040013 1 2024-03-13T14:50:05.003 INFO:tasks.workunit.client.0.smithi176.stdout:5/891: write d5/d2c/d40/dc7/f79 [835698,89707] 0 2024-03-13T14:50:05.004 INFO:tasks.workunit.client.0.smithi176.stdout:4/929: stat d7/d8/l51 0 2024-03-13T14:50:05.005 INFO:tasks.workunit.client.1.smithi196.stdout:8/933: chown d0/d10/d12/df5/d47/l5b 0 1 2024-03-13T14:50:05.007 INFO:tasks.workunit.client.0.smithi176.stdout:7/865: mknod d1/d1c/d26/d41/db6/c117 0 2024-03-13T14:50:05.008 INFO:tasks.workunit.client.0.smithi176.stdout:8/850: dread d0/d5/ffd [0,4194304] 0 2024-03-13T14:50:05.012 INFO:tasks.workunit.client.0.smithi176.stdout:5/892: creat d5/d2c/d40/f130 x:0 0 0 2024-03-13T14:50:05.016 INFO:tasks.workunit.client.0.smithi176.stdout:7/866: symlink d1/d1c/d26/d41/db6/df1/l118 0 2024-03-13T14:50:05.019 INFO:tasks.workunit.client.0.smithi176.stdout:0/812: dwrite d2/db/d35/daa/fb1 [0,4194304] 0 2024-03-13T14:50:05.026 INFO:tasks.workunit.client.1.smithi196.stdout:8/934: write d0/d10/d12/df5/d47/f80 [459455,117283] 0 2024-03-13T14:50:05.026 INFO:tasks.workunit.client.0.smithi176.stdout:7/867: dread d1/d28/f75 [0,4194304] 0 2024-03-13T14:50:05.027 INFO:tasks.workunit.client.0.smithi176.stdout:8/851: write d0/d5/d11/f68 [2226912,76098] 0 2024-03-13T14:50:05.028 INFO:tasks.workunit.client.0.smithi176.stdout:0/813: getdents d2/db 0 2024-03-13T14:50:05.031 INFO:tasks.workunit.client.1.smithi196.stdout:8/935: symlink d0/d10/d12/d1e/dc5/l135 0 2024-03-13T14:50:05.031 INFO:tasks.workunit.client.0.smithi176.stdout:0/814: stat d2/d74/d61/dfa 0 2024-03-13T14:50:05.034 INFO:tasks.workunit.client.0.smithi176.stdout:0/815: chown d2/d5/d1e/c89 12018713 1 2024-03-13T14:50:05.034 INFO:tasks.workunit.client.1.smithi196.stdout:8/936: creat d0/d16/d3d/d7f/dda/f136 x:0 0 0 2024-03-13T14:50:05.035 INFO:tasks.workunit.client.1.smithi196.stdout:8/937: fsync d0/d1c/f7c 0 2024-03-13T14:50:05.037 INFO:tasks.workunit.client.1.smithi196.stdout:8/938: write d0/d16/f11b [748101,84496] 0 2024-03-13T14:50:05.037 INFO:tasks.workunit.client.0.smithi176.stdout:8/852: rename d0/db/d39/d56/f9f to d0/f114 0 2024-03-13T14:50:05.038 INFO:tasks.workunit.client.0.smithi176.stdout:0/816: truncate d2/f82 135222 0 2024-03-13T14:50:05.038 INFO:tasks.workunit.client.1.smithi196.stdout:8/939: chown d0/d10/d12/d1e/db1 679738 1 2024-03-13T14:50:05.039 INFO:tasks.workunit.client.0.smithi176.stdout:8/853: chown d0/d24/d98/dc6/ff2 92035415 1 2024-03-13T14:50:05.042 INFO:tasks.workunit.client.1.smithi196.stdout:8/940: creat d0/d16/d71/d68/f137 x:0 0 0 2024-03-13T14:50:05.046 INFO:tasks.workunit.client.0.smithi176.stdout:8/854: creat d0/d24/d29/f115 x:0 0 0 2024-03-13T14:50:05.047 INFO:tasks.workunit.client.1.smithi196.stdout:2/943: sync 2024-03-13T14:50:05.049 INFO:tasks.workunit.client.0.smithi176.stdout:5/893: dread d5/d2b/f5b [0,4194304] 0 2024-03-13T14:50:05.053 INFO:tasks.workunit.client.0.smithi176.stdout:5/894: fsync d5/f9a 0 2024-03-13T14:50:05.054 INFO:tasks.workunit.client.0.smithi176.stdout:8/855: dread d0/d24/d7b/dab/f92 [0,4194304] 0 2024-03-13T14:50:05.055 INFO:tasks.workunit.client.0.smithi176.stdout:0/817: write d2/d74/f6e [1379107,11211] 0 2024-03-13T14:50:05.057 INFO:tasks.workunit.client.1.smithi196.stdout:8/941: dwrite d0/d10/f26 [4194304,4194304] 0 2024-03-13T14:50:05.057 INFO:tasks.workunit.client.0.smithi176.stdout:8/856: chown d0/d24/d98/dc6/l4b 3710 1 2024-03-13T14:50:05.059 INFO:tasks.workunit.client.1.smithi196.stdout:2/944: write d0/d6/d4b/f64 [411181,13522] 0 2024-03-13T14:50:05.063 INFO:tasks.workunit.client.0.smithi176.stdout:0/818: mkdir d2/d5/d28/d8c/d9d/d102 0 2024-03-13T14:50:05.064 INFO:tasks.workunit.client.0.smithi176.stdout:7/868: dread d1/d1c/d26/d41/db6/df1/dfc/d81/fe0 [0,4194304] 0 2024-03-13T14:50:05.064 INFO:tasks.workunit.client.0.smithi176.stdout:0/819: chown d2/d74/df8/cfb 1 1 2024-03-13T14:50:05.073 INFO:tasks.workunit.client.1.smithi196.stdout:2/945: write d0/d6/d4b/d5f/dce/ff1 [967364,84140] 0 2024-03-13T14:50:05.074 INFO:tasks.workunit.client.1.smithi196.stdout:2/946: write d0/d7/d9/d61/d83/fad [169203,66110] 0 2024-03-13T14:50:05.077 INFO:tasks.workunit.client.1.smithi196.stdout:2/947: symlink d0/d6/d34/l134 0 2024-03-13T14:50:05.085 INFO:tasks.workunit.client.1.smithi196.stdout:8/942: dwrite d0/d63/f94 [0,4194304] 0 2024-03-13T14:50:05.090 INFO:tasks.workunit.client.1.smithi196.stdout:8/943: creat d0/d10/d12/df5/f138 x:0 0 0 2024-03-13T14:50:05.092 INFO:tasks.workunit.client.1.smithi196.stdout:8/944: unlink d0/lc 0 2024-03-13T14:50:05.095 INFO:tasks.workunit.client.1.smithi196.stdout:2/948: dwrite d0/d7/d9/f48 [0,4194304] 0 2024-03-13T14:50:05.097 INFO:tasks.workunit.client.1.smithi196.stdout:8/945: mkdir d0/d10/da5/d110/d139 0 2024-03-13T14:50:05.103 INFO:tasks.workunit.client.1.smithi196.stdout:8/946: mkdir d0/d10/d12/d17/def/d13a 0 2024-03-13T14:50:05.111 INFO:tasks.workunit.client.0.smithi176.stdout:5/895: write d5/d2e/d7e/dad/d7b/da7/d111/d15/f21 [398328,22028] 0 2024-03-13T14:50:05.112 INFO:tasks.workunit.client.0.smithi176.stdout:0/820: readlink d2/d5/d28/l70 0 2024-03-13T14:50:05.115 INFO:tasks.workunit.client.1.smithi196.stdout:8/947: write d0/d10/d12/d17/d42/f130 [1902960,36691] 0 2024-03-13T14:50:05.116 INFO:tasks.workunit.client.0.smithi176.stdout:5/896: truncate d5/d2e/d5f/fdc 6576 0 2024-03-13T14:50:05.116 INFO:tasks.workunit.client.0.smithi176.stdout:7/869: dread d1/d28/d3a/d1b/d8f/dc1/dbb/f9f [0,4194304] 0 2024-03-13T14:50:05.118 INFO:tasks.workunit.client.1.smithi196.stdout:8/948: read d0/d16/d3d/d8b/d48/f5f [3109770,45755] 0 2024-03-13T14:50:05.118 INFO:tasks.workunit.client.0.smithi176.stdout:8/857: dread d0/db/fd [0,4194304] 0 2024-03-13T14:50:05.119 INFO:tasks.workunit.client.0.smithi176.stdout:0/821: fdatasync d2/d5/d1e/f72 0 2024-03-13T14:50:05.122 INFO:tasks.workunit.client.0.smithi176.stdout:5/897: rename d5/d2c/l117 to d5/d31/def/d10e/d118/l131 0 2024-03-13T14:50:05.124 INFO:tasks.workunit.client.0.smithi176.stdout:0/822: unlink d2/d5/d1e/f52 0 2024-03-13T14:50:05.125 INFO:tasks.workunit.client.0.smithi176.stdout:7/870: symlink d1/d28/d3a/d1b/l119 0 2024-03-13T14:50:05.126 INFO:tasks.workunit.client.0.smithi176.stdout:8/858: symlink d0/db/d39/d56/d93/l116 0 2024-03-13T14:50:05.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:04 smithi176 ceph-mon[29064]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 186 MiB/s wr, 404 op/s 2024-03-13T14:50:05.129 INFO:tasks.workunit.client.1.smithi196.stdout:2/949: dread d0/d7/d9/d42/d98/de4/d104/f23 [0,4194304] 0 2024-03-13T14:50:05.130 INFO:tasks.workunit.client.0.smithi176.stdout:5/898: creat d5/d2e/d7e/d96/dca/f132 x:0 0 0 2024-03-13T14:50:05.131 INFO:tasks.workunit.client.0.smithi176.stdout:8/859: rename d0/d24/d98/dc6/d40/f48 to d0/db/d39/d56/d93/f117 0 2024-03-13T14:50:05.132 INFO:tasks.workunit.client.1.smithi196.stdout:2/950: dread - d0/d7/d9/d42/ff8 zero size 2024-03-13T14:50:05.134 INFO:tasks.workunit.client.0.smithi176.stdout:5/899: read d5/d31/f3e [46235,89273] 0 2024-03-13T14:50:05.135 INFO:tasks.workunit.client.0.smithi176.stdout:8/860: creat d0/d24/d7b/f118 x:0 0 0 2024-03-13T14:50:05.137 INFO:tasks.workunit.client.1.smithi196.stdout:2/951: rename d0/d6/d4b to d0/d7/d9/d5a/dbc/d135 0 2024-03-13T14:50:05.139 INFO:tasks.workunit.client.0.smithi176.stdout:8/861: rename d0/d24/d29/d70/cac to d0/d24/d7b/c119 0 2024-03-13T14:50:05.141 INFO:tasks.workunit.client.1.smithi196.stdout:2/952: creat d0/d7/d9/d5a/dbc/d135/d5f/d12b/f136 x:0 0 0 2024-03-13T14:50:05.142 INFO:tasks.workunit.client.0.smithi176.stdout:8/862: fsync d0/d24/d29/f67 0 2024-03-13T14:50:05.145 INFO:tasks.workunit.client.1.smithi196.stdout:2/953: read d0/d7/d9/d5a/dbc/d135/d5f/f81 [1569928,127204] 0 2024-03-13T14:50:05.145 INFO:tasks.workunit.client.0.smithi176.stdout:8/863: truncate d0/d24/f32 2809147 0 2024-03-13T14:50:05.146 INFO:tasks.workunit.client.0.smithi176.stdout:8/864: chown d0/db/d53 2 1 2024-03-13T14:50:05.149 INFO:tasks.workunit.client.1.smithi196.stdout:2/954: unlink d0/d7/d9/d5a/dbc/d135/d5f/dce/ff9 0 2024-03-13T14:50:05.149 INFO:tasks.workunit.client.0.smithi176.stdout:8/865: readlink d0/db/d53/df7/lfe 0 2024-03-13T14:50:05.151 INFO:tasks.workunit.client.1.smithi196.stdout:2/955: chown d0/l2f 8670 1 2024-03-13T14:50:05.152 INFO:tasks.workunit.client.0.smithi176.stdout:8/866: unlink d0/db/d39/d56/d93/f110 0 2024-03-13T14:50:05.153 INFO:tasks.workunit.client.1.smithi196.stdout:2/956: creat d0/d7/d40/f137 x:0 0 0 2024-03-13T14:50:05.155 INFO:tasks.workunit.client.0.smithi176.stdout:8/867: write d0/d24/d29/fbd [546482,20929] 0 2024-03-13T14:50:05.173 INFO:tasks.workunit.client.1.smithi196.stdout:2/957: dwrite d0/d7/d9/f48 [4194304,4194304] 0 2024-03-13T14:50:05.183 INFO:tasks.workunit.client.1.smithi196.stdout:2/958: unlink d0/d7/d9/d42/d98/de4/d67/fcc 0 2024-03-13T14:50:05.186 INFO:tasks.workunit.client.1.smithi196.stdout:2/959: creat d0/d7/d40/f138 x:0 0 0 2024-03-13T14:50:05.190 INFO:tasks.workunit.client.1.smithi196.stdout:2/960: chown d0/d7/d9/d61/c74 34 1 2024-03-13T14:50:05.196 INFO:tasks.workunit.client.0.smithi176.stdout:7/871: truncate d1/d1c/d26/d41/db6/df1/dfc/d81/f8e 1522002 0 2024-03-13T14:50:05.198 INFO:tasks.workunit.client.0.smithi176.stdout:5/900: write d5/d31/d47/d80/f62 [3832627,118635] 0 2024-03-13T14:50:05.199 INFO:tasks.workunit.client.1.smithi196.stdout:8/949: dwrite d0/d16/d3d/d8b/f38 [0,4194304] 0 2024-03-13T14:50:05.204 INFO:tasks.workunit.client.1.smithi196.stdout:8/950: rmdir d0/d16/d3d/d8b/d48 39 2024-03-13T14:50:05.206 INFO:tasks.workunit.client.0.smithi176.stdout:0/823: dwrite d2/f82 [0,4194304] 0 2024-03-13T14:50:05.208 INFO:tasks.workunit.client.0.smithi176.stdout:8/868: write d0/d24/d98/dc6/fb1 [720868,96330] 0 2024-03-13T14:50:05.211 INFO:tasks.workunit.client.1.smithi196.stdout:2/961: write d0/d7/d9/d5a/dbc/d135/f7d [5007284,112165] 0 2024-03-13T14:50:05.211 INFO:tasks.workunit.client.1.smithi196.stdout:8/951: rename d0/d16/d71/cb6 to d0/d10/d12/d17/d42/dbe/c13b 0 2024-03-13T14:50:05.219 INFO:tasks.workunit.client.0.smithi176.stdout:5/901: write d5/d2c/d40/f8f [490316,126806] 0 2024-03-13T14:50:05.220 INFO:tasks.workunit.client.1.smithi196.stdout:8/952: link d0/d16/d3d/d7f/fbf d0/d10/d12/f13c 0 2024-03-13T14:50:05.222 INFO:tasks.workunit.client.1.smithi196.stdout:8/953: readlink d0/d10/d12/df5/l99 0 2024-03-13T14:50:05.228 INFO:tasks.workunit.client.0.smithi176.stdout:8/869: dwrite d0/f5c [0,4194304] 0 2024-03-13T14:50:05.229 INFO:tasks.workunit.client.1.smithi196.stdout:2/962: dwrite d0/d7/d9/d5a/dbc/d135/d5f/fb7 [0,4194304] 0 2024-03-13T14:50:05.234 INFO:tasks.workunit.client.0.smithi176.stdout:8/870: creat d0/d24/d7b/dab/f11a x:0 0 0 2024-03-13T14:50:05.238 INFO:tasks.workunit.client.0.smithi176.stdout:5/902: dwrite d5/d2c/d8a/fd4 [0,4194304] 0 2024-03-13T14:50:05.241 INFO:tasks.workunit.client.0.smithi176.stdout:3/932: sync 2024-03-13T14:50:05.243 INFO:tasks.workunit.client.0.smithi176.stdout:5/903: read - d5/d2c/d40/dc7/d53/de9/f124 zero size 2024-03-13T14:50:05.247 INFO:tasks.workunit.client.0.smithi176.stdout:8/871: link d0/ccf d0/db/d39/d56/d93/da7/c11b 0 2024-03-13T14:50:05.247 INFO:tasks.workunit.client.1.smithi196.stdout:8/954: dwrite d0/d10/d12/d17/f10f [0,4194304] 0 2024-03-13T14:50:05.255 INFO:tasks.workunit.client.1.smithi196.stdout:2/963: write d0/d7/d9/d42/d98/de4/d104/f51 [4662105,109258] 0 2024-03-13T14:50:05.257 INFO:tasks.workunit.client.0.smithi176.stdout:0/824: dread d2/f84 [0,4194304] 0 2024-03-13T14:50:05.258 INFO:tasks.workunit.client.1.smithi196.stdout:8/955: read d0/d10/d12/df5/f65 [3964203,11445] 0 2024-03-13T14:50:05.258 INFO:tasks.workunit.client.0.smithi176.stdout:5/904: dread d5/f24 [0,4194304] 0 2024-03-13T14:50:05.260 INFO:tasks.workunit.client.0.smithi176.stdout:8/872: mknod d0/d24/d29/d70/dca/c11c 0 2024-03-13T14:50:05.261 INFO:tasks.workunit.client.1.smithi196.stdout:2/964: mkdir d0/d7/d9/d5a/d139 0 2024-03-13T14:50:05.264 INFO:tasks.workunit.client.0.smithi176.stdout:0/825: creat d2/d5/d28/dac/f103 x:0 0 0 2024-03-13T14:50:05.264 INFO:tasks.workunit.client.0.smithi176.stdout:3/933: dwrite d4/d13/d81/f139 [0,4194304] 0 2024-03-13T14:50:05.264 INFO:tasks.workunit.client.0.smithi176.stdout:5/905: dread - d5/d2c/d40/dc7/ff0 zero size 2024-03-13T14:50:05.266 INFO:tasks.workunit.client.0.smithi176.stdout:8/873: unlink d0/d24/d98/dc6/d40/fcd 0 2024-03-13T14:50:05.268 INFO:tasks.workunit.client.1.smithi196.stdout:2/965: rename d0/d6/d34/l134 to d0/d7/d9/d5a/d139/l13a 0 2024-03-13T14:50:05.273 INFO:tasks.workunit.client.0.smithi176.stdout:5/906: unlink d5/d2e/d7e/dad/d7b/da7/d111/lc1 0 2024-03-13T14:50:05.273 INFO:tasks.workunit.client.1.smithi196.stdout:2/966: mkdir d0/d7/d9/d9a/d13b 0 2024-03-13T14:50:05.274 INFO:tasks.workunit.client.0.smithi176.stdout:8/874: mkdir d0/d24/d29/d82/d11d 0 2024-03-13T14:50:05.276 INFO:tasks.workunit.client.0.smithi176.stdout:3/934: getdents d4/d13/d81 0 2024-03-13T14:50:05.277 INFO:tasks.workunit.client.0.smithi176.stdout:3/935: stat d4/d28/d31 0 2024-03-13T14:50:05.278 INFO:tasks.workunit.client.0.smithi176.stdout:3/936: fdatasync d4/f1d 0 2024-03-13T14:50:05.279 INFO:tasks.workunit.client.1.smithi196.stdout:2/967: creat d0/d7/d9/d5a/dbc/d135/d5f/dce/f13c x:0 0 0 2024-03-13T14:50:05.282 INFO:tasks.workunit.client.0.smithi176.stdout:3/937: chown d4/d13/d2d 14987668 1 2024-03-13T14:50:05.282 INFO:tasks.workunit.client.0.smithi176.stdout:3/938: write d4/d28/fc0 [2824883,124259] 0 2024-03-13T14:50:05.282 INFO:tasks.workunit.client.0.smithi176.stdout:5/907: rename d5/d2e/d7e/dad/d7b/da7/d111/fa6 to d5/d2e/d7e/dad/d127/f133 0 2024-03-13T14:50:05.287 INFO:tasks.workunit.client.0.smithi176.stdout:3/939: symlink d4/d13/d81/d89/d5b/dc5/df0/l144 0 2024-03-13T14:50:05.291 INFO:tasks.workunit.client.0.smithi176.stdout:8/875: dwrite d0/d24/d29/f115 [0,4194304] 0 2024-03-13T14:50:05.292 INFO:tasks.workunit.client.1.smithi196.stdout:2/968: readlink d0/d7/d9/d5a/d139/l13a 0 2024-03-13T14:50:05.293 INFO:tasks.workunit.client.0.smithi176.stdout:8/876: write d0/d24/d7b/dab/d86/f104 [636100,81325] 0 2024-03-13T14:50:05.293 INFO:tasks.workunit.client.1.smithi196.stdout:2/969: stat d0/d7/d9/d5a/dbc/l10b 0 2024-03-13T14:50:05.297 INFO:tasks.workunit.client.1.smithi196.stdout:8/956: dwrite d0/d16/d3d/d7f/f114 [0,4194304] 0 2024-03-13T14:50:05.300 INFO:tasks.workunit.client.0.smithi176.stdout:0/826: dwrite d2/db/d39/fc8 [0,4194304] 0 2024-03-13T14:50:05.304 INFO:tasks.workunit.client.0.smithi176.stdout:5/908: write d5/d2e/f58 [27899,78685] 0 2024-03-13T14:50:05.304 INFO:tasks.workunit.client.1.smithi196.stdout:2/970: fsync d0/d7/d9/d42/d98/de4/d104/f7a 0 2024-03-13T14:50:05.306 INFO:tasks.workunit.client.1.smithi196.stdout:2/971: chown d0/d7/l1e 6482 1 2024-03-13T14:50:05.307 INFO:tasks.workunit.client.1.smithi196.stdout:2/972: chown d0/d6/fb3 82661 1 2024-03-13T14:50:05.308 INFO:tasks.workunit.client.0.smithi176.stdout:3/940: truncate d4/d13/d81/f139 304617 0 2024-03-13T14:50:05.312 INFO:tasks.workunit.client.0.smithi176.stdout:5/909: write d5/d2e/d7e/dad/d7b/da7/d111/d15/f21 [1051035,80316] 0 2024-03-13T14:50:05.312 INFO:tasks.workunit.client.0.smithi176.stdout:3/941: dread - d4/d28/d31/d45/d82/d12b/df5/ff8 zero size 2024-03-13T14:50:05.314 INFO:tasks.workunit.client.0.smithi176.stdout:0/827: link d2/d5/f64 d2/db/d31/f104 0 2024-03-13T14:50:05.315 INFO:tasks.workunit.client.1.smithi196.stdout:2/973: creat d0/d7/d9/d61/d83/d124/d133/f13d x:0 0 0 2024-03-13T14:50:05.317 INFO:tasks.workunit.client.1.smithi196.stdout:8/957: dwrite d0/d10/d79/f118 [0,4194304] 0 2024-03-13T14:50:05.318 INFO:tasks.workunit.client.0.smithi176.stdout:0/828: read d2/d5/d28/d8c/fcc [681580,54944] 0 2024-03-13T14:50:05.318 INFO:tasks.workunit.client.0.smithi176.stdout:5/910: stat d5/d2e/d7e/dad/d7b/da7/d111/l12 0 2024-03-13T14:50:05.321 INFO:tasks.workunit.client.0.smithi176.stdout:0/829: dread - d2/d5/dc/d59/fe0 zero size 2024-03-13T14:50:05.321 INFO:tasks.workunit.client.1.smithi196.stdout:8/958: write d0/d10/da5/d110/f11a [295984,96882] 0 2024-03-13T14:50:05.322 INFO:tasks.workunit.client.0.smithi176.stdout:8/877: dwrite d0/db/d53/d80/f97 [0,4194304] 0 2024-03-13T14:50:05.325 INFO:tasks.workunit.client.1.smithi196.stdout:8/959: chown d0/d10/d12/d1e/d2d/f10e 21154 1 2024-03-13T14:50:05.326 INFO:tasks.workunit.client.0.smithi176.stdout:3/942: truncate d4/f1d 2330253 0 2024-03-13T14:50:05.327 INFO:tasks.workunit.client.0.smithi176.stdout:5/911: symlink d5/d31/l134 0 2024-03-13T14:50:05.329 INFO:tasks.workunit.client.0.smithi176.stdout:8/878: fdatasync d0/d24/d98/dc6/f13 0 2024-03-13T14:50:05.330 INFO:tasks.workunit.client.1.smithi196.stdout:8/960: fdatasync d0/d10/d12/df5/d47/f52 0 2024-03-13T14:50:05.331 INFO:tasks.workunit.client.0.smithi176.stdout:3/943: write d4/d28/d34/dde/f117 [785851,113216] 0 2024-03-13T14:50:05.332 INFO:tasks.workunit.client.0.smithi176.stdout:0/830: dread d2/f68 [0,4194304] 0 2024-03-13T14:50:05.334 INFO:tasks.workunit.client.1.smithi196.stdout:8/961: truncate d0/d16/d71/f43 1326563 0 2024-03-13T14:50:05.336 INFO:tasks.workunit.client.1.smithi196.stdout:8/962: read - d0/d10/d79/f100 zero size 2024-03-13T14:50:05.337 INFO:tasks.workunit.client.0.smithi176.stdout:3/944: creat d4/d13/d16/db6/f145 x:0 0 0 2024-03-13T14:50:05.341 INFO:tasks.workunit.client.0.smithi176.stdout:3/945: mkdir d4/d28/d34/d146 0 2024-03-13T14:50:05.348 INFO:tasks.workunit.client.0.smithi176.stdout:8/879: dwrite d0/db/f19 [4194304,4194304] 0 2024-03-13T14:50:05.349 INFO:tasks.workunit.client.1.smithi196.stdout:2/974: write d0/d7/d9/d42/d98/de4/d104/f27 [4483505,55867] 0 2024-03-13T14:50:05.362 INFO:tasks.workunit.client.0.smithi176.stdout:8/880: write d0/fc7 [359762,63259] 0 2024-03-13T14:50:05.362 INFO:tasks.workunit.client.1.smithi196.stdout:8/963: dwrite d0/d16/d3d/d8b/f91 [0,4194304] 0 2024-03-13T14:50:05.364 INFO:tasks.workunit.client.1.smithi196.stdout:8/964: stat d0/d10/d12/c20 0 2024-03-13T14:50:05.365 INFO:tasks.workunit.client.0.smithi176.stdout:5/912: dwrite d5/d2c/d40/dc7/d89/fc4 [0,4194304] 0 2024-03-13T14:50:05.373 INFO:tasks.workunit.client.1.smithi196.stdout:2/975: dwrite d0/d6/fc4 [0,4194304] 0 2024-03-13T14:50:05.373 INFO:tasks.workunit.client.0.smithi176.stdout:8/881: truncate d0/d24/d29/f9d 1001318 0 2024-03-13T14:50:05.375 INFO:tasks.workunit.client.0.smithi176.stdout:3/946: dwrite d4/d13/d16/d3d/f4c [4194304,4194304] 0 2024-03-13T14:50:05.375 INFO:tasks.workunit.client.0.smithi176.stdout:5/913: getdents d5/d31/d47/d80 0 2024-03-13T14:50:05.376 INFO:tasks.workunit.client.1.smithi196.stdout:2/976: readlink d0/d7/d9/d42/d98/de4/l8d 0 2024-03-13T14:50:05.378 INFO:tasks.workunit.client.0.smithi176.stdout:8/882: truncate d0/db/fdc 415267 0 2024-03-13T14:50:05.381 INFO:tasks.workunit.client.0.smithi176.stdout:3/947: chown d4/f25 31987 1 2024-03-13T14:50:05.383 INFO:tasks.workunit.client.1.smithi196.stdout:2/977: creat d0/d7/d9/d61/d83/f13e x:0 0 0 2024-03-13T14:50:05.388 INFO:tasks.workunit.client.0.smithi176.stdout:0/831: dread d2/db/d31/f7f [0,4194304] 0 2024-03-13T14:50:05.388 INFO:tasks.workunit.client.1.smithi196.stdout:2/978: readlink d0/d50/lfb 0 2024-03-13T14:50:05.390 INFO:tasks.workunit.client.0.smithi176.stdout:8/883: dread d0/d5/f87 [0,4194304] 0 2024-03-13T14:50:05.394 INFO:tasks.workunit.client.1.smithi196.stdout:8/965: dwrite d0/d10/d12/df5/fa6 [0,4194304] 0 2024-03-13T14:50:05.397 INFO:tasks.workunit.client.0.smithi176.stdout:5/914: dwrite d5/d2e/d7e/ff5 [0,4194304] 0 2024-03-13T14:50:05.400 INFO:tasks.workunit.client.1.smithi196.stdout:8/966: rename d0/d16/d3d/d8b/c2b to d0/d10/d12/d1e/db1/d117/d133/c13d 0 2024-03-13T14:50:05.402 INFO:tasks.workunit.client.0.smithi176.stdout:5/915: rmdir d5/d2e/d7e/d96/dca 39 2024-03-13T14:50:05.404 INFO:tasks.workunit.client.1.smithi196.stdout:8/967: mknod d0/d1c/c13e 0 2024-03-13T14:50:05.406 INFO:tasks.workunit.client.1.smithi196.stdout:8/968: chown d0/d16/d71/d68/c90 3605296 1 2024-03-13T14:50:05.407 INFO:tasks.workunit.client.0.smithi176.stdout:0/832: dwrite d2/f68 [0,4194304] 0 2024-03-13T14:50:05.408 INFO:tasks.workunit.client.1.smithi196.stdout:8/969: chown d0/d10/d79/f118 911 1 2024-03-13T14:50:05.410 INFO:tasks.workunit.client.0.smithi176.stdout:5/916: fdatasync d5/d31/d47/d80/fbf 0 2024-03-13T14:50:05.410 INFO:tasks.workunit.client.0.smithi176.stdout:8/884: dwrite d0/d5/d11/f68 [0,4194304] 0 2024-03-13T14:50:05.418 INFO:tasks.workunit.client.0.smithi176.stdout:0/833: symlink d2/d74/d61/dfa/l105 0 2024-03-13T14:50:05.419 INFO:tasks.workunit.client.0.smithi176.stdout:5/917: symlink d5/d2c/d40/dc7/d53/l135 0 2024-03-13T14:50:05.420 INFO:tasks.workunit.client.0.smithi176.stdout:2/944: sync 2024-03-13T14:50:05.420 INFO:tasks.workunit.client.0.smithi176.stdout:6/944: sync 2024-03-13T14:50:05.422 INFO:tasks.workunit.client.0.smithi176.stdout:0/834: creat d2/d74/d9f/f106 x:0 0 0 2024-03-13T14:50:05.424 INFO:tasks.workunit.client.0.smithi176.stdout:2/945: write d1/dc/d2d/fca [6236233,66785] 0 2024-03-13T14:50:05.427 INFO:tasks.workunit.client.0.smithi176.stdout:2/946: chown d1/dc/d2d/d90/dcf/da0 26501557 1 2024-03-13T14:50:05.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:04 smithi196 ceph-mon[37486]: pgmap v21: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 79 MiB/s rd, 186 MiB/s wr, 404 op/s 2024-03-13T14:50:05.430 INFO:tasks.workunit.client.0.smithi176.stdout:8/885: dwrite d0/fc7 [0,4194304] 0 2024-03-13T14:50:05.433 INFO:tasks.workunit.client.0.smithi176.stdout:0/835: rmdir d2/db/d31 39 2024-03-13T14:50:05.433 INFO:tasks.workunit.client.0.smithi176.stdout:2/947: chown d1/dc/d2d/d59/d8b/daf/db2/cc6 262244 1 2024-03-13T14:50:05.434 INFO:tasks.workunit.client.1.smithi196.stdout:2/979: dwrite d0/d7/d56/dc8/ffa [0,4194304] 0 2024-03-13T14:50:05.439 INFO:tasks.workunit.client.0.smithi176.stdout:5/918: read d5/d2e/d7e/dad/d7b/da7/d111/f29 [63086,11555] 0 2024-03-13T14:50:05.440 INFO:tasks.workunit.client.0.smithi176.stdout:2/948: unlink d1/dc/d2d/d4f/fac 0 2024-03-13T14:50:05.441 INFO:tasks.workunit.client.0.smithi176.stdout:0/836: unlink d2/d5/d1e/f42 0 2024-03-13T14:50:05.441 INFO:tasks.workunit.client.1.smithi196.stdout:2/980: creat d0/d7/d9/d42/d98/f13f x:0 0 0 2024-03-13T14:50:05.442 INFO:tasks.workunit.client.0.smithi176.stdout:8/886: creat d0/d24/d98/f11e x:0 0 0 2024-03-13T14:50:05.444 INFO:tasks.workunit.client.0.smithi176.stdout:0/837: unlink d2/d5/dc/d59/f80 0 2024-03-13T14:50:05.445 INFO:tasks.workunit.client.0.smithi176.stdout:5/919: mkdir d5/d136 0 2024-03-13T14:50:05.446 INFO:tasks.workunit.client.1.smithi196.stdout:2/981: symlink d0/d7/d9/d42/d98/de4/d104/d131/l140 0 2024-03-13T14:50:05.448 INFO:tasks.workunit.client.0.smithi176.stdout:8/887: dread - d0/d24/d7b/dab/d86/fa8 zero size 2024-03-13T14:50:05.451 INFO:tasks.workunit.client.0.smithi176.stdout:3/948: dread f2 [8388608,4194304] 0 2024-03-13T14:50:05.451 INFO:tasks.workunit.client.0.smithi176.stdout:2/949: rename d1/dc/d2d/d59/d80/d5c/d78/fed to d1/f13f 0 2024-03-13T14:50:05.452 INFO:tasks.workunit.client.1.smithi196.stdout:8/970: write d0/d1c/f7b [1105477,3413] 0 2024-03-13T14:50:05.456 INFO:tasks.workunit.client.0.smithi176.stdout:6/945: write d0/de/f49 [1512855,123820] 0 2024-03-13T14:50:05.457 INFO:tasks.workunit.client.1.smithi196.stdout:2/982: link d0/fbb d0/d7/d9/d5a/dbc/d135/d5f/d12b/f141 0 2024-03-13T14:50:05.458 INFO:tasks.workunit.client.0.smithi176.stdout:6/946: chown d0/d9/l22 434507093 1 2024-03-13T14:50:05.461 INFO:tasks.workunit.client.0.smithi176.stdout:5/920: getdents d5/d2c/d74 0 2024-03-13T14:50:05.462 INFO:tasks.workunit.client.0.smithi176.stdout:2/950: creat d1/dc/f140 x:0 0 0 2024-03-13T14:50:05.463 INFO:tasks.workunit.client.1.smithi196.stdout:2/983: mknod d0/d7/d9/d5a/dbc/d135/d5f/d12b/c142 0 2024-03-13T14:50:05.464 INFO:tasks.workunit.client.0.smithi176.stdout:6/947: mkdir d0/d10/d19/d12b 0 2024-03-13T14:50:05.465 INFO:tasks.workunit.client.0.smithi176.stdout:0/838: link d2/db/c98 d2/d5/d28/d8c/d9d/c107 0 2024-03-13T14:50:05.466 INFO:tasks.workunit.client.1.smithi196.stdout:2/984: unlink d0/f77 0 2024-03-13T14:50:05.467 INFO:tasks.workunit.client.0.smithi176.stdout:3/949: dwrite d4/d13/d81/fb5 [4194304,4194304] 0 2024-03-13T14:50:05.469 INFO:tasks.workunit.client.0.smithi176.stdout:6/948: creat d0/d9/f12c x:0 0 0 2024-03-13T14:50:05.471 INFO:tasks.workunit.client.1.smithi196.stdout:2/985: creat d0/d7/d9/d5a/dbc/d135/d5f/dce/f143 x:0 0 0 2024-03-13T14:50:05.472 INFO:tasks.workunit.client.0.smithi176.stdout:8/888: write d0/fc7 [4870244,123064] 0 2024-03-13T14:50:05.473 INFO:tasks.workunit.client.0.smithi176.stdout:0/839: rename d2/d5/fb2 to d2/d74/df8/f108 0 2024-03-13T14:50:05.476 INFO:tasks.workunit.client.0.smithi176.stdout:3/950: unlink d4/d13/d81/d89/d5b/dc5/fa5 0 2024-03-13T14:50:05.478 INFO:tasks.workunit.client.0.smithi176.stdout:2/951: dread d1/dc/d2d/d59/d80/d7e/fe3 [0,4194304] 0 2024-03-13T14:50:05.479 INFO:tasks.workunit.client.0.smithi176.stdout:5/921: dwrite d5/d2e/d5f/f6f [0,4194304] 0 2024-03-13T14:50:05.479 INFO:tasks.workunit.client.0.smithi176.stdout:7/872: sync 2024-03-13T14:50:05.480 INFO:tasks.workunit.client.0.smithi176.stdout:4/930: sync 2024-03-13T14:50:05.480 INFO:tasks.workunit.client.0.smithi176.stdout:1/902: sync 2024-03-13T14:50:05.481 INFO:tasks.workunit.client.1.smithi196.stdout:2/986: mkdir d0/d7/d9/d42/d98/de4/d67/d116/d144 0 2024-03-13T14:50:05.481 INFO:tasks.workunit.client.0.smithi176.stdout:5/922: stat d5/d31/f61 0 2024-03-13T14:50:05.483 INFO:tasks.workunit.client.0.smithi176.stdout:0/840: mknod d2/d74/d61/dfa/c109 0 2024-03-13T14:50:05.483 INFO:tasks.workunit.client.0.smithi176.stdout:5/923: fdatasync d5/d2c/d40/dc7/d89/df2/f123 0 2024-03-13T14:50:05.485 INFO:tasks.workunit.client.0.smithi176.stdout:6/949: getdents d0/d10/d19/d12b 0 2024-03-13T14:50:05.487 INFO:tasks.workunit.client.1.smithi196.stdout:8/971: dwrite d0/d63/f109 [0,4194304] 0 2024-03-13T14:50:05.489 INFO:tasks.workunit.client.0.smithi176.stdout:4/931: rmdir d7/d8/d33 39 2024-03-13T14:50:05.490 INFO:tasks.workunit.client.0.smithi176.stdout:4/932: fsync d7/d8/d2f/fb4 0 2024-03-13T14:50:05.491 INFO:tasks.workunit.client.0.smithi176.stdout:7/873: readlink d1/d1c/d26/d41/d2b/d40/l49 0 2024-03-13T14:50:05.493 INFO:tasks.workunit.client.0.smithi176.stdout:1/903: rename d2/de/d10/d16/d40/d10b to d2/d30/d10f 0 2024-03-13T14:50:05.493 INFO:tasks.workunit.client.0.smithi176.stdout:1/904: chown d2/d30 945 1 2024-03-13T14:50:05.503 INFO:tasks.workunit.client.0.smithi176.stdout:6/950: symlink d0/d10/d19/d27/d2b/d2e/d3e/l12d 0 2024-03-13T14:50:05.507 INFO:tasks.workunit.client.0.smithi176.stdout:4/933: mknod d7/db9/dd8/c13b 0 2024-03-13T14:50:05.507 INFO:tasks.workunit.client.1.smithi196.stdout:8/972: dwrite d0/d16/f11b [0,4194304] 0 2024-03-13T14:50:05.509 INFO:tasks.workunit.client.1.smithi196.stdout:8/973: chown d0/d10/f77 216455804 1 2024-03-13T14:50:05.509 INFO:tasks.workunit.client.0.smithi176.stdout:7/874: rmdir d1/d1c/d26/d41/db6/df1/dfc/d81 39 2024-03-13T14:50:05.510 INFO:tasks.workunit.client.0.smithi176.stdout:1/905: dwrite d2/de/d3f/d50/d83/dcb/fb7 [4194304,4194304] 0 2024-03-13T14:50:05.511 INFO:tasks.workunit.client.1.smithi196.stdout:2/987: rmdir d0/d7/d9 39 2024-03-13T14:50:05.512 INFO:tasks.workunit.client.1.smithi196.stdout:2/988: dread - d0/d7/fd4 zero size 2024-03-13T14:50:05.513 INFO:tasks.workunit.client.0.smithi176.stdout:8/889: link d0/db/d39/d56/d91/cad d0/db/d39/d56/d93/c11f 0 2024-03-13T14:50:05.515 INFO:tasks.workunit.client.0.smithi176.stdout:8/890: chown d0/d24/l85 117187 1 2024-03-13T14:50:05.517 INFO:tasks.workunit.client.1.smithi196.stdout:8/974: fsync d0/d10/d12/df5/d47/f98 0 2024-03-13T14:50:05.523 INFO:tasks.workunit.client.0.smithi176.stdout:6/951: symlink d0/de/d38/d6b/da3/l12e 0 2024-03-13T14:50:05.524 INFO:tasks.workunit.client.0.smithi176.stdout:5/924: write d5/d2e/d7e/dad/d7b/da7/d111/d15/f16 [1418861,86331] 0 2024-03-13T14:50:05.525 INFO:tasks.workunit.client.0.smithi176.stdout:6/952: chown d0 7 1 2024-03-13T14:50:05.526 INFO:tasks.workunit.client.0.smithi176.stdout:1/906: dwrite d2/de/d10/d16/d40/f77 [0,4194304] 0 2024-03-13T14:50:05.530 INFO:tasks.workunit.client.0.smithi176.stdout:3/951: getdents d4/d13/d16 0 2024-03-13T14:50:05.531 INFO:tasks.workunit.client.0.smithi176.stdout:7/875: creat d1/d1c/d26/d41/d2b/d40/d64/d94/f11a x:0 0 0 2024-03-13T14:50:05.535 INFO:tasks.workunit.client.0.smithi176.stdout:8/891: truncate d0/d5/d11/f76 177098 0 2024-03-13T14:50:05.536 INFO:tasks.workunit.client.0.smithi176.stdout:4/934: symlink d7/d10/d46/d4a/dcc/dd0/d11d/l13c 0 2024-03-13T14:50:05.536 INFO:tasks.workunit.client.1.smithi196.stdout:2/989: dwrite d0/d6/d34/f7e [0,4194304] 0 2024-03-13T14:50:05.537 INFO:tasks.workunit.client.0.smithi176.stdout:1/907: symlink d2/d30/d63/d22/l110 0 2024-03-13T14:50:05.542 INFO:tasks.workunit.client.0.smithi176.stdout:7/876: chown d1/d1c/d26/d41/db6/c117 64611324 1 2024-03-13T14:50:05.546 INFO:tasks.workunit.client.0.smithi176.stdout:3/952: creat d4/d28/d31/d45/f147 x:0 0 0 2024-03-13T14:50:05.547 INFO:tasks.workunit.client.1.smithi196.stdout:2/990: symlink d0/d7/d9/d5a/l145 0 2024-03-13T14:50:05.549 INFO:tasks.workunit.client.0.smithi176.stdout:7/877: rmdir d1/d1c 39 2024-03-13T14:50:05.550 INFO:tasks.workunit.client.0.smithi176.stdout:2/952: dwrite d1/dc/f34 [4194304,4194304] 0 2024-03-13T14:50:05.552 INFO:tasks.workunit.client.1.smithi196.stdout:8/975: write d0/d10/da5/d110/f76 [103718,126237] 0 2024-03-13T14:50:05.563 INFO:tasks.workunit.client.0.smithi176.stdout:4/935: mkdir d7/d8/d33/d96/d13d 0 2024-03-13T14:50:05.568 INFO:tasks.workunit.client.0.smithi176.stdout:0/841: dwrite d2/db/d39/fa2 [0,4194304] 0 2024-03-13T14:50:05.569 INFO:tasks.workunit.client.0.smithi176.stdout:6/953: dread d0/de/d38/d88/d9d/f104 [0,4194304] 0 2024-03-13T14:50:05.572 INFO:tasks.workunit.client.0.smithi176.stdout:0/842: chown d2/d5/d28/dac/f103 8542 1 2024-03-13T14:50:05.572 INFO:tasks.workunit.client.0.smithi176.stdout:5/925: dwrite d5/d2c/d40/dc7/f6e [0,4194304] 0 2024-03-13T14:50:05.574 INFO:tasks.workunit.client.0.smithi176.stdout:8/892: dwrite d0/f52 [0,4194304] 0 2024-03-13T14:50:05.577 INFO:tasks.workunit.client.0.smithi176.stdout:3/953: write d4/d13/d81/d89/f7c [459884,84377] 0 2024-03-13T14:50:05.583 INFO:tasks.workunit.client.1.smithi196.stdout:2/991: truncate d0/d7/d9/d42/d98/de4/d104/f23 1971774 0 2024-03-13T14:50:05.583 INFO:tasks.workunit.client.0.smithi176.stdout:2/953: dread d1/dc/fae [0,4194304] 0 2024-03-13T14:50:05.590 INFO:tasks.workunit.client.1.smithi196.stdout:2/992: mkdir d0/d7/d9/d5a/dbc/d135/d5f/d146 0 2024-03-13T14:50:05.600 INFO:tasks.workunit.client.0.smithi176.stdout:6/954: dread - d0/d10/d19/d7d/fdf zero size 2024-03-13T14:50:05.603 INFO:tasks.workunit.client.0.smithi176.stdout:5/926: mkdir d5/d31/d86/dc0/d11a/d120/d137 0 2024-03-13T14:50:05.606 INFO:tasks.workunit.client.0.smithi176.stdout:8/893: mkdir d0/d5/d11/d47/db7/de0/d120 0 2024-03-13T14:50:05.606 INFO:tasks.workunit.client.0.smithi176.stdout:3/954: rmdir d4/d13/d81/d89/d5b/dc5/df0/d100 39 2024-03-13T14:50:05.607 INFO:tasks.workunit.client.0.smithi176.stdout:1/908: dread d2/fb8 [0,4194304] 0 2024-03-13T14:50:05.612 INFO:tasks.workunit.client.1.smithi196.stdout:2/993: dwrite d0/d7/d9/d42/d98/de4/fe6 [0,4194304] 0 2024-03-13T14:50:05.613 INFO:tasks.workunit.client.0.smithi176.stdout:3/955: rmdir d4/d13 39 2024-03-13T14:50:05.614 INFO:tasks.workunit.client.0.smithi176.stdout:7/878: dread d1/d1c/d26/d41/d2b/f85 [0,4194304] 0 2024-03-13T14:50:05.614 INFO:tasks.workunit.client.0.smithi176.stdout:8/894: mknod d0/d24/d98/dc5/c121 0 2024-03-13T14:50:05.616 INFO:tasks.workunit.client.0.smithi176.stdout:1/909: creat d2/de/dbf/f111 x:0 0 0 2024-03-13T14:50:05.617 INFO:tasks.workunit.client.0.smithi176.stdout:0/843: dread d2/db/d35/daa/fb1 [0,4194304] 0 2024-03-13T14:50:05.620 INFO:tasks.workunit.client.1.smithi196.stdout:2/994: mknod d0/d7/d9/d61/dd3/c147 0 2024-03-13T14:50:05.620 INFO:tasks.workunit.client.1.smithi196.stdout:2/995: readlink d0/d6/l58 0 2024-03-13T14:50:05.624 INFO:tasks.workunit.client.0.smithi176.stdout:5/927: getdents d5/d2e/d43/d12b 0 2024-03-13T14:50:05.632 INFO:tasks.workunit.client.0.smithi176.stdout:5/928: symlink d5/d2c/d74/l138 0 2024-03-13T14:50:05.634 INFO:tasks.workunit.client.1.smithi196.stdout:2/996: getdents d0/d7/d9/d42/d98/de4/d67 0 2024-03-13T14:50:05.634 INFO:tasks.workunit.client.0.smithi176.stdout:6/955: dread d0/d10/d19/d27/d2b/d2e/f7a [0,4194304] 0 2024-03-13T14:50:05.635 INFO:tasks.workunit.client.1.smithi196.stdout:2/997: chown d0/d6/d34/d5b 742509 1 2024-03-13T14:50:05.636 INFO:tasks.workunit.client.1.smithi196.stdout:2/998: truncate d0/d7/f89 4991460 0 2024-03-13T14:50:05.637 INFO:tasks.workunit.client.0.smithi176.stdout:5/929: mkdir d5/d2e/d7e/dad/d127/d139 0 2024-03-13T14:50:05.641 INFO:tasks.workunit.client.0.smithi176.stdout:1/910: rename d2/de/d3f/d3e/d86/db3/fc9 to d2/de/d3f/d50/f112 0 2024-03-13T14:50:05.643 INFO:tasks.workunit.client.1.smithi196.stdout:2/999: symlink d0/d7/l148 0 2024-03-13T14:50:05.644 INFO:tasks.workunit.client.0.smithi176.stdout:7/879: dread d1/d1c/d4a/f8d [0,4194304] 0 2024-03-13T14:50:05.644 INFO:tasks.workunit.client.0.smithi176.stdout:5/930: creat d5/d31/d86/dc0/d11a/d120/d12c/f13a x:0 0 0 2024-03-13T14:50:05.644 INFO:tasks.workunit.client.0.smithi176.stdout:1/911: creat d2/de/d10/d16/d47/f113 x:0 0 0 2024-03-13T14:50:05.646 INFO:tasks.workunit.client.0.smithi176.stdout:3/956: dread d4/d13/d16/d80/faf [0,4194304] 0 2024-03-13T14:50:05.647 INFO:tasks.workunit.client.0.smithi176.stdout:5/931: creat d5/d2e/d7e/dad/d7b/f13b x:0 0 0 2024-03-13T14:50:05.649 INFO:tasks.workunit.client.0.smithi176.stdout:7/880: fsync d1/d1c/d26/d41/db6/df1/dfc/d4b/fc2 0 2024-03-13T14:50:05.656 INFO:tasks.workunit.client.0.smithi176.stdout:4/936: dwrite d7/d2d/fa4 [0,4194304] 0 2024-03-13T14:50:05.656 INFO:tasks.workunit.client.0.smithi176.stdout:1/912: dread d2/de/d10/d16/d47/f5c [0,4194304] 0 2024-03-13T14:50:05.660 INFO:tasks.workunit.client.0.smithi176.stdout:7/881: mknod d1/d28/c11b 0 2024-03-13T14:50:05.660 INFO:tasks.workunit.client.0.smithi176.stdout:8/895: truncate d0/fc7 2708505 0 2024-03-13T14:50:05.662 INFO:tasks.workunit.client.0.smithi176.stdout:1/913: unlink d2/de/d10/d16/d4a/d5f/c60 0 2024-03-13T14:50:05.662 INFO:tasks.workunit.client.0.smithi176.stdout:0/844: write d2/d5/d1e/d97/dcf/fec [3476788,101194] 0 2024-03-13T14:50:05.665 INFO:tasks.workunit.client.0.smithi176.stdout:3/957: write d4/d28/d31/d45/d82/d12b/df5/ff8 [264519,57277] 0 2024-03-13T14:50:05.667 INFO:tasks.workunit.client.0.smithi176.stdout:4/937: truncate d7/d10/d2c/df1/f104 473102 0 2024-03-13T14:50:05.672 INFO:tasks.workunit.client.0.smithi176.stdout:1/914: chown d2/de/d3f/d19/lf1 21 1 2024-03-13T14:50:05.677 INFO:tasks.workunit.client.0.smithi176.stdout:2/954: dwrite d1/dc/d2d/d90/fc9 [0,4194304] 0 2024-03-13T14:50:05.681 INFO:tasks.workunit.client.0.smithi176.stdout:0/845: creat d2/d5/dc/d20/f10a x:0 0 0 2024-03-13T14:50:05.681 INFO:tasks.workunit.client.0.smithi176.stdout:2/955: readlink d1/dc/d1c/d107/l10c 0 2024-03-13T14:50:05.684 INFO:tasks.workunit.client.0.smithi176.stdout:4/938: symlink d7/d2d/da7/l13e 0 2024-03-13T14:50:05.684 INFO:tasks.workunit.client.0.smithi176.stdout:2/956: read - d1/dc/d2d/ff7 zero size 2024-03-13T14:50:05.685 INFO:tasks.workunit.client.0.smithi176.stdout:5/932: dwrite d5/d31/f61 [0,4194304] 0 2024-03-13T14:50:05.685 INFO:tasks.workunit.client.0.smithi176.stdout:3/958: read d4/d13/d81/d89/f60 [408602,34835] 0 2024-03-13T14:50:05.694 INFO:tasks.workunit.client.0.smithi176.stdout:3/959: fdatasync d4/d13/d16/f84 0 2024-03-13T14:50:05.696 INFO:tasks.workunit.client.0.smithi176.stdout:0/846: rename d2/d5/d28/d8c/d9d/fc7 to d2/d5/f10b 0 2024-03-13T14:50:05.697 INFO:tasks.workunit.client.0.smithi176.stdout:5/933: symlink d5/d2e/d7e/dad/d7b/da7/l13c 0 2024-03-13T14:50:05.700 INFO:tasks.workunit.client.0.smithi176.stdout:2/957: creat d1/dc/d61/d104/d105/f141 x:0 0 0 2024-03-13T14:50:05.702 INFO:tasks.workunit.client.0.smithi176.stdout:0/847: write d2/d5/d28/dac/f103 [827105,126841] 0 2024-03-13T14:50:05.702 INFO:tasks.workunit.client.0.smithi176.stdout:3/960: write d4/d13/f19 [7063234,118501] 0 2024-03-13T14:50:05.703 INFO:tasks.workunit.client.0.smithi176.stdout:7/882: dwrite d1/d1c/d26/d41/d2b/d40/d64/f60 [0,4194304] 0 2024-03-13T14:50:05.706 INFO:tasks.workunit.client.0.smithi176.stdout:5/934: rmdir d5/d2e/d7e/dad/d127/d139 0 2024-03-13T14:50:05.711 INFO:tasks.workunit.client.0.smithi176.stdout:2/958: fsync d1/dc/d2d/d59/d80/fbb 0 2024-03-13T14:50:05.711 INFO:tasks.workunit.client.0.smithi176.stdout:5/935: rmdir d5/d2c/d74 39 2024-03-13T14:50:05.711 INFO:tasks.workunit.client.0.smithi176.stdout:4/939: dwrite d7/ffb [0,4194304] 0 2024-03-13T14:50:05.715 INFO:tasks.workunit.client.0.smithi176.stdout:5/936: chown d5/c11 90575796 1 2024-03-13T14:50:05.718 INFO:tasks.workunit.client.0.smithi176.stdout:0/848: rename d2/f11 to d2/d5/d28/f10c 0 2024-03-13T14:50:05.719 INFO:tasks.workunit.client.0.smithi176.stdout:3/961: dread d4/d28/d31/f75 [0,4194304] 0 2024-03-13T14:50:05.722 INFO:tasks.workunit.client.0.smithi176.stdout:3/962: write d4/d13/d16/db6/de8/f13c [592405,89792] 0 2024-03-13T14:50:05.722 INFO:tasks.workunit.client.0.smithi176.stdout:0/849: creat d2/db/d39/d9c/f10d x:0 0 0 2024-03-13T14:50:05.728 INFO:tasks.workunit.client.0.smithi176.stdout:3/963: fsync d4/d13/d16/dad/fe6 0 2024-03-13T14:50:05.730 INFO:tasks.workunit.client.0.smithi176.stdout:0/850: mknod d2/d5/d28/dac/c10e 0 2024-03-13T14:50:05.734 INFO:tasks.workunit.client.0.smithi176.stdout:2/959: write d1/d2a/fbf [12838,80919] 0 2024-03-13T14:50:05.735 INFO:tasks.workunit.client.0.smithi176.stdout:0/851: mkdir d2/d74/d10f 0 2024-03-13T14:50:05.738 INFO:tasks.workunit.client.0.smithi176.stdout:0/852: mkdir d2/db/d39/d9c/d110 0 2024-03-13T14:50:05.740 INFO:tasks.workunit.client.0.smithi176.stdout:7/883: dread d1/d1c/d26/d41/db6/df1/dfc/d81/f8e [0,4194304] 0 2024-03-13T14:50:05.744 INFO:tasks.workunit.client.0.smithi176.stdout:0/853: rename d2/d5/d28/f56 to d2/d5/dc/d20/f111 0 2024-03-13T14:50:05.746 INFO:tasks.workunit.client.0.smithi176.stdout:4/940: dwrite d7/d8/d15/d94/f12f [4194304,4194304] 0 2024-03-13T14:50:05.750 INFO:tasks.workunit.client.0.smithi176.stdout:7/884: write d1/d1c/d26/d41/db6/df1/dfc/d65/ff7 [485435,28296] 0 2024-03-13T14:50:05.752 INFO:tasks.workunit.client.0.smithi176.stdout:0/854: chown d2/db/c27 3 1 2024-03-13T14:50:05.753 INFO:tasks.workunit.client.0.smithi176.stdout:4/941: fdatasync d7/d10/f44 0 2024-03-13T14:50:05.755 INFO:tasks.workunit.client.0.smithi176.stdout:3/964: dwrite d4/d28/d4f/ffd [0,4194304] 0 2024-03-13T14:50:05.756 INFO:tasks.workunit.client.0.smithi176.stdout:7/885: fdatasync d1/d28/d3a/d1b/d8f/dd1/df4/ffd 0 2024-03-13T14:50:05.759 INFO:tasks.workunit.client.0.smithi176.stdout:3/965: chown d4/d13/d16/d3d/f4c 347 1 2024-03-13T14:50:05.759 INFO:tasks.workunit.client.0.smithi176.stdout:4/942: mkdir d7/d8/d15/d23/d13f 0 2024-03-13T14:50:05.762 INFO:tasks.workunit.client.0.smithi176.stdout:3/966: stat d4/d13/d81/d89/d5b/c7b 0 2024-03-13T14:50:05.766 INFO:tasks.workunit.client.0.smithi176.stdout:2/960: dread d1/dc/d2d/f7a [0,4194304] 0 2024-03-13T14:50:05.766 INFO:tasks.workunit.client.0.smithi176.stdout:0/855: creat d2/d5/d28/d8c/d9d/d102/f112 x:0 0 0 2024-03-13T14:50:05.767 INFO:tasks.workunit.client.0.smithi176.stdout:4/943: mkdir d7/d140 0 2024-03-13T14:50:05.769 INFO:tasks.workunit.client.0.smithi176.stdout:3/967: rename d4/d13/d81 to d4/d13/d16/db6/d148 0 2024-03-13T14:50:05.770 INFO:tasks.workunit.client.0.smithi176.stdout:2/961: mknod d1/dc/d2d/d59/d80/d77/c142 0 2024-03-13T14:50:05.771 INFO:tasks.workunit.client.0.smithi176.stdout:7/886: dread d1/d28/d3a/d1b/d8f/dc1/ddb/f104 [0,4194304] 0 2024-03-13T14:50:05.778 INFO:tasks.workunit.client.0.smithi176.stdout:2/962: fdatasync d1/dc/d2d/d4b/f63 0 2024-03-13T14:50:05.779 INFO:tasks.workunit.client.0.smithi176.stdout:7/887: mknod d1/d1c/d26/d41/d2b/d40/c11c 0 2024-03-13T14:50:05.780 INFO:tasks.workunit.client.0.smithi176.stdout:2/963: mknod d1/dc/de0/c143 0 2024-03-13T14:50:05.780 INFO:tasks.workunit.client.0.smithi176.stdout:7/888: dread - d1/f103 zero size 2024-03-13T14:50:05.783 INFO:tasks.workunit.client.0.smithi176.stdout:0/856: dwrite d2/db/d39/fc8 [0,4194304] 0 2024-03-13T14:50:05.783 INFO:tasks.workunit.client.0.smithi176.stdout:2/964: readlink d1/d2a/l72 0 2024-03-13T14:50:05.784 INFO:tasks.workunit.client.0.smithi176.stdout:0/857: chown d2/db/d35/cdf 7040 1 2024-03-13T14:50:05.785 INFO:tasks.workunit.client.0.smithi176.stdout:7/889: mkdir d1/d1c/d26/d41/db6/df1/dfc/d76/da5/d101/d11d 0 2024-03-13T14:50:05.789 INFO:tasks.workunit.client.0.smithi176.stdout:2/965: write d1/dc/d2d/d59/d80/d7e/da3/f11a [235084,42865] 0 2024-03-13T14:50:05.789 INFO:tasks.workunit.client.0.smithi176.stdout:0/858: symlink d2/db/d39/l113 0 2024-03-13T14:50:05.794 INFO:tasks.workunit.client.0.smithi176.stdout:2/966: dread - d1/dc/d2d/d59/f12a zero size 2024-03-13T14:50:05.794 INFO:tasks.workunit.client.0.smithi176.stdout:3/968: dwrite d4/d28/d31/d45/d82/fb3 [0,4194304] 0 2024-03-13T14:50:05.797 INFO:tasks.workunit.client.0.smithi176.stdout:2/967: stat d1/dc/d2d/d38/d5d/l9f 0 2024-03-13T14:50:05.798 INFO:tasks.workunit.client.0.smithi176.stdout:3/969: chown d4/d28/d31/d45/d82/d12b/df5/f3c 159349 1 2024-03-13T14:50:05.803 INFO:tasks.workunit.client.0.smithi176.stdout:3/970: creat d4/d28/d34/d11c/f149 x:0 0 0 2024-03-13T14:50:05.805 INFO:tasks.workunit.client.0.smithi176.stdout:2/968: rename d1/dc/d2d/d90/dcf to d1/dc/d2d/d90/d99/d144 0 2024-03-13T14:50:05.807 INFO:tasks.workunit.client.0.smithi176.stdout:3/971: mknod d4/d28/c14a 0 2024-03-13T14:50:05.807 INFO:tasks.workunit.client.0.smithi176.stdout:4/944: dread d7/d10/d46/f8f [0,4194304] 0 2024-03-13T14:50:05.810 INFO:tasks.workunit.client.0.smithi176.stdout:7/890: dwrite d1/d1c/d26/d41/db6/df1/dfc/d81/f102 [0,4194304] 0 2024-03-13T14:50:05.814 INFO:tasks.workunit.client.0.smithi176.stdout:2/969: rmdir d1/dc/d2d/d59 39 2024-03-13T14:50:05.818 INFO:tasks.workunit.client.0.smithi176.stdout:3/972: mknod d4/d28/d31/d45/d82/d87/c14b 0 2024-03-13T14:50:05.819 INFO:tasks.workunit.client.0.smithi176.stdout:7/891: mknod d1/d1c/d26/d41/c11e 0 2024-03-13T14:50:05.821 INFO:tasks.workunit.client.0.smithi176.stdout:7/892: chown d1/d1c/d26/d41/db6/df1 27 1 2024-03-13T14:50:05.823 INFO:tasks.workunit.client.0.smithi176.stdout:3/973: read - d4/d13/d16/d113/fe3 zero size 2024-03-13T14:50:05.825 INFO:tasks.workunit.client.0.smithi176.stdout:2/970: fdatasync d1/dc/d2d/d59/d80/d7e/da3/fb9 0 2024-03-13T14:50:05.827 INFO:tasks.workunit.client.0.smithi176.stdout:7/893: rename d1/d1c/d26/d41/d2b/d40/d64/l10c to d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/d7a/l11f 0 2024-03-13T14:50:05.878 INFO:tasks.workunit.client.0.smithi176.stdout:2/971: unlink d1/dc/d2d/d59/d80/d76/f112 0 2024-03-13T14:50:05.880 INFO:tasks.workunit.client.0.smithi176.stdout:3/974: getdents d4/d13/d16/d113 0 2024-03-13T14:50:05.880 INFO:tasks.workunit.client.1.smithi196.stdout:8/976: sync 2024-03-13T14:50:05.881 INFO:tasks.workunit.client.0.smithi176.stdout:7/894: mkdir d1/d1c/d26/d41/d2b/d120 0 2024-03-13T14:50:05.885 INFO:tasks.workunit.client.0.smithi176.stdout:4/945: dwrite d7/d8/d33/fd2 [0,4194304] 0 2024-03-13T14:50:05.885 INFO:tasks.workunit.client.1.smithi196.stdout:8/977: mkdir d0/d10/d79/d13f 0 2024-03-13T14:50:05.899 INFO:tasks.workunit.client.0.smithi176.stdout:7/895: mkdir d1/d28/d3a/d1b/d8f/d121 0 2024-03-13T14:50:05.902 INFO:tasks.workunit.client.0.smithi176.stdout:3/975: write d4/d13/d5e/f9a [2053101,50625] 0 2024-03-13T14:50:05.906 INFO:tasks.workunit.client.0.smithi176.stdout:4/946: dwrite d7/f120 [0,4194304] 0 2024-03-13T14:50:05.906 INFO:tasks.workunit.client.0.smithi176.stdout:7/896: mknod d1/d28/d3a/d1b/c122 0 2024-03-13T14:50:05.908 INFO:tasks.workunit.client.0.smithi176.stdout:4/947: stat d7/db9/fe7 0 2024-03-13T14:50:05.910 INFO:tasks.workunit.client.0.smithi176.stdout:3/976: rmdir d4/d13/d16/db6/d148/d89/d5b/dc5/df0/d100 39 2024-03-13T14:50:05.926 INFO:tasks.workunit.client.1.smithi196.stdout:8/978: dwrite d0/f51 [4194304,4194304] 0 2024-03-13T14:50:05.928 INFO:tasks.workunit.client.0.smithi176.stdout:7/897: rename d1/d1c/d26/d41/d2b/d40/c7d to d1/d1c/d26/d41/db6/df1/dfc/d65/c123 0 2024-03-13T14:50:05.933 INFO:tasks.workunit.client.0.smithi176.stdout:4/948: getdents d7/d8/d33/d96/d111 0 2024-03-13T14:50:05.933 INFO:tasks.workunit.client.0.smithi176.stdout:3/977: dwrite d4/d28/d31/d45/d82/d12b/df5/ff8 [0,4194304] 0 2024-03-13T14:50:05.942 INFO:tasks.workunit.client.0.smithi176.stdout:4/949: unlink d7/d10/d2c/d124/f12c 0 2024-03-13T14:50:05.957 INFO:tasks.workunit.client.0.smithi176.stdout:3/978: getdents d4/d28/d31/d52/de9 0 2024-03-13T14:50:05.961 INFO:tasks.workunit.client.0.smithi176.stdout:7/898: dread d1/d1c/d26/d41/d2b/d40/d64/d94/f83 [0,4194304] 0 2024-03-13T14:50:05.972 INFO:tasks.workunit.client.0.smithi176.stdout:7/899: truncate d1/d28/d3a/d1b/d8f/dc1/db7/fd0 27308 0 2024-03-13T14:50:05.973 INFO:tasks.workunit.client.0.smithi176.stdout:4/950: getdents d7/d10/d2c/d124 0 2024-03-13T14:50:05.984 INFO:tasks.workunit.client.0.smithi176.stdout:7/900: creat d1/d1c/d26/d41/d2b/f124 x:0 0 0 2024-03-13T14:50:05.999 INFO:tasks.workunit.client.0.smithi176.stdout:3/979: write d4/d13/d16/db6/d148/d89/d5b/dc5/df0/d100/f110 [197521,64571] 0 2024-03-13T14:50:06.005 INFO:tasks.workunit.client.0.smithi176.stdout:4/951: dread d7/d8/d45/f5c [0,4194304] 0 2024-03-13T14:50:06.006 INFO:tasks.workunit.client.1.smithi196.stdout:8/979: mkdir d0/d10/d12/d17/d42/dbe/d140 0 2024-03-13T14:50:06.008 INFO:tasks.workunit.client.0.smithi176.stdout:3/980: dread d4/d13/d5e/f79 [0,4194304] 0 2024-03-13T14:50:06.011 INFO:tasks.workunit.client.1.smithi196.stdout:8/980: write d0/d10/da5/de4/f12d [1506160,115973] 0 2024-03-13T14:50:06.012 INFO:tasks.workunit.client.0.smithi176.stdout:7/901: dread d1/d1c/d26/d41/d2b/d40/d64/d94/f7e [0,4194304] 0 2024-03-13T14:50:06.013 INFO:tasks.workunit.client.0.smithi176.stdout:3/981: symlink d4/d13/d16/db6/d148/l14c 0 2024-03-13T14:50:06.016 INFO:tasks.workunit.client.0.smithi176.stdout:6/956: sync 2024-03-13T14:50:06.016 INFO:tasks.workunit.client.0.smithi176.stdout:7/902: chown d1/d1c/d26/d41/db6/df1/dfc/d76/c2e 223 1 2024-03-13T14:50:06.018 INFO:tasks.workunit.client.1.smithi196.stdout:8/981: truncate d0/d10/d12/d1e/fbb 4895939 0 2024-03-13T14:50:06.025 INFO:tasks.workunit.client.1.smithi196.stdout:8/982: rename d0/d10/d12/d17/d42/dbe/d132 to d0/d10/d12/d17/d42/dbe/d141 0 2024-03-13T14:50:06.029 INFO:tasks.workunit.client.0.smithi176.stdout:6/957: rename d0/de/d8d/fbd to d0/d10/d19/d27/d2b/de0/f12f 0 2024-03-13T14:50:06.031 INFO:tasks.workunit.client.0.smithi176.stdout:3/982: dwrite d4/d13/d16/db6/fc4 [4194304,4194304] 0 2024-03-13T14:50:06.033 INFO:tasks.workunit.client.0.smithi176.stdout:6/958: stat d0/d10/d19/d27/d2b/d2e/dc4/dc5/fe7 0 2024-03-13T14:50:06.036 INFO:tasks.workunit.client.0.smithi176.stdout:4/952: write d7/d8/d2f/f58 [369281,95086] 0 2024-03-13T14:50:06.039 INFO:tasks.workunit.client.0.smithi176.stdout:3/983: fdatasync d4/d13/d16/db6/d148/f139 0 2024-03-13T14:50:06.040 INFO:tasks.workunit.client.0.smithi176.stdout:4/953: chown d7/d8/d45/d11a 191432869 1 2024-03-13T14:50:06.041 INFO:tasks.workunit.client.0.smithi176.stdout:7/903: write d1/d1c/d26/fa9 [915116,93667] 0 2024-03-13T14:50:06.049 INFO:tasks.workunit.client.0.smithi176.stdout:3/984: rmdir d4/d28/d34/d12f 0 2024-03-13T14:50:06.049 INFO:tasks.workunit.client.0.smithi176.stdout:7/904: readlink d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/d7a/l7f 0 2024-03-13T14:50:06.053 INFO:tasks.workunit.client.0.smithi176.stdout:3/985: fdatasync d4/d13/f2a 0 2024-03-13T14:50:06.055 INFO:tasks.workunit.client.0.smithi176.stdout:3/986: chown d4/d13/d2d/d105 2602907 1 2024-03-13T14:50:06.055 INFO:tasks.workunit.client.0.smithi176.stdout:4/954: rename d7/d10/d46/d4a/d5d/da1/de8/d10e to d7/d10/d2c/d124/d141 0 2024-03-13T14:50:06.056 INFO:tasks.workunit.client.0.smithi176.stdout:7/905: mknod d1/d28/d3a/d1b/d8f/dd1/df4/c125 0 2024-03-13T14:50:06.060 INFO:tasks.workunit.client.0.smithi176.stdout:6/959: dwrite d0/de/d38/d6b/da3/d128/f11b [0,4194304] 0 2024-03-13T14:50:06.061 INFO:tasks.workunit.client.0.smithi176.stdout:3/987: symlink d4/d13/l14d 0 2024-03-13T14:50:06.061 INFO:tasks.workunit.client.0.smithi176.stdout:7/906: write d1/d1c/d26/d41/d2b/ddc/fc0 [573782,95939] 0 2024-03-13T14:50:06.062 INFO:tasks.workunit.client.0.smithi176.stdout:4/955: mknod d7/d10/d46/d4a/dcc/dd0/d11d/c142 0 2024-03-13T14:50:06.066 INFO:tasks.workunit.client.0.smithi176.stdout:3/988: fsync d4/d13/f57 0 2024-03-13T14:50:06.068 INFO:tasks.workunit.client.0.smithi176.stdout:7/907: chown d1/d28/d3a/d1b/l72 1492 1 2024-03-13T14:50:06.076 INFO:tasks.workunit.client.0.smithi176.stdout:7/908: truncate d1/d1c/d4a/f62 1292099 0 2024-03-13T14:50:06.082 INFO:tasks.workunit.client.0.smithi176.stdout:7/909: mkdir d1/d1c/d26/d41/db6/df1/dfc/d76/da5/d126 0 2024-03-13T14:50:06.084 INFO:tasks.workunit.client.0.smithi176.stdout:3/989: dwrite d4/d13/d16/db6/d148/d89/f27 [4194304,4194304] 0 2024-03-13T14:50:06.086 INFO:tasks.workunit.client.0.smithi176.stdout:6/960: write d0/d10/d19/d27/d2b/de0/ffa [197317,125377] 0 2024-03-13T14:50:06.088 INFO:tasks.workunit.client.0.smithi176.stdout:7/910: dread d1/d28/f2a [0,4194304] 0 2024-03-13T14:50:06.097 INFO:tasks.workunit.client.0.smithi176.stdout:4/956: dwrite d7/d10/d46/d4a/d5d/fda [0,4194304] 0 2024-03-13T14:50:06.099 INFO:tasks.workunit.client.0.smithi176.stdout:6/961: getdents d0/d10/d19/d27/d2b/d2e/d3e/db4 0 2024-03-13T14:50:06.101 INFO:tasks.workunit.client.0.smithi176.stdout:6/962: stat d0/de/d38/d88/d9d/dfe 0 2024-03-13T14:50:06.111 INFO:tasks.workunit.client.0.smithi176.stdout:7/911: truncate d1/d1c/d26/d41/db6/df1/dfc/d81/fe0 977862 0 2024-03-13T14:50:06.115 INFO:tasks.workunit.client.0.smithi176.stdout:2/972: sync 2024-03-13T14:50:06.116 INFO:tasks.workunit.client.0.smithi176.stdout:5/937: sync 2024-03-13T14:50:06.116 INFO:tasks.workunit.client.0.smithi176.stdout:8/896: sync 2024-03-13T14:50:06.116 INFO:tasks.workunit.client.0.smithi176.stdout:1/915: sync 2024-03-13T14:50:06.116 INFO:tasks.workunit.client.0.smithi176.stdout:0/859: sync 2024-03-13T14:50:06.116 INFO:tasks.workunit.client.0.smithi176.stdout:7/912: symlink d1/d28/l127 0 2024-03-13T14:50:06.118 INFO:tasks.workunit.client.0.smithi176.stdout:8/897: chown d0/d24/d98/dc6/fb8 641940 1 2024-03-13T14:50:06.122 INFO:tasks.workunit.client.0.smithi176.stdout:4/957: write d7/d10/d46/d4a/dcc/dff/dbb/fd3 [149506,29681] 0 2024-03-13T14:50:06.123 INFO:tasks.workunit.client.0.smithi176.stdout:2/973: chown d1/dc/d2d/d59/d80/d7e/da3/l110 401965567 1 2024-03-13T14:50:06.126 INFO:tasks.workunit.client.0.smithi176.stdout:3/990: dread d4/d13/d16/d3d/f11e [0,4194304] 0 2024-03-13T14:50:06.126 INFO:tasks.workunit.client.0.smithi176.stdout:1/916: truncate d2/de/d10/d16/d4a/d5f/ff9 999951 0 2024-03-13T14:50:06.126 INFO:tasks.workunit.client.0.smithi176.stdout:8/898: mkdir d0/d24/d29/d70/d122 0 2024-03-13T14:50:06.128 INFO:tasks.workunit.client.0.smithi176.stdout:2/974: dread - d1/dc/d2d/d59/d8b/f134 zero size 2024-03-13T14:50:06.129 INFO:tasks.workunit.client.0.smithi176.stdout:7/913: creat d1/d1c/d26/d41/db6/df1/dfc/d65/dfa/f128 x:0 0 0 2024-03-13T14:50:06.134 INFO:tasks.workunit.client.0.smithi176.stdout:1/917: mkdir d2/de/d3f/d50/d83/d114 0 2024-03-13T14:50:06.135 INFO:tasks.workunit.client.0.smithi176.stdout:4/958: write d7/d2d/f2e [2193011,61674] 0 2024-03-13T14:50:06.137 INFO:tasks.workunit.client.0.smithi176.stdout:2/975: creat d1/dc/d61/d104/f145 x:0 0 0 2024-03-13T14:50:06.139 INFO:tasks.workunit.client.0.smithi176.stdout:7/914: creat d1/d28/d3a/d1b/d8f/dc1/f129 x:0 0 0 2024-03-13T14:50:06.141 INFO:tasks.workunit.client.0.smithi176.stdout:3/991: write d4/d28/f29 [3200159,87234] 0 2024-03-13T14:50:06.142 INFO:tasks.workunit.client.0.smithi176.stdout:1/918: mknod d2/de/d3f/d3e/d86/db3/c115 0 2024-03-13T14:50:06.143 INFO:tasks.workunit.client.0.smithi176.stdout:4/959: read d7/d8/f28 [1824083,118976] 0 2024-03-13T14:50:06.147 INFO:tasks.workunit.client.0.smithi176.stdout:6/963: dread d0/d10/d19/d27/d2b/de0/f12f [0,4194304] 0 2024-03-13T14:50:06.150 INFO:tasks.workunit.client.0.smithi176.stdout:5/938: dwrite d5/d2e/d5f/f102 [0,4194304] 0 2024-03-13T14:50:06.151 INFO:tasks.workunit.client.0.smithi176.stdout:1/919: creat d2/de/d10/d16/d47/f116 x:0 0 0 2024-03-13T14:50:06.152 INFO:tasks.workunit.client.0.smithi176.stdout:7/915: creat d1/d28/f12a x:0 0 0 2024-03-13T14:50:06.154 INFO:tasks.workunit.client.0.smithi176.stdout:4/960: write d7/d10/d46/d4a/dcc/dff/fe2 [4986949,50746] 0 2024-03-13T14:50:06.156 INFO:tasks.workunit.client.0.smithi176.stdout:0/860: dread d2/d5/d1e/f72 [0,4194304] 0 2024-03-13T14:50:06.156 INFO:tasks.workunit.client.0.smithi176.stdout:3/992: creat d4/d28/d31/d45/d82/d87/f14e x:0 0 0 2024-03-13T14:50:06.158 INFO:tasks.workunit.client.0.smithi176.stdout:1/920: write d2/fb8 [1778015,2532] 0 2024-03-13T14:50:06.161 INFO:tasks.workunit.client.0.smithi176.stdout:4/961: creat d7/d8/d45/f143 x:0 0 0 2024-03-13T14:50:06.162 INFO:tasks.workunit.client.0.smithi176.stdout:0/861: mknod d2/db/d39/c114 0 2024-03-13T14:50:06.163 INFO:tasks.workunit.client.0.smithi176.stdout:1/921: readlink d2/d30/d63/d22/l110 0 2024-03-13T14:50:06.166 INFO:tasks.workunit.client.0.smithi176.stdout:7/916: creat d1/d1c/d26/d41/db6/df1/dfc/d76/f12b x:0 0 0 2024-03-13T14:50:06.168 INFO:tasks.workunit.client.0.smithi176.stdout:3/993: creat d4/d13/d16/db6/d148/d89/d5b/dc5/df0/d100/f14f x:0 0 0 2024-03-13T14:50:06.171 INFO:tasks.workunit.client.0.smithi176.stdout:6/964: truncate d0/d10/d19/d27/d2b/d2e/d3e/d9a/da1/fc2 500750 0 2024-03-13T14:50:06.174 INFO:tasks.workunit.client.0.smithi176.stdout:2/976: dwrite d1/f31 [0,4194304] 0 2024-03-13T14:50:06.175 INFO:tasks.workunit.client.0.smithi176.stdout:4/962: creat d7/d10/d46/d4a/dcc/dff/dbb/f144 x:0 0 0 2024-03-13T14:50:06.178 INFO:tasks.workunit.client.0.smithi176.stdout:8/899: dread d0/d5/f113 [0,4194304] 0 2024-03-13T14:50:06.180 INFO:tasks.workunit.client.0.smithi176.stdout:6/965: truncate d0/de/d38/fe9 1256950 0 2024-03-13T14:50:06.181 INFO:tasks.workunit.client.0.smithi176.stdout:1/922: truncate d2/de/d10/d16/d47/f5c 272636 0 2024-03-13T14:50:06.187 INFO:tasks.workunit.client.0.smithi176.stdout:2/977: creat d1/dc/d2d/d90/d99/d144/dea/f146 x:0 0 0 2024-03-13T14:50:06.187 INFO:tasks.workunit.client.0.smithi176.stdout:7/917: rename d1/d1c/l80 to d1/d1c/d26/d41/d2b/l12c 0 2024-03-13T14:50:06.188 INFO:tasks.workunit.client.0.smithi176.stdout:6/966: fdatasync d0/de/d38/f80 0 2024-03-13T14:50:06.190 INFO:tasks.workunit.client.0.smithi176.stdout:4/963: dread d7/d8/d15/d94/fa6 [0,4194304] 0 2024-03-13T14:50:06.197 INFO:tasks.workunit.client.0.smithi176.stdout:4/964: stat d7/d10/d2c/f66 0 2024-03-13T14:50:06.200 INFO:tasks.workunit.client.0.smithi176.stdout:0/862: write d2/d74/d61/f66 [2610605,108467] 0 2024-03-13T14:50:06.203 INFO:tasks.workunit.client.0.smithi176.stdout:0/863: write d2/db/d39/ffc [357772,49745] 0 2024-03-13T14:50:06.204 INFO:tasks.workunit.client.1.smithi196.stdout:8/983: sync 2024-03-13T14:50:06.207 INFO:tasks.workunit.client.0.smithi176.stdout:3/994: write d4/d13/d16/db6/d148/d89/f6d [1285333,128823] 0 2024-03-13T14:50:06.208 INFO:tasks.workunit.client.0.smithi176.stdout:0/864: write d2/d74/f6e [415632,78067] 0 2024-03-13T14:50:06.208 INFO:tasks.workunit.client.0.smithi176.stdout:1/923: write d2/de/d3f/d50/d83/d97/fcf [327335,91936] 0 2024-03-13T14:50:06.211 INFO:tasks.workunit.client.0.smithi176.stdout:0/865: chown d2/d74/df8 22858866 1 2024-03-13T14:50:06.211 INFO:tasks.workunit.client.0.smithi176.stdout:1/924: stat d2/d29/d73/cdc 0 2024-03-13T14:50:06.212 INFO:tasks.workunit.client.0.smithi176.stdout:4/965: truncate d7/d8/d45/f9f 192782 0 2024-03-13T14:50:06.213 INFO:tasks.workunit.client.0.smithi176.stdout:8/900: dwrite d0/d24/d98/fe7 [0,4194304] 0 2024-03-13T14:50:06.216 INFO:tasks.workunit.client.0.smithi176.stdout:1/925: mkdir d2/d30/d117 0 2024-03-13T14:50:06.216 INFO:tasks.workunit.client.0.smithi176.stdout:8/901: chown d0/db/d53/cff 65385691 1 2024-03-13T14:50:06.224 INFO:tasks.workunit.client.0.smithi176.stdout:0/866: creat d2/d74/d9f/f115 x:0 0 0 2024-03-13T14:50:06.224 INFO:tasks.workunit.client.0.smithi176.stdout:1/926: chown d2/d29/f70 1551 1 2024-03-13T14:50:06.224 INFO:tasks.workunit.client.0.smithi176.stdout:6/967: write d0/de/d54/f5e [4580995,109873] 0 2024-03-13T14:50:06.227 INFO:tasks.workunit.client.0.smithi176.stdout:8/902: fsync d0/d5/d11/d47/f7a 0 2024-03-13T14:50:06.238 INFO:tasks.workunit.client.0.smithi176.stdout:1/927: fdatasync d2/de/d10/f6d 0 2024-03-13T14:50:06.239 INFO:tasks.workunit.client.0.smithi176.stdout:6/968: creat d0/d10/d19/d27/d2b/d2e/dc4/dc5/f130 x:0 0 0 2024-03-13T14:50:06.241 INFO:tasks.workunit.client.0.smithi176.stdout:7/918: dwrite d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/f10a [0,4194304] 0 2024-03-13T14:50:06.242 INFO:tasks.workunit.client.0.smithi176.stdout:2/978: dwrite d1/d2a/d33/f5b [0,4194304] 0 2024-03-13T14:50:06.251 INFO:tasks.workunit.client.0.smithi176.stdout:2/979: mknod d1/dc/d1c/d107/c147 0 2024-03-13T14:50:06.253 INFO:tasks.workunit.client.0.smithi176.stdout:6/969: write d0/de/d38/d88/fc3 [4887422,93276] 0 2024-03-13T14:50:06.253 INFO:tasks.workunit.client.0.smithi176.stdout:8/903: dread d0/db/d39/d56/f65 [0,4194304] 0 2024-03-13T14:50:06.258 INFO:tasks.workunit.client.0.smithi176.stdout:4/966: dwrite d7/d8/d15/d23/f7d [0,4194304] 0 2024-03-13T14:50:06.260 INFO:tasks.workunit.client.0.smithi176.stdout:1/928: link d2/de/dd1/cf6 d2/de/d10/d16/d47/c118 0 2024-03-13T14:50:06.266 INFO:tasks.workunit.client.0.smithi176.stdout:7/919: dread d1/d1c/d26/d41/db6/df1/dfc/d4b/fc2 [0,4194304] 0 2024-03-13T14:50:06.273 INFO:tasks.workunit.client.0.smithi176.stdout:1/929: read d2/de/d3f/d50/d83/dcb/f9d [102913,62747] 0 2024-03-13T14:50:06.276 INFO:tasks.workunit.client.0.smithi176.stdout:7/920: write d1/d1c/d26/d41/d2b/fee [1659213,70016] 0 2024-03-13T14:50:06.279 INFO:tasks.workunit.client.0.smithi176.stdout:1/930: chown d2/d30/d63/d107 79995 1 2024-03-13T14:50:06.280 INFO:tasks.workunit.client.0.smithi176.stdout:0/867: dwrite d2/d5/f64 [0,4194304] 0 2024-03-13T14:50:06.282 INFO:tasks.workunit.client.0.smithi176.stdout:8/904: link d0/d24/d7b/dab/fe5 d0/d5/d11/d47/db7/f123 0 2024-03-13T14:50:06.286 INFO:tasks.workunit.client.0.smithi176.stdout:7/921: creat d1/d1c/d26/d41/db6/f12d x:0 0 0 2024-03-13T14:50:06.290 INFO:tasks.workunit.client.0.smithi176.stdout:0/868: creat d2/d5/dc/d20/da8/f116 x:0 0 0 2024-03-13T14:50:06.293 INFO:tasks.workunit.client.0.smithi176.stdout:1/931: read d2/de/d3f/d50/d83/d97/fcf [190371,71462] 0 2024-03-13T14:50:06.295 INFO:tasks.workunit.client.0.smithi176.stdout:7/922: mkdir d1/d1c/d26/d41/db6/df1/dfc/d76/da5/d101/d11d/d12e 0 2024-03-13T14:50:06.297 INFO:tasks.workunit.client.0.smithi176.stdout:2/980: truncate d1/d2a/d33/ff9 4100596 0 2024-03-13T14:50:06.302 INFO:tasks.workunit.client.0.smithi176.stdout:7/923: chown d1/d28/d3a/d1b/d8f/dc1/dbb/cd5 6 1 2024-03-13T14:50:06.303 INFO:tasks.workunit.client.0.smithi176.stdout:2/981: mknod d1/dc/d2d/d59/d8c/c148 0 2024-03-13T14:50:06.304 INFO:tasks.workunit.client.0.smithi176.stdout:7/924: dread - d1/f103 zero size 2024-03-13T14:50:06.308 INFO:tasks.workunit.client.0.smithi176.stdout:7/925: stat d1/d1c/d26/d41/db6/df1/dfc/d65/dfa/f128 0 2024-03-13T14:50:06.309 INFO:tasks.workunit.client.0.smithi176.stdout:4/967: dwrite d7/d10/d46/f8f [0,4194304] 0 2024-03-13T14:50:06.312 INFO:tasks.workunit.client.0.smithi176.stdout:2/982: mknod d1/dc/d2d/d59/d80/d7e/da3/d111/c149 0 2024-03-13T14:50:06.314 INFO:tasks.workunit.client.0.smithi176.stdout:4/968: symlink d7/d8/d45/l145 0 2024-03-13T14:50:06.314 INFO:tasks.workunit.client.0.smithi176.stdout:1/932: link d2/c58 d2/de/d10/d16/d4a/c119 0 2024-03-13T14:50:06.321 INFO:tasks.workunit.client.0.smithi176.stdout:7/926: creat d1/d1c/d26/d41/d2b/f12f x:0 0 0 2024-03-13T14:50:06.321 INFO:tasks.workunit.client.0.smithi176.stdout:4/969: mkdir d7/d8/d33/d146 0 2024-03-13T14:50:06.322 INFO:tasks.workunit.client.1.smithi196.stdout:8/984: sync 2024-03-13T14:50:06.323 INFO:tasks.workunit.client.0.smithi176.stdout:8/905: write d0/d24/d7b/dab/d86/fa8 [1029685,62180] 0 2024-03-13T14:50:06.326 INFO:tasks.workunit.client.0.smithi176.stdout:8/906: read - d0/d24/d7b/dec/f10e zero size 2024-03-13T14:50:06.328 INFO:tasks.workunit.client.0.smithi176.stdout:7/927: creat d1/d1c/d26/d41/d2b/ddc/f130 x:0 0 0 2024-03-13T14:50:06.330 INFO:tasks.workunit.client.0.smithi176.stdout:4/970: dread - d7/d8/d15/ff4 zero size 2024-03-13T14:50:06.333 INFO:tasks.workunit.client.0.smithi176.stdout:5/939: sync 2024-03-13T14:50:06.334 INFO:tasks.workunit.client.0.smithi176.stdout:1/933: link d2/d30/d63/d22/f2b d2/f11a 0 2024-03-13T14:50:06.339 INFO:tasks.workunit.client.1.smithi196.stdout:8/985: dread d0/d10/d12/d1e/dc5/ff2 [0,4194304] 0 2024-03-13T14:50:06.341 INFO:tasks.workunit.client.0.smithi176.stdout:2/983: write d1/dc/d2d/d59/d8c/f8f [1569760,125405] 0 2024-03-13T14:50:06.342 INFO:tasks.workunit.client.0.smithi176.stdout:5/940: rmdir d5/d31/d86/dc0/d11a 39 2024-03-13T14:50:06.343 INFO:tasks.workunit.client.0.smithi176.stdout:1/934: rmdir d2/de/d10/d16/d4a/d5f/d66 39 2024-03-13T14:50:06.345 INFO:tasks.workunit.client.0.smithi176.stdout:8/907: truncate d0/d5/d11/d47/db7/f123 273618 0 2024-03-13T14:50:06.346 INFO:tasks.workunit.client.0.smithi176.stdout:0/869: dwrite d2/db/d31/dc4/fff [0,4194304] 0 2024-03-13T14:50:06.347 INFO:tasks.workunit.client.0.smithi176.stdout:7/928: link d1/d1c/d26/d41/d2b/d40/d64/f60 d1/d1c/d4a/dda/f131 0 2024-03-13T14:50:06.347 INFO:tasks.workunit.client.0.smithi176.stdout:2/984: rmdir d1/dc/d2d/d59/d80 39 2024-03-13T14:50:06.348 INFO:tasks.workunit.client.0.smithi176.stdout:6/970: dread d0/f4e [0,4194304] 0 2024-03-13T14:50:06.349 INFO:tasks.workunit.client.0.smithi176.stdout:5/941: mkdir d5/d2c/d40/dc7/d89/df2/d13d 0 2024-03-13T14:50:06.351 INFO:tasks.workunit.client.0.smithi176.stdout:4/971: write d7/d8/d45/f99 [362049,98282] 0 2024-03-13T14:50:06.351 INFO:tasks.workunit.client.0.smithi176.stdout:7/929: chown d1/d1c/d4a/fe7 0 1 2024-03-13T14:50:06.352 INFO:tasks.workunit.client.0.smithi176.stdout:1/935: stat d2/de/d10/d16/d4a/d5f/d66/df7 0 2024-03-13T14:50:06.352 INFO:tasks.workunit.client.1.smithi196.stdout:8/986: dwrite d0/d1c/f7b [0,4194304] 0 2024-03-13T14:50:06.356 INFO:tasks.workunit.client.1.smithi196.stdout:8/987: rmdir d0/d10/d12 39 2024-03-13T14:50:06.358 INFO:tasks.workunit.client.0.smithi176.stdout:7/930: write d1/d1c/d26/d41/d2b/f12f [505884,68563] 0 2024-03-13T14:50:06.359 INFO:tasks.workunit.client.0.smithi176.stdout:4/972: unlink d7/d10/c12e 0 2024-03-13T14:50:06.362 INFO:tasks.workunit.client.0.smithi176.stdout:6/971: symlink d0/de/d38/d88/d9d/d121/l131 0 2024-03-13T14:50:06.362 INFO:tasks.workunit.client.1.smithi196.stdout:8/988: dread d0/d10/f77 [0,4194304] 0 2024-03-13T14:50:06.362 INFO:tasks.workunit.client.0.smithi176.stdout:0/870: truncate d2/db/d39/fa9 493371 0 2024-03-13T14:50:06.364 INFO:tasks.workunit.client.0.smithi176.stdout:4/973: creat d7/d8/d15/d23/f147 x:0 0 0 2024-03-13T14:50:06.365 INFO:tasks.workunit.client.0.smithi176.stdout:7/931: creat d1/d7/f132 x:0 0 0 2024-03-13T14:50:06.366 INFO:tasks.workunit.client.1.smithi196.stdout:8/989: mkdir d0/d10/d70/d142 0 2024-03-13T14:50:06.368 INFO:tasks.workunit.client.0.smithi176.stdout:0/871: read d2/db/d35/f3f [450032,103390] 0 2024-03-13T14:50:06.369 INFO:tasks.workunit.client.0.smithi176.stdout:6/972: creat d0/de/d38/d70/d109/f132 x:0 0 0 2024-03-13T14:50:06.372 INFO:tasks.workunit.client.1.smithi196.stdout:8/990: mknod d0/d10/d12/d17/d42/dbe/c143 0 2024-03-13T14:50:06.372 INFO:tasks.workunit.client.0.smithi176.stdout:1/936: dread d2/d30/d63/d107/ffc [0,4194304] 0 2024-03-13T14:50:06.376 INFO:tasks.workunit.client.0.smithi176.stdout:8/908: dwrite d0/db/d39/d56/ffa [0,4194304] 0 2024-03-13T14:50:06.380 INFO:tasks.workunit.client.0.smithi176.stdout:7/932: mknod d1/d28/d3a/d1b/d8f/dc1/c133 0 2024-03-13T14:50:06.380 INFO:tasks.workunit.client.0.smithi176.stdout:5/942: write d5/d2e/f3f [2286162,123475] 0 2024-03-13T14:50:06.380 INFO:tasks.workunit.client.0.smithi176.stdout:2/985: write d1/dc/d2d/d4b/f63 [1395755,126874] 0 2024-03-13T14:50:06.385 INFO:tasks.workunit.client.0.smithi176.stdout:1/937: truncate d2/de/d3f/f6f 747434 0 2024-03-13T14:50:06.386 INFO:tasks.workunit.client.0.smithi176.stdout:3/995: sync 2024-03-13T14:50:06.387 INFO:tasks.workunit.client.0.smithi176.stdout:4/974: creat d7/d8/d15/f148 x:0 0 0 2024-03-13T14:50:06.388 INFO:tasks.workunit.client.0.smithi176.stdout:1/938: stat d2/de/d10/d16/d40/dba 0 2024-03-13T14:50:06.389 INFO:tasks.workunit.client.0.smithi176.stdout:2/986: mkdir d1/dc/d2d/d38/d12e/d14a 0 2024-03-13T14:50:06.390 INFO:tasks.workunit.client.0.smithi176.stdout:4/975: readlink d7/d8/d15/l22 0 2024-03-13T14:50:06.391 INFO:tasks.workunit.client.0.smithi176.stdout:7/933: mkdir d1/d1c/d26/d41/db6/df1/dfc/d65/dfa/d134 0 2024-03-13T14:50:06.392 INFO:tasks.workunit.client.0.smithi176.stdout:0/872: creat d2/d5/f117 x:0 0 0 2024-03-13T14:50:06.392 INFO:tasks.workunit.client.0.smithi176.stdout:8/909: dread d0/d24/d98/dc6/db9/fdb [0,4194304] 0 2024-03-13T14:50:06.396 INFO:tasks.workunit.client.0.smithi176.stdout:6/973: write d0/d10/d19/d27/fa9 [2435825,20910] 0 2024-03-13T14:50:06.398 INFO:tasks.workunit.client.0.smithi176.stdout:5/943: creat d5/d2c/f13e x:0 0 0 2024-03-13T14:50:06.400 INFO:tasks.workunit.client.0.smithi176.stdout:2/987: rename d1/dc/d1c/l60 to d1/dc/d2d/d4b/d118/l14b 0 2024-03-13T14:50:06.400 INFO:tasks.workunit.client.0.smithi176.stdout:0/873: symlink d2/d5/d1e/l118 0 2024-03-13T14:50:06.404 INFO:tasks.workunit.client.0.smithi176.stdout:8/910: creat d0/d5/d11/d47/db7/de0/df0/f124 x:0 0 0 2024-03-13T14:50:06.405 INFO:tasks.workunit.client.0.smithi176.stdout:5/944: truncate d5/f81 1197101 0 2024-03-13T14:50:06.406 INFO:tasks.workunit.client.0.smithi176.stdout:5/945: chown d5/d2e 6109007 1 2024-03-13T14:50:06.407 INFO:tasks.workunit.client.0.smithi176.stdout:0/874: creat d2/db/d31/dc4/f119 x:0 0 0 2024-03-13T14:50:06.410 INFO:tasks.workunit.client.0.smithi176.stdout:8/911: chown d0/d5/cb5 7698611 1 2024-03-13T14:50:06.412 INFO:tasks.workunit.client.0.smithi176.stdout:2/988: rmdir d1/dc/d2d/d38/d12e/d14a 0 2024-03-13T14:50:06.414 INFO:tasks.workunit.client.0.smithi176.stdout:1/939: dwrite d2/de/dbf/f111 [0,4194304] 0 2024-03-13T14:50:06.414 INFO:tasks.workunit.client.0.smithi176.stdout:6/974: link d0/d10/d19/d27/fbe d0/d10/d19/d27/d2b/d2e/d3e/db4/f133 0 2024-03-13T14:50:06.418 INFO:tasks.workunit.client.0.smithi176.stdout:3/996: dwrite d4/d28/fa2 [0,4194304] 0 2024-03-13T14:50:06.419 INFO:tasks.workunit.client.0.smithi176.stdout:0/875: read d2/d5/dc/d20/f48 [1760192,72365] 0 2024-03-13T14:50:06.421 INFO:tasks.workunit.client.0.smithi176.stdout:3/997: chown d4/d28/d31 124356 1 2024-03-13T14:50:06.423 INFO:tasks.workunit.client.0.smithi176.stdout:2/989: rename d1/dc/f16 to d1/d2a/d33/f14c 0 2024-03-13T14:50:06.424 INFO:tasks.workunit.client.0.smithi176.stdout:4/976: truncate d7/d10/d46/d4a/dcc/dff/fe2 5625307 0 2024-03-13T14:50:06.426 INFO:tasks.workunit.client.0.smithi176.stdout:1/940: symlink d2/d29/l11b 0 2024-03-13T14:50:06.427 INFO:tasks.workunit.client.0.smithi176.stdout:4/977: stat d7/d8/f28 0 2024-03-13T14:50:06.428 INFO:tasks.workunit.client.0.smithi176.stdout:3/998: mkdir d4/d28/d34/d146/d150 0 2024-03-13T14:50:06.434 INFO:tasks.workunit.client.0.smithi176.stdout:2/990: mkdir d1/dc/d61/d104/d105/d14d 0 2024-03-13T14:50:06.436 INFO:tasks.workunit.client.0.smithi176.stdout:4/978: mkdir d7/d10/d46/d149 0 2024-03-13T14:50:06.436 INFO:tasks.workunit.client.0.smithi176.stdout:8/912: dwrite d0/d24/d7b/dec/f10e [0,4194304] 0 2024-03-13T14:50:06.440 INFO:tasks.workunit.client.0.smithi176.stdout:8/913: fdatasync d0/d24/d98/fe7 0 2024-03-13T14:50:06.441 INFO:tasks.workunit.client.0.smithi176.stdout:5/946: link d5/d2e/d7e/dad/d7b/da7/d111/d15/fec d5/d2e/d7e/d96/f13f 0 2024-03-13T14:50:06.441 INFO:tasks.workunit.client.0.smithi176.stdout:3/999: mkdir d4/d151 0 2024-03-13T14:50:06.442 INFO:tasks.workunit.client.0.smithi176.stdout:1/941: fsync d2/de/d10/d16/d47/f5c 0 2024-03-13T14:50:06.445 INFO:tasks.workunit.client.0.smithi176.stdout:7/934: write d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/fb4 [1450166,87575] 0 2024-03-13T14:50:06.445 INFO:tasks.workunit.client.0.smithi176.stdout:4/979: mkdir d7/d10/d46/d4a/dcc/dff/d14a 0 2024-03-13T14:50:06.453 INFO:tasks.workunit.client.0.smithi176.stdout:7/935: chown d1/d28/d3a/d1b/l119 249003518 1 2024-03-13T14:50:06.453 INFO:tasks.workunit.client.0.smithi176.stdout:5/947: write d5/d2e/d7e/dad/d7b/da7/d111/d15/f21 [2112728,45206] 0 2024-03-13T14:50:06.457 INFO:tasks.workunit.client.0.smithi176.stdout:4/980: creat d7/d8/f14b x:0 0 0 2024-03-13T14:50:06.461 INFO:tasks.workunit.client.0.smithi176.stdout:5/948: chown d5/d2e/d7e/dad/d7b/daa 7257 1 2024-03-13T14:50:06.463 INFO:tasks.workunit.client.1.smithi196.stdout:8/991: sync 2024-03-13T14:50:06.463 INFO:tasks.workunit.client.0.smithi176.stdout:5/949: unlink d5/c11 0 2024-03-13T14:50:06.464 INFO:tasks.workunit.client.0.smithi176.stdout:6/975: dwrite d0/d10/d19/d27/f68 [0,4194304] 0 2024-03-13T14:50:06.466 INFO:tasks.workunit.client.1.smithi196.stdout:8/992: write d0/d63/dcb/ff6 [4654940,85087] 0 2024-03-13T14:50:06.468 INFO:tasks.workunit.client.0.smithi176.stdout:6/976: fsync d0/d10/d19/d27/d2b/d2e/f62 0 2024-03-13T14:50:06.469 INFO:tasks.workunit.client.0.smithi176.stdout:4/981: rename d7/d8/d33/d96/d111/f127 to d7/db9/dd8/f14c 0 2024-03-13T14:50:06.469 INFO:tasks.workunit.client.0.smithi176.stdout:0/876: dread d2/d5/dc/d20/f48 [0,4194304] 0 2024-03-13T14:50:06.471 INFO:tasks.workunit.client.0.smithi176.stdout:4/982: write d7/fe [8466754,40582] 0 2024-03-13T14:50:06.472 INFO:tasks.workunit.client.0.smithi176.stdout:5/950: write d5/d31/fa2 [3926040,4693] 0 2024-03-13T14:50:06.477 INFO:tasks.workunit.client.1.smithi196.stdout:8/993: creat d0/d1c/f144 x:0 0 0 2024-03-13T14:50:06.477 INFO:tasks.workunit.client.0.smithi176.stdout:4/983: unlink d7/d8/d15/l12a 0 2024-03-13T14:50:06.481 INFO:tasks.workunit.client.0.smithi176.stdout:7/936: write d1/d28/d3a/d1b/d8f/dc1/db7/fd0 [999457,59259] 0 2024-03-13T14:50:06.481 INFO:tasks.workunit.client.0.smithi176.stdout:0/877: dread - d2/d5/dc/d59/fb3 zero size 2024-03-13T14:50:06.482 INFO:tasks.workunit.client.1.smithi196.stdout:8/994: creat d0/d10/d12/d17/d42/f145 x:0 0 0 2024-03-13T14:50:06.484 INFO:tasks.workunit.client.0.smithi176.stdout:5/951: creat d5/d2e/d43/d12b/f140 x:0 0 0 2024-03-13T14:50:06.486 INFO:tasks.workunit.client.0.smithi176.stdout:4/984: mkdir d7/d8/d33/d96/d111/d14d 0 2024-03-13T14:50:06.487 INFO:tasks.workunit.client.0.smithi176.stdout:0/878: creat d2/db/d39/da5/f11a x:0 0 0 2024-03-13T14:50:06.487 INFO:tasks.workunit.client.1.smithi196.stdout:8/995: mknod d0/d10/d70/dc9/df9/c146 0 2024-03-13T14:50:06.489 INFO:tasks.workunit.client.0.smithi176.stdout:5/952: mknod d5/d2e/d7e/dad/d7b/da7/c141 0 2024-03-13T14:50:06.490 INFO:tasks.workunit.client.0.smithi176.stdout:7/937: getdents d1/d1c/d26/d41/db6/df1/dfc/d65/dfa/d134 0 2024-03-13T14:50:06.491 INFO:tasks.workunit.client.1.smithi196.stdout:8/996: mknod d0/d10/d12/d17/d42/dbe/d106/c147 0 2024-03-13T14:50:06.492 INFO:tasks.workunit.client.0.smithi176.stdout:7/938: dread - d1/f103 zero size 2024-03-13T14:50:06.493 INFO:tasks.workunit.client.0.smithi176.stdout:6/977: dwrite d0/de/d38/d88/d9d/dfe/f111 [0,4194304] 0 2024-03-13T14:50:06.495 INFO:tasks.workunit.client.0.smithi176.stdout:8/914: dread d0/d5/d11/fda [0,4194304] 0 2024-03-13T14:50:06.497 INFO:tasks.workunit.client.1.smithi196.stdout:8/997: rename d0/d1c/f7b to d0/d10/d70/dc9/f148 0 2024-03-13T14:50:06.500 INFO:tasks.workunit.client.0.smithi176.stdout:0/879: rename d2/d74/d61/lf5 to d2/d5/d1e/d97/dcf/l11b 0 2024-03-13T14:50:06.502 INFO:tasks.workunit.client.0.smithi176.stdout:6/978: creat d0/de/d38/d70/f134 x:0 0 0 2024-03-13T14:50:06.506 INFO:tasks.workunit.client.0.smithi176.stdout:6/979: chown d0/d10/d19/d27/d2b/d2e/d3e/db4/ff7 2 1 2024-03-13T14:50:06.506 INFO:tasks.workunit.client.1.smithi196.stdout:8/998: creat d0/d16/d3d/d9c/d87/d8c/da9/db8/f149 x:0 0 0 2024-03-13T14:50:06.509 INFO:tasks.workunit.client.0.smithi176.stdout:8/915: fdatasync d0/d24/d98/dc6/d40/f62 0 2024-03-13T14:50:06.511 INFO:tasks.workunit.client.0.smithi176.stdout:7/939: creat d1/d1c/d26/d41/d2b/d120/f135 x:0 0 0 2024-03-13T14:50:06.512 INFO:tasks.workunit.client.0.smithi176.stdout:0/880: mkdir d2/db/d39/d9c/d11c 0 2024-03-13T14:50:06.515 INFO:tasks.workunit.client.1.smithi196.stdout:8/999: creat d0/d10/d12/d17/def/d13a/f14a x:0 0 0 2024-03-13T14:50:06.518 INFO:tasks.workunit.client.0.smithi176.stdout:7/940: symlink d1/d1c/d26/d41/d2b/d120/l136 0 2024-03-13T14:50:06.519 INFO:tasks.workunit.client.0.smithi176.stdout:0/881: mknod d2/db/d35/daa/c11d 0 2024-03-13T14:50:06.520 INFO:tasks.workunit.client.0.smithi176.stdout:8/916: rename d0/d5/c90 to d0/d5/d3b/c125 0 2024-03-13T14:50:06.522 INFO:tasks.workunit.client.0.smithi176.stdout:6/980: creat d0/d10/d19/d27/f135 x:0 0 0 2024-03-13T14:50:06.523 INFO:tasks.workunit.client.0.smithi176.stdout:4/985: dread d7/d10/f44 [0,4194304] 0 2024-03-13T14:50:06.523 INFO:tasks.workunit.client.1.smithi196.stderr:+ rm -rf -- ./tmp.n2NxSjmiRJ 2024-03-13T14:50:06.527 INFO:tasks.workunit.client.0.smithi176.stdout:8/917: creat d0/d5/dc2/f126 x:0 0 0 2024-03-13T14:50:06.529 INFO:tasks.workunit.client.0.smithi176.stdout:5/953: dread d5/d2e/d7e/d9f/f4e [0,4194304] 0 2024-03-13T14:50:06.529 INFO:tasks.workunit.client.0.smithi176.stdout:4/986: fsync d7/d10/d2c/f77 0 2024-03-13T14:50:06.530 INFO:tasks.workunit.client.0.smithi176.stdout:5/954: dread - d5/d31/f12d zero size 2024-03-13T14:50:06.530 INFO:tasks.workunit.client.0.smithi176.stdout:8/918: chown d0/d24/d29/d82/cee 1147 1 2024-03-13T14:50:06.534 INFO:tasks.workunit.client.0.smithi176.stdout:5/955: chown d5/d2e/d7e/dad/d7b/da7/d111/d15/l1c 13863 1 2024-03-13T14:50:06.534 INFO:tasks.workunit.client.0.smithi176.stdout:4/987: mknod d7/d10/d46/d4a/dcc/dff/c14e 0 2024-03-13T14:50:06.536 INFO:tasks.workunit.client.0.smithi176.stdout:6/981: write d0/de/d54/f75 [851227,8277] 0 2024-03-13T14:50:06.537 INFO:tasks.workunit.client.0.smithi176.stdout:8/919: symlink d0/d5/d11/d47/db7/de0/d120/l127 0 2024-03-13T14:50:06.541 INFO:tasks.workunit.client.0.smithi176.stdout:7/941: dwrite d1/d1c/d26/d41/d2b/d40/d64/d94/f7e [0,4194304] 0 2024-03-13T14:50:06.550 INFO:tasks.workunit.client.0.smithi176.stdout:5/956: rename d5/d2e/d7e/d9f/l9c to d5/d31/d86/dc0/d11a/d120/d12c/l142 0 2024-03-13T14:50:06.550 INFO:tasks.workunit.client.0.smithi176.stdout:0/882: dread d2/d5/f46 [0,4194304] 0 2024-03-13T14:50:06.553 INFO:tasks.workunit.client.0.smithi176.stdout:0/883: dread - d2/d74/d9f/f115 zero size 2024-03-13T14:50:06.558 INFO:tasks.workunit.client.0.smithi176.stdout:0/884: read d2/d5/d1e/d97/dcf/fec [2561667,16529] 0 2024-03-13T14:50:06.571 INFO:tasks.workunit.client.0.smithi176.stdout:6/982: dwrite d0/d10/d19/d27/d2b/de0/f74 [0,4194304] 0 2024-03-13T14:50:06.573 INFO:tasks.workunit.client.0.smithi176.stdout:8/920: dwrite d0/d24/d29/f67 [0,4194304] 0 2024-03-13T14:50:06.579 INFO:tasks.workunit.client.0.smithi176.stdout:7/942: dread d1/d1c/d26/d41/db6/df1/dfc/d65/f97 [0,4194304] 0 2024-03-13T14:50:06.581 INFO:tasks.workunit.client.0.smithi176.stdout:5/957: dwrite d5/d2e/d7e/d9f/f4e [4194304,4194304] 0 2024-03-13T14:50:06.608 INFO:tasks.workunit.client.0.smithi176.stdout:0/885: creat d2/db/d39/f11e x:0 0 0 2024-03-13T14:50:06.610 INFO:tasks.workunit.client.0.smithi176.stdout:6/983: fdatasync d0/d10/d19/d27/d2b/d2e/f7b 0 2024-03-13T14:50:06.623 INFO:tasks.workunit.client.0.smithi176.stdout:7/943: getdents d1/d1c/d26/d41/db6/df1/dfc/d65/dfa/d134 0 2024-03-13T14:50:06.624 INFO:tasks.workunit.client.0.smithi176.stdout:7/944: stat d1/d28/d3a/d1b/d8f/dd1/df4 0 2024-03-13T14:50:06.628 INFO:tasks.workunit.client.0.smithi176.stdout:5/958: creat d5/d2e/d7e/d96/f143 x:0 0 0 2024-03-13T14:50:06.629 INFO:tasks.workunit.client.0.smithi176.stdout:5/959: chown d5/d2e/l109 2053 1 2024-03-13T14:50:06.630 INFO:tasks.workunit.client.0.smithi176.stdout:7/945: rmdir d1/d1c/d26 39 2024-03-13T14:50:06.633 INFO:tasks.workunit.client.0.smithi176.stdout:5/960: creat d5/d31/def/f144 x:0 0 0 2024-03-13T14:50:06.637 INFO:tasks.workunit.client.0.smithi176.stdout:5/961: fdatasync d5/d2c/d40/dc7/ff0 0 2024-03-13T14:50:06.638 INFO:tasks.workunit.client.0.smithi176.stdout:0/886: dread d2/db/d39/fa2 [0,4194304] 0 2024-03-13T14:50:06.639 INFO:tasks.workunit.client.0.smithi176.stdout:7/946: getdents d1/d1c/d26/d41/db6/df1/dfc/d76/da5/d126 0 2024-03-13T14:50:06.641 INFO:tasks.workunit.client.0.smithi176.stdout:5/962: mknod d5/d2e/d5f/c145 0 2024-03-13T14:50:06.645 INFO:tasks.workunit.client.0.smithi176.stdout:0/887: unlink d2/db/d39/f11e 0 2024-03-13T14:50:06.646 INFO:tasks.workunit.client.0.smithi176.stdout:0/888: write d2/d5/dc/d20/f10a [605340,96766] 0 2024-03-13T14:50:06.650 INFO:tasks.workunit.client.0.smithi176.stdout:0/889: truncate d2/db/d35/ffd 776219 0 2024-03-13T14:50:06.653 INFO:tasks.workunit.client.0.smithi176.stdout:5/963: mkdir d5/d31/d86/dc0/d146 0 2024-03-13T14:50:06.654 INFO:tasks.workunit.client.0.smithi176.stdout:7/947: dwrite d1/d1c/d26/d41/db6/df1/dfc/fab [0,4194304] 0 2024-03-13T14:50:06.658 INFO:tasks.workunit.client.0.smithi176.stdout:5/964: fdatasync d5/d31/d47/f7f 0 2024-03-13T14:50:06.658 INFO:tasks.workunit.client.0.smithi176.stdout:7/948: readlink d1/d1c/d26/d41/db6/df1/l118 0 2024-03-13T14:50:06.659 INFO:tasks.workunit.client.0.smithi176.stdout:6/984: write d0/d10/d19/d27/d2b/de0/d76/f85 [106816,31929] 0 2024-03-13T14:50:06.659 INFO:tasks.workunit.client.0.smithi176.stdout:0/890: getdents d2/d5/dc/d20/da8 0 2024-03-13T14:50:06.667 INFO:tasks.workunit.client.0.smithi176.stdout:8/921: dwrite d0/db/d39/fae [0,4194304] 0 2024-03-13T14:50:06.676 INFO:tasks.workunit.client.0.smithi176.stdout:0/891: dwrite d2/d5/d1e/f4e [0,4194304] 0 2024-03-13T14:50:06.676 INFO:tasks.workunit.client.0.smithi176.stdout:6/985: rename d0/d10/dca/dd2/fe3 to d0/de/d38/d88/d116/f136 0 2024-03-13T14:50:06.700 INFO:tasks.workunit.client.0.smithi176.stdout:5/965: dwrite d5/d2e/d7e/dad/d7b/daa/fb8 [0,4194304] 0 2024-03-13T14:50:06.708 INFO:tasks.workunit.client.0.smithi176.stdout:8/922: dwrite d0/d5/d11/f5f [0,4194304] 0 2024-03-13T14:50:06.712 INFO:tasks.workunit.client.0.smithi176.stdout:6/986: write d0/d10/d19/d27/d2b/db1/fd3 [458500,47689] 0 2024-03-13T14:50:06.713 INFO:tasks.workunit.client.0.smithi176.stdout:8/923: write d0/d5/d11/f68 [2271743,62514] 0 2024-03-13T14:50:06.714 INFO:tasks.workunit.client.0.smithi176.stdout:5/966: unlink d5/d2e/d7e/dad/d7b/da7/d111/d15/f84 0 2024-03-13T14:50:06.719 INFO:tasks.workunit.client.0.smithi176.stdout:8/924: fsync d0/f6c 0 2024-03-13T14:50:06.720 INFO:tasks.workunit.client.0.smithi176.stdout:7/949: dread d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/d7a/fa0 [0,4194304] 0 2024-03-13T14:50:06.723 INFO:tasks.workunit.client.0.smithi176.stdout:5/967: rename d5/d2e/d7e/dad/d7b/da7/c141 to d5/d31/d86/dc0/d11a/c147 0 2024-03-13T14:50:06.727 INFO:tasks.workunit.client.0.smithi176.stdout:8/925: mkdir d0/d5/d11/d47/db7/de0/d120/d128 0 2024-03-13T14:50:06.801 INFO:tasks.workunit.client.0.smithi176.stdout:2/991: sync 2024-03-13T14:50:06.805 INFO:tasks.workunit.client.0.smithi176.stdout:6/987: creat d0/d10/d19/d27/d2b/f137 x:0 0 0 2024-03-13T14:50:06.836 INFO:tasks.workunit.client.0.smithi176.stdout:1/942: sync 2024-03-13T14:50:06.836 INFO:tasks.workunit.client.0.smithi176.stdout:0/892: sync 2024-03-13T14:50:06.836 INFO:tasks.workunit.client.0.smithi176.stdout:4/988: sync 2024-03-13T14:50:06.868 INFO:tasks.workunit.client.0.smithi176.stdout:7/950: rename d1/d28/d3a/d1b/d8f/dd1/cd4 to d1/d1c/d26/d41/db6/df1/dfc/d81/c137 0 2024-03-13T14:50:06.871 INFO:tasks.workunit.client.0.smithi176.stdout:7/951: write d1/d28/d3a/d1b/d8f/dd1/f106 [285627,19850] 0 2024-03-13T14:50:06.879 INFO:tasks.workunit.client.0.smithi176.stdout:0/893: creat d2/db/d35/daa/f11f x:0 0 0 2024-03-13T14:50:06.884 INFO:tasks.workunit.client.0.smithi176.stdout:5/968: symlink d5/d2c/d40/dc7/d89/df2/d13d/l148 0 2024-03-13T14:50:06.885 INFO:tasks.workunit.client.0.smithi176.stdout:5/969: chown d5/d2c/d8a 2029 1 2024-03-13T14:50:06.887 INFO:tasks.workunit.client.0.smithi176.stdout:7/952: chown d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/d7a/l11f 40 1 2024-03-13T14:50:06.891 INFO:tasks.workunit.client.0.smithi176.stdout:0/894: mkdir d2/d5/dc/d59/d120 0 2024-03-13T14:50:06.893 INFO:tasks.workunit.client.0.smithi176.stdout:4/989: mknod d7/d10/d46/d149/c14f 0 2024-03-13T14:50:06.896 INFO:tasks.workunit.client.0.smithi176.stdout:2/992: dread d1/dc/d2d/d59/d80/d77/f84 [4194304,4194304] 0 2024-03-13T14:50:06.900 INFO:tasks.workunit.client.0.smithi176.stdout:7/953: rename d1/d7/ff8 to d1/d7/df6/f138 0 2024-03-13T14:50:06.903 INFO:tasks.workunit.client.0.smithi176.stdout:0/895: truncate d2/db/f93 9050196 0 2024-03-13T14:50:06.912 INFO:tasks.workunit.client.0.smithi176.stdout:5/970: rmdir d5/d136 0 2024-03-13T14:50:06.915 INFO:tasks.workunit.client.0.smithi176.stdout:2/993: mkdir d1/dc/d2d/d135/d14e 0 2024-03-13T14:50:06.918 INFO:tasks.workunit.client.0.smithi176.stdout:5/971: rmdir d5/d2b 39 2024-03-13T14:50:06.919 INFO:tasks.workunit.client.0.smithi176.stdout:2/994: read d1/dc/d1c/d107/d130/f13d [3146710,72464] 0 2024-03-13T14:50:06.926 INFO:tasks.workunit.client.0.smithi176.stdout:5/972: mkdir d5/d2c/d40/dc7/d53/de9/d149 0 2024-03-13T14:50:06.928 INFO:tasks.workunit.client.0.smithi176.stdout:5/973: chown d5/d2e/d7e/d9f/c68 56778 1 2024-03-13T14:50:06.932 INFO:tasks.workunit.client.0.smithi176.stdout:5/974: dread d5/d31/d47/f59 [0,4194304] 0 2024-03-13T14:50:07.000 INFO:tasks.workunit.client.0.smithi176.stdout:6/988: creat d0/d10/d19/d27/d2b/de0/d76/f138 x:0 0 0 2024-03-13T14:50:07.005 INFO:tasks.workunit.client.0.smithi176.stdout:6/989: mknod d0/d10/d19/d106/d11c/c139 0 2024-03-13T14:50:07.008 INFO:tasks.workunit.client.0.smithi176.stdout:6/990: chown d0/de/d54/f5e 121771154 1 2024-03-13T14:50:07.011 INFO:tasks.workunit.client.0.smithi176.stdout:6/991: stat d0/d10/d19/d27/d2b/de0/d76/d7c/f9b 0 2024-03-13T14:50:07.013 INFO:tasks.workunit.client.0.smithi176.stdout:6/992: fdatasync d0/de/d54/f75 0 2024-03-13T14:50:07.015 INFO:tasks.workunit.client.0.smithi176.stdout:6/993: chown d0/d10/d19/d27/d2b/d2e/d3e/db4/ff7 4221 1 2024-03-13T14:50:07.018 INFO:tasks.workunit.client.0.smithi176.stdout:6/994: dread - d0/d10/d19/d27/d2b/d2e/f123 zero size 2024-03-13T14:50:07.021 INFO:tasks.workunit.client.0.smithi176.stdout:6/995: write d0/d10/d19/d27/d2b/de0/d76/d7c/dfd/d122/f12a [823320,28571] 0 2024-03-13T14:50:07.047 INFO:tasks.workunit.client.0.smithi176.stdout:8/926: symlink d0/d5/d11/d47/db7/l129 0 2024-03-13T14:50:07.050 INFO:tasks.workunit.client.0.smithi176.stdout:8/927: creat d0/d24/dc3/f12a x:0 0 0 2024-03-13T14:50:07.052 INFO:tasks.workunit.client.0.smithi176.stdout:8/928: symlink d0/d24/dc3/l12b 0 2024-03-13T14:50:07.060 INFO:tasks.workunit.client.0.smithi176.stdout:8/929: link d0/db/l10d d0/d24/d29/d70/dca/l12c 0 2024-03-13T14:50:07.061 INFO:tasks.workunit.client.0.smithi176.stdout:8/930: stat d0/d24/d29/d82/f105 0 2024-03-13T14:50:07.065 INFO:tasks.workunit.client.0.smithi176.stdout:4/990: write d7/d10/d46/d4a/d5d/f69 [5545557,72783] 0 2024-03-13T14:50:07.069 INFO:tasks.workunit.client.0.smithi176.stdout:4/991: creat d7/d8/d33/d96/f150 x:0 0 0 2024-03-13T14:50:07.072 INFO:tasks.workunit.client.0.smithi176.stdout:4/992: stat d7/d8/d45/c48 0 2024-03-13T14:50:07.076 INFO:tasks.workunit.client.0.smithi176.stdout:7/954: dwrite d1/d1c/d26/d41/db6/df1/dfc/d81/fc5 [0,4194304] 0 2024-03-13T14:50:07.082 INFO:tasks.workunit.client.0.smithi176.stdout:2/995: dwrite d1/dc/d2d/d4f/f11b [0,4194304] 0 2024-03-13T14:50:07.082 INFO:tasks.workunit.client.0.smithi176.stdout:7/955: symlink d1/d1c/d26/d41/d2b/ddc/l139 0 2024-03-13T14:50:07.096 INFO:tasks.workunit.client.0.smithi176.stdout:7/956: creat d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/f13a x:0 0 0 2024-03-13T14:50:07.100 INFO:tasks.workunit.client.0.smithi176.stdout:2/996: symlink d1/dc/d1c/d107/l14f 0 2024-03-13T14:50:07.115 INFO:tasks.workunit.client.0.smithi176.stdout:2/997: dwrite d1/dc/d2d/d59/d80/d5c/f121 [0,4194304] 0 2024-03-13T14:50:07.127 INFO:tasks.workunit.client.0.smithi176.stdout:1/943: unlink d2/de/d3f/d50/f42 0 2024-03-13T14:50:07.131 INFO:tasks.workunit.client.0.smithi176.stdout:1/944: mknod d2/de/d10/d16/c11c 0 2024-03-13T14:50:07.138 INFO:tasks.workunit.client.0.smithi176.stdout:1/945: rmdir d2/d29/d73/ddb 0 2024-03-13T14:50:07.139 INFO:tasks.workunit.client.0.smithi176.stdout:2/998: dread d1/f7f [0,4194304] 0 2024-03-13T14:50:07.141 INFO:tasks.workunit.client.0.smithi176.stdout:2/999: dread - d1/dc/d2d/d38/d12e/f131 zero size 2024-03-13T14:50:07.144 INFO:tasks.workunit.client.0.smithi176.stdout:8/931: write d0/d5/d11/d47/f6a [2433258,8672] 0 2024-03-13T14:50:07.152 INFO:tasks.workunit.client.0.smithi176.stdout:0/896: rename d2/f68 to d2/db/d35/f121 0 2024-03-13T14:50:07.154 INFO:tasks.workunit.client.0.smithi176.stdout:5/975: rename d5/d2c/d40/dc7/d89/df2/d13d/l148 to d5/d2c/d40/dc7/d89/df2/l14a 0 2024-03-13T14:50:07.155 INFO:tasks.workunit.client.0.smithi176.stdout:1/946: dwrite d2/de/d3f/d3e/d86/fe0 [0,4194304] 0 2024-03-13T14:50:07.158 INFO:tasks.workunit.client.0.smithi176.stdout:7/957: write d1/d1c/d4a/f62 [79761,111166] 0 2024-03-13T14:50:07.159 INFO:tasks.workunit.client.0.smithi176.stdout:6/996: rename d0/cce to d0/d10/d19/d27/d2b/d2e/dc4/dc5/c13a 0 2024-03-13T14:50:07.162 INFO:tasks.workunit.client.0.smithi176.stdout:1/947: creat d2/de/d3f/d50/d83/d97/f11d x:0 0 0 2024-03-13T14:50:07.162 INFO:tasks.workunit.client.0.smithi176.stdout:6/997: chown d0/d10/dca/dd2/f108 408695 1 2024-03-13T14:50:07.165 INFO:tasks.workunit.client.0.smithi176.stdout:6/998: dread - d0/de/d38/d70/d109/f132 zero size 2024-03-13T14:50:07.175 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:07 smithi176 ceph-mon[29064]: pgmap v22: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 110 MiB/s rd, 255 MiB/s wr, 582 op/s 2024-03-13T14:50:07.181 INFO:tasks.workunit.client.0.smithi176.stdout:5/976: rename d5/d31/d86/dc0/d146 to d5/d2c/d40/d14b 0 2024-03-13T14:50:07.184 INFO:tasks.workunit.client.0.smithi176.stdout:5/977: chown d5/d31/d47/d80/fd6 157644040 1 2024-03-13T14:50:07.187 INFO:tasks.workunit.client.0.smithi176.stdout:8/932: dwrite d0/d5/d11/d47/f63 [0,4194304] 0 2024-03-13T14:50:07.187 INFO:tasks.workunit.client.0.smithi176.stdout:0/897: dread d2/d5/dc/f65 [0,4194304] 0 2024-03-13T14:50:07.187 INFO:tasks.workunit.client.0.smithi176.stdout:6/999: write d0/d10/d19/fa8 [658696,62485] 0 2024-03-13T14:50:07.194 INFO:tasks.workunit.client.0.smithi176.stdout:1/948: rename d2/de/d3f/fcc to d2/de/d10/d16/d4a/d5f/f11e 0 2024-03-13T14:50:07.196 INFO:tasks.workunit.client.0.smithi176.stdout:7/958: dread d1/d1c/d26/d41/db6/df1/dfc/d4b/d63/fb3 [0,4194304] 0 2024-03-13T14:50:07.200 INFO:tasks.workunit.client.0.smithi176.stdout:7/959: stat d1/d1c/d26/d41/db6/df1/dfc/d4b 0 2024-03-13T14:50:07.204 INFO:tasks.workunit.client.0.smithi176.stdout:5/978: dwrite d5/d2c/d40/d76/f95 [0,4194304] 0 2024-03-13T14:50:07.205 INFO:tasks.workunit.client.0.smithi176.stdout:8/933: dread d0/d5/d11/fda [0,4194304] 0 2024-03-13T14:50:07.208 INFO:tasks.workunit.client.0.smithi176.stdout:7/960: unlink d1/d1c/d26/d41/db6/df1/dfc/d76/ld8 0 2024-03-13T14:50:07.210 INFO:tasks.workunit.client.0.smithi176.stdout:5/979: creat d5/d2c/d40/dc7/d53/f14c x:0 0 0 2024-03-13T14:50:07.213 INFO:tasks.workunit.client.0.smithi176.stdout:7/961: mkdir d1/d1c/d26/d41/db6/df1/dfc/d13b 0 2024-03-13T14:50:07.214 INFO:tasks.workunit.client.0.smithi176.stdout:8/934: unlink d0/d5/d3b/f81 0 2024-03-13T14:50:07.218 INFO:tasks.workunit.client.0.smithi176.stdout:5/980: readlink d5/d2e/d43/l129 0 2024-03-13T14:50:07.219 INFO:tasks.workunit.client.0.smithi176.stdout:1/949: dwrite d2/de/d10/d16/f10c [0,4194304] 0 2024-03-13T14:50:07.224 INFO:tasks.workunit.client.0.smithi176.stdout:7/962: symlink d1/d28/d3a/d1b/d8f/dc1/db7/df9/l13c 0 2024-03-13T14:50:07.227 INFO:tasks.workunit.client.0.smithi176.stdout:1/950: write d2/de/d10/d16/f27 [9324709,123885] 0 2024-03-13T14:50:07.237 INFO:tasks.workunit.client.0.smithi176.stdout:8/935: dwrite d0/d5/d11/f5f [4194304,4194304] 0 2024-03-13T14:50:07.238 INFO:tasks.workunit.client.0.smithi176.stdout:1/951: mkdir d2/d30/dd3/d11f 0 2024-03-13T14:50:07.240 INFO:tasks.workunit.client.0.smithi176.stdout:1/952: readlink d2/de/lc5 0 2024-03-13T14:50:07.243 INFO:tasks.workunit.client.0.smithi176.stdout:8/936: symlink d0/d24/d7b/l12d 0 2024-03-13T14:50:07.245 INFO:tasks.workunit.client.0.smithi176.stdout:5/981: dwrite d5/d2e/d7e/dad/f88 [0,4194304] 0 2024-03-13T14:50:07.247 INFO:tasks.workunit.client.0.smithi176.stdout:1/953: creat d2/d30/d63/d107/f120 x:0 0 0 2024-03-13T14:50:07.250 INFO:tasks.workunit.client.0.smithi176.stdout:7/963: dwrite d1/d1c/d26/d41/d2b/ddc/fc0 [0,4194304] 0 2024-03-13T14:50:07.252 INFO:tasks.workunit.client.0.smithi176.stdout:1/954: chown d2/de/d10/d16/d4a/l88 1 1 2024-03-13T14:50:07.253 INFO:tasks.workunit.client.0.smithi176.stdout:5/982: creat d5/d2c/d40/dc7/d53/de9/f14d x:0 0 0 2024-03-13T14:50:07.258 INFO:tasks.workunit.client.0.smithi176.stdout:1/955: mkdir d2/de/d3f/d19/d121 0 2024-03-13T14:50:07.261 INFO:tasks.workunit.client.0.smithi176.stdout:7/964: creat d1/d1c/d26/d41/d2b/ddc/f13d x:0 0 0 2024-03-13T14:50:07.263 INFO:tasks.workunit.client.0.smithi176.stdout:5/983: unlink d5/ld 0 2024-03-13T14:50:07.265 INFO:tasks.workunit.client.0.smithi176.stdout:1/956: fdatasync d2/de/d10/d16/d40/fcd 0 2024-03-13T14:50:07.269 INFO:tasks.workunit.client.0.smithi176.stdout:5/984: fsync d5/d75/fe2 0 2024-03-13T14:50:07.271 INFO:tasks.workunit.client.0.smithi176.stdout:7/965: truncate d1/d28/f2a 1469186 0 2024-03-13T14:50:07.274 INFO:tasks.workunit.client.0.smithi176.stdout:1/957: symlink d2/de/d10/d16/d40/l122 0 2024-03-13T14:50:07.276 INFO:tasks.workunit.client.0.smithi176.stdout:5/985: mkdir d5/d2e/d7e/dad/d7b/d14e 0 2024-03-13T14:50:07.278 INFO:tasks.workunit.client.0.smithi176.stdout:8/937: dread d0/db/f19 [4194304,4194304] 0 2024-03-13T14:50:07.281 INFO:tasks.workunit.client.0.smithi176.stdout:5/986: stat d5/d2b/fcd 0 2024-03-13T14:50:07.282 INFO:tasks.workunit.client.0.smithi176.stdout:5/987: dread - d5/d2c/d40/d76/ff4 zero size 2024-03-13T14:50:07.284 INFO:tasks.workunit.client.0.smithi176.stdout:1/958: link d2/d29/d73/f94 d2/d30/d63/d107/f123 0 2024-03-13T14:50:07.285 INFO:tasks.workunit.client.0.smithi176.stdout:8/938: chown d0/db/d39/d56/d91/cad 8514554 1 2024-03-13T14:50:07.287 INFO:tasks.workunit.client.0.smithi176.stdout:8/939: chown d0/d5/d11/f5f 19664 1 2024-03-13T14:50:07.290 INFO:tasks.workunit.client.0.smithi176.stdout:8/940: chown d0/d5/d11/f76 261768982 1 2024-03-13T14:50:07.293 INFO:tasks.workunit.client.0.smithi176.stdout:8/941: truncate d0/db/d53/d80/f97 4442946 0 2024-03-13T14:50:07.297 INFO:tasks.workunit.client.0.smithi176.stdout:5/988: dwrite d5/d2c/d40/dc7/d89/df2/f123 [0,4194304] 0 2024-03-13T14:50:07.299 INFO:tasks.workunit.client.0.smithi176.stdout:5/989: rename d5/d31 to d5/d31/d14f 22 2024-03-13T14:50:07.300 INFO:tasks.workunit.client.0.smithi176.stdout:4/993: sync 2024-03-13T14:50:07.304 INFO:tasks.workunit.client.0.smithi176.stdout:4/994: unlink d7/d10/d46/d4a/dcc/dff/dbb/f144 0 2024-03-13T14:50:07.306 INFO:tasks.workunit.client.0.smithi176.stdout:8/942: creat d0/d24/d98/dc6/db9/def/f12e x:0 0 0 2024-03-13T14:50:07.306 INFO:tasks.workunit.client.0.smithi176.stdout:1/959: write d2/de/d3f/d6a/fe2 [3160674,38979] 0 2024-03-13T14:50:07.315 INFO:tasks.workunit.client.0.smithi176.stdout:4/995: creat d7/d10/d2c/d124/d141/f151 x:0 0 0 2024-03-13T14:50:07.316 INFO:tasks.workunit.client.0.smithi176.stdout:7/966: dwrite d1/d1c/d4a/f8d [0,4194304] 0 2024-03-13T14:50:07.322 INFO:tasks.workunit.client.0.smithi176.stdout:5/990: dwrite d5/d2e/d7e/d96/f143 [0,4194304] 0 2024-03-13T14:50:07.325 INFO:tasks.workunit.client.0.smithi176.stdout:5/991: stat d5/d31/def/c114 0 2024-03-13T14:50:07.326 INFO:tasks.workunit.client.0.smithi176.stdout:8/943: rename d0/f52 to d0/f12f 0 2024-03-13T14:50:07.326 INFO:tasks.workunit.client.0.smithi176.stdout:4/996: symlink d7/d10/d46/d4a/dcc/dd0/d11d/dba/l152 0 2024-03-13T14:50:07.330 INFO:tasks.workunit.client.0.smithi176.stdout:4/997: write d7/fe [4872706,92825] 0 2024-03-13T14:50:07.333 INFO:tasks.workunit.client.0.smithi176.stdout:1/960: dwrite d2/d30/dd3/feb [0,4194304] 0 2024-03-13T14:50:07.334 INFO:tasks.workunit.client.0.smithi176.stdout:5/992: rmdir d5/d2c/d40/dc7/d89/df2 39 2024-03-13T14:50:07.342 INFO:tasks.workunit.client.0.smithi176.stdout:8/944: symlink d0/d24/d98/dc6/l130 0 2024-03-13T14:50:07.342 INFO:tasks.workunit.client.0.smithi176.stdout:7/967: truncate d1/d28/d3a/d1b/d8f/dd1/df4/f114 780084 0 2024-03-13T14:50:07.344 INFO:tasks.workunit.client.0.smithi176.stdout:8/945: readlink d0/d24/d29/l111 0 2024-03-13T14:50:07.345 INFO:tasks.workunit.client.0.smithi176.stdout:1/961: creat d2/de/d3f/d3e/d86/db3/f124 x:0 0 0 2024-03-13T14:50:07.349 INFO:tasks.workunit.client.0.smithi176.stdout:4/998: creat d7/f153 x:0 0 0 2024-03-13T14:50:07.350 INFO:tasks.workunit.client.0.smithi176.stdout:7/968: creat d1/d1c/f13e x:0 0 0 2024-03-13T14:50:07.350 INFO:tasks.workunit.client.0.smithi176.stdout:8/946: creat d0/d5/d11/d47/db7/d10a/f131 x:0 0 0 2024-03-13T14:50:07.352 INFO:tasks.workunit.client.0.smithi176.stdout:1/962: fdatasync d2/de/d3f/d3e/d86/fe0 0 2024-03-13T14:50:07.358 INFO:tasks.workunit.client.0.smithi176.stdout:5/993: dread d5/d2c/d40/dc7/fb9 [0,4194304] 0 2024-03-13T14:50:07.358 INFO:tasks.workunit.client.0.smithi176.stdout:8/947: write d0/d5/d11/d47/f63 [2291046,73956] 0 2024-03-13T14:50:07.359 INFO:tasks.workunit.client.0.smithi176.stdout:4/999: fsync d7/d8/d15/d94/fa6 0 2024-03-13T14:50:07.364 INFO:tasks.workunit.client.0.smithi176.stdout:8/948: truncate d0/d24/d98/dc6/f6b 9289111 0 2024-03-13T14:50:07.369 INFO:tasks.workunit.client.0.smithi176.stdout:7/969: dwrite d1/d1c/f13e [0,4194304] 0 2024-03-13T14:50:07.374 INFO:tasks.workunit.client.0.smithi176.stdout:8/949: dread d0/d24/d7b/dab/f92 [0,4194304] 0 2024-03-13T14:50:07.377 INFO:tasks.workunit.client.0.smithi176.stdout:5/994: write d5/d31/d47/fed [2752892,58101] 0 2024-03-13T14:50:07.386 INFO:tasks.workunit.client.0.smithi176.stdout:5/995: unlink d5/c36 0 2024-03-13T14:50:07.390 INFO:tasks.workunit.client.0.smithi176.stdout:5/996: fdatasync d5/d2e/d5f/fd3 0 2024-03-13T14:50:07.393 INFO:tasks.workunit.client.0.smithi176.stdout:5/997: chown d5/d2c/d40/dc7/fda 159346881 1 2024-03-13T14:50:07.398 INFO:tasks.workunit.client.0.smithi176.stdout:7/970: dread d1/d28/d3a/d1b/d8f/dc1/dbb/f10d [0,4194304] 0 2024-03-13T14:50:07.400 INFO:tasks.workunit.client.0.smithi176.stdout:7/971: fsync d1/d1c/d26/d41/db6/df1/dfc/fab 0 2024-03-13T14:50:07.406 INFO:tasks.workunit.client.0.smithi176.stdout:8/950: dread d0/d24/d7b/dec/f108 [0,4194304] 0 2024-03-13T14:50:07.407 INFO:tasks.workunit.client.0.smithi176.stdout:7/972: getdents d1/d7/df6 0 2024-03-13T14:50:07.410 INFO:tasks.workunit.client.0.smithi176.stdout:8/951: dread d0/d5/f113 [4194304,4194304] 0 2024-03-13T14:50:07.411 INFO:tasks.workunit.client.0.smithi176.stdout:7/973: mkdir d1/d28/d3a/d1b/d8f/dc1/dbb/de5/d13f 0 2024-03-13T14:50:07.415 INFO:tasks.workunit.client.0.smithi176.stdout:8/952: dread d0/d24/d98/dc6/f2a [0,4194304] 0 2024-03-13T14:50:07.416 INFO:tasks.workunit.client.0.smithi176.stdout:7/974: fdatasync d1/d28/d3a/d1b/d8f/dc1/dbb/f9f 0 2024-03-13T14:50:07.416 INFO:tasks.workunit.client.0.smithi176.stdout:7/975: stat d1/d28/d3a 0 2024-03-13T14:50:07.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:07 smithi196 ceph-mon[37486]: pgmap v22: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 110 MiB/s rd, 255 MiB/s wr, 582 op/s 2024-03-13T14:50:07.467 INFO:tasks.workunit.client.0.smithi176.stdout:0/898: sync 2024-03-13T14:50:07.468 INFO:tasks.workunit.client.0.smithi176.stdout:8/953: truncate d0/f3 4715277 0 2024-03-13T14:50:07.469 INFO:tasks.workunit.client.0.smithi176.stdout:7/976: chown d1/d1c/d26/d41/db6/df1/dfc/fab 51 1 2024-03-13T14:50:07.472 INFO:tasks.workunit.client.0.smithi176.stdout:0/899: rmdir d2/db/d31 39 2024-03-13T14:50:07.488 INFO:tasks.workunit.client.0.smithi176.stdout:8/954: dwrite d0/d24/d98/fed [0,4194304] 0 2024-03-13T14:50:07.490 INFO:tasks.workunit.client.0.smithi176.stdout:0/900: dwrite d2/fa6 [0,4194304] 0 2024-03-13T14:50:07.499 INFO:tasks.workunit.client.0.smithi176.stdout:7/977: dread d1/d1c/d26/d41/db6/df1/dfc/d76/da5/ff2 [0,4194304] 0 2024-03-13T14:50:07.503 INFO:tasks.workunit.client.0.smithi176.stdout:0/901: rename d2/db/d31/f44 to d2/db/d35/f122 0 2024-03-13T14:50:07.508 INFO:tasks.workunit.client.0.smithi176.stdout:7/978: rmdir d1/d28/d3a/d1b/d8f/dc1/dbb/de5/d13f 0 2024-03-13T14:50:07.509 INFO:tasks.workunit.client.0.smithi176.stdout:0/902: symlink d2/db/d39/d9c/d11c/l123 0 2024-03-13T14:50:07.515 INFO:tasks.workunit.client.0.smithi176.stdout:8/955: dwrite d0/f3 [0,4194304] 0 2024-03-13T14:50:07.515 INFO:tasks.workunit.client.0.smithi176.stdout:7/979: rename d1/d28/l127 to d1/d1c/l140 0 2024-03-13T14:50:07.515 INFO:tasks.workunit.client.0.smithi176.stdout:0/903: read d2/d5/d1e/f72 [1304009,107150] 0 2024-03-13T14:50:07.519 INFO:tasks.workunit.client.0.smithi176.stdout:7/980: fdatasync d1/d1c/d26/d41/d2b/ddc/f13d 0 2024-03-13T14:50:07.520 INFO:tasks.workunit.client.0.smithi176.stdout:0/904: dread - d2/db/d39/f73 zero size 2024-03-13T14:50:07.522 INFO:tasks.workunit.client.0.smithi176.stdout:0/905: stat d2/d5/d28/dac/fc0 0 2024-03-13T14:50:07.524 INFO:tasks.workunit.client.0.smithi176.stdout:8/956: creat d0/d5/d11/f132 x:0 0 0 2024-03-13T14:50:07.534 INFO:tasks.workunit.client.0.smithi176.stdout:0/906: mknod d2/db/c124 0 2024-03-13T14:50:07.534 INFO:tasks.workunit.client.0.smithi176.stdout:0/907: mknod d2/db/d31/dc4/c125 0 2024-03-13T14:50:07.534 INFO:tasks.workunit.client.0.smithi176.stdout:8/957: read d0/d24/dc3/fe3 [1794958,92391] 0 2024-03-13T14:50:07.538 INFO:tasks.workunit.client.0.smithi176.stdout:0/908: mkdir d2/d5/d28/d126 0 2024-03-13T14:50:07.548 INFO:tasks.workunit.client.0.smithi176.stdout:7/981: write d1/d1c/d26/d41/d2b/d40/d64/f60 [1839047,42090] 0 2024-03-13T14:50:07.552 INFO:tasks.workunit.client.0.smithi176.stdout:7/982: truncate d1/d1c/d4a/dda/f131 4533797 0 2024-03-13T14:50:07.555 INFO:tasks.workunit.client.0.smithi176.stdout:7/983: truncate d1/d1c/d4a/f8d 4590513 0 2024-03-13T14:50:07.555 INFO:tasks.workunit.client.0.smithi176.stdout:8/958: write d0/d5/d11/f7f [4872241,65320] 0 2024-03-13T14:50:07.556 INFO:tasks.workunit.client.0.smithi176.stdout:7/984: symlink d1/d7/l141 0 2024-03-13T14:50:07.564 INFO:tasks.workunit.client.0.smithi176.stdout:0/909: dwrite d2/d5/fd1 [0,4194304] 0 2024-03-13T14:50:07.573 INFO:tasks.workunit.client.0.smithi176.stdout:0/910: mkdir d2/d74/d127 0 2024-03-13T14:50:07.573 INFO:tasks.workunit.client.0.smithi176.stdout:7/985: write d1/d28/d3a/d1b/d8f/dc1/dbb/fd7 [901707,67141] 0 2024-03-13T14:50:07.575 INFO:tasks.workunit.client.0.smithi176.stdout:7/986: dread - d1/d7/f132 zero size 2024-03-13T14:50:07.586 INFO:tasks.workunit.client.0.smithi176.stdout:8/959: dread d0/db/d39/d56/f5e [0,4194304] 0 2024-03-13T14:50:07.589 INFO:tasks.workunit.client.0.smithi176.stdout:8/960: truncate d0/d24/d98/f11e 977387 0 2024-03-13T14:50:07.623 INFO:tasks.workunit.client.0.smithi176.stdout:7/987: mknod d1/d1c/d26/d41/db6/df1/dfc/d4b/c142 0 2024-03-13T14:50:07.624 INFO:tasks.workunit.client.0.smithi176.stdout:0/911: creat d2/d5/d1e/f128 x:0 0 0 2024-03-13T14:50:07.629 INFO:tasks.workunit.client.0.smithi176.stdout:0/912: mkdir d2/d74/d10f/d129 0 2024-03-13T14:50:07.633 INFO:tasks.workunit.client.0.smithi176.stdout:1/963: sync 2024-03-13T14:50:07.633 INFO:tasks.workunit.client.0.smithi176.stdout:5/998: sync 2024-03-13T14:50:07.635 INFO:tasks.workunit.client.0.smithi176.stdout:0/913: mkdir d2/db/d35/daa/daf/d12a 0 2024-03-13T14:50:07.637 INFO:tasks.workunit.client.0.smithi176.stdout:1/964: write d2/de/d3f/d3e/f105 [994161,21945] 0 2024-03-13T14:50:07.637 INFO:tasks.workunit.client.0.smithi176.stdout:5/999: dread - d5/d2c/d8a/f105 zero size 2024-03-13T14:50:07.642 INFO:tasks.workunit.client.0.smithi176.stdout:8/961: dwrite d0/d24/d7b/dec/f108 [0,4194304] 0 2024-03-13T14:50:07.645 INFO:tasks.workunit.client.0.smithi176.stdout:1/965: creat d2/d30/f125 x:0 0 0 2024-03-13T14:50:07.648 INFO:tasks.workunit.client.0.smithi176.stdout:1/966: creat d2/de/f126 x:0 0 0 2024-03-13T14:50:07.661 INFO:tasks.workunit.client.0.smithi176.stdout:1/967: fsync d2/de/d3f/f6f 0 2024-03-13T14:50:07.661 INFO:tasks.workunit.client.0.smithi176.stdout:0/914: dwrite d2/d74/fdd [0,4194304] 0 2024-03-13T14:50:07.665 INFO:tasks.workunit.client.0.smithi176.stdout:7/988: write d1/d28/d3a/d1b/fa6 [4406672,125673] 0 2024-03-13T14:50:07.668 INFO:tasks.workunit.client.0.smithi176.stdout:7/989: chown d1/d1c/d26/d41/d2b/ddc/f13d 0 1 2024-03-13T14:50:07.670 INFO:tasks.workunit.client.0.smithi176.stdout:7/990: fdatasync d1/d1c/d26/d41/db6/df1/dfc/d76/f12b 0 2024-03-13T14:50:07.673 INFO:tasks.workunit.client.0.smithi176.stdout:1/968: symlink d2/de/d10/d16/d4a/l127 0 2024-03-13T14:50:07.677 INFO:tasks.workunit.client.0.smithi176.stdout:8/962: truncate d0/db/d39/d56/d93/da7/fbf 519879 0 2024-03-13T14:50:07.678 INFO:tasks.workunit.client.0.smithi176.stdout:1/969: chown d2/lb 1 1 2024-03-13T14:50:07.678 INFO:tasks.workunit.client.0.smithi176.stdout:0/915: link d2/db/cbc d2/d74/d10f/d129/c12b 0 2024-03-13T14:50:07.681 INFO:tasks.workunit.client.0.smithi176.stdout:1/970: rmdir d2/d29/d73 39 2024-03-13T14:50:07.683 INFO:tasks.workunit.client.0.smithi176.stdout:8/963: creat d0/db/d39/d56/d91/f133 x:0 0 0 2024-03-13T14:50:07.688 INFO:tasks.workunit.client.0.smithi176.stdout:1/971: dread d2/de/d10/d16/d4a/d5f/ff9 [0,4194304] 0 2024-03-13T14:50:07.690 INFO:tasks.workunit.client.0.smithi176.stdout:7/991: dwrite d1/d1c/d26/d41/db6/df1/dfc/d65/ff7 [0,4194304] 0 2024-03-13T14:50:07.694 INFO:tasks.workunit.client.0.smithi176.stdout:8/964: mkdir d0/d5/d11/d47/db7/d10a/d134 0 2024-03-13T14:50:07.702 INFO:tasks.workunit.client.0.smithi176.stdout:0/916: getdents d2/db/d39/da5 0 2024-03-13T14:50:07.703 INFO:tasks.workunit.client.0.smithi176.stdout:0/917: fdatasync d2/f82 0 2024-03-13T14:50:07.706 INFO:tasks.workunit.client.0.smithi176.stdout:1/972: mkdir d2/d30/d63/dd5/d128 0 2024-03-13T14:50:07.710 INFO:tasks.workunit.client.0.smithi176.stdout:7/992: rename d1/d1c/d26/d41/db6/df1/dfc/d76/cb5 to d1/d1c/d26/d41/db6/df1/dfc/d76/da5/d126/c143 0 2024-03-13T14:50:07.711 INFO:tasks.workunit.client.0.smithi176.stdout:8/965: symlink d0/d5/d11/d47/db7/d10a/d134/l135 0 2024-03-13T14:50:07.719 INFO:tasks.workunit.client.0.smithi176.stdout:1/973: creat d2/de/d3f/d19/d121/f129 x:0 0 0 2024-03-13T14:50:07.720 INFO:tasks.workunit.client.0.smithi176.stdout:7/993: fsync d1/d28/f2a 0 2024-03-13T14:50:07.721 INFO:tasks.workunit.client.0.smithi176.stdout:1/974: chown d2/d30/dd3/cf5 1 1 2024-03-13T14:50:07.722 INFO:tasks.workunit.client.0.smithi176.stdout:0/918: rename d2/d5/d28/d8c/d9d/d102 to d2/d74/d12c 0 2024-03-13T14:50:07.723 INFO:tasks.workunit.client.0.smithi176.stdout:7/994: write d1/d7/f132 [607953,24639] 0 2024-03-13T14:50:07.725 INFO:tasks.workunit.client.0.smithi176.stdout:1/975: chown d2/de/d3f/d50/c49 26010 1 2024-03-13T14:50:07.726 INFO:tasks.workunit.client.0.smithi176.stdout:8/966: dwrite d0/d24/dc3/f12a [0,4194304] 0 2024-03-13T14:50:07.732 INFO:tasks.workunit.client.0.smithi176.stdout:1/976: rmdir d2/de/d10/d16/d40 39 2024-03-13T14:50:07.734 INFO:tasks.workunit.client.0.smithi176.stdout:0/919: getdents d2/d5/d1e/d97/dcf 0 2024-03-13T14:50:07.737 INFO:tasks.workunit.client.0.smithi176.stdout:8/967: rename d0/f12f to d0/f136 0 2024-03-13T14:50:07.739 INFO:tasks.workunit.client.0.smithi176.stdout:0/920: mknod d2/d74/d10f/c12d 0 2024-03-13T14:50:07.743 INFO:tasks.workunit.client.0.smithi176.stdout:8/968: rename d0/d5/d11/d47/db7/l129 to d0/d24/l137 0 2024-03-13T14:50:07.744 INFO:tasks.workunit.client.0.smithi176.stdout:1/977: dread d2/de/d10/d16/d59/f6e [0,4194304] 0 2024-03-13T14:50:07.745 INFO:tasks.workunit.client.0.smithi176.stdout:8/969: dread - d0/db/d39/d56/d91/f133 zero size 2024-03-13T14:50:07.746 INFO:tasks.workunit.client.0.smithi176.stdout:0/921: rename d2/d5/dc/dfe to d2/db/d35/daa/daf/d12e 0 2024-03-13T14:50:07.747 INFO:tasks.workunit.client.0.smithi176.stdout:1/978: rmdir d2/de/d10/d16/d4a 39 2024-03-13T14:50:07.747 INFO:tasks.workunit.client.0.smithi176.stdout:0/922: chown d2/d74/d9f 38497990 1 2024-03-13T14:50:07.749 INFO:tasks.workunit.client.0.smithi176.stdout:1/979: fsync d2/d30/f125 0 2024-03-13T14:50:07.752 INFO:tasks.workunit.client.0.smithi176.stdout:1/980: truncate d2/de/d10/f72 655940 0 2024-03-13T14:50:07.754 INFO:tasks.workunit.client.0.smithi176.stdout:1/981: write d2/de/d10/d16/d47/f116 [732674,125238] 0 2024-03-13T14:50:07.756 INFO:tasks.workunit.client.0.smithi176.stdout:7/995: dread d1/d1c/d26/d41/db6/df1/dfc/d65/fe4 [0,4194304] 0 2024-03-13T14:50:07.758 INFO:tasks.workunit.client.0.smithi176.stdout:1/982: fsync d2/de/d3f/d3e/f5b 0 2024-03-13T14:50:07.763 INFO:tasks.workunit.client.0.smithi176.stdout:8/970: dwrite d0/db/f19 [4194304,4194304] 0 2024-03-13T14:50:07.786 INFO:tasks.workunit.client.0.smithi176.stdout:1/983: fdatasync d2/de/d3f/d6a/fda 0 2024-03-13T14:50:07.792 INFO:tasks.workunit.client.0.smithi176.stdout:8/971: chown d0/d24/d98/dc6/lde 314703827 1 2024-03-13T14:50:07.799 INFO:tasks.workunit.client.0.smithi176.stdout:1/984: dread d2/d29/f103 [0,4194304] 0 2024-03-13T14:50:07.804 INFO:tasks.workunit.client.0.smithi176.stdout:8/972: dread d0/d5/d11/fda [0,4194304] 0 2024-03-13T14:50:07.807 INFO:tasks.workunit.client.0.smithi176.stdout:7/996: dread d1/d1c/d4a/f98 [0,4194304] 0 2024-03-13T14:50:07.809 INFO:tasks.workunit.client.0.smithi176.stdout:8/973: symlink d0/d24/d7b/dab/d101/l138 0 2024-03-13T14:50:07.817 INFO:tasks.workunit.client.0.smithi176.stdout:7/997: write d1/f1f [190768,109241] 0 2024-03-13T14:50:07.821 INFO:tasks.workunit.client.0.smithi176.stdout:8/974: dread d0/d24/d29/fbd [0,4194304] 0 2024-03-13T14:50:07.822 INFO:tasks.workunit.client.0.smithi176.stdout:7/998: creat d1/d1c/d26/d41/db6/df1/dfc/d76/da5/d101/d11d/d12e/f144 x:0 0 0 2024-03-13T14:50:07.833 INFO:tasks.workunit.client.0.smithi176.stdout:8/975: write d0/db/f25 [5112625,20010] 0 2024-03-13T14:50:07.835 INFO:tasks.workunit.client.0.smithi176.stdout:8/976: chown d0/d5/d11/d47/ce4 12 1 2024-03-13T14:50:07.837 INFO:tasks.workunit.client.0.smithi176.stdout:8/977: chown d0/d5/d11/d47/db7/cbe 994173735 1 2024-03-13T14:50:07.840 INFO:tasks.workunit.client.0.smithi176.stdout:8/978: symlink d0/d5/d11/d47/db7/de0/l139 0 2024-03-13T14:50:07.842 INFO:tasks.workunit.client.0.smithi176.stdout:8/979: chown d0/d24/d7b/dab/f11a 6 1 2024-03-13T14:50:07.843 INFO:tasks.workunit.client.0.smithi176.stdout:8/980: read d0/d5/d11/fda [2965948,64068] 0 2024-03-13T14:50:07.869 INFO:tasks.workunit.client.0.smithi176.stdout:8/981: dread d0/d5/d11/d47/f6a [0,4194304] 0 2024-03-13T14:50:07.877 INFO:tasks.workunit.client.0.smithi176.stdout:8/982: mkdir d0/d24/d29/d13a 0 2024-03-13T14:50:07.885 INFO:tasks.workunit.client.0.smithi176.stdout:8/983: unlink d0/db/d53/d80/f97 0 2024-03-13T14:50:07.917 INFO:tasks.workunit.client.0.smithi176.stdout:8/984: dread d0/d24/d98/dc6/d40/f62 [0,4194304] 0 2024-03-13T14:50:07.921 INFO:tasks.workunit.client.0.smithi176.stdout:8/985: creat d0/db/f13b x:0 0 0 2024-03-13T14:50:07.931 INFO:tasks.workunit.client.0.smithi176.stdout:8/986: creat d0/d5/f13c x:0 0 0 2024-03-13T14:50:07.940 INFO:tasks.workunit.client.0.smithi176.stdout:8/987: creat d0/d24/d29/dce/f13d x:0 0 0 2024-03-13T14:50:08.002 INFO:tasks.workunit.client.0.smithi176.stdout:8/988: dwrite d0/d24/d7b/dab/fe5 [0,4194304] 0 2024-03-13T14:50:08.012 INFO:tasks.workunit.client.0.smithi176.stdout:8/989: dread d0/d24/d29/f36 [0,4194304] 0 2024-03-13T14:50:08.018 INFO:tasks.workunit.client.0.smithi176.stdout:8/990: rename d0/db/d39/d56/f79 to d0/db/d53/df7/f13e 0 2024-03-13T14:50:08.020 INFO:tasks.workunit.client.0.smithi176.stdout:8/991: write d0/f5c [3510142,108687] 0 2024-03-13T14:50:08.021 INFO:tasks.workunit.client.0.smithi176.stdout:8/992: write d0/fdf [3415628,102576] 0 2024-03-13T14:50:08.024 INFO:tasks.workunit.client.0.smithi176.stdout:8/993: symlink d0/db/l13f 0 2024-03-13T14:50:08.028 INFO:tasks.workunit.client.0.smithi176.stdout:7/999: sync 2024-03-13T14:50:08.028 INFO:tasks.workunit.client.0.smithi176.stdout:1/985: sync 2024-03-13T14:50:08.028 INFO:tasks.workunit.client.0.smithi176.stdout:0/923: sync 2024-03-13T14:50:08.030 INFO:tasks.workunit.client.0.smithi176.stdout:1/986: fsync d2/de/d3f/f52 0 2024-03-13T14:50:08.032 INFO:tasks.workunit.client.0.smithi176.stdout:1/987: chown d2/de/d3f/d3e/d86 353907 1 2024-03-13T14:50:08.035 INFO:tasks.workunit.client.0.smithi176.stdout:8/994: rename d0/db/d39/d56/l59 to d0/d5/d11/d47/l140 0 2024-03-13T14:50:08.037 INFO:tasks.workunit.client.0.smithi176.stdout:1/988: creat d2/de/d3f/d19/f12a x:0 0 0 2024-03-13T14:50:08.041 INFO:tasks.workunit.client.0.smithi176.stdout:0/924: rename d2/d74/d61/f66 to d2/d5/f12f 0 2024-03-13T14:50:08.043 INFO:tasks.workunit.client.0.smithi176.stdout:0/925: write d2/db/d31/dc4/f119 [37928,80757] 0 2024-03-13T14:50:08.043 INFO:tasks.workunit.client.0.smithi176.stdout:1/989: symlink d2/de/d10/d16/d47/db4/l12b 0 2024-03-13T14:50:08.051 INFO:tasks.workunit.client.0.smithi176.stdout:0/926: mkdir d2/d130 0 2024-03-13T14:50:08.053 INFO:tasks.workunit.client.0.smithi176.stdout:0/927: truncate d2/d5/d1e/f4e 4577073 0 2024-03-13T14:50:08.065 INFO:tasks.workunit.client.0.smithi176.stdout:0/928: getdents d2/db/d31/dc4 0 2024-03-13T14:50:08.079 INFO:tasks.workunit.client.0.smithi176.stdout:1/990: dwrite d2/de/d10/d16/d4a/f8a [0,4194304] 0 2024-03-13T14:50:08.094 INFO:tasks.workunit.client.0.smithi176.stdout:0/929: dwrite d2/d5/d1e/f92 [0,4194304] 0 2024-03-13T14:50:08.103 INFO:tasks.workunit.client.0.smithi176.stdout:1/991: dwrite d2/de/d3f/d6a/fe2 [0,4194304] 0 2024-03-13T14:50:08.109 INFO:tasks.workunit.client.0.smithi176.stdout:0/930: getdents d2/db 0 2024-03-13T14:50:08.112 INFO:tasks.workunit.client.0.smithi176.stdout:1/992: write d2/de/d3f/d19/f102 [155386,78876] 0 2024-03-13T14:50:08.119 INFO:tasks.workunit.client.0.smithi176.stdout:1/993: write d2/de/dbf/f111 [703275,36202] 0 2024-03-13T14:50:08.146 INFO:tasks.workunit.client.0.smithi176.stdout:0/931: dread d2/db/d31/f7f [0,4194304] 0 2024-03-13T14:50:08.146 INFO:tasks.workunit.client.0.smithi176.stdout:1/994: dread d2/d30/f65 [0,4194304] 0 2024-03-13T14:50:08.147 INFO:tasks.workunit.client.0.smithi176.stdout:0/932: dread - d2/db/d39/da5/f11a zero size 2024-03-13T14:50:08.150 INFO:tasks.workunit.client.0.smithi176.stdout:0/933: creat d2/d74/d9f/f131 x:0 0 0 2024-03-13T14:50:08.151 INFO:tasks.workunit.client.0.smithi176.stdout:1/995: dread d2/de/d10/d16/d47/f5c [0,4194304] 0 2024-03-13T14:50:08.152 INFO:tasks.workunit.client.0.smithi176.stdout:1/996: chown d2/de/d10/d16/d59 1 1 2024-03-13T14:50:08.152 INFO:tasks.workunit.client.0.smithi176.stdout:0/934: mkdir d2/db/d39/d9c/d11c/d132 0 2024-03-13T14:50:08.154 INFO:tasks.workunit.client.0.smithi176.stdout:1/997: write d2/de/d3f/d50/f8e [1905021,31249] 0 2024-03-13T14:50:08.155 INFO:tasks.workunit.client.0.smithi176.stdout:0/935: dread - d2/f99 zero size 2024-03-13T14:50:08.161 INFO:tasks.workunit.client.0.smithi176.stdout:0/936: dread d2/d74/df8/f108 [0,4194304] 0 2024-03-13T14:50:08.163 INFO:tasks.workunit.client.0.smithi176.stdout:0/937: fdatasync d2/d5/dc/d20/f37 0 2024-03-13T14:50:08.171 INFO:tasks.workunit.client.0.smithi176.stdout:1/998: truncate d2/de/d10/d16/f27 8911748 0 2024-03-13T14:50:08.179 INFO:tasks.workunit.client.0.smithi176.stdout:0/938: dwrite d2/d74/f91 [0,4194304] 0 2024-03-13T14:50:08.182 INFO:tasks.workunit.client.0.smithi176.stdout:0/939: chown d2/d5/d1e/d97/dcf/ld6 802 1 2024-03-13T14:50:08.187 INFO:tasks.workunit.client.0.smithi176.stdout:8/995: sync 2024-03-13T14:50:08.188 INFO:tasks.workunit.client.0.smithi176.stdout:0/940: getdents d2/db/d35/d101 0 2024-03-13T14:50:08.189 INFO:tasks.workunit.client.0.smithi176.stdout:0/941: chown d2/db/l4a 667 1 2024-03-13T14:50:08.194 INFO:tasks.workunit.client.0.smithi176.stdout:8/996: dread d0/d24/d29/f9d [0,4194304] 0 2024-03-13T14:50:08.199 INFO:tasks.workunit.client.0.smithi176.stdout:8/997: mknod d0/c141 0 2024-03-13T14:50:08.200 INFO:tasks.workunit.client.0.smithi176.stdout:0/942: write d2/d5/dc/d20/feb [99055,65817] 0 2024-03-13T14:50:08.200 INFO:tasks.workunit.client.0.smithi176.stdout:8/998: chown d0/f3 6080 1 2024-03-13T14:50:08.202 INFO:tasks.workunit.client.0.smithi176.stdout:8/999: chown d0/d24/d98/fed 0 1 2024-03-13T14:50:08.204 INFO:tasks.workunit.client.0.smithi176.stdout:0/943: rename d2/db/d31/cb4 to d2/d74/d9f/c133 0 2024-03-13T14:50:08.205 INFO:tasks.workunit.client.0.smithi176.stdout:0/944: readlink d2/db/d31/dc4/le3 0 2024-03-13T14:50:08.217 INFO:tasks.workunit.client.0.smithi176.stdout:0/945: dread d2/d5/d1e/f4e [0,4194304] 0 2024-03-13T14:50:08.248 INFO:tasks.workunit.client.0.smithi176.stdout:0/946: dread d2/d5/f95 [0,4194304] 0 2024-03-13T14:50:08.254 INFO:tasks.workunit.client.0.smithi176.stdout:0/947: mknod d2/d5/dc/c134 0 2024-03-13T14:50:08.260 INFO:tasks.workunit.client.0.smithi176.stdout:0/948: read d2/f6 [3487402,89879] 0 2024-03-13T14:50:08.266 INFO:tasks.workunit.client.0.smithi176.stdout:0/949: fsync d2/db/d39/fa9 0 2024-03-13T14:50:08.274 INFO:tasks.workunit.client.0.smithi176.stdout:0/950: creat d2/db/d35/d101/f135 x:0 0 0 2024-03-13T14:50:08.276 INFO:tasks.workunit.client.0.smithi176.stdout:0/951: read d2/d74/f91 [794228,64103] 0 2024-03-13T14:50:08.281 INFO:tasks.workunit.client.0.smithi176.stdout:0/952: readlink d2/d5/l7e 0 2024-03-13T14:50:08.287 INFO:tasks.workunit.client.0.smithi176.stdout:0/953: mkdir d2/d5/d28/d8c/d9d/d136 0 2024-03-13T14:50:08.292 INFO:tasks.workunit.client.0.smithi176.stdout:0/954: write d2/d5/d28/d8c/d9d/ff0 [269120,118212] 0 2024-03-13T14:50:08.301 INFO:tasks.workunit.client.0.smithi176.stdout:0/955: rename d2/d5/d28/ce4 to d2/d74/d127/c137 0 2024-03-13T14:50:08.303 INFO:tasks.workunit.client.0.smithi176.stdout:1/999: sync 2024-03-13T14:50:08.307 INFO:tasks.workunit.client.0.smithi176.stdout:0/956: mknod d2/d5/dc/d59/d120/c138 0 2024-03-13T14:50:08.334 INFO:tasks.workunit.client.0.smithi176.stdout:0/957: write d2/db/d35/f121 [2028470,42617] 0 2024-03-13T14:50:08.354 INFO:tasks.workunit.client.0.smithi176.stdout:0/958: truncate d2/fa6 1666973 0 2024-03-13T14:50:08.359 INFO:tasks.workunit.client.0.smithi176.stdout:0/959: write d2/db/d39/fa2 [1657138,98447] 0 2024-03-13T14:50:08.363 INFO:tasks.workunit.client.0.smithi176.stdout:0/960: chown d2/db/d39/ffc 757 1 2024-03-13T14:50:08.379 INFO:tasks.workunit.client.0.smithi176.stdout:0/961: write d2/d74/d9f/fbf [270365,27243] 0 2024-03-13T14:50:08.386 INFO:tasks.workunit.client.0.smithi176.stdout:0/962: mkdir d2/db/d39/d9c/d139 0 2024-03-13T14:50:08.393 INFO:tasks.workunit.client.0.smithi176.stdout:0/963: unlink d2/d5/c3c 0 2024-03-13T14:50:08.436 INFO:tasks.workunit.client.0.smithi176.stdout:0/964: dread d2/d5/dc/d20/f37 [0,4194304] 0 2024-03-13T14:50:08.470 INFO:tasks.workunit.client.0.smithi176.stdout:0/965: dwrite d2/d5/dc/fd8 [0,4194304] 0 2024-03-13T14:50:08.480 INFO:tasks.workunit.client.0.smithi176.stdout:0/966: readlink d2/d5/d1e/l58 0 2024-03-13T14:50:08.514 INFO:tasks.workunit.client.0.smithi176.stdout:0/967: dwrite d2/d74/d9f/f106 [0,4194304] 0 2024-03-13T14:50:08.525 INFO:tasks.workunit.client.0.smithi176.stdout:0/968: getdents d2/db/d39/d9c/d139 0 2024-03-13T14:50:08.530 INFO:tasks.workunit.client.0.smithi176.stdout:0/969: mknod d2/d130/c13a 0 2024-03-13T14:50:08.558 INFO:tasks.workunit.client.0.smithi176.stdout:0/970: dwrite d2/d5/f12f [0,4194304] 0 2024-03-13T14:50:08.767 INFO:tasks.workunit.client.0.smithi176.stdout:0/971: sync 2024-03-13T14:50:08.774 INFO:tasks.workunit.client.0.smithi176.stdout:0/972: mknod d2/db/d35/daa/daf/d12e/c13b 0 2024-03-13T14:50:08.779 INFO:tasks.workunit.client.0.smithi176.stdout:0/973: mkdir d2/db/d31/d13c 0 2024-03-13T14:50:08.814 INFO:tasks.workunit.client.0.smithi176.stdout:0/974: dwrite d2/db/d35/d101/f135 [0,4194304] 0 2024-03-13T14:50:08.815 INFO:tasks.workunit.client.0.smithi176.stdout:0/975: stat d2 0 2024-03-13T14:50:08.866 INFO:tasks.workunit.client.0.smithi176.stdout:0/976: dwrite d2/d5/f10b [0,4194304] 0 2024-03-13T14:50:08.884 INFO:tasks.workunit.client.0.smithi176.stdout:0/977: creat d2/d74/f13d x:0 0 0 2024-03-13T14:50:08.898 INFO:tasks.workunit.client.0.smithi176.stdout:0/978: dread d2/d5/d28/d8c/fcc [0,4194304] 0 2024-03-13T14:50:08.925 INFO:tasks.workunit.client.0.smithi176.stdout:0/979: write d2/f50 [846502,130569] 0 2024-03-13T14:50:08.935 INFO:tasks.workunit.client.0.smithi176.stdout:0/980: creat d2/d74/d12c/f13e x:0 0 0 2024-03-13T14:50:08.945 INFO:tasks.workunit.client.0.smithi176.stdout:0/981: creat d2/d5/d28/d8c/d9d/f13f x:0 0 0 2024-03-13T14:50:08.967 INFO:tasks.workunit.client.0.smithi176.stdout:0/982: dwrite d2/d74/ff4 [0,4194304] 0 2024-03-13T14:50:08.981 INFO:tasks.workunit.client.0.smithi176.stdout:0/983: creat d2/db/d39/d9c/f140 x:0 0 0 2024-03-13T14:50:08.982 INFO:tasks.workunit.client.0.smithi176.stdout:0/984: readlink d2/db/d39/d9c/la0 0 2024-03-13T14:50:08.992 INFO:tasks.workunit.client.0.smithi176.stdout:0/985: unlink d2/f3 0 2024-03-13T14:50:09.004 INFO:tasks.workunit.client.0.smithi176.stdout:0/986: rename d2/db/d39/d9c/fb0 to d2/db/d35/d101/f141 0 2024-03-13T14:50:09.006 INFO:tasks.workunit.client.0.smithi176.stdout:0/987: stat d2/db/d39/da5/f11a 0 2024-03-13T14:50:09.007 INFO:tasks.workunit.client.0.smithi176.stdout:0/988: fsync d2/d74/d9f/fbf 0 2024-03-13T14:50:09.016 INFO:tasks.workunit.client.0.smithi176.stdout:0/989: unlink d2/d5/dc/c77 0 2024-03-13T14:50:09.018 INFO:tasks.workunit.client.0.smithi176.stdout:0/990: chown d2/db/f3b 1 1 2024-03-13T14:50:09.042 INFO:tasks.workunit.client.0.smithi176.stdout:0/991: dwrite d2/d74/d12c/f13e [0,4194304] 0 2024-03-13T14:50:09.049 INFO:tasks.workunit.client.0.smithi176.stdout:0/992: symlink d2/db/d35/ded/l142 0 2024-03-13T14:50:09.051 INFO:tasks.workunit.client.0.smithi176.stdout:0/993: fsync d2/d74/f6e 0 2024-03-13T14:50:09.055 INFO:tasks.workunit.client.0.smithi176.stdout:0/994: chown d2/d74/d9f 2 1 2024-03-13T14:50:09.092 INFO:tasks.workunit.client.0.smithi176.stdout:0/995: dwrite d2/d5/dc/d20/f37 [4194304,4194304] 0 2024-03-13T14:50:09.101 INFO:tasks.workunit.client.0.smithi176.stdout:0/996: creat d2/d5/dc/d59/d120/f143 x:0 0 0 2024-03-13T14:50:09.139 INFO:tasks.workunit.client.0.smithi176.stdout:0/997: dwrite d2/db/d31/f7f [0,4194304] 0 2024-03-13T14:50:09.186 INFO:tasks.workunit.client.0.smithi176.stdout:0/998: dread d2/db/d39/f7a [0,4194304] 0 2024-03-13T14:50:09.239 INFO:tasks.workunit.client.0.smithi176.stdout:0/999: dwrite d2/d5/d1e/f3e [4194304,4194304] 0 2024-03-13T14:50:09.258 INFO:tasks.workunit.client.0.smithi176.stderr:+ rm -rf -- ./tmp.VggskuEev9 2024-03-13T14:50:09.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:09 smithi176 ceph-mon[29064]: pgmap v23: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 170 MiB/s wr, 396 op/s 2024-03-13T14:50:09.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:09 smithi196 ceph-mon[37486]: pgmap v23: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 73 MiB/s rd, 170 MiB/s wr, 396 op/s 2024-03-13T14:50:11.314 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:11 smithi176 ceph-mon[29064]: pgmap v24: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 101 MiB/s rd, 232 MiB/s wr, 563 op/s 2024-03-13T14:50:11.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:11 smithi196 ceph-mon[37486]: pgmap v24: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 101 MiB/s rd, 232 MiB/s wr, 563 op/s 2024-03-13T14:50:13.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:13 smithi176 ceph-mon[29064]: pgmap v25: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 65 MiB/s rd, 145 MiB/s wr, 363 op/s 2024-03-13T14:50:13.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:13.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:13.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:13.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:50:13.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:50:13.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:13 smithi196 ceph-mon[37486]: pgmap v25: 65 pgs: 65 active+clean; 4.1 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 65 MiB/s rd, 145 MiB/s wr, 363 op/s 2024-03-13T14:50:13.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:13.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:13.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:13.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:50:13.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:50:14.372 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:14 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:50:14.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:14 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:50:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:15 smithi176 ceph-mon[29064]: pgmap v26: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 65 MiB/s rd, 146 MiB/s wr, 366 op/s 2024-03-13T14:50:15.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:15 smithi196 ceph-mon[37486]: pgmap v26: 65 pgs: 65 active+clean; 4.0 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 65 MiB/s rd, 146 MiB/s wr, 366 op/s 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 -- 172.21.15.176:0/1893123533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ff2f0 msgr2=0x7f56bc0ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 --2- 172.21.15.176:0/1893123533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc0ff710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009910 tx=0x7f56ac009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 -- 172.21.15.176:0/1893123533 shutdown_connections 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 --2- 172.21.15.176:0/1893123533 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ffc50 0x7f56bc1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 --2- 172.21.15.176:0/1893123533 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc0ff710 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 -- 172.21.15.176:0/1893123533 >> 172.21.15.176:0/1893123533 conn(0x7f56bc0faed0 msgr2=0x7f56bc0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 -- 172.21.15.176:0/1893123533 shutdown_connections 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.354+0000 7f56c0c00700 1 -- 172.21.15.176:0/1893123533 wait complete. 2024-03-13T14:50:16.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.356+0000 7f56c0c00700 1 Processor -- start 2024-03-13T14:50:16.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.356+0000 7f56c0c00700 1 -- start start 2024-03-13T14:50:16.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.356+0000 7f56c0c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.356+0000 7f56c0c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ffc50 0x7f56bc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.356+0000 7f56c0c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc198f30 con 0x7f56bc0ffc50 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.356+0000 7f56c0c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc199070 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56bb400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56baa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ffc50 0x7f56bc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56bb400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:42466/0 (socket says 172.21.15.176:42466) 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56bb400700 1 -- 172.21.15.176:0/3952762891 learned_addr learned my addr 172.21.15.176:0/3952762891 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56bb400700 1 -- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ffc50 msgr2=0x7f56bc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56bb400700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ffc50 0x7f56bc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56bb400700 1 -- 172.21.15.176:0/3952762891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac0095f0 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.357+0000 7f56baa00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ffc50 0x7f56bc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.358+0000 7f56bb400700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc1983d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0092e0 tx=0x7f56ac00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:16.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.358+0000 7f56b3e00700 1 -- 172.21.15.176:0/3952762891 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac01d070 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.358+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc19dac0 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.358+0000 7f56b3e00700 1 -- 172.21.15.176:0/3952762891 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f56ac005dc0 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.358+0000 7f56b3e00700 1 -- 172.21.15.176:0/3952762891 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac017630 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.358+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc19dfb0 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.360+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc10b770 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.360+0000 7f56b3e00700 1 -- 172.21.15.176:0/3952762891 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f56ac003680 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.361+0000 7f56b3e00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f56a006dd50 0x7f56a0070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.361+0000 7f56b3e00700 1 -- 172.21.15.176:0/3952762891 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f56ac08ef40 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.362+0000 7f56baa00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f56a006dd50 0x7f56a0070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:16.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.362+0000 7f56baa00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f56a006dd50 0x7f56a0070210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f56a8011490 tx=0x7f56a8009260 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:16.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.379+0000 7f56b3e00700 1 -- 172.21.15.176:0/3952762891 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f56ac05c180 con 0x7f56bc0ff2f0 2024-03-13T14:50:16.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:16 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:50:16.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:16 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:50:16.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:16 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:50:16.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.637+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56bc109040 con 0x7f56a006dd50 2024-03-13T14:50:16.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:16 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:50:16.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:16 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:50:16.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:16 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:50:16.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.639+0000 7f56b3e00700 1 -- 172.21.15.176:0/3952762891 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f56bc109040 con 0x7f56a006dd50 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.641+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f56a006dd50 msgr2=0x7f56a0070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.641+0000 7f56c0c00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f56a006dd50 0x7f56a0070210 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f56a8011490 tx=0x7f56a8009260 comp rx=0 tx=0).stop 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.641+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ff2f0 msgr2=0x7f56bc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.641+0000 7f56c0c00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc1983d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0092e0 tx=0x7f56ac00f690 comp rx=0 tx=0).stop 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.642+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 shutdown_connections 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.642+0000 7f56c0c00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f56a006dd50 0x7f56a0070210 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.642+0000 7f56c0c00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f56bc0ff2f0 0x7f56bc1983d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.642+0000 7f56c0c00700 1 --2- 172.21.15.176:0/3952762891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f56bc0ffc50 0x7f56bc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.642+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 >> 172.21.15.176:0/3952762891 conn(0x7f56bc0faed0 msgr2=0x7f56bc107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.642+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 shutdown_connections 2024-03-13T14:50:16.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.642+0000 7f56c0c00700 1 -- 172.21.15.176:0/3952762891 wait complete. 2024-03-13T14:50:16.646 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:50:16.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 -- 172.21.15.176:0/2027186874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e0103fd0 msgr2=0x7f64e01063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:16.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 --2- 172.21.15.176:0/2027186874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e0103fd0 0x7f64e01063c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009910 tx=0x7f64cc009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:16.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 -- 172.21.15.176:0/2027186874 shutdown_connections 2024-03-13T14:50:16.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 --2- 172.21.15.176:0/2027186874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e0103fd0 0x7f64e01063c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 --2- 172.21.15.176:0/2027186874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e01016a0 0x7f64e0103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 -- 172.21.15.176:0/2027186874 >> 172.21.15.176:0/2027186874 conn(0x7f64e00fb030 msgr2=0x7f64e00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:16.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 -- 172.21.15.176:0/2027186874 shutdown_connections 2024-03-13T14:50:16.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.814+0000 7f64e5800700 1 -- 172.21.15.176:0/2027186874 wait complete. 2024-03-13T14:50:16.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.815+0000 7f64e5800700 1 Processor -- start 2024-03-13T14:50:16.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.816+0000 7f64e5800700 1 -- start start 2024-03-13T14:50:16.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.816+0000 7f64e5800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e01016a0 0x7f64e0196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:16.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.816+0000 7f64e5800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e0103fd0 0x7f64e0196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:16.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.816+0000 7f64e5800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0196e90 con 0x7f64e01016a0 2024-03-13T14:50:16.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.816+0000 7f64e5800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e0196fd0 con 0x7f64e0103fd0 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64dea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e01016a0 0x7f64e0196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64dea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e01016a0 0x7f64e0196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37492/0 (socket says 172.21.15.176:37492) 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64dea00700 1 -- 172.21.15.176:0/1217841387 learned_addr learned my addr 172.21.15.176:0/1217841387 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64de000700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e0103fd0 0x7f64e0196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64dea00700 1 -- 172.21.15.176:0/1217841387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e0103fd0 msgr2=0x7f64e0196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64dea00700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e0103fd0 0x7f64e0196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64dea00700 1 -- 172.21.15.176:0/1217841387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0095f0 con 0x7f64e01016a0 2024-03-13T14:50:16.805 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.817+0000 7f64de000700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e0103fd0 0x7f64e0196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:50:16.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.818+0000 7f64dea00700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e01016a0 0x7f64e0196330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e920 tx=0x7f64d000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:16.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.818+0000 7f64d7400700 1 -- 172.21.15.176:0/1217841387 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d0009790 con 0x7f64e01016a0 2024-03-13T14:50:16.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.818+0000 7f64d7400700 1 -- 172.21.15.176:0/1217841387 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64d000cba0 con 0x7f64e01016a0 2024-03-13T14:50:16.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.818+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e019ba80 con 0x7f64e01016a0 2024-03-13T14:50:16.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.818+0000 7f64d7400700 1 -- 172.21.15.176:0/1217841387 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64d0018630 con 0x7f64e01016a0 2024-03-13T14:50:16.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.818+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e019bfd0 con 0x7f64e01016a0 2024-03-13T14:50:16.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.820+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e0190520 con 0x7f64e01016a0 2024-03-13T14:50:16.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.822+0000 7f64d7400700 1 -- 172.21.15.176:0/1217841387 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f64d000cd10 con 0x7f64e01016a0 2024-03-13T14:50:16.811 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.823+0000 7f64d7400700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f64c406dd50 0x7f64c4070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:16.811 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.824+0000 7f64d7400700 1 -- 172.21.15.176:0/1217841387 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f64d0014070 con 0x7f64e01016a0 2024-03-13T14:50:16.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.824+0000 7f64de000700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f64c406dd50 0x7f64c4070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:16.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.825+0000 7f64de000700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f64c406dd50 0x7f64c4070210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009fd0 tx=0x7f64cc005fb0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:16.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:16.829+0000 7f64d7400700 1 -- 172.21.15.176:0/1217841387 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f64d005b2d0 con 0x7f64e01016a0 2024-03-13T14:50:17.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.155+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f64e00611d0 con 0x7f64c406dd50 2024-03-13T14:50:17.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.157+0000 7f64d7400700 1 -- 172.21.15.176:0/1217841387 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f64e00611d0 con 0x7f64c406dd50 2024-03-13T14:50:17.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.159+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f64c406dd50 msgr2=0x7f64c4070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.159+0000 7f64e5800700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f64c406dd50 0x7f64c4070210 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009fd0 tx=0x7f64cc005fb0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.159+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e01016a0 msgr2=0x7f64e0196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.159+0000 7f64e5800700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e01016a0 0x7f64e0196330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f64d000e920 tx=0x7f64d000ece0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.160+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 shutdown_connections 2024-03-13T14:50:17.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.160+0000 7f64e5800700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f64c406dd50 0x7f64c4070210 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.160+0000 7f64e5800700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f64e01016a0 0x7f64e0196330 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.160+0000 7f64e5800700 1 --2- 172.21.15.176:0/1217841387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64e0103fd0 0x7f64e0196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.160+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 >> 172.21.15.176:0/1217841387 conn(0x7f64e00fb030 msgr2=0x7f64e00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:17.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.160+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 shutdown_connections 2024-03-13T14:50:17.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.161+0000 7f64e5800700 1 -- 172.21.15.176:0/1217841387 wait complete. 2024-03-13T14:50:17.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 -- 172.21.15.176:0/2439412349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 msgr2=0x7fd738103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 --2- 172.21.15.176:0/2439412349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd738103230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd72c009910 tx=0x7fd72c009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:17.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 -- 172.21.15.176:0/2439412349 shutdown_connections 2024-03-13T14:50:17.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 --2- 172.21.15.176:0/2439412349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd738103f70 0x7fd7381043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 --2- 172.21.15.176:0/2439412349 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd738103230 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 -- 172.21.15.176:0/2439412349 >> 172.21.15.176:0/2439412349 conn(0x7fd7380fe3d0 msgr2=0x7fd7381007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:17.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 -- 172.21.15.176:0/2439412349 shutdown_connections 2024-03-13T14:50:17.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.318+0000 7fd740400700 1 -- 172.21.15.176:0/2439412349 wait complete. 2024-03-13T14:50:17.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.319+0000 7fd740400700 1 Processor -- start 2024-03-13T14:50:17.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.319+0000 7fd740400700 1 -- start start 2024-03-13T14:50:17.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd740400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd7381985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:17.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd740400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd738103f70 0x7fd738198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:17.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd740400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738199150 con 0x7fd738102e10 2024-03-13T14:50:17.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd740400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738199290 con 0x7fd738103f70 2024-03-13T14:50:17.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd73da00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd7381985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:17.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd73d000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd738103f70 0x7fd738198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:17.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd73da00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd7381985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37508/0 (socket says 172.21.15.176:37508) 2024-03-13T14:50:17.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.320+0000 7fd73da00700 1 -- 172.21.15.176:0/1285500919 learned_addr learned my addr 172.21.15.176:0/1285500919 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:17.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd73da00700 1 -- 172.21.15.176:0/1285500919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd738103f70 msgr2=0x7fd738198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd73da00700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd738103f70 0x7fd738198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd73da00700 1 -- 172.21.15.176:0/1285500919 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c0095f0 con 0x7fd738102e10 2024-03-13T14:50:17.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd73d000700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd738103f70 0x7fd738198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T14:50:17.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd73da00700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd7381985f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005b40 tx=0x7fd72c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:17.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd736000700 1 -- 172.21.15.176:0/1285500919 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd72c01d070 con 0x7fd738102e10 2024-03-13T14:50:17.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd736000700 1 -- 172.21.15.176:0/1285500919 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd72c022470 con 0x7fd738102e10 2024-03-13T14:50:17.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73819dce0 con 0x7fd738102e10 2024-03-13T14:50:17.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.321+0000 7fd736000700 1 -- 172.21.15.176:0/1285500919 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd72c00f690 con 0x7fd738102e10 2024-03-13T14:50:17.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.322+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73819e200 con 0x7fd738102e10 2024-03-13T14:50:17.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.323+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd73810b970 con 0x7fd738102e10 2024-03-13T14:50:17.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.324+0000 7fd736000700 1 -- 172.21.15.176:0/1285500919 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fd72c022a10 con 0x7fd738102e10 2024-03-13T14:50:17.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.325+0000 7fd736000700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd72006dd50 0x7fd720070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:17.313 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.325+0000 7fd736000700 1 -- 172.21.15.176:0/1285500919 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd72c08f0b0 con 0x7fd738102e10 2024-03-13T14:50:17.313 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.325+0000 7fd73d000700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd72006dd50 0x7fd720070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:17.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.326+0000 7fd73d000700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd72006dd50 0x7fd720070210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd724009de0 tx=0x7fd724009190 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:17.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.330+0000 7fd736000700 1 -- 172.21.15.176:0/1285500919 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fd72c05c4c0 con 0x7fd738102e10 2024-03-13T14:50:17.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:17 smithi196 ceph-mon[37486]: Upgrade: Updating mgr.smithi196.vdktbk 2024-03-13T14:50:17.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:17 smithi196 ceph-mon[37486]: Deploying daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:50:17.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:17 smithi196 ceph-mon[37486]: pgmap v27: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 60 MiB/s rd, 134 MiB/s wr, 402 op/s 2024-03-13T14:50:17.557 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.569+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd738108950 con 0x7fd72006dd50 2024-03-13T14:50:17.557 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:17 smithi176 ceph-mon[29064]: Upgrade: Updating mgr.smithi196.vdktbk 2024-03-13T14:50:17.557 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:17 smithi176 ceph-mon[29064]: Deploying daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:50:17.557 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:17 smithi176 ceph-mon[29064]: pgmap v27: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 60 MiB/s rd, 134 MiB/s wr, 402 op/s 2024-03-13T14:50:17.573 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.585+0000 7fd736000700 1 -- 172.21.15.176:0/1285500919 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd738108950 con 0x7fd72006dd50 2024-03-13T14:50:17.573 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:50:17.573 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (9m) 34s ago 11m 13.9M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:50:17.573 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (12m) 34s ago 12m 7784k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:50:17.573 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (10m) 37s ago 10m 7696k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (12m) 34s ago 12m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (10m) 37s ago 10m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (9m) 34s ago 11m 74.9M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (4m) 34s ago 4m 108M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (4m) 34s ago 4m 13.8M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (4m) 37s ago 4m 166M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (4m) 37s ago 4m 13.8M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (66s) 34s ago 13m 473M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (2m) 37s ago 10m 71.3M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5187659249a7 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (13m) 34s ago 13m 51.7M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (10m) 37s ago 10m 41.0M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (11m) 34s ago 11m 7680k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (10m) 37s ago 10m 8082k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:50:17.574 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (9m) 34s ago 9m 139M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:50:17.575 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (8m) 34s ago 8m 156M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:50:17.575 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (8m) 34s ago 8m 138M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:50:17.575 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (7m) 37s ago 7m 178M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:50:17.575 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (6m) 37s ago 6m 166M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:50:17.575 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (6m) 37s ago 6m 139M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:50:17.575 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 starting - - - - 2024-03-13T14:50:17.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd72006dd50 msgr2=0x7fd720070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd72006dd50 0x7fd720070210 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd724009de0 tx=0x7fd724009190 comp rx=0 tx=0).stop 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 msgr2=0x7fd7381985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd7381985f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd72c005b40 tx=0x7fd72c00bd70 comp rx=0 tx=0).stop 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 shutdown_connections 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd72006dd50 0x7fd720070210 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd738102e10 0x7fd7381985f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.588+0000 7fd740400700 1 --2- 172.21.15.176:0/1285500919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd738103f70 0x7fd738198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.589+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 >> 172.21.15.176:0/1285500919 conn(0x7fd7380fe3d0 msgr2=0x7fd738107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.589+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 shutdown_connections 2024-03-13T14:50:17.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.589+0000 7fd740400700 1 -- 172.21.15.176:0/1285500919 wait complete. 2024-03-13T14:50:17.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.738+0000 7fc3b6000700 1 -- 172.21.15.176:0/2228749792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0102c20 msgr2=0x7fc3b0103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.738+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2228749792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0102c20 0x7fc3b0103040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009910 tx=0x7fc3a4009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.739+0000 7fc3b6000700 1 -- 172.21.15.176:0/2228749792 shutdown_connections 2024-03-13T14:50:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.739+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2228749792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3b0103e20 0x7fc3b0104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.739+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2228749792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0102c20 0x7fc3b0103040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.739+0000 7fc3b6000700 1 -- 172.21.15.176:0/2228749792 >> 172.21.15.176:0/2228749792 conn(0x7fc3b00fe1a0 msgr2=0x7fc3b0100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.739+0000 7fc3b6000700 1 -- 172.21.15.176:0/2228749792 shutdown_connections 2024-03-13T14:50:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.739+0000 7fc3b6000700 1 -- 172.21.15.176:0/2228749792 wait complete. 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3b6000700 1 Processor -- start 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3b6000700 1 -- start start 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3b6000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3b0102c20 0x7fc3b01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3b6000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0103e20 0x7fc3b0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3b6000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b0198f30 con 0x7fc3b0103e20 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3b6000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b0199070 con 0x7fc3b0102c20 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0103e20 0x7fc3b0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0103e20 0x7fc3b0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37524/0 (socket says 172.21.15.176:37524) 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.740+0000 7fc3afe00700 1 -- 172.21.15.176:0/2978637201 learned_addr learned my addr 172.21.15.176:0/2978637201 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3b4c00700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3b0102c20 0x7fc3b01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3afe00700 1 -- 172.21.15.176:0/2978637201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3b0102c20 msgr2=0x7fc3b01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3afe00700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3b0102c20 0x7fc3b01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3afe00700 1 -- 172.21.15.176:0/2978637201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a40095f0 con 0x7fc3b0103e20 2024-03-13T14:50:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3afe00700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0103e20 0x7fc3b0198910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c8a0 tx=0x7fc39c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:17.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3ad600700 1 -- 172.21.15.176:0/2978637201 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc39c00cea0 con 0x7fc3b0103e20 2024-03-13T14:50:17.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3ad600700 1 -- 172.21.15.176:0/2978637201 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc39c004830 con 0x7fc3b0103e20 2024-03-13T14:50:17.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.741+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b019db20 con 0x7fc3b0103e20 2024-03-13T14:50:17.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.742+0000 7fc3ad600700 1 -- 172.21.15.176:0/2978637201 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc39c0055b0 con 0x7fc3b0103e20 2024-03-13T14:50:17.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.742+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b019e070 con 0x7fc3b0103e20 2024-03-13T14:50:17.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.743+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b010b780 con 0x7fc3b0103e20 2024-03-13T14:50:17.731 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.744+0000 7fc3ad600700 1 -- 172.21.15.176:0/2978637201 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fc39c007ad0 con 0x7fc3b0103e20 2024-03-13T14:50:17.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.745+0000 7fc3ad600700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc39806dd50 0x7fc398070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:17.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.745+0000 7fc3ad600700 1 -- 172.21.15.176:0/2978637201 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc39c08d7b0 con 0x7fc3b0103e20 2024-03-13T14:50:17.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.745+0000 7fc3b4c00700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc39806dd50 0x7fc398070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:17.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.746+0000 7fc3b4c00700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc39806dd50 0x7fc398070210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4006010 tx=0x7fc3a400b540 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:17.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:17.751+0000 7fc3ad600700 1 -- 172.21.15.176:0/2978637201 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fc39c05aa10 con 0x7fc3b0103e20 2024-03-13T14:50:18.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.134+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc3b010b970 con 0x7fc3b0103e20 2024-03-13T14:50:18.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.135+0000 7fc3ad600700 1 -- 172.21.15.176:0/2978637201 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fc39c05a5a0 con 0x7fc3b0103e20 2024-03-13T14:50:18.123 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:50:18.124 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:50:18.125 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:50:18.125 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:50:18.125 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:50:18.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc39806dd50 msgr2=0x7fc398070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:18.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc39806dd50 0x7fc398070210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4006010 tx=0x7fc3a400b540 comp rx=0 tx=0).stop 2024-03-13T14:50:18.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0103e20 msgr2=0x7fc3b0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:18.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0103e20 0x7fc3b0198910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c8a0 tx=0x7fc39c00cc60 comp rx=0 tx=0).stop 2024-03-13T14:50:18.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 shutdown_connections 2024-03-13T14:50:18.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fc39806dd50 0x7fc398070210 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3b0102c20 0x7fc3b01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 --2- 172.21.15.176:0/2978637201 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3b0103e20 0x7fc3b0198910 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00c8a0 tx=0x7fc39c00cc60 comp rx=0 tx=0).stop 2024-03-13T14:50:18.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 >> 172.21.15.176:0/2978637201 conn(0x7fc3b00fe1a0 msgr2=0x7fc3b0107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:18.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 shutdown_connections 2024-03-13T14:50:18.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.138+0000 7fc3b6000700 1 -- 172.21.15.176:0/2978637201 wait complete. 2024-03-13T14:50:18.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 -- 172.21.15.176:0/1936158042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78101640 msgr2=0x7fec78103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:18.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1936158042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78101640 0x7fec78103a30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fec68009910 tx=0x7fec68009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:18.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 -- 172.21.15.176:0/1936158042 shutdown_connections 2024-03-13T14:50:18.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1936158042 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78103f70 0x7fec78106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1936158042 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78101640 0x7fec78103a30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 -- 172.21.15.176:0/1936158042 >> 172.21.15.176:0/1936158042 conn(0x7fec780fb050 msgr2=0x7fec780fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:18.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 -- 172.21.15.176:0/1936158042 shutdown_connections 2024-03-13T14:50:18.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.315+0000 7fec7cc00700 1 -- 172.21.15.176:0/1936158042 wait complete. 2024-03-13T14:50:18.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.316+0000 7fec7cc00700 1 Processor -- start 2024-03-13T14:50:18.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.316+0000 7fec7cc00700 1 -- start start 2024-03-13T14:50:18.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.317+0000 7fec7cc00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78101640 0x7fec78198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:18.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.317+0000 7fec7cc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78103f70 0x7fec78198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:18.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.317+0000 7fec7cc00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec78199060 con 0x7fec78103f70 2024-03-13T14:50:18.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.317+0000 7fec7cc00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec781991a0 con 0x7fec78101640 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.317+0000 7fec75e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78101640 0x7fec78198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.317+0000 7fec75e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78101640 0x7fec78198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:42546/0 (socket says 172.21.15.176:42546) 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.317+0000 7fec75e00700 1 -- 172.21.15.176:0/1496640351 learned_addr learned my addr 172.21.15.176:0/1496640351 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec75400700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78103f70 0x7fec78198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec75e00700 1 -- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78103f70 msgr2=0x7fec78198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec75e00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78103f70 0x7fec78198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec75e00700 1 -- 172.21.15.176:0/1496640351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec680095f0 con 0x7fec78101640 2024-03-13T14:50:18.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec75e00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78101640 0x7fec78198500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec68005b40 tx=0x7fec6800be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:18.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec6ea00700 1 -- 172.21.15.176:0/1496640351 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec6801d070 con 0x7fec78101640 2024-03-13T14:50:18.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7819dbf0 con 0x7fec78101640 2024-03-13T14:50:18.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec6ea00700 1 -- 172.21.15.176:0/1496640351 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec6800f460 con 0x7fec78101640 2024-03-13T14:50:18.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.318+0000 7fec6ea00700 1 -- 172.21.15.176:0/1496640351 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec68021640 con 0x7fec78101640 2024-03-13T14:50:18.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.319+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7819e0e0 con 0x7fec78101640 2024-03-13T14:50:18.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.320+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec780fcc20 con 0x7fec78101640 2024-03-13T14:50:18.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.321+0000 7fec6ea00700 1 -- 172.21.15.176:0/1496640351 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 30) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fec6800fa90 con 0x7fec78101640 2024-03-13T14:50:18.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.323+0000 7fec6ea00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fec5c06dd50 0x7fec5c070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:18.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.323+0000 7fec6ea00700 1 -- 172.21.15.176:0/1496640351 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fec6808ede0 con 0x7fec78101640 2024-03-13T14:50:18.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.323+0000 7fec75400700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fec5c06dd50 0x7fec5c070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:18.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.324+0000 7fec75400700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fec5c06dd50 0x7fec5c070210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec64005950 tx=0x7fec640058e0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:18.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.331+0000 7fec6ea00700 1 -- 172.21.15.176:0/1496640351 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fec6805c020 con 0x7fec78101640 2024-03-13T14:50:18.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:18 smithi176 ceph-mon[29064]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:18.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:18 smithi176 ceph-mon[29064]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:18.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:18 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2978637201' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:18.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:18 smithi196 ceph-mon[37486]: from='client.24437 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:18.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:18 smithi196 ceph-mon[37486]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:18.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:18 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2978637201' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:18.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.607+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec780611d0 con 0x7fec5c06dd50 2024-03-13T14:50:18.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.611+0000 7fec6ea00700 1 -- 172.21.15.176:0/1496640351 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fec780611d0 con 0x7fec5c06dd50 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "2/2 daemons upgraded", 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mgr daemons", 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:50:18.599 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:50:18.601 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.613+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fec5c06dd50 msgr2=0x7fec5c070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:18.601 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.613+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fec5c06dd50 0x7fec5c070210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fec64005950 tx=0x7fec640058e0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.601 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.613+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78101640 msgr2=0x7fec78198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:18.601 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.613+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78101640 0x7fec78198500 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec68005b40 tx=0x7fec6800be20 comp rx=0 tx=0).stop 2024-03-13T14:50:18.602 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.614+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 shutdown_connections 2024-03-13T14:50:18.602 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.614+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fec5c06dd50 0x7fec5c070210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.602 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.614+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78101640 0x7fec78198500 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.603 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.614+0000 7fec7cc00700 1 --2- 172.21.15.176:0/1496640351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fec78103f70 0x7fec78198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:18.603 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.614+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 >> 172.21.15.176:0/1496640351 conn(0x7fec780fb050 msgr2=0x7fec78100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:18.603 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.614+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 shutdown_connections 2024-03-13T14:50:18.603 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:18.614+0000 7fec7cc00700 1 -- 172.21.15.176:0/1496640351 wait complete. 2024-03-13T14:50:19.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:19 smithi196 ceph-mon[37486]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:19.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:19 smithi196 ceph-mon[37486]: pgmap v28: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 29 MiB/s rd, 65 MiB/s wr, 224 op/s 2024-03-13T14:50:19.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:19 smithi176 ceph-mon[29064]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:19.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:19 smithi176 ceph-mon[29064]: pgmap v28: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 29 MiB/s rd, 65 MiB/s wr, 224 op/s 2024-03-13T14:50:20.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:20 smithi196 ceph-mon[37486]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:20.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:20 smithi176 ceph-mon[29064]: from='client.24447 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:21.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:21 smithi196 ceph-mon[37486]: pgmap v29: 65 pgs: 65 active+clean; 3.0 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 29 MiB/s rd, 65 MiB/s wr, 287 op/s 2024-03-13T14:50:21.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:21 smithi176 ceph-mon[29064]: pgmap v29: 65 pgs: 65 active+clean; 3.0 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 29 MiB/s rd, 65 MiB/s wr, 287 op/s 2024-03-13T14:50:22.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:22 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:22.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:22 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:22.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:22 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:50:22.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:22 smithi196 ceph-mon[37486]: pgmap v30: 65 pgs: 65 active+clean; 3.0 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 120 op/s 2024-03-13T14:50:22.558 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:22 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:22.558 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:22 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:22.558 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:22 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:50:22.558 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:22 smithi176 ceph-mon[29064]: pgmap v30: 65 pgs: 65 active+clean; 3.0 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 120 op/s 2024-03-13T14:50:23.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:23 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:23.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:23 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:23 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:23 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:24.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:24 smithi196 ceph-mon[37486]: pgmap v31: 65 pgs: 65 active+clean; 2.9 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 125 op/s 2024-03-13T14:50:24.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:24 smithi176 ceph-mon[29064]: pgmap v31: 65 pgs: 65 active+clean; 2.9 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 125 op/s 2024-03-13T14:50:27.206 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:27 smithi176 ceph-mon[29064]: pgmap v32: 65 pgs: 65 active+clean; 2.2 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-03-13T14:50:27.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:27 smithi196 ceph-mon[37486]: pgmap v32: 65 pgs: 65 active+clean; 2.2 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-03-13T14:50:28.577 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:28.577 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:28.577 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:28 smithi196 ceph-mon[37486]: pgmap v33: 65 pgs: 65 active+clean; 2.2 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 886 KiB/s rd, 851 KiB/s wr, 117 op/s 2024-03-13T14:50:28.577 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:28.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:28.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:28.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:28 smithi176 ceph-mon[29064]: pgmap v33: 65 pgs: 65 active+clean; 2.2 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 886 KiB/s rd, 851 KiB/s wr, 117 op/s 2024-03-13T14:50:28.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:30.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:30 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:30 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:30 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:30 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:30 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:30 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:30 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:30 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:30.498 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-13T14:50:30.498 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-03-13T14:50:31.335 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:31 smithi196 ceph-mon[37486]: pgmap v34: 65 pgs: 65 active+clean; 1.4 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-03-13T14:50:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:31 smithi176 ceph-mon[29064]: pgmap v34: 65 pgs: 65 active+clean; 1.4 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-03-13T14:50:33.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:33 smithi176 ceph-mon[29064]: pgmap v35: 65 pgs: 65 active+clean; 1.4 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 803 KiB/s rd, 845 KiB/s wr, 108 op/s 2024-03-13T14:50:33.414 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:33 smithi196 ceph-mon[37486]: pgmap v35: 65 pgs: 65 active+clean; 1.4 GiB data, 4.9 GiB used, 532 GiB / 536 GiB avail; 803 KiB/s rd, 845 KiB/s wr, 108 op/s 2024-03-13T14:50:34.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:34 smithi176 ceph-mon[29064]: Standby manager daemon smithi196.vdktbk restarted 2024-03-13T14:50:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:34 smithi176 ceph-mon[29064]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:50:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:34 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:50:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:34 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:50:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:34 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:50:34.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:34 smithi176 ceph-mon[29064]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:50:34.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:34 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk restarted 2024-03-13T14:50:34.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:34 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:50:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:34 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:50:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:34 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:50:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:34 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:50:34.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:34 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/3859524675' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:50:35.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:35 smithi176 ceph-mon[29064]: pgmap v36: 65 pgs: 65 active+clean; 1.3 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 803 KiB/s rd, 845 KiB/s wr, 113 op/s 2024-03-13T14:50:35.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:35 smithi176 ceph-mon[29064]: mgrmap e31: smithi176.tzfdwd(active, since 66s), standbys: smithi196.vdktbk 2024-03-13T14:50:35.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:35 smithi196 ceph-mon[37486]: pgmap v36: 65 pgs: 65 active+clean; 1.3 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 803 KiB/s rd, 845 KiB/s wr, 113 op/s 2024-03-13T14:50:35.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:35 smithi196 ceph-mon[37486]: mgrmap e31: smithi176.tzfdwd(active, since 66s), standbys: smithi196.vdktbk 2024-03-13T14:50:37.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:37 smithi176 ceph-mon[29064]: pgmap v37: 65 pgs: 65 active+clean; 681 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-03-13T14:50:37.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:37 smithi196 ceph-mon[37486]: pgmap v37: 65 pgs: 65 active+clean; 681 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-03-13T14:50:37.621 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-13T14:50:37.621 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-13T14:50:39.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:39 smithi196 ceph-mon[37486]: pgmap v38: 65 pgs: 65 active+clean; 681 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 766 KiB/s rd, 816 KiB/s wr, 106 op/s 2024-03-13T14:50:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:39 smithi176 ceph-mon[29064]: pgmap v38: 65 pgs: 65 active+clean; 681 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 766 KiB/s rd, 816 KiB/s wr, 106 op/s 2024-03-13T14:50:41.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:41 smithi196 ceph-mon[37486]: pgmap v39: 65 pgs: 65 active+clean; 367 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-03-13T14:50:41.432 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:41 smithi176 ceph-mon[29064]: pgmap v39: 65 pgs: 65 active+clean; 367 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 191 op/s 2024-03-13T14:50:43.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:43 smithi196 ceph-mon[37486]: pgmap v40: 65 pgs: 65 active+clean; 367 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 730 KiB/s rd, 730 KiB/s wr, 138 op/s 2024-03-13T14:50:43.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:43.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:43 smithi176 ceph-mon[29064]: pgmap v40: 65 pgs: 65 active+clean; 367 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 730 KiB/s rd, 730 KiB/s wr, 138 op/s 2024-03-13T14:50:43.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: pgmap v41: 65 pgs: 65 active+clean; 320 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 730 KiB/s rd, 730 KiB/s wr, 156 op/s 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all mgr 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.tzfdwd"}]: dispatch 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.tzfdwd"}]': finished 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi196.vdktbk"}]: dispatch 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi196.vdktbk"}]': finished 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all rgw 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all rbd-mirror 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.597 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-13T14:50:44.858 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: pgmap v41: 65 pgs: 65 active+clean; 320 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 730 KiB/s rd, 730 KiB/s wr, 156 op/s 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all mgr 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.tzfdwd"}]: dispatch 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi176.tzfdwd"}]': finished 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi196.vdktbk"}]: dispatch 2024-03-13T14:50:44.859 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi196.vdktbk"}]': finished 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all rgw 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all rbd-mirror 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all cephfs-mirror 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all iscsi 2024-03-13T14:50:44.860 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.861 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.861 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all nfs 2024-03-13T14:50:44.861 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.861 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.861 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all nvmeof 2024-03-13T14:50:44.861 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:44 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all iscsi 2024-03-13T14:50:44.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all nfs 2024-03-13T14:50:44.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:44.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:44.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all nvmeof 2024-03-13T14:50:44.932 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:44 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:47.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:47 smithi176 ceph-mon[29064]: pgmap v42: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1005 KiB/s rd, 1.0 MiB/s wr, 210 op/s 2024-03-13T14:50:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:47 smithi196 ceph-mon[37486]: pgmap v42: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1005 KiB/s rd, 1.0 MiB/s wr, 210 op/s 2024-03-13T14:50:48.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:48 smithi176 ceph-mon[29064]: Upgrade: Updating node-exporter.smithi176 (1/2) 2024-03-13T14:50:48.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:48 smithi176 ceph-mon[29064]: Deploying daemon node-exporter.smithi176 on smithi176 2024-03-13T14:50:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:48 smithi196 ceph-mon[37486]: Upgrade: Updating node-exporter.smithi176 (1/2) 2024-03-13T14:50:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:48 smithi196 ceph-mon[37486]: Deploying daemon node-exporter.smithi176 on smithi176 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 -- 172.21.15.176:0/3729808026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ff490 msgr2=0x7fd3300ff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 --2- 172.21.15.176:0/3729808026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ff490 0x7fd3300ff8b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd324009910 tx=0x7fd324009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 -- 172.21.15.176:0/3729808026 shutdown_connections 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 --2- 172.21.15.176:0/3729808026 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3300ffdf0 0x7fd330100270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 --2- 172.21.15.176:0/3729808026 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ff490 0x7fd3300ff8b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 -- 172.21.15.176:0/3729808026 >> 172.21.15.176:0/3729808026 conn(0x7fd3300fb030 msgr2=0x7fd3300fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 -- 172.21.15.176:0/3729808026 shutdown_connections 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.828+0000 7fd338400700 1 -- 172.21.15.176:0/3729808026 wait complete. 2024-03-13T14:50:48.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.829+0000 7fd338400700 1 Processor -- start 2024-03-13T14:50:48.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.829+0000 7fd338400700 1 -- start start 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd338400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3300ff490 0x7fd330198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd338400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ffdf0 0x7fd330198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd338400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd330199080 con 0x7fd3300ffdf0 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd338400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3301991c0 con 0x7fd3300ff490 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd335000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ffdf0 0x7fd330198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd335000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ffdf0 0x7fd330198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60030/0 (socket says 172.21.15.176:60030) 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd335000700 1 -- 172.21.15.176:0/1055390544 learned_addr learned my addr 172.21.15.176:0/1055390544 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.830+0000 7fd335a00700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3300ff490 0x7fd330198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:48.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd335000700 1 -- 172.21.15.176:0/1055390544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3300ff490 msgr2=0x7fd330198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd335000700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3300ff490 0x7fd330198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd335000700 1 -- 172.21.15.176:0/1055390544 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240095f0 con 0x7fd3300ffdf0 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd335000700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ffdf0 0x7fd330198a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00e920 tx=0x7fd31c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd32e000700 1 -- 172.21.15.176:0/1055390544 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd31c00c9d0 con 0x7fd3300ffdf0 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33019dc70 con 0x7fd3300ffdf0 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd32e000700 1 -- 172.21.15.176:0/1055390544 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd31c010430 con 0x7fd3300ffdf0 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd32e000700 1 -- 172.21.15.176:0/1055390544 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd31c018650 con 0x7fd3300ffdf0 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.831+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33019e1c0 con 0x7fd3300ffdf0 2024-03-13T14:50:48.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.833+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33010b910 con 0x7fd3300ffdf0 2024-03-13T14:50:48.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.834+0000 7fd32e000700 1 -- 172.21.15.176:0/1055390544 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fd31c0105a0 con 0x7fd3300ffdf0 2024-03-13T14:50:48.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.835+0000 7fd32e000700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd31806dd50 0x7fd318070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:48.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.835+0000 7fd32e000700 1 -- 172.21.15.176:0/1055390544 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd31c014070 con 0x7fd3300ffdf0 2024-03-13T14:50:48.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.835+0000 7fd335a00700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd31806dd50 0x7fd318070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:48.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.836+0000 7fd335a00700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd31806dd50 0x7fd318070210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd324009fd0 tx=0x7fd324005fd0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:48.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:48.840+0000 7fd32e000700 1 -- 172.21.15.176:0/1055390544 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fd31c052f30 con 0x7fd3300ffdf0 2024-03-13T14:50:49.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.112+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3301091e0 con 0x7fd31806dd50 2024-03-13T14:50:49.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.115+0000 7fd32e000700 1 -- 172.21.15.176:0/1055390544 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fd3301091e0 con 0x7fd31806dd50 2024-03-13T14:50:49.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.118+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd31806dd50 msgr2=0x7fd318070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.118+0000 7fd338400700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd31806dd50 0x7fd318070210 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd324009fd0 tx=0x7fd324005fd0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.118+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ffdf0 msgr2=0x7fd330198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.106 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.118+0000 7fd338400700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ffdf0 0x7fd330198a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00e920 tx=0x7fd31c00ec30 comp rx=0 tx=0).stop 2024-03-13T14:50:49.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.120+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 shutdown_connections 2024-03-13T14:50:49.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.120+0000 7fd338400700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd31806dd50 0x7fd318070210 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.120+0000 7fd338400700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3300ff490 0x7fd330198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.120+0000 7fd338400700 1 --2- 172.21.15.176:0/1055390544 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd3300ffdf0 0x7fd330198a60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.120+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 >> 172.21.15.176:0/1055390544 conn(0x7fd3300fb030 msgr2=0x7fd330107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:49.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.122+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 shutdown_connections 2024-03-13T14:50:49.112 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.124+0000 7fd338400700 1 -- 172.21.15.176:0/1055390544 wait complete. 2024-03-13T14:50:49.129 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 -- 172.21.15.176:0/2646243529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c100a00 msgr2=0x7f667c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 --2- 172.21.15.176:0/2646243529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c100a00 0x7f667c100e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f666c009910 tx=0x7f666c009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 -- 172.21.15.176:0/2646243529 shutdown_connections 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 --2- 172.21.15.176:0/2646243529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f667c101c00 0x7f667c102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 --2- 172.21.15.176:0/2646243529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c100a00 0x7f667c100e20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 -- 172.21.15.176:0/2646243529 >> 172.21.15.176:0/2646243529 conn(0x7f667c0fbf60 msgr2=0x7f667c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 -- 172.21.15.176:0/2646243529 shutdown_connections 2024-03-13T14:50:49.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.291+0000 7f667be00700 1 -- 172.21.15.176:0/2646243529 wait complete. 2024-03-13T14:50:49.280 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.293+0000 7f667be00700 1 Processor -- start 2024-03-13T14:50:49.280 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.293+0000 7f667be00700 1 -- start start 2024-03-13T14:50:49.281 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.293+0000 7f667be00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f667c100a00 0x7f667c074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:49.281 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.293+0000 7f667be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c101c00 0x7f667c073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:49.281 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.293+0000 7f667be00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0736c0 con 0x7f667c101c00 2024-03-13T14:50:49.281 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.293+0000 7f667be00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c073800 con 0x7f667c100a00 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f6673e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c101c00 0x7f667c073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f667aa00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f667c100a00 0x7f667c074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f6673e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c101c00 0x7f667c073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60056/0 (socket says 172.21.15.176:60056) 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f6673e00700 1 -- 172.21.15.176:0/2152824282 learned_addr learned my addr 172.21.15.176:0/2152824282 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f667aa00700 1 -- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c101c00 msgr2=0x7f667c073180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f667aa00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c101c00 0x7f667c073180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f667aa00700 1 -- 172.21.15.176:0/2152824282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0095f0 con 0x7f667c100a00 2024-03-13T14:50:49.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.294+0000 7f6673e00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c101c00 0x7f667c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-13T14:50:49.283 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.295+0000 7f667aa00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f667c100a00 0x7f667c074b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f666c009910 tx=0x7f666c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:49.283 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.295+0000 7f6673400700 1 -- 172.21.15.176:0/2152824282 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c01d070 con 0x7f667c100a00 2024-03-13T14:50:49.283 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.295+0000 7f6673400700 1 -- 172.21.15.176:0/2152824282 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f666c005dc0 con 0x7f667c100a00 2024-03-13T14:50:49.283 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.295+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c073a80 con 0x7f667c100a00 2024-03-13T14:50:49.283 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.295+0000 7f6673400700 1 -- 172.21.15.176:0/2152824282 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c017400 con 0x7f667c100a00 2024-03-13T14:50:49.284 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.295+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c073f70 con 0x7f667c100a00 2024-03-13T14:50:49.284 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.297+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c109560 con 0x7f667c100a00 2024-03-13T14:50:49.285 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.298+0000 7f6673400700 1 -- 172.21.15.176:0/2152824282 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f666c003810 con 0x7f667c100a00 2024-03-13T14:50:49.287 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.299+0000 7f6673400700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f666006dd50 0x7f6660070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:49.287 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.299+0000 7f6673400700 1 -- 172.21.15.176:0/2152824282 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f666c08ee90 con 0x7f667c100a00 2024-03-13T14:50:49.287 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.299+0000 7f6673e00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f666006dd50 0x7f6660070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:49.287 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.300+0000 7f6673e00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f666006dd50 0x7f6660070210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6664011550 tx=0x7f6664009260 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:49.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.314+0000 7f6673400700 1 -- 172.21.15.176:0/2152824282 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f666c05c050 con 0x7f667c100a00 2024-03-13T14:50:49.359 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:49 smithi176 ceph-mon[29064]: pgmap v43: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 594 KiB/s rd, 648 KiB/s wr, 162 op/s 2024-03-13T14:50:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:49 smithi196 ceph-mon[37486]: pgmap v43: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 594 KiB/s rd, 648 KiB/s wr, 162 op/s 2024-03-13T14:50:49.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.622+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f667c106550 con 0x7f666006dd50 2024-03-13T14:50:49.612 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.625+0000 7f6673400700 1 -- 172.21.15.176:0/2152824282 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f667c106550 con 0x7f666006dd50 2024-03-13T14:50:49.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f666006dd50 msgr2=0x7f6660070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f666006dd50 0x7f6660070210 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6664011550 tx=0x7f6664009260 comp rx=0 tx=0).stop 2024-03-13T14:50:49.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f667c100a00 msgr2=0x7f667c074b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f667c100a00 0x7f667c074b30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f666c009910 tx=0x7f666c00f690 comp rx=0 tx=0).stop 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 shutdown_connections 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f666006dd50 0x7f6660070210 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f667c100a00 0x7f667c074b30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 --2- 172.21.15.176:0/2152824282 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f667c101c00 0x7f667c073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 >> 172.21.15.176:0/2152824282 conn(0x7f667c0fbf60 msgr2=0x7f667c104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 shutdown_connections 2024-03-13T14:50:49.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.628+0000 7f667be00700 1 -- 172.21.15.176:0/2152824282 wait complete. 2024-03-13T14:50:49.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 -- 172.21.15.176:0/2561183934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 msgr2=0x7f6898104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 --2- 172.21.15.176:0/2561183934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 0x7f6898104230 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6884009910 tx=0x7f6884009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:49.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 -- 172.21.15.176:0/2561183934 shutdown_connections 2024-03-13T14:50:49.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 --2- 172.21.15.176:0/2561183934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 0x7f6898104230 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 --2- 172.21.15.176:0/2561183934 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68980ff430 0x7f6898101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 -- 172.21.15.176:0/2561183934 >> 172.21.15.176:0/2561183934 conn(0x7f68980fb030 msgr2=0x7f68980fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:49.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 -- 172.21.15.176:0/2561183934 shutdown_connections 2024-03-13T14:50:49.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.836+0000 7f68a0200700 1 -- 172.21.15.176:0/2561183934 wait complete. 2024-03-13T14:50:49.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.837+0000 7f68a0200700 1 Processor -- start 2024-03-13T14:50:49.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.837+0000 7f68a0200700 1 -- start start 2024-03-13T14:50:49.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.838+0000 7f68a0200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68980ff430 0x7f689819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.838+0000 7f68a0200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 0x7f689819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.838+0000 7f68a0200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689819d450 con 0x7f68980ff430 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.838+0000 7f68a0200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689819d590 con 0x7f6898101da0 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.839+0000 7f689ce00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 0x7f689819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.839+0000 7f689ce00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 0x7f689819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:54016/0 (socket says 172.21.15.176:54016) 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.839+0000 7f689ce00700 1 -- 172.21.15.176:0/4200643529 learned_addr learned my addr 172.21.15.176:0/4200643529 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.839+0000 7f689d800700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68980ff430 0x7f689819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.840+0000 7f689d800700 1 -- 172.21.15.176:0/4200643529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 msgr2=0x7f689819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.840+0000 7f689d800700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 0x7f689819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:49.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.840+0000 7f689d800700 1 -- 172.21.15.176:0/4200643529 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68840095f0 con 0x7f68980ff430 2024-03-13T14:50:49.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.841+0000 7f689d800700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68980ff430 0x7f689819c8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d6d0 tx=0x7f688c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:49.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.842+0000 7f6896000700 1 -- 172.21.15.176:0/4200643529 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c0041f0 con 0x7f68980ff430 2024-03-13T14:50:49.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.842+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68981a1fe0 con 0x7f68980ff430 2024-03-13T14:50:49.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.842+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68981a2500 con 0x7f68980ff430 2024-03-13T14:50:49.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.843+0000 7f6896000700 1 -- 172.21.15.176:0/4200643529 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f688c004ce0 con 0x7f68980ff430 2024-03-13T14:50:49.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.843+0000 7f6896000700 1 -- 172.21.15.176:0/4200643529 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f688c005710 con 0x7f68980ff430 2024-03-13T14:50:49.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.844+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6898196b10 con 0x7f68980ff430 2024-03-13T14:50:49.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.845+0000 7f6896000700 1 -- 172.21.15.176:0/4200643529 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f688c00f690 con 0x7f68980ff430 2024-03-13T14:50:49.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.846+0000 7f6896000700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f688006dd50 0x7f6880070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:49.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.846+0000 7f6896000700 1 -- 172.21.15.176:0/4200643529 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f688c08d7e0 con 0x7f68980ff430 2024-03-13T14:50:49.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.847+0000 7f689ce00700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f688006dd50 0x7f6880070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:49.836 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.848+0000 7f689ce00700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f688006dd50 0x7f6880070210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f688400b5c0 tx=0x7f6884005a90 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:49.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:49.852+0000 7f6896000700 1 -- 172.21.15.176:0/4200643529 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f688c05aac0 con 0x7f68980ff430 2024-03-13T14:50:50.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.120+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f68980611d0 con 0x7f688006dd50 2024-03-13T14:50:50.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.133+0000 7f6896000700 1 -- 172.21.15.176:0/4200643529 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f68980611d0 con 0x7f688006dd50 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (10m) 21s ago 12m 13.9M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (12m) 21s ago 12m 7852k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (11m) 22s ago 11m 7772k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (12m) 21s ago 12m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (11m) 22s ago 11m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (10m) 21s ago 11m 77.5M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:50:50.121 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (4m) 21s ago 4m 141M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (5m) 21s ago 5m 14.2M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (5m) 22s ago 5m 219M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (4m) 22s ago 4m 14.3M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (99s) 21s ago 14m 492M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (29s) 22s ago 10m 93.0M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (14m) 21s ago 14m 53.8M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (10m) 22s ago 10m 42.3M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (12m) 21s ago 12m 7898k - 1.5.0 0da6a335fe13 ec161d1825d5 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (10m) 22s ago 10m 7864k - 1.5.0 0da6a335fe13 30d9c0cf473f 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (9m) 21s ago 9m 198M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (9m) 21s ago 9m 207M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (8m) 21s ago 8m 177M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (8m) 22s ago 8m 240M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:50:50.122 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (7m) 22s ago 7m 217M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:50:50.123 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (6m) 22s ago 6m 206M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:50:50.123 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (37s) 21s ago 11m 46.9M - 2.43.0 a07b618ecd1d b4eb817c4249 2024-03-13T14:50:50.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.136+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f688006dd50 msgr2=0x7f6880070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.136+0000 7f68a0200700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f688006dd50 0x7f6880070210 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f688400b5c0 tx=0x7f6884005a90 comp rx=0 tx=0).stop 2024-03-13T14:50:50.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.136+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68980ff430 msgr2=0x7f689819c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.136+0000 7f68a0200700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68980ff430 0x7f689819c8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f688c00d6d0 tx=0x7f688c00da90 comp rx=0 tx=0).stop 2024-03-13T14:50:50.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.137+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 shutdown_connections 2024-03-13T14:50:50.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.137+0000 7f68a0200700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f688006dd50 0x7f6880070210 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.137+0000 7f68a0200700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68980ff430 0x7f689819c8f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.137+0000 7f68a0200700 1 --2- 172.21.15.176:0/4200643529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6898101da0 0x7f689819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.137+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 >> 172.21.15.176:0/4200643529 conn(0x7f68980fb030 msgr2=0x7f68980fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:50.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.137+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 shutdown_connections 2024-03-13T14:50:50.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.138+0000 7f68a0200700 1 -- 172.21.15.176:0/4200643529 wait complete. 2024-03-13T14:50:50.288 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 -- 172.21.15.176:0/2566067436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 msgr2=0x7fea10107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.288 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 --2- 172.21.15.176:0/2566067436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea10107620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fea00009910 tx=0x7fea00009c20 comp rx=0 tx=0).stop 2024-03-13T14:50:50.288 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 -- 172.21.15.176:0/2566067436 shutdown_connections 2024-03-13T14:50:50.288 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 --2- 172.21.15.176:0/2566067436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea10108400 0x7fea10108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.288 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 --2- 172.21.15.176:0/2566067436 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea10107620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.289 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 -- 172.21.15.176:0/2566067436 >> 172.21.15.176:0/2566067436 conn(0x7fea100761b0 msgr2=0x7fea10078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:50.289 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 -- 172.21.15.176:0/2566067436 shutdown_connections 2024-03-13T14:50:50.289 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.300+0000 7fea15a00700 1 -- 172.21.15.176:0/2566067436 wait complete. 2024-03-13T14:50:50.289 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.301+0000 7fea15a00700 1 Processor -- start 2024-03-13T14:50:50.289 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea15a00700 1 -- start start 2024-03-13T14:50:50.290 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea15a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea1019c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:50.290 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea15a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea10108400 0x7fea1019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:50.290 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea15a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1019d510 con 0x7fea10107200 2024-03-13T14:50:50.290 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea15a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1019d650 con 0x7fea10108400 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea0ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea1019c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea0ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea1019c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60090/0 (socket says 172.21.15.176:60090) 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.302+0000 7fea0ea00700 1 -- 172.21.15.176:0/2522194417 learned_addr learned my addr 172.21.15.176:0/2522194417 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.303+0000 7fea0e000700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea10108400 0x7fea1019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.303+0000 7fea0ea00700 1 -- 172.21.15.176:0/2522194417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea10108400 msgr2=0x7fea1019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.303+0000 7fea0ea00700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea10108400 0x7fea1019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.303+0000 7fea0ea00700 1 -- 172.21.15.176:0/2522194417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000095f0 con 0x7fea10107200 2024-03-13T14:50:50.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.303+0000 7fea0ea00700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea1019c9b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fea0000bda0 tx=0x7fea0000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:50.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.304+0000 7fea07400700 1 -- 172.21.15.176:0/2522194417 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea0001d070 con 0x7fea10107200 2024-03-13T14:50:50.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.304+0000 7fea07400700 1 -- 172.21.15.176:0/2522194417 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fea0000f460 con 0x7fea10107200 2024-03-13T14:50:50.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.304+0000 7fea07400700 1 -- 172.21.15.176:0/2522194417 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea00021620 con 0x7fea10107200 2024-03-13T14:50:50.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.304+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea101a20a0 con 0x7fea10107200 2024-03-13T14:50:50.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.304+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea101a2590 con 0x7fea10107200 2024-03-13T14:50:50.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.305+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1010fd60 con 0x7fea10107200 2024-03-13T14:50:50.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.306+0000 7fea07400700 1 -- 172.21.15.176:0/2522194417 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7fea0000f5d0 con 0x7fea10107200 2024-03-13T14:50:50.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.307+0000 7fea07400700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fe9f406dd50 0x7fe9f4070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:50.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.307+0000 7fea07400700 1 -- 172.21.15.176:0/2522194417 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fea0008ed30 con 0x7fea10107200 2024-03-13T14:50:50.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.308+0000 7fea0e000700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fe9f406dd50 0x7fe9f4070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:50.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.309+0000 7fea0e000700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fe9f406dd50 0x7fe9f4070210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc005950 tx=0x7fe9fc0058e0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:50.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.312+0000 7fea07400700 1 -- 172.21.15.176:0/2522194417 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fea0005bf90 con 0x7fea10107200 2024-03-13T14:50:50.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:50 smithi176 ceph-mon[29064]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:50.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:50 smithi196 ceph-mon[37486]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:50.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.666+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fea101a2c20 con 0x7fea10107200 2024-03-13T14:50:50.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.667+0000 7fea07400700 1 -- 172.21.15.176:0/2522194417 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fea0005bb20 con 0x7fea10107200 2024-03-13T14:50:50.655 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:50:50.655 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:50:50.656 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:50:50.657 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.670+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fe9f406dd50 msgr2=0x7fe9f4070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.670+0000 7fea15a00700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fe9f406dd50 0x7fe9f4070210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc005950 tx=0x7fe9fc0058e0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 msgr2=0x7fea1019c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea1019c9b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fea0000bda0 tx=0x7fea0000be80 comp rx=0 tx=0).stop 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 shutdown_connections 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fe9f406dd50 0x7fe9f4070210 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fea10107200 0x7fea1019c9b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 --2- 172.21.15.176:0/2522194417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea10108400 0x7fea1019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 >> 172.21.15.176:0/2522194417 conn(0x7fea100761b0 msgr2=0x7fea1010b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 shutdown_connections 2024-03-13T14:50:50.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.671+0000 7fea15a00700 1 -- 172.21.15.176:0/2522194417 wait complete. 2024-03-13T14:50:50.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.855+0000 7f81a7200700 1 -- 172.21.15.176:0/1542438444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 msgr2=0x7f81a010a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.855+0000 7f81a7200700 1 --2- 172.21.15.176:0/1542438444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 0x7f81a010a7d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f818c0098a0 tx=0x7f818c009bb0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.855+0000 7f81a7200700 1 -- 172.21.15.176:0/1542438444 shutdown_connections 2024-03-13T14:50:50.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.855+0000 7f81a7200700 1 --2- 172.21.15.176:0/1542438444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 0x7f81a010a7d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.855+0000 7f81a7200700 1 --2- 172.21.15.176:0/1542438444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0072b80 0x7f81a0109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.855+0000 7f81a7200700 1 -- 172.21.15.176:0/1542438444 >> 172.21.15.176:0/1542438444 conn(0x7f81a006d9f0 msgr2=0x7f81a006fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:50.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.855+0000 7f81a7200700 1 -- 172.21.15.176:0/1542438444 shutdown_connections 2024-03-13T14:50:50.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.856+0000 7f81a7200700 1 -- 172.21.15.176:0/1542438444 wait complete. 2024-03-13T14:50:50.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.857+0000 7f81a7200700 1 Processor -- start 2024-03-13T14:50:50.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.857+0000 7f81a7200700 1 -- start start 2024-03-13T14:50:50.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.858+0000 7f81a7200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0072b80 0x7f81a01a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:50.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.858+0000 7f81a7200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 0x7f81a01a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:50.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.858+0000 7f81a7200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01a5cf0 con 0x7f81a010a370 2024-03-13T14:50:50.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.858+0000 7f81a7200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01a5e30 con 0x7f81a0072b80 2024-03-13T14:50:50.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.859+0000 7f819f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 0x7f81a01a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:50.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.860+0000 7f819fe00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0072b80 0x7f81a01a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:50.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.860+0000 7f819fe00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0072b80 0x7f81a01a5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:54060/0 (socket says 172.21.15.176:54060) 2024-03-13T14:50:50.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.860+0000 7f819fe00700 1 -- 172.21.15.176:0/2529327181 learned_addr learned my addr 172.21.15.176:0/2529327181 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:50:50.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.860+0000 7f819f400700 1 -- 172.21.15.176:0/2529327181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0072b80 msgr2=0x7f81a01a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:50.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.860+0000 7f819f400700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0072b80 0x7f81a01a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:50.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.860+0000 7f819f400700 1 -- 172.21.15.176:0/2529327181 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c009520 con 0x7f81a010a370 2024-03-13T14:50:50.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.861+0000 7f819f400700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 0x7f81a01a56d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f818c009230 tx=0x7f818c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:50.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.861+0000 7f819cc00700 1 -- 172.21.15.176:0/2529327181 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818c01c070 con 0x7f81a010a370 2024-03-13T14:50:50.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.862+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a01aa880 con 0x7f81a010a370 2024-03-13T14:50:50.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.862+0000 7f819cc00700 1 -- 172.21.15.176:0/2529327181 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f818c00fe10 con 0x7f81a010a370 2024-03-13T14:50:50.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.867+0000 7f819cc00700 1 -- 172.21.15.176:0/2529327181 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f818c0173f0 con 0x7f81a010a370 2024-03-13T14:50:50.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.867+0000 7f819cc00700 1 -- 172.21.15.176:0/2529327181 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 31) v1 ==== 91798+0+0 (secure 0 0 0) 0x7f818c020070 con 0x7f81a010a370 2024-03-13T14:50:50.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.868+0000 7f819cc00700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f818406dd50 0x7f8184070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:50:50.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.869+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a01aad30 con 0x7f81a010a370 2024-03-13T14:50:50.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.869+0000 7f819cc00700 1 -- 172.21.15.176:0/2529327181 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f818c05d6e0 con 0x7f81a010a370 2024-03-13T14:50:50.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.870+0000 7f819fe00700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f818406dd50 0x7f8184070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:50:50.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.871+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a01101e0 con 0x7f81a010a370 2024-03-13T14:50:50.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.871+0000 7f819fe00700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f818406dd50 0x7f8184070210 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81980062a0 tx=0x7f819800b040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:50:50.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:50.880+0000 7f819cc00700 1 -- 172.21.15.176:0/2529327181 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f818c027020 con 0x7f81a010a370 2024-03-13T14:50:51.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.163+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81a00611d0 con 0x7f818406dd50 2024-03-13T14:50:51.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.164+0000 7f819cc00700 1 -- 172.21.15.176:0/2529327181 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f81a00611d0 con 0x7f818406dd50 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "2/2 daemons upgraded", 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading node-exporter daemons", 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:50:51.153 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:50:51.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.167+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f818406dd50 msgr2=0x7f8184070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:51.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.167+0000 7f81a7200700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f818406dd50 0x7f8184070210 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81980062a0 tx=0x7f819800b040 comp rx=0 tx=0).stop 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.167+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 msgr2=0x7f81a01a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.167+0000 7f81a7200700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 0x7f81a01a56d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f818c009230 tx=0x7f818c00f740 comp rx=0 tx=0).stop 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.168+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 shutdown_connections 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.168+0000 7f81a7200700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f818406dd50 0x7f8184070210 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.168+0000 7f81a7200700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0072b80 0x7f81a01a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.168+0000 7f81a7200700 1 --2- 172.21.15.176:0/2529327181 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f81a010a370 0x7f81a01a56d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.168+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 >> 172.21.15.176:0/2529327181 conn(0x7f81a006d9f0 msgr2=0x7f81a006e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.168+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 shutdown_connections 2024-03-13T14:50:51.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:50:51.168+0000 7f81a7200700 1 -- 172.21.15.176:0/2529327181 wait complete. 2024-03-13T14:50:51.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:51 smithi176 ceph-mon[29064]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:51.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:51 smithi176 ceph-mon[29064]: pgmap v44: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 270 op/s 2024-03-13T14:50:51.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:51 smithi176 ceph-mon[29064]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:51.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:51 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2522194417' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:51.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:51 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:51.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:51 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:51 smithi196 ceph-mon[37486]: from='client.24459 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:51 smithi196 ceph-mon[37486]: pgmap v44: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 270 op/s 2024-03-13T14:50:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:51 smithi196 ceph-mon[37486]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:51 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2522194417' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:50:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:51 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:51 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:52.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:52 smithi196 ceph-mon[37486]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:52.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:52 smithi176 ceph-mon[29064]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:50:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:53 smithi196 ceph-mon[37486]: pgmap v45: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 717 KiB/s rd, 726 KiB/s wr, 185 op/s 2024-03-13T14:50:53.497 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:53 smithi176 ceph-mon[29064]: pgmap v45: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 717 KiB/s rd, 726 KiB/s wr, 185 op/s 2024-03-13T14:50:55.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:55 smithi196 ceph-mon[37486]: Upgrade: Updating node-exporter.smithi196 (2/2) 2024-03-13T14:50:55.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:55 smithi196 ceph-mon[37486]: Deploying daemon node-exporter.smithi196 on smithi196 2024-03-13T14:50:55.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:55 smithi196 ceph-mon[37486]: pgmap v46: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 717 KiB/s rd, 726 KiB/s wr, 201 op/s 2024-03-13T14:50:55.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:55 smithi176 ceph-mon[29064]: Upgrade: Updating node-exporter.smithi196 (2/2) 2024-03-13T14:50:55.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:55 smithi176 ceph-mon[29064]: Deploying daemon node-exporter.smithi196 on smithi196 2024-03-13T14:50:55.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:55 smithi176 ceph-mon[29064]: pgmap v46: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 717 KiB/s rd, 726 KiB/s wr, 201 op/s 2024-03-13T14:50:57.418 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:57 smithi196 ceph-mon[37486]: pgmap v47: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 266 op/s 2024-03-13T14:50:57.564 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:57 smithi176 ceph-mon[29064]: pgmap v47: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 266 op/s 2024-03-13T14:50:59.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:58 smithi176 ceph-mon[29064]: pgmap v48: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 860 KiB/s rd, 906 KiB/s wr, 207 op/s 2024-03-13T14:50:59.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:59.114 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:59.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:59.115 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:50:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:50:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:58 smithi196 ceph-mon[37486]: pgmap v48: 65 pgs: 65 active+clean; 289 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 860 KiB/s rd, 906 KiB/s wr, 207 op/s 2024-03-13T14:50:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:50:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:50:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:50:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:00.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:00 smithi176 ceph-mon[29064]: mgrmap e32: smithi176.tzfdwd(active, since 91s), standbys: smithi196.vdktbk 2024-03-13T14:51:00.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:00 smithi196 ceph-mon[37486]: mgrmap e32: smithi176.tzfdwd(active, since 91s), standbys: smithi196.vdktbk 2024-03-13T14:51:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:01 smithi196 ceph-mon[37486]: pgmap v49: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 349 op/s 2024-03-13T14:51:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:01 smithi176 ceph-mon[29064]: pgmap v49: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 349 op/s 2024-03-13T14:51:03.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:03 smithi176 ceph-mon[29064]: pgmap v50: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 909 KiB/s rd, 951 KiB/s wr, 241 op/s 2024-03-13T14:51:03.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:03 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:03.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:03 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:03.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:03 smithi196 ceph-mon[37486]: pgmap v50: 65 pgs: 65 active+clean; 291 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 909 KiB/s rd, 951 KiB/s wr, 241 op/s 2024-03-13T14:51:03.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:03 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:03.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:03 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:05.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:05 smithi196 ceph-mon[37486]: pgmap v51: 65 pgs: 65 active+clean; 290 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 909 KiB/s rd, 951 KiB/s wr, 278 op/s 2024-03-13T14:51:05.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:05 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:05.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:05 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:05.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:05 smithi176 ceph-mon[29064]: pgmap v51: 65 pgs: 65 active+clean; 290 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 909 KiB/s rd, 951 KiB/s wr, 278 op/s 2024-03-13T14:51:05.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:05 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:05.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:05 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:06.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:06 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:06.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:06 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:06.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:06 smithi196 ceph-mon[37486]: pgmap v52: 65 pgs: 65 active+clean; 290 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-03-13T14:51:07.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:06 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:07.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:06 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:07.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:06 smithi176 ceph-mon[29064]: pgmap v52: 65 pgs: 65 active+clean; 290 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 350 op/s 2024-03-13T14:51:09.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:08 smithi176 ceph-mon[29064]: pgmap v53: 65 pgs: 65 active+clean; 290 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 996 KiB/s rd, 988 KiB/s wr, 266 op/s 2024-03-13T14:51:09.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:08 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:09.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:08 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:09.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:08 smithi196 ceph-mon[37486]: pgmap v53: 65 pgs: 65 active+clean; 290 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 996 KiB/s rd, 988 KiB/s wr, 266 op/s 2024-03-13T14:51:09.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:08 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:09.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:08 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:11.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:11 smithi176 ceph-mon[29064]: pgmap v54: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 419 op/s 2024-03-13T14:51:11.390 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:11 smithi196 ceph-mon[37486]: pgmap v54: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 419 op/s 2024-03-13T14:51:13.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:13 smithi176 ceph-mon[29064]: pgmap v55: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 278 op/s 2024-03-13T14:51:13.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:13.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:13 smithi196 ceph-mon[37486]: pgmap v55: 65 pgs: 65 active+clean; 285 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 278 op/s 2024-03-13T14:51:13.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:14.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:14 smithi176 ceph-mon[29064]: pgmap v56: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-03-13T14:51:14.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:14 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:14.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:14 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:14.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:14 smithi196 ceph-mon[37486]: pgmap v56: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 441 op/s 2024-03-13T14:51:14.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:14 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:14.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:14 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:17.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:17 smithi176 ceph-mon[29064]: pgmap v57: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 597 op/s 2024-03-13T14:51:17.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:17 smithi196 ceph-mon[37486]: pgmap v57: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 597 op/s 2024-03-13T14:51:19.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:19 smithi176 ceph-mon[29064]: pgmap v58: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 896 KiB/s rd, 951 KiB/s wr, 510 op/s 2024-03-13T14:51:19.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:19 smithi196 ceph-mon[37486]: pgmap v58: 65 pgs: 65 active+clean; 283 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 896 KiB/s rd, 951 KiB/s wr, 510 op/s 2024-03-13T14:51:20.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:20 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:20.521 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:20 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 -- 172.21.15.176:0/3102688685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 msgr2=0x7f0728101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 --2- 172.21.15.176:0/3102688685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f0728101690 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0718009910 tx=0x7f0718009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 -- 172.21.15.176:0/3102688685 shutdown_connections 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 --2- 172.21.15.176:0/3102688685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0728101bd0 0x7f0728104060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 --2- 172.21.15.176:0/3102688685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f0728101690 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 -- 172.21.15.176:0/3102688685 >> 172.21.15.176:0/3102688685 conn(0x7f07280fae40 msgr2=0x7f07280fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 -- 172.21.15.176:0/3102688685 shutdown_connections 2024-03-13T14:51:21.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.332+0000 7f072d200700 1 -- 172.21.15.176:0/3102688685 wait complete. 2024-03-13T14:51:21.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.333+0000 7f072d200700 1 Processor -- start 2024-03-13T14:51:21.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.333+0000 7f072d200700 1 -- start start 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f072d200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f072819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f072d200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0728101bd0 0x7f072819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f072d200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072819d310 con 0x7f07280ff260 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f072d200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072819d450 con 0x7f0728101bd0 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f0727400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f072819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f0727400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f072819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55436/0 (socket says 172.21.15.176:55436) 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f0727400700 1 -- 172.21.15.176:0/356974331 learned_addr learned my addr 172.21.15.176:0/356974331 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.334+0000 7f071ea00700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0728101bd0 0x7f072819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:21.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.335+0000 7f0727400700 1 -- 172.21.15.176:0/356974331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0728101bd0 msgr2=0x7f072819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:21.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.335+0000 7f0727400700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0728101bd0 0x7f072819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.335+0000 7f0727400700 1 -- 172.21.15.176:0/356974331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07180095f0 con 0x7f07280ff260 2024-03-13T14:51:21.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.335+0000 7f0727400700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f072819c7b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f071800b5c0 tx=0x7f071800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:21.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.336+0000 7f0724c00700 1 -- 172.21.15.176:0/356974331 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f071801d070 con 0x7f07280ff260 2024-03-13T14:51:21.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.336+0000 7f0724c00700 1 -- 172.21.15.176:0/356974331 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f071800f460 con 0x7f07280ff260 2024-03-13T14:51:21.325 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.336+0000 7f0724c00700 1 -- 172.21.15.176:0/356974331 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0718021640 con 0x7f07280ff260 2024-03-13T14:51:21.325 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.336+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07281a1ea0 con 0x7f07280ff260 2024-03-13T14:51:21.325 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.336+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07281a2310 con 0x7f07280ff260 2024-03-13T14:51:21.325 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.338+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0728196940 con 0x7f07280ff260 2024-03-13T14:51:21.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.339+0000 7f0724c00700 1 -- 172.21.15.176:0/356974331 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f071800fa60 con 0x7f07280ff260 2024-03-13T14:51:21.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.340+0000 7f0724c00700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f070c06dc70 0x7f070c070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:21.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.340+0000 7f0724c00700 1 -- 172.21.15.176:0/356974331 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f071808ea40 con 0x7f07280ff260 2024-03-13T14:51:21.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.340+0000 7f071ea00700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f070c06dc70 0x7f070c070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:21.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.341+0000 7f071ea00700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f070c06dc70 0x7f070c070130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0710005950 tx=0x7f071000b410 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.346+0000 7f0724c00700 1 -- 172.21.15.176:0/356974331 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f071805bf20 con 0x7f07280ff260 2024-03-13T14:51:21.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:21 smithi196 ceph-mon[37486]: pgmap v59: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 829 op/s 2024-03-13T14:51:21.604 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.616+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f07280611d0 con 0x7f070c06dc70 2024-03-13T14:51:21.604 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:21 smithi176 ceph-mon[29064]: pgmap v59: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 829 op/s 2024-03-13T14:51:21.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.619+0000 7f0724c00700 1 -- 172.21.15.176:0/356974331 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f07280611d0 con 0x7f070c06dc70 2024-03-13T14:51:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.622+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f070c06dc70 msgr2=0x7f070c070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.622+0000 7f072d200700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f070c06dc70 0x7f070c070130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0710005950 tx=0x7f071000b410 comp rx=0 tx=0).stop 2024-03-13T14:51:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.622+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 msgr2=0x7f072819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.622+0000 7f072d200700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f072819c7b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f071800b5c0 tx=0x7f071800bd70 comp rx=0 tx=0).stop 2024-03-13T14:51:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.623+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 shutdown_connections 2024-03-13T14:51:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.623+0000 7f072d200700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f070c06dc70 0x7f070c070130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.623+0000 7f072d200700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07280ff260 0x7f072819c7b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.623+0000 7f072d200700 1 --2- 172.21.15.176:0/356974331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0728101bd0 0x7f072819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.623+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 >> 172.21.15.176:0/356974331 conn(0x7f07280fae40 msgr2=0x7f07280fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.623+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 shutdown_connections 2024-03-13T14:51:21.612 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.623+0000 7f072d200700 1 -- 172.21.15.176:0/356974331 wait complete. 2024-03-13T14:51:21.626 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:51:21.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.831+0000 7f7665a00700 1 -- 172.21.15.176:0/507200413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660101470 msgr2=0x7f7660103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.831+0000 7f7665a00700 1 --2- 172.21.15.176:0/507200413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660101470 0x7f7660103860 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7650009910 tx=0x7f7650009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.832+0000 7f7665a00700 1 -- 172.21.15.176:0/507200413 shutdown_connections 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.832+0000 7f7665a00700 1 --2- 172.21.15.176:0/507200413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7660103da0 0x7f7660106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.832+0000 7f7665a00700 1 --2- 172.21.15.176:0/507200413 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660101470 0x7f7660103860 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.832+0000 7f7665a00700 1 -- 172.21.15.176:0/507200413 >> 172.21.15.176:0/507200413 conn(0x7f76600fae40 msgr2=0x7f76600fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.832+0000 7f7665a00700 1 -- 172.21.15.176:0/507200413 shutdown_connections 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.832+0000 7f7665a00700 1 -- 172.21.15.176:0/507200413 wait complete. 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.833+0000 7f7665a00700 1 Processor -- start 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.833+0000 7f7665a00700 1 -- start start 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.833+0000 7f7665a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7660101470 0x7f7660198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:21.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.833+0000 7f7665a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660103da0 0x7f76601988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:21.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.833+0000 7f7665a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7660198ee0 con 0x7f7660103da0 2024-03-13T14:51:21.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.833+0000 7f7665a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7660199020 con 0x7f7660101470 2024-03-13T14:51:21.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660103da0 0x7f76601988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:21.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660103da0 0x7f76601988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55444/0 (socket says 172.21.15.176:55444) 2024-03-13T14:51:21.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765f400700 1 -- 172.21.15.176:0/4184773440 learned_addr learned my addr 172.21.15.176:0/4184773440 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:21.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765fe00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7660101470 0x7f7660198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:21.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765f400700 1 -- 172.21.15.176:0/4184773440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7660101470 msgr2=0x7f7660198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:21.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765f400700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7660101470 0x7f7660198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:21.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765f400700 1 -- 172.21.15.176:0/4184773440 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76500095f0 con 0x7f7660103da0 2024-03-13T14:51:21.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.834+0000 7f765f400700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660103da0 0x7f76601988c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f764c00e920 tx=0x7f764c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:21.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.835+0000 7f765cc00700 1 -- 172.21.15.176:0/4184773440 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764c00c9d0 con 0x7f7660103da0 2024-03-13T14:51:21.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.835+0000 7f765cc00700 1 -- 172.21.15.176:0/4184773440 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f764c010430 con 0x7f7660103da0 2024-03-13T14:51:21.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.835+0000 7f765cc00700 1 -- 172.21.15.176:0/4184773440 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764c018650 con 0x7f7660103da0 2024-03-13T14:51:21.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.835+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f766019dad0 con 0x7f7660103da0 2024-03-13T14:51:21.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.835+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7660106dc0 con 0x7f7660103da0 2024-03-13T14:51:21.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.836+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7660192550 con 0x7f7660103da0 2024-03-13T14:51:21.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.838+0000 7f765cc00700 1 -- 172.21.15.176:0/4184773440 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f764c0105a0 con 0x7f7660103da0 2024-03-13T14:51:21.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.839+0000 7f765cc00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f764406dc20 0x7f76440700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:21.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.839+0000 7f765cc00700 1 -- 172.21.15.176:0/4184773440 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f764c014070 con 0x7f7660103da0 2024-03-13T14:51:21.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.839+0000 7f765fe00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f764406dc20 0x7f76440700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:21.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.840+0000 7f765fe00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f764406dc20 0x7f76440700e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7650009fd0 tx=0x7f7650005fd0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:21.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:21.843+0000 7f765cc00700 1 -- 172.21.15.176:0/4184773440 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f764c05b7b0 con 0x7f7660103da0 2024-03-13T14:51:22.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.221+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f76600611d0 con 0x7f764406dc20 2024-03-13T14:51:22.213 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.225+0000 7f765cc00700 1 -- 172.21.15.176:0/4184773440 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f76600611d0 con 0x7f764406dc20 2024-03-13T14:51:22.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f764406dc20 msgr2=0x7f76440700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f764406dc20 0x7f76440700e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7650009fd0 tx=0x7f7650005fd0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660103da0 msgr2=0x7f76601988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660103da0 0x7f76601988c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f764c00e920 tx=0x7f764c00ec30 comp rx=0 tx=0).stop 2024-03-13T14:51:22.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 shutdown_connections 2024-03-13T14:51:22.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f764406dc20 0x7f76440700e0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7650009fd0 tx=0x7f7650005fd0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7660101470 0x7f7660198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 --2- 172.21.15.176:0/4184773440 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7660103da0 0x7f76601988c0 secure :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f764c00e920 tx=0x7f764c00ec30 comp rx=0 tx=0).stop 2024-03-13T14:51:22.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.230+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 >> 172.21.15.176:0/4184773440 conn(0x7f76600fae40 msgr2=0x7f76600fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:22.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.231+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 shutdown_connections 2024-03-13T14:51:22.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.232+0000 7f7665a00700 1 -- 172.21.15.176:0/4184773440 wait complete. 2024-03-13T14:51:22.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.403+0000 7faa8a000700 1 -- 172.21.15.176:0/3581236474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 msgr2=0x7faa84102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.403+0000 7faa8a000700 1 --2- 172.21.15.176:0/3581236474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84102060 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faa70009910 tx=0x7faa70009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:22.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.403+0000 7faa8a000700 1 -- 172.21.15.176:0/3581236474 shutdown_connections 2024-03-13T14:51:22.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.403+0000 7faa8a000700 1 --2- 172.21.15.176:0/3581236474 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84102060 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.403+0000 7faa8a000700 1 --2- 172.21.15.176:0/3581236474 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84100a00 0x7faa84100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.403+0000 7faa8a000700 1 -- 172.21.15.176:0/3581236474 >> 172.21.15.176:0/3581236474 conn(0x7faa840fbf60 msgr2=0x7faa840fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:22.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.403+0000 7faa8a000700 1 -- 172.21.15.176:0/3581236474 shutdown_connections 2024-03-13T14:51:22.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.404+0000 7faa8a000700 1 -- 172.21.15.176:0/3581236474 wait complete. 2024-03-13T14:51:22.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.405+0000 7faa8a000700 1 Processor -- start 2024-03-13T14:51:22.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.405+0000 7faa8a000700 1 -- start start 2024-03-13T14:51:22.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.405+0000 7faa8a000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84100a00 0x7faa84192540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:22.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.405+0000 7faa8a000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84192a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:22.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.405+0000 7faa8a000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8418f030 con 0x7faa84101c00 2024-03-13T14:51:22.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.405+0000 7faa8a000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8418f1a0 con 0x7faa84100a00 2024-03-13T14:51:22.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.406+0000 7faa88c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84100a00 0x7faa84192540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:22.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.406+0000 7faa83e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84192a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:22.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.406+0000 7faa83e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84192a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46646/0 (socket says 172.21.15.176:46646) 2024-03-13T14:51:22.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.406+0000 7faa83e00700 1 -- 172.21.15.176:0/3278259998 learned_addr learned my addr 172.21.15.176:0/3278259998 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:22.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.407+0000 7faa83e00700 1 -- 172.21.15.176:0/3278259998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84100a00 msgr2=0x7faa84192540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.407+0000 7faa83e00700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84100a00 0x7faa84192540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.407+0000 7faa83e00700 1 -- 172.21.15.176:0/3278259998 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa700095f0 con 0x7faa84101c00 2024-03-13T14:51:22.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.407+0000 7faa88c00700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84100a00 0x7faa84192540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:51:22.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.407+0000 7faa83e00700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84192a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faa70005b40 tx=0x7faa7000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:22.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.408+0000 7faa81600700 1 -- 172.21.15.176:0/3278259998 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa7001d070 con 0x7faa84101c00 2024-03-13T14:51:22.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.408+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8418f420 con 0x7faa84101c00 2024-03-13T14:51:22.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.408+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8418f910 con 0x7faa84101c00 2024-03-13T14:51:22.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.408+0000 7faa81600700 1 -- 172.21.15.176:0/3278259998 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faa7000f460 con 0x7faa84101c00 2024-03-13T14:51:22.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.408+0000 7faa81600700 1 -- 172.21.15.176:0/3278259998 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa70021640 con 0x7faa84101c00 2024-03-13T14:51:22.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.409+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa64005320 con 0x7faa84101c00 2024-03-13T14:51:22.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.410+0000 7faa81600700 1 -- 172.21.15.176:0/3278259998 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7faa70003a60 con 0x7faa84101c00 2024-03-13T14:51:22.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.411+0000 7faa81600700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7faa6c06dc20 0x7faa6c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:22.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.412+0000 7faa81600700 1 -- 172.21.15.176:0/3278259998 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faa7008ead0 con 0x7faa84101c00 2024-03-13T14:51:22.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.412+0000 7faa88c00700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7faa6c06dc20 0x7faa6c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:22.400 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.413+0000 7faa88c00700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7faa6c06dc20 0x7faa6c0700e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa78005fd0 tx=0x7faa78005f00 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:22.404 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.416+0000 7faa81600700 1 -- 172.21.15.176:0/3278259998 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7faa7005bf40 con 0x7faa84101c00 2024-03-13T14:51:22.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.697+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faa64000bf0 con 0x7faa6c06dc20 2024-03-13T14:51:22.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.720+0000 7faa81600700 1 -- 172.21.15.176:0/3278259998 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7faa64000bf0 con 0x7faa6c06dc20 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (10m) 17s ago 12m 13.7M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (13m) 17s ago 13m 7898k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (11m) 19s ago 11m 7868k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (13m) 17s ago 13m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (11m) 19s ago 11m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (10m) 17s ago 12m 74.7M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (5m) 17s ago 5m 141M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (5m) 17s ago 5m 14.5M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (5m) 19s ago 5m 216M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (5m) 19s ago 5m 14.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:51:22.708 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (2m) 17s ago 14m 494M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (62s) 19s ago 11m 447M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (14m) 17s ago 14m 54.7M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (11m) 19s ago 11m 44.1M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (32s) 17s ago 13m 6832k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (25s) 19s ago 11m 2847k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (10m) 17s ago 10m 199M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (9m) 17s ago 9m 209M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (9m) 17s ago 9m 181M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (8m) 19s ago 8m 247M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (7m) 19s ago 7m 220M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (7m) 19s ago 7m 194M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:51:22.709 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (70s) 17s ago 12m 54.7M - 2.43.0 a07b618ecd1d b4eb817c4249 2024-03-13T14:51:22.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.722+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7faa6c06dc20 msgr2=0x7faa6c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.722+0000 7faa8a000700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7faa6c06dc20 0x7faa6c0700e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7faa78005fd0 tx=0x7faa78005f00 comp rx=0 tx=0).stop 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.722+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 msgr2=0x7faa84192a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.722+0000 7faa8a000700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84192a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faa70005b40 tx=0x7faa7000be20 comp rx=0 tx=0).stop 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.723+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 shutdown_connections 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.723+0000 7faa8a000700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7faa6c06dc20 0x7faa6c0700e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.723+0000 7faa8a000700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa84100a00 0x7faa84192540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.723+0000 7faa8a000700 1 --2- 172.21.15.176:0/3278259998 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa84101c00 0x7faa84192a80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.723+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 >> 172.21.15.176:0/3278259998 conn(0x7faa840fbf60 msgr2=0x7faa84102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.723+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 shutdown_connections 2024-03-13T14:51:22.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.723+0000 7faa8a000700 1 -- 172.21.15.176:0/3278259998 wait complete. 2024-03-13T14:51:22.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 -- 172.21.15.176:0/1282157517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 msgr2=0x7f36500731a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 --2- 172.21.15.176:0/1282157517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 0x7f36500731a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3640009910 tx=0x7f3640009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:22.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 -- 172.21.15.176:0/1282157517 shutdown_connections 2024-03-13T14:51:22.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 --2- 172.21.15.176:0/1282157517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3650073770 0x7f3650073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 --2- 172.21.15.176:0/1282157517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 0x7f36500731a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 -- 172.21.15.176:0/1282157517 >> 172.21.15.176:0/1282157517 conn(0x7f36500fc110 msgr2=0x7f36500fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:22.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 -- 172.21.15.176:0/1282157517 shutdown_connections 2024-03-13T14:51:22.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.873+0000 7f3656800700 1 -- 172.21.15.176:0/1282157517 wait complete. 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.874+0000 7f3656800700 1 Processor -- start 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.874+0000 7f3656800700 1 -- start start 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f3656800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3650073770 0x7f3650198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f3656800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 0x7f3650198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f3656800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501990b0 con 0x7f3650074d40 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f3656800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36501991f0 con 0x7f3650073770 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f364ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 0x7f3650198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f364f400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3650073770 0x7f3650198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f364f400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3650073770 0x7f3650198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:53132/0 (socket says 172.21.15.176:53132) 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f364f400700 1 -- 172.21.15.176:0/389801732 learned_addr learned my addr 172.21.15.176:0/389801732 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f364ea00700 1 -- 172.21.15.176:0/389801732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3650073770 msgr2=0x7f3650198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.875+0000 7f364ea00700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3650073770 0x7f3650198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:22.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.876+0000 7f364ea00700 1 -- 172.21.15.176:0/389801732 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36400095f0 con 0x7f3650074d40 2024-03-13T14:51:22.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.876+0000 7f364ea00700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 0x7f3650198a90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f363c00d8d0 tx=0x7f363c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:22.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.876+0000 7f3647e00700 1 -- 172.21.15.176:0/389801732 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f363c00f860 con 0x7f3650074d40 2024-03-13T14:51:22.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.876+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365019dca0 con 0x7f3650074d40 2024-03-13T14:51:22.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.876+0000 7f3647e00700 1 -- 172.21.15.176:0/389801732 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f363c00fe70 con 0x7f3650074d40 2024-03-13T14:51:22.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.876+0000 7f3647e00700 1 -- 172.21.15.176:0/389801732 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f363c00e5e0 con 0x7f3650074d40 2024-03-13T14:51:22.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.876+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365019e1f0 con 0x7f3650074d40 2024-03-13T14:51:22.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.877+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365010b900 con 0x7f3650074d40 2024-03-13T14:51:22.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.879+0000 7f3647e00700 1 -- 172.21.15.176:0/389801732 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f363c010460 con 0x7f3650074d40 2024-03-13T14:51:22.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.880+0000 7f3647e00700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f363406dc20 0x7f36340700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:22.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.880+0000 7f3647e00700 1 -- 172.21.15.176:0/389801732 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f363c08d450 con 0x7f3650074d40 2024-03-13T14:51:22.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.881+0000 7f364f400700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f363406dc20 0x7f36340700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:22.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.882+0000 7f364f400700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f363406dc20 0x7f36340700e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3640006010 tx=0x7f3640005ab0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:22.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:22.889+0000 7f3647e00700 1 -- 172.21.15.176:0/389801732 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f363c05a830 con 0x7f3650074d40 2024-03-13T14:51:23.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.247+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f365019e800 con 0x7f3650074d40 2024-03-13T14:51:23.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.248+0000 7f3647e00700 1 -- 172.21.15.176:0/389801732 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f363c05a3c0 con 0x7f3650074d40 2024-03-13T14:51:23.236 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:51:23.236 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:51:23.237 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:51:23.238 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:51:23.238 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:51:23.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.251+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f363406dc20 msgr2=0x7f36340700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.251+0000 7f3656800700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f363406dc20 0x7f36340700e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3640006010 tx=0x7f3640005ab0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.251+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 msgr2=0x7f3650198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.251+0000 7f3656800700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 0x7f3650198a90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f363c00d8d0 tx=0x7f363c00dbe0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.252+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 shutdown_connections 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.252+0000 7f3656800700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f363406dc20 0x7f36340700e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.252+0000 7f3656800700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3650073770 0x7f3650198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.252+0000 7f3656800700 1 --2- 172.21.15.176:0/389801732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3650074d40 0x7f3650198a90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.252+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 >> 172.21.15.176:0/389801732 conn(0x7f36500fc110 msgr2=0x7f3650106a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:23.240 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.252+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 shutdown_connections 2024-03-13T14:51:23.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.252+0000 7f3656800700 1 -- 172.21.15.176:0/389801732 wait complete. 2024-03-13T14:51:23.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 -- 172.21.15.176:0/3530456335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 msgr2=0x7f5814107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 --2- 172.21.15.176:0/3530456335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f5814107620 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5804009910 tx=0x7f5804009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 -- 172.21.15.176:0/3530456335 shutdown_connections 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 --2- 172.21.15.176:0/3530456335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5814108400 0x7f5814108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 --2- 172.21.15.176:0/3530456335 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f5814107620 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 -- 172.21.15.176:0/3530456335 >> 172.21.15.176:0/3530456335 conn(0x7f58140761b0 msgr2=0x7f5814078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 -- 172.21.15.176:0/3530456335 shutdown_connections 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.400+0000 7f5819c00700 1 -- 172.21.15.176:0/3530456335 wait complete. 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.401+0000 7f5819c00700 1 Processor -- start 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.401+0000 7f5819c00700 1 -- start start 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.401+0000 7f5819c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f581419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.401+0000 7f5819c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5814108400 0x7f581419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:23.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.401+0000 7f5819c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419d510 con 0x7f5814107200 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.401+0000 7f5819c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581419d650 con 0x7f5814108400 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f581419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f581419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46698/0 (socket says 172.21.15.176:46698) 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812a00700 1 -- 172.21.15.176:0/507568929 learned_addr learned my addr 172.21.15.176:0/507568929 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812000700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5814108400 0x7f581419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812a00700 1 -- 172.21.15.176:0/507568929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5814108400 msgr2=0x7f581419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812a00700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5814108400 0x7f581419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812a00700 1 -- 172.21.15.176:0/507568929 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58040095f0 con 0x7f5814107200 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.402+0000 7f5812a00700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f581419c9b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f580400bd70 tx=0x7f580400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.403+0000 7f580b400700 1 -- 172.21.15.176:0/507568929 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580401d070 con 0x7f5814107200 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.403+0000 7f580b400700 1 -- 172.21.15.176:0/507568929 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f580400f460 con 0x7f5814107200 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.403+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58141a20a0 con 0x7f5814107200 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.403+0000 7f580b400700 1 -- 172.21.15.176:0/507568929 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5804021640 con 0x7f5814107200 2024-03-13T14:51:23.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.403+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58141a2590 con 0x7f5814107200 2024-03-13T14:51:23.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.404+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581410fd60 con 0x7f5814107200 2024-03-13T14:51:23.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.405+0000 7f580b400700 1 -- 172.21.15.176:0/507568929 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f580400fab0 con 0x7f5814107200 2024-03-13T14:51:23.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.406+0000 7f580b400700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f57f806dc20 0x7f57f80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:23.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.406+0000 7f580b400700 1 -- 172.21.15.176:0/507568929 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f580408ec50 con 0x7f5814107200 2024-03-13T14:51:23.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.406+0000 7f5812000700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f57f806dc20 0x7f57f80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:23.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.407+0000 7f5812000700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f57f806dc20 0x7f57f80700e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5800005fd0 tx=0x7f580000a560 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:23.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.412+0000 7f580b400700 1 -- 172.21.15.176:0/507568929 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f580405c0b0 con 0x7f5814107200 2024-03-13T14:51:23.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:23 smithi196 ceph-mon[37486]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:23.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:23 smithi196 ceph-mon[37486]: pgmap v60: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 651 KiB/s rd, 771 KiB/s wr, 676 op/s 2024-03-13T14:51:23.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:23 smithi196 ceph-mon[37486]: Upgrade: Updating prometheus.smithi176 2024-03-13T14:51:23.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:23 smithi196 ceph-mon[37486]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:23.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:23 smithi176 ceph-mon[29064]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:23.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:23 smithi176 ceph-mon[29064]: pgmap v60: 65 pgs: 65 active+clean; 277 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 651 KiB/s rd, 771 KiB/s wr, 676 op/s 2024-03-13T14:51:23.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:23 smithi176 ceph-mon[29064]: Upgrade: Updating prometheus.smithi176 2024-03-13T14:51:23.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:23 smithi176 ceph-mon[29064]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:23.719 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.732+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f581410cd50 con 0x7f57f806dc20 2024-03-13T14:51:23.720 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.733+0000 7f580b400700 1 -- 172.21.15.176:0/507568929 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f581410cd50 con 0x7f57f806dc20 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "2/2 daemons upgraded", 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading prometheus daemons", 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:51:23.721 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:51:23.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f57f806dc20 msgr2=0x7f57f80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:23.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f57f806dc20 0x7f57f80700e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5800005fd0 tx=0x7f580000a560 comp rx=0 tx=0).stop 2024-03-13T14:51:23.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 msgr2=0x7f581419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f581419c9b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f580400bd70 tx=0x7f580400be50 comp rx=0 tx=0).stop 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 shutdown_connections 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f57f806dc20 0x7f57f80700e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5814107200 0x7f581419c9b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 --2- 172.21.15.176:0/507568929 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5814108400 0x7f581419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 >> 172.21.15.176:0/507568929 conn(0x7f58140761b0 msgr2=0x7f581410b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 shutdown_connections 2024-03-13T14:51:23.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:23.736+0000 7f5819c00700 1 -- 172.21.15.176:0/507568929 wait complete. 2024-03-13T14:51:24.506 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:24 smithi176 ceph-mon[29064]: Deploying daemon prometheus.smithi176 on smithi176 2024-03-13T14:51:24.507 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:24 smithi176 ceph-mon[29064]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:24.507 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:24 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/389801732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:24.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:24 smithi196 ceph-mon[37486]: Deploying daemon prometheus.smithi176 on smithi176 2024-03-13T14:51:24.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:24 smithi196 ceph-mon[37486]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:24.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:24 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/389801732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:25.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:25 smithi176 ceph-mon[29064]: pgmap v61: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 998 KiB/s rd, 1.1 MiB/s wr, 790 op/s 2024-03-13T14:51:25.291 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:25 smithi176 ceph-mon[29064]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:25 smithi196 ceph-mon[37486]: pgmap v61: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 998 KiB/s rd, 1.1 MiB/s wr, 790 op/s 2024-03-13T14:51:25.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:25 smithi196 ceph-mon[37486]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:27.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:27 smithi176 ceph-mon[29064]: pgmap v62: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1012 KiB/s rd, 1.2 MiB/s wr, 749 op/s 2024-03-13T14:51:27.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:27 smithi196 ceph-mon[37486]: pgmap v62: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1012 KiB/s rd, 1.2 MiB/s wr, 749 op/s 2024-03-13T14:51:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:29.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:29 smithi176 ceph-mon[29064]: pgmap v63: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 725 KiB/s rd, 855 KiB/s wr, 555 op/s 2024-03-13T14:51:29.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:29 smithi196 ceph-mon[37486]: pgmap v63: 65 pgs: 65 active+clean; 270 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 725 KiB/s rd, 855 KiB/s wr, 555 op/s 2024-03-13T14:51:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:31 smithi176 ceph-mon[29064]: pgmap v64: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 713 op/s 2024-03-13T14:51:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:31 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:31 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:31 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:31.638 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:31 smithi196 ceph-mon[37486]: pgmap v64: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 713 op/s 2024-03-13T14:51:31.639 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:31 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:31.639 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:31 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:31.639 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:31 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:33.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:33 smithi196 ceph-mon[37486]: pgmap v65: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 827 KiB/s rd, 921 KiB/s wr, 394 op/s 2024-03-13T14:51:33.499 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:33 smithi176 ceph-mon[29064]: pgmap v65: 65 pgs: 65 active+clean; 264 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 827 KiB/s rd, 921 KiB/s wr, 394 op/s 2024-03-13T14:51:35.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:35 smithi196 ceph-mon[37486]: pgmap v66: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 464 op/s 2024-03-13T14:51:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:35 smithi176 ceph-mon[29064]: pgmap v66: 65 pgs: 65 active+clean; 262 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 464 op/s 2024-03-13T14:51:37.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:37 smithi176 ceph-mon[29064]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 823 KiB/s rd, 961 KiB/s wr, 404 op/s 2024-03-13T14:51:37.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:37 smithi196 ceph-mon[37486]: pgmap v67: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 823 KiB/s rd, 961 KiB/s wr, 404 op/s 2024-03-13T14:51:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:39 smithi176 ceph-mon[29064]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 797 KiB/s rd, 850 KiB/s wr, 282 op/s 2024-03-13T14:51:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:39 smithi196 ceph-mon[37486]: pgmap v68: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 797 KiB/s rd, 850 KiB/s wr, 282 op/s 2024-03-13T14:51:40.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:40 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:40.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:40 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:40.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:40 smithi176 ceph-mon[29064]: pgmap v69: 65 pgs: 65 active+clean; 255 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-03-13T14:51:40.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:40 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:40.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:40 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:40.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:40 smithi196 ceph-mon[37486]: pgmap v69: 65 pgs: 65 active+clean; 255 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-03-13T14:51:42.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:42 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:42.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:42 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:42.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:42 smithi176 ceph-mon[29064]: pgmap v70: 65 pgs: 65 active+clean; 255 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 743 KiB/s rd, 784 KiB/s wr, 238 op/s 2024-03-13T14:51:42.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:42 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:42.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:42 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:42.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:42 smithi196 ceph-mon[37486]: pgmap v70: 65 pgs: 65 active+clean; 255 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 743 KiB/s rd, 784 KiB/s wr, 238 op/s 2024-03-13T14:51:43.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:43.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:44 smithi176 ceph-mon[29064]: pgmap v71: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 754 KiB/s rd, 796 KiB/s wr, 284 op/s 2024-03-13T14:51:45.159 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:44 smithi196 ceph-mon[37486]: pgmap v71: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 754 KiB/s rd, 796 KiB/s wr, 284 op/s 2024-03-13T14:51:47.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:47 smithi176 ceph-mon[29064]: pgmap v72: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 815 KiB/s rd, 785 KiB/s wr, 285 op/s 2024-03-13T14:51:47.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:47 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:47.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:47 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:47.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:47 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:51:47.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:47 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:51:47.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:47 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:47 smithi196 ceph-mon[37486]: pgmap v72: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 815 KiB/s rd, 785 KiB/s wr, 285 op/s 2024-03-13T14:51:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:47 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:47 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:47 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:51:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:47 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:51:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:47 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:48.412 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:51:48.412 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:48.412 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.412 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.412 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:48.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:49 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:51:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:49 smithi196 ceph-mon[37486]: pgmap v73: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 815 KiB/s rd, 785 KiB/s wr, 231 op/s 2024-03-13T14:51:49.472 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:49 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-13T14:51:49.472 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:49 smithi176 ceph-mon[29064]: pgmap v73: 65 pgs: 65 active+clean; 253 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 815 KiB/s rd, 785 KiB/s wr, 231 op/s 2024-03-13T14:51:51.195 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:51 smithi176 ceph-mon[29064]: Upgrade: Updating alertmanager.smithi176 2024-03-13T14:51:51.196 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:51 smithi176 ceph-mon[29064]: Deploying daemon alertmanager.smithi176 on smithi176 2024-03-13T14:51:51.196 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:51 smithi176 ceph-mon[29064]: pgmap v74: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 344 op/s 2024-03-13T14:51:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:51 smithi196 ceph-mon[37486]: Upgrade: Updating alertmanager.smithi176 2024-03-13T14:51:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:51 smithi196 ceph-mon[37486]: Deploying daemon alertmanager.smithi176 on smithi176 2024-03-13T14:51:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:51 smithi196 ceph-mon[37486]: pgmap v74: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 344 op/s 2024-03-13T14:51:53.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:53 smithi176 ceph-mon[29064]: pgmap v75: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 723 KiB/s rd, 768 KiB/s wr, 230 op/s 2024-03-13T14:51:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:53 smithi196 ceph-mon[37486]: pgmap v75: 65 pgs: 65 active+clean; 243 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 723 KiB/s rd, 768 KiB/s wr, 230 op/s 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 -- 172.21.15.176:0/4249503527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 msgr2=0x7f97f4104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 --2- 172.21.15.176:0/4249503527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4104450 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009910 tx=0x7f97e0009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 -- 172.21.15.176:0/4249503527 shutdown_connections 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 --2- 172.21.15.176:0/4249503527 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4104450 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 --2- 172.21.15.176:0/4249503527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f4102df0 0x7f97f4103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 -- 172.21.15.176:0/4249503527 >> 172.21.15.176:0/4249503527 conn(0x7f97f40fe390 msgr2=0x7f97f41007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 -- 172.21.15.176:0/4249503527 shutdown_connections 2024-03-13T14:51:53.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.913+0000 7f97f8c00700 1 -- 172.21.15.176:0/4249503527 wait complete. 2024-03-13T14:51:53.902 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.914+0000 7f97f8c00700 1 Processor -- start 2024-03-13T14:51:53.902 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f8c00700 1 -- start start 2024-03-13T14:51:53.902 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f8c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f4102df0 0x7f97f41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:53.902 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f8c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:53.902 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f8c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4199100 con 0x7f97f4103ff0 2024-03-13T14:51:53.903 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f8c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f4199240 con 0x7f97f4102df0 2024-03-13T14:51:53.903 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:53.903 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55766/0 (socket says 172.21.15.176:55766) 2024-03-13T14:51:53.903 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.915+0000 7f97f1400700 1 -- 172.21.15.176:0/2907016940 learned_addr learned my addr 172.21.15.176:0/2907016940 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:53.903 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.916+0000 7f97f1e00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f4102df0 0x7f97f41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:53.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.916+0000 7f97f1400700 1 -- 172.21.15.176:0/2907016940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f4102df0 msgr2=0x7f97f41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:53.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.916+0000 7f97f1400700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f4102df0 0x7f97f41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:53.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.916+0000 7f97f1400700 1 -- 172.21.15.176:0/2907016940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e00095f0 con 0x7f97f4103ff0 2024-03-13T14:51:53.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.916+0000 7f97f1400700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4198ae0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005b40 tx=0x7f97e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:53.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.917+0000 7f97eaa00700 1 -- 172.21.15.176:0/2907016940 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e001d070 con 0x7f97f4103ff0 2024-03-13T14:51:53.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.917+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f419dc90 con 0x7f97f4103ff0 2024-03-13T14:51:53.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.917+0000 7f97eaa00700 1 -- 172.21.15.176:0/2907016940 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97e000f460 con 0x7f97f4103ff0 2024-03-13T14:51:53.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.917+0000 7f97eaa00700 1 -- 172.21.15.176:0/2907016940 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e0021640 con 0x7f97f4103ff0 2024-03-13T14:51:53.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.917+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f419e180 con 0x7f97f4103ff0 2024-03-13T14:51:53.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.918+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d4005320 con 0x7f97f4103ff0 2024-03-13T14:51:53.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.919+0000 7f97eaa00700 1 -- 172.21.15.176:0/2907016940 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f97e0003a60 con 0x7f97f4103ff0 2024-03-13T14:51:53.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.920+0000 7f97eaa00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f97d806dc20 0x7f97d80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:53.908 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.920+0000 7f97eaa00700 1 -- 172.21.15.176:0/2907016940 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f97e008eb90 con 0x7f97f4103ff0 2024-03-13T14:51:53.908 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.921+0000 7f97f1e00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f97d806dc20 0x7f97d80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:53.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.921+0000 7f97f1e00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f97d806dc20 0x7f97d80700e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005950 tx=0x7f97e40058e0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:53.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:53.926+0000 7f97eaa00700 1 -- 172.21.15.176:0/2907016940 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f97e005bf70 con 0x7f97f4103ff0 2024-03-13T14:51:54.195 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.207+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97d4000bf0 con 0x7f97d806dc20 2024-03-13T14:51:54.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.209+0000 7f97eaa00700 1 -- 172.21.15.176:0/2907016940 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f97d4000bf0 con 0x7f97d806dc20 2024-03-13T14:51:54.200 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.212+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f97d806dc20 msgr2=0x7f97d80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.200 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.212+0000 7f97f8c00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f97d806dc20 0x7f97d80700e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005950 tx=0x7f97e40058e0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.212+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 msgr2=0x7f97f4198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.212+0000 7f97f8c00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4198ae0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f97e0005b40 tx=0x7f97e000be20 comp rx=0 tx=0).stop 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.213+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 shutdown_connections 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.213+0000 7f97f8c00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f97d806dc20 0x7f97d80700e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.213+0000 7f97f8c00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f4102df0 0x7f97f41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.213+0000 7f97f8c00700 1 --2- 172.21.15.176:0/2907016940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f97f4103ff0 0x7f97f4198ae0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.213+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 >> 172.21.15.176:0/2907016940 conn(0x7f97f40fe390 msgr2=0x7f97f4107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.213+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 shutdown_connections 2024-03-13T14:51:54.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.213+0000 7f97f8c00700 1 -- 172.21.15.176:0/2907016940 wait complete. 2024-03-13T14:51:54.216 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:51:54.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 -- 172.21.15.176:0/2902943410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 msgr2=0x7ff2e8102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2902943410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8102210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009910 tx=0x7ff2d4009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 -- 172.21.15.176:0/2902943410 shutdown_connections 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2902943410 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8102210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2902943410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e8100bb0 0x7ff2e8100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 -- 172.21.15.176:0/2902943410 >> 172.21.15.176:0/2902943410 conn(0x7ff2e80fc110 msgr2=0x7ff2e80fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 -- 172.21.15.176:0/2902943410 shutdown_connections 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.424+0000 7ff2ee200700 1 -- 172.21.15.176:0/2902943410 wait complete. 2024-03-13T14:51:54.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.425+0000 7ff2ee200700 1 Processor -- start 2024-03-13T14:51:54.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.425+0000 7ff2ee200700 1 -- start start 2024-03-13T14:51:54.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2ee200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e8100bb0 0x7ff2e8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:54.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2ee200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:54.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2ee200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e81990d0 con 0x7ff2e8101db0 2024-03-13T14:51:54.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2ee200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e8199210 con 0x7ff2e8100bb0 2024-03-13T14:51:54.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2e6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:54.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2e6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55778/0 (socket says 172.21.15.176:55778) 2024-03-13T14:51:54.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2e6a00700 1 -- 172.21.15.176:0/2252973383 learned_addr learned my addr 172.21.15.176:0/2252973383 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:54.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.426+0000 7ff2e7400700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e8100bb0 0x7ff2e8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2e6a00700 1 -- 172.21.15.176:0/2252973383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e8100bb0 msgr2=0x7ff2e8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2e6a00700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e8100bb0 0x7ff2e8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2e6a00700 1 -- 172.21.15.176:0/2252973383 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40095f0 con 0x7ff2e8101db0 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2e6a00700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8198ab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4000c00 tx=0x7ff2d400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2dfe00700 1 -- 172.21.15.176:0/2252973383 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d401d070 con 0x7ff2e8101db0 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2dfe00700 1 -- 172.21.15.176:0/2252973383 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff2d4005dc0 con 0x7ff2e8101db0 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e819dc60 con 0x7ff2e8101db0 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2dfe00700 1 -- 172.21.15.176:0/2252973383 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2d40226a0 con 0x7ff2e8101db0 2024-03-13T14:51:54.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.427+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e819e150 con 0x7ff2e8101db0 2024-03-13T14:51:54.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.429+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e8105e90 con 0x7ff2e8101db0 2024-03-13T14:51:54.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.430+0000 7ff2dfe00700 1 -- 172.21.15.176:0/2252973383 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7ff2d40038f0 con 0x7ff2e8101db0 2024-03-13T14:51:54.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.430+0000 7ff2dfe00700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff2cc06dc20 0x7ff2cc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:54.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.431+0000 7ff2dfe00700 1 -- 172.21.15.176:0/2252973383 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff2d408edd0 con 0x7ff2e8101db0 2024-03-13T14:51:54.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.431+0000 7ff2e7400700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff2cc06dc20 0x7ff2cc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:54.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.432+0000 7ff2e7400700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff2cc06dc20 0x7ff2cc0700e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8101c10 tx=0x7ff2d8009260 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:54.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.435+0000 7ff2dfe00700 1 -- 172.21.15.176:0/2252973383 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7ff2d405c1b0 con 0x7ff2e8101db0 2024-03-13T14:51:54.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.700+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2e80611d0 con 0x7ff2cc06dc20 2024-03-13T14:51:54.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.702+0000 7ff2dfe00700 1 -- 172.21.15.176:0/2252973383 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7ff2e80611d0 con 0x7ff2cc06dc20 2024-03-13T14:51:54.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff2cc06dc20 msgr2=0x7ff2cc0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff2cc06dc20 0x7ff2cc0700e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8101c10 tx=0x7ff2d8009260 comp rx=0 tx=0).stop 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 msgr2=0x7ff2e8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8198ab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4000c00 tx=0x7ff2d400f690 comp rx=0 tx=0).stop 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 shutdown_connections 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff2cc06dc20 0x7ff2cc0700e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2e8100bb0 0x7ff2e8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 --2- 172.21.15.176:0/2252973383 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2e8101db0 0x7ff2e8198ab0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 >> 172.21.15.176:0/2252973383 conn(0x7ff2e80fc110 msgr2=0x7ff2e8102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.705+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 shutdown_connections 2024-03-13T14:51:54.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.706+0000 7ff2ee200700 1 -- 172.21.15.176:0/2252973383 wait complete. 2024-03-13T14:51:54.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.871+0000 7f430b000700 1 -- 172.21.15.176:0/847026099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304100bb0 msgr2=0x7f4304100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.871+0000 7f430b000700 1 --2- 172.21.15.176:0/847026099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304100bb0 0x7f4304100fd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009910 tx=0x7f42f4009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:54.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.871+0000 7f430b000700 1 -- 172.21.15.176:0/847026099 shutdown_connections 2024-03-13T14:51:54.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.871+0000 7f430b000700 1 --2- 172.21.15.176:0/847026099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4304101db0 0x7f4304102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.871+0000 7f430b000700 1 --2- 172.21.15.176:0/847026099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304100bb0 0x7f4304100fd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.871+0000 7f430b000700 1 -- 172.21.15.176:0/847026099 >> 172.21.15.176:0/847026099 conn(0x7f43040fc110 msgr2=0x7f43040fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:54.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.871+0000 7f430b000700 1 -- 172.21.15.176:0/847026099 shutdown_connections 2024-03-13T14:51:54.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.872+0000 7f430b000700 1 -- 172.21.15.176:0/847026099 wait complete. 2024-03-13T14:51:54.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f430b000700 1 Processor -- start 2024-03-13T14:51:54.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f430b000700 1 -- start start 2024-03-13T14:51:54.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f430b000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4304100bb0 0x7f4304198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:54.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f430b000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304101db0 0x7f4304198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f430b000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4304199170 con 0x7f4304101db0 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f430b000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43041992b0 con 0x7f4304100bb0 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f4303400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304101db0 0x7f4304198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f4303400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304101db0 0x7f4304198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55788/0 (socket says 172.21.15.176:55788) 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.873+0000 7f4303400700 1 -- 172.21.15.176:0/3622572535 learned_addr learned my addr 172.21.15.176:0/3622572535 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.874+0000 7f4303e00700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4304100bb0 0x7f4304198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.874+0000 7f4303400700 1 -- 172.21.15.176:0/3622572535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4304100bb0 msgr2=0x7f4304198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.874+0000 7f4303400700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4304100bb0 0x7f4304198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:54.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.874+0000 7f4303400700 1 -- 172.21.15.176:0/3622572535 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40095f0 con 0x7f4304101db0 2024-03-13T14:51:54.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.874+0000 7f4303400700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304101db0 0x7f4304198b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b4b0 tx=0x7f42f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:54.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.875+0000 7f4300c00700 1 -- 172.21.15.176:0/3622572535 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f0011800 con 0x7f4304101db0 2024-03-13T14:51:54.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.875+0000 7f4300c00700 1 -- 172.21.15.176:0/3622572535 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f42f0011e10 con 0x7f4304101db0 2024-03-13T14:51:54.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.875+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430419dd60 con 0x7f4304101db0 2024-03-13T14:51:54.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.875+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430419e330 con 0x7f4304101db0 2024-03-13T14:51:54.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.875+0000 7f4300c00700 1 -- 172.21.15.176:0/3622572535 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f000f2e0 con 0x7f4304101db0 2024-03-13T14:51:54.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.876+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e4005320 con 0x7f4304101db0 2024-03-13T14:51:54.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.877+0000 7f4300c00700 1 -- 172.21.15.176:0/3622572535 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f42f0011960 con 0x7f4304101db0 2024-03-13T14:51:54.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.878+0000 7f4300c00700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f42e806dc20 0x7f42e80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:54.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.878+0000 7f4300c00700 1 -- 172.21.15.176:0/3622572535 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f42f008d300 con 0x7f4304101db0 2024-03-13T14:51:54.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.878+0000 7f4303e00700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f42e806dc20 0x7f42e80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:54.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.879+0000 7f4303e00700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f42e806dc20 0x7f42e80700e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42f4006010 tx=0x7f42f400b540 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:54.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:54.882+0000 7f4300c00700 1 -- 172.21.15.176:0/3622572535 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f42f005a760 con 0x7f4304101db0 2024-03-13T14:51:55.118 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.130+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f42e4000bf0 con 0x7f42e806dc20 2024-03-13T14:51:55.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.153+0000 7f4300c00700 1 -- 172.21.15.176:0/3622572535 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f42e4000bf0 con 0x7f42e806dc20 2024-03-13T14:51:55.140 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (11m) 15s ago 13m 13.7M - 0.25.0 c8568f914cd2 0c51f6123d33 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (13m) 15s ago 13m 7948k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (12m) 52s ago 12m 7868k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (13m) 15s ago 13m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (12m) 52s ago 12m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (11m) 15s ago 12m 74.7M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (6m) 15s ago 6m 125M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (6m) 15s ago 6m 14.7M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (6m) 52s ago 6m 216M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (5m) 52s ago 5m 14.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (2m) 15s ago 15m 497M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (94s) 52s ago 11m 447M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (15m) 15s ago 15m 56.0M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (11m) 52s ago 11m 44.1M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:51:55.141 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (65s) 15s ago 13m 6962k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (57s) 52s ago 11m 2847k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (10m) 15s ago 10m 218M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (10m) 15s ago 10m 229M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (9m) 15s ago 9m 198M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (9m) 52s ago 9m 247M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (8m) 52s ago 8m 220M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (7m) 52s ago 7m 194M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:51:55.142 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (24s) 15s ago 12m 41.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f42e806dc20 msgr2=0x7f42e80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f42e806dc20 0x7f42e80700e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42f4006010 tx=0x7f42f400b540 comp rx=0 tx=0).stop 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304101db0 msgr2=0x7f4304198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304101db0 0x7f4304198b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f42f000b4b0 tx=0x7f42f000b870 comp rx=0 tx=0).stop 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 shutdown_connections 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f42e806dc20 0x7f42e80700e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4304100bb0 0x7f4304198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 --2- 172.21.15.176:0/3622572535 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4304101db0 0x7f4304198b50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 >> 172.21.15.176:0/3622572535 conn(0x7f43040fc110 msgr2=0x7f4304102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:55.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.155+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 shutdown_connections 2024-03-13T14:51:55.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.156+0000 7f430b000700 1 -- 172.21.15.176:0/3622572535 wait complete. 2024-03-13T14:51:55.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.339+0000 7ffa0dc00700 1 -- 172.21.15.176:0/2409637707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa08074dc0 msgr2=0x7ffa08073220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.339+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/2409637707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa08074dc0 0x7ffa08073220 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009910 tx=0x7ff9f8009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:55.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.339+0000 7ffa0dc00700 1 -- 172.21.15.176:0/2409637707 shutdown_connections 2024-03-13T14:51:55.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.339+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/2409637707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa080737f0 0x7ffa08073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.339+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/2409637707 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa08074dc0 0x7ffa08073220 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009910 tx=0x7ff9f8009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:55.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.339+0000 7ffa0dc00700 1 -- 172.21.15.176:0/2409637707 >> 172.21.15.176:0/2409637707 conn(0x7ffa080fc110 msgr2=0x7ffa080fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:55.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.340+0000 7ffa0dc00700 1 -- 172.21.15.176:0/2409637707 shutdown_connections 2024-03-13T14:51:55.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.340+0000 7ffa0dc00700 1 -- 172.21.15.176:0/2409637707 wait complete. 2024-03-13T14:51:55.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.341+0000 7ffa0dc00700 1 Processor -- start 2024-03-13T14:51:55.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.341+0000 7ffa0dc00700 1 -- start start 2024-03-13T14:51:55.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa0dc00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa080737f0 0x7ffa0819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:55.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa0dc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa0819d120 0x7ffa081a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:55.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa0dc00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0819d630 con 0x7ffa0819d120 2024-03-13T14:51:55.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa0dc00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0819d7a0 con 0x7ffa080737f0 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa0819d120 0x7ffa081a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa080737f0 0x7ffa0819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa080737f0 0x7ffa0819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:41874/0 (socket says 172.21.15.176:41874) 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06a00700 1 -- 172.21.15.176:0/1464151820 learned_addr learned my addr 172.21.15.176:0/1464151820 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa0819d120 0x7ffa081a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55796/0 (socket says 172.21.15.176:55796) 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06000700 1 -- 172.21.15.176:0/1464151820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa080737f0 msgr2=0x7ffa0819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06000700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa080737f0 0x7ffa0819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.342+0000 7ffa06000700 1 -- 172.21.15.176:0/1464151820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80095f0 con 0x7ffa0819d120 2024-03-13T14:51:55.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.343+0000 7ffa06000700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa0819d120 0x7ffa081a2190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e940 tx=0x7ff9f400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:55.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.343+0000 7ff9ff400700 1 -- 172.21.15.176:0/1464151820 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f400eee0 con 0x7ffa0819d120 2024-03-13T14:51:55.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.343+0000 7ff9ff400700 1 -- 172.21.15.176:0/1464151820 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff9f40044e0 con 0x7ffa0819d120 2024-03-13T14:51:55.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.343+0000 7ff9ff400700 1 -- 172.21.15.176:0/1464151820 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f4003d90 con 0x7ffa0819d120 2024-03-13T14:51:55.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.343+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa081a2730 con 0x7ffa0819d120 2024-03-13T14:51:55.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.344+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa081a2c80 con 0x7ffa0819d120 2024-03-13T14:51:55.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.345+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa08107a10 con 0x7ffa0819d120 2024-03-13T14:51:55.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.348+0000 7ff9ff400700 1 -- 172.21.15.176:0/1464151820 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7ff9f400cba0 con 0x7ffa0819d120 2024-03-13T14:51:55.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.349+0000 7ff9ff400700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff9ec06dc70 0x7ff9ec070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:55.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.349+0000 7ff9ff400700 1 -- 172.21.15.176:0/1464151820 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff9f4014070 con 0x7ffa0819d120 2024-03-13T14:51:55.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.349+0000 7ffa06a00700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff9ec06dc70 0x7ff9ec070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:55.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.350+0000 7ffa06a00700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff9ec06dc70 0x7ff9ec070130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f8005fd0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:55.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.364+0000 7ff9ff400700 1 -- 172.21.15.176:0/1464151820 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7ff9f405b030 con 0x7ffa0819d120 2024-03-13T14:51:55.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:55 smithi176 ceph-mon[29064]: pgmap v76: 65 pgs: 65 active+clean; 245 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-03-13T14:51:55.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:55 smithi176 ceph-mon[29064]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:55.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:55 smithi196 ceph-mon[37486]: pgmap v76: 65 pgs: 65 active+clean; 245 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 290 op/s 2024-03-13T14:51:55.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:55 smithi196 ceph-mon[37486]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:55.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.716+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffa0802cc70 con 0x7ffa0819d120 2024-03-13T14:51:55.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.717+0000 7ff9ff400700 1 -- 172.21.15.176:0/1464151820 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7ff9f405abc0 con 0x7ffa0819d120 2024-03-13T14:51:55.704 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:51:55.704 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:51:55.705 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:51:55.706 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff9ec06dc70 msgr2=0x7ff9ec070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff9ec06dc70 0x7ff9ec070130 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009fd0 tx=0x7ff9f8005fd0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa0819d120 msgr2=0x7ffa081a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa0819d120 0x7ffa081a2190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e940 tx=0x7ff9f400ed00 comp rx=0 tx=0).stop 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 shutdown_connections 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ff9ec06dc70 0x7ff9ec070130 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa080737f0 0x7ffa0819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 --2- 172.21.15.176:0/1464151820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffa0819d120 0x7ffa081a2190 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 >> 172.21.15.176:0/1464151820 conn(0x7ffa080fc110 msgr2=0x7ffa08104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:55.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 shutdown_connections 2024-03-13T14:51:55.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.720+0000 7ffa0dc00700 1 -- 172.21.15.176:0/1464151820 wait complete. 2024-03-13T14:51:55.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.877+0000 7ffb37000700 1 -- 172.21.15.176:0/3727373714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30103ff0 msgr2=0x7ffb30104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.877+0000 7ffb37000700 1 --2- 172.21.15.176:0/3727373714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30103ff0 0x7ffb30104450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009910 tx=0x7ffb1c009c20 comp rx=0 tx=0).stop 2024-03-13T14:51:55.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.878+0000 7ffb37000700 1 -- 172.21.15.176:0/3727373714 shutdown_connections 2024-03-13T14:51:55.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.878+0000 7ffb37000700 1 --2- 172.21.15.176:0/3727373714 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30103ff0 0x7ffb30104450 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.878+0000 7ffb37000700 1 --2- 172.21.15.176:0/3727373714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb30102df0 0x7ffb30103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.878+0000 7ffb37000700 1 -- 172.21.15.176:0/3727373714 >> 172.21.15.176:0/3727373714 conn(0x7ffb300fe390 msgr2=0x7ffb301007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:55.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.878+0000 7ffb37000700 1 -- 172.21.15.176:0/3727373714 shutdown_connections 2024-03-13T14:51:55.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.878+0000 7ffb37000700 1 -- 172.21.15.176:0/3727373714 wait complete. 2024-03-13T14:51:55.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.879+0000 7ffb37000700 1 Processor -- start 2024-03-13T14:51:55.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.879+0000 7ffb37000700 1 -- start start 2024-03-13T14:51:55.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb37000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30102df0 0x7ffb301987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:55.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb37000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb30198d00 0x7ffb3019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:55.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb37000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30199210 con 0x7ffb30102df0 2024-03-13T14:51:55.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb37000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb30199380 con 0x7ffb30198d00 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb2fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30102df0 0x7ffb301987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb2fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30102df0 0x7ffb301987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55822/0 (socket says 172.21.15.176:55822) 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb2fe00700 1 -- 172.21.15.176:0/2235538566 learned_addr learned my addr 172.21.15.176:0/2235538566 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb2f400700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb30198d00 0x7ffb3019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb2fe00700 1 -- 172.21.15.176:0/2235538566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb30198d00 msgr2=0x7ffb3019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.880+0000 7ffb2fe00700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb30198d00 0x7ffb3019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.881+0000 7ffb2fe00700 1 -- 172.21.15.176:0/2235538566 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0095f0 con 0x7ffb30102df0 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.881+0000 7ffb2fe00700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30102df0 0x7ffb301987c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000e8f0 tx=0x7ffb2000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.881+0000 7ffb2cc00700 1 -- 172.21.15.176:0/2235538566 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb20009790 con 0x7ffb30102df0 2024-03-13T14:51:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.881+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3019e310 con 0x7ffb30102df0 2024-03-13T14:51:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.881+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3019e860 con 0x7ffb30102df0 2024-03-13T14:51:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.881+0000 7ffb2cc00700 1 -- 172.21.15.176:0/2235538566 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb200044d0 con 0x7ffb30102df0 2024-03-13T14:51:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.881+0000 7ffb2cc00700 1 -- 172.21.15.176:0/2235538566 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb20003da0 con 0x7ffb30102df0 2024-03-13T14:51:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.883+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb10005320 con 0x7ffb30102df0 2024-03-13T14:51:55.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.884+0000 7ffb2cc00700 1 -- 172.21.15.176:0/2235538566 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7ffb2000cba0 con 0x7ffb30102df0 2024-03-13T14:51:55.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.885+0000 7ffb2cc00700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ffb1406dc20 0x7ffb140700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:51:55.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.885+0000 7ffb2cc00700 1 -- 172.21.15.176:0/2235538566 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ffb20014070 con 0x7ffb30102df0 2024-03-13T14:51:55.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.885+0000 7ffb2f400700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ffb1406dc20 0x7ffb140700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:51:55.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.886+0000 7ffb2f400700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ffb1406dc20 0x7ffb140700e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00b6f0 tx=0x7ffb1c009f70 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:51:55.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:55.890+0000 7ffb2cc00700 1 -- 172.21.15.176:0/2235538566 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7ffb2005aff0 con 0x7ffb30102df0 2024-03-13T14:51:56.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.156+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb10000bf0 con 0x7ffb1406dc20 2024-03-13T14:51:56.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.158+0000 7ffb2cc00700 1 -- 172.21.15.176:0/2235538566 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7ffb10000bf0 con 0x7ffb1406dc20 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:51:56.146 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "2/2 daemons upgraded", 2024-03-13T14:51:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading alertmanager daemons", 2024-03-13T14:51:56.147 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:51:56.147 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:51:56.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.160+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ffb1406dc20 msgr2=0x7ffb140700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:56.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.160+0000 7ffb37000700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ffb1406dc20 0x7ffb140700e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00b6f0 tx=0x7ffb1c009f70 comp rx=0 tx=0).stop 2024-03-13T14:51:56.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.160+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30102df0 msgr2=0x7ffb301987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:51:56.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.160+0000 7ffb37000700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30102df0 0x7ffb301987c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000e8f0 tx=0x7ffb2000ecb0 comp rx=0 tx=0).stop 2024-03-13T14:51:56.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.161+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 shutdown_connections 2024-03-13T14:51:56.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.161+0000 7ffb37000700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7ffb1406dc20 0x7ffb140700e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:56.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.161+0000 7ffb37000700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb30102df0 0x7ffb301987c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:56.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.161+0000 7ffb37000700 1 --2- 172.21.15.176:0/2235538566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb30198d00 0x7ffb3019dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:51:56.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.161+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 >> 172.21.15.176:0/2235538566 conn(0x7ffb300fe390 msgr2=0x7ffb30107220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:51:56.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.161+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 shutdown_connections 2024-03-13T14:51:56.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:51:56.161+0000 7ffb37000700 1 -- 172.21.15.176:0/2235538566 wait complete. 2024-03-13T14:51:56.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:56 smithi176 ceph-mon[29064]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:56.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:56 smithi176 ceph-mon[29064]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:56.413 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:56 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/1464151820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:56 smithi196 ceph-mon[37486]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:56 smithi196 ceph-mon[37486]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:56 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1464151820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:51:57.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:57 smithi176 ceph-mon[29064]: pgmap v77: 65 pgs: 65 active+clean; 236 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 286 op/s 2024-03-13T14:51:57.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:57 smithi176 ceph-mon[29064]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:57.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:57 smithi196 ceph-mon[37486]: pgmap v77: 65 pgs: 65 active+clean; 236 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 286 op/s 2024-03-13T14:51:57.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:57 smithi196 ceph-mon[37486]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:51:59.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:58 smithi176 ceph-mon[29064]: pgmap v78: 65 pgs: 65 active+clean; 236 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 693 KiB/s wr, 215 op/s 2024-03-13T14:51:59.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:59.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:59.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:59.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:51:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:51:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:58 smithi196 ceph-mon[37486]: pgmap v78: 65 pgs: 65 active+clean; 236 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 655 KiB/s rd, 693 KiB/s wr, 215 op/s 2024-03-13T14:51:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:51:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:51:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:51:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:01 smithi196 ceph-mon[37486]: pgmap v79: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 299 op/s 2024-03-13T14:52:01.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:01 smithi176 ceph-mon[29064]: pgmap v79: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 299 op/s 2024-03-13T14:52:03.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:03 smithi196 ceph-mon[37486]: pgmap v80: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 713 KiB/s rd, 691 KiB/s wr, 186 op/s 2024-03-13T14:52:03.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:03 smithi176 ceph-mon[29064]: pgmap v80: 65 pgs: 65 active+clean; 234 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 713 KiB/s rd, 691 KiB/s wr, 186 op/s 2024-03-13T14:52:05.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:05 smithi196 ceph-mon[37486]: pgmap v81: 65 pgs: 65 active+clean; 230 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 725 KiB/s rd, 704 KiB/s wr, 236 op/s 2024-03-13T14:52:05.478 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:05 smithi176 ceph-mon[29064]: pgmap v81: 65 pgs: 65 active+clean; 230 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 725 KiB/s rd, 704 KiB/s wr, 236 op/s 2024-03-13T14:52:07.396 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-03-13T14:52:07.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-03-13T14:52:07.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:07 smithi196 ceph-mon[37486]: pgmap v82: 65 pgs: 65 active+clean; 230 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 716 KiB/s rd, 763 KiB/s wr, 253 op/s 2024-03-13T14:52:07.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:07 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:07.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:07 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:07.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:07 smithi176 ceph-mon[29064]: pgmap v82: 65 pgs: 65 active+clean; 230 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 716 KiB/s rd, 763 KiB/s wr, 253 op/s 2024-03-13T14:52:07.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:07 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:07.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:07 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:08.539 DEBUG:teuthology.parallel:result is None 2024-03-13T14:52:09.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:09 smithi196 ceph-mon[37486]: pgmap v83: 65 pgs: 65 active+clean; 230 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 716 KiB/s rd, 762 KiB/s wr, 212 op/s 2024-03-13T14:52:09.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:09 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:09.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:09 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:09 smithi176 ceph-mon[29064]: pgmap v83: 65 pgs: 65 active+clean; 230 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 716 KiB/s rd, 762 KiB/s wr, 212 op/s 2024-03-13T14:52:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:09 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:09 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:11.319 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:11 smithi176 ceph-mon[29064]: pgmap v84: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 307 op/s 2024-03-13T14:52:11.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:11 smithi196 ceph-mon[37486]: pgmap v84: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 307 op/s 2024-03-13T14:52:13.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:13 smithi196 ceph-mon[37486]: pgmap v85: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 681 KiB/s rd, 714 KiB/s wr, 222 op/s 2024-03-13T14:52:13.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:52:13.455 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:13 smithi176 ceph-mon[29064]: pgmap v85: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 681 KiB/s rd, 714 KiB/s wr, 222 op/s 2024-03-13T14:52:13.455 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: pgmap v86: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 862 KiB/s rd, 891 KiB/s wr, 262 op/s 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:52:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:15 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: pgmap v86: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 862 KiB/s rd, 891 KiB/s wr, 262 op/s 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:15.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:15 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:16.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:16 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:52:16.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:16 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-13T14:52:17.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:17 smithi176 ceph-mon[29064]: pgmap v87: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 850 KiB/s rd, 878 KiB/s wr, 234 op/s 2024-03-13T14:52:17.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:17 smithi196 ceph-mon[37486]: pgmap v87: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 850 KiB/s rd, 878 KiB/s wr, 234 op/s 2024-03-13T14:52:18.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:18 smithi176 ceph-mon[29064]: Upgrade: Updating grafana.smithi176 2024-03-13T14:52:18.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:18 smithi176 ceph-mon[29064]: Deploying daemon grafana.smithi176 on smithi176 2024-03-13T14:52:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:18 smithi196 ceph-mon[37486]: Upgrade: Updating grafana.smithi176 2024-03-13T14:52:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:18 smithi196 ceph-mon[37486]: Deploying daemon grafana.smithi176 on smithi176 2024-03-13T14:52:19.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:19 smithi176 ceph-mon[29064]: pgmap v88: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 513 KiB/s rd, 486 KiB/s wr, 157 op/s 2024-03-13T14:52:19.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:19 smithi196 ceph-mon[37486]: pgmap v88: 65 pgs: 65 active+clean; 217 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 513 KiB/s rd, 486 KiB/s wr, 157 op/s 2024-03-13T14:52:21.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:21 smithi176 ceph-mon[29064]: pgmap v89: 65 pgs: 65 active+clean; 212 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 682 KiB/s rd, 659 KiB/s wr, 211 op/s 2024-03-13T14:52:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:21 smithi196 ceph-mon[37486]: pgmap v89: 65 pgs: 65 active+clean; 212 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 682 KiB/s rd, 659 KiB/s wr, 211 op/s 2024-03-13T14:52:23.497 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:23 smithi176 ceph-mon[29064]: pgmap v90: 65 pgs: 65 active+clean; 212 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 351 KiB/s rd, 350 KiB/s wr, 116 op/s 2024-03-13T14:52:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:23 smithi196 ceph-mon[37486]: pgmap v90: 65 pgs: 65 active+clean; 212 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 351 KiB/s rd, 350 KiB/s wr, 116 op/s 2024-03-13T14:52:24.805 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-03-13T14:52:24.806 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-13T14:52:25.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:25 smithi176 ceph-mon[29064]: pgmap v91: 65 pgs: 65 active+clean; 211 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 499 KiB/s rd, 529 KiB/s wr, 147 op/s 2024-03-13T14:52:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:25 smithi196 ceph-mon[37486]: pgmap v91: 65 pgs: 65 active+clean; 211 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 499 KiB/s rd, 529 KiB/s wr, 147 op/s 2024-03-13T14:52:26.236 DEBUG:teuthology.parallel:result is None 2024-03-13T14:52:26.236 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-13T14:52:26.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.338+0000 7f1a01800700 1 -- 172.21.15.176:0/888005450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc101db0 msgr2=0x7f19fc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:26.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.338+0000 7f1a01800700 1 --2- 172.21.15.176:0/888005450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc101db0 0x7f19fc102210 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009910 tx=0x7f19e8009c20 comp rx=0 tx=0).stop 2024-03-13T14:52:26.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.339+0000 7f1a01800700 1 -- 172.21.15.176:0/888005450 shutdown_connections 2024-03-13T14:52:26.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.339+0000 7f1a01800700 1 --2- 172.21.15.176:0/888005450 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc101db0 0x7f19fc102210 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.339+0000 7f1a01800700 1 --2- 172.21.15.176:0/888005450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc100bb0 0x7f19fc100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.339+0000 7f1a01800700 1 -- 172.21.15.176:0/888005450 >> 172.21.15.176:0/888005450 conn(0x7f19fc0fc110 msgr2=0x7f19fc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:26.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.339+0000 7f1a01800700 1 -- 172.21.15.176:0/888005450 shutdown_connections 2024-03-13T14:52:26.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.339+0000 7f1a01800700 1 -- 172.21.15.176:0/888005450 wait complete. 2024-03-13T14:52:26.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.341+0000 7f1a01800700 1 Processor -- start 2024-03-13T14:52:26.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.341+0000 7f1a01800700 1 -- start start 2024-03-13T14:52:26.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.341+0000 7f1a01800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc100bb0 0x7f19fc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:26.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.341+0000 7f1a01800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc101db0 0x7f19fc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:26.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.341+0000 7f1a01800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc199140 con 0x7f19fc100bb0 2024-03-13T14:52:26.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.341+0000 7f1a01800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc199280 con 0x7f19fc101db0 2024-03-13T14:52:26.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.342+0000 7f19faa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc100bb0 0x7f19fc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:26.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.342+0000 7f19faa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc100bb0 0x7f19fc1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37994/0 (socket says 172.21.15.176:37994) 2024-03-13T14:52:26.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.342+0000 7f19faa00700 1 -- 172.21.15.176:0/768257059 learned_addr learned my addr 172.21.15.176:0/768257059 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:52:26.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.342+0000 7f19fa000700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc101db0 0x7f19fc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:26.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.342+0000 7f19faa00700 1 -- 172.21.15.176:0/768257059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc101db0 msgr2=0x7f19fc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:26.330 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.342+0000 7f19faa00700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc101db0 0x7f19fc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.342+0000 7f19faa00700 1 -- 172.21.15.176:0/768257059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80095f0 con 0x7f19fc100bb0 2024-03-13T14:52:26.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.343+0000 7f19faa00700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc100bb0 0x7f19fc1985e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b480 tx=0x7f19ec00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:26.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.343+0000 7f19f3400700 1 -- 172.21.15.176:0/768257059 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19ec011800 con 0x7f19fc100bb0 2024-03-13T14:52:26.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.343+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc19dd30 con 0x7f19fc100bb0 2024-03-13T14:52:26.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.343+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc19e2a0 con 0x7f19fc100bb0 2024-03-13T14:52:26.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.343+0000 7f19f3400700 1 -- 172.21.15.176:0/768257059 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f19ec011e10 con 0x7f19fc100bb0 2024-03-13T14:52:26.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.343+0000 7f19f3400700 1 -- 172.21.15.176:0/768257059 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19ec010ea0 con 0x7f19fc100bb0 2024-03-13T14:52:26.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.345+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19fc105e90 con 0x7f19fc100bb0 2024-03-13T14:52:26.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.348+0000 7f19f3400700 1 -- 172.21.15.176:0/768257059 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f19ec011960 con 0x7f19fc100bb0 2024-03-13T14:52:26.336 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-13T14:52:26.337 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-03-13T14:52:26.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.349+0000 7f19f3400700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f19e006dc20 0x7f19e00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:26.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.349+0000 7f19f3400700 1 -- 172.21.15.176:0/768257059 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f19ec08d890 con 0x7f19fc100bb0 2024-03-13T14:52:26.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.349+0000 7f19fa000700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f19e006dc20 0x7f19e00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:26.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.350+0000 7f19fa000700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f19e006dc20 0x7f19e00700e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19e8006010 tx=0x7f19e800b560 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:26.339 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.352+0000 7f19f3400700 1 -- 172.21.15.176:0/768257059 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f19ec05ad70 con 0x7f19fc100bb0 2024-03-13T14:52:26.448 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-03-13T14:52:26.449 DEBUG:teuthology.parallel:result is None 2024-03-13T14:52:26.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.672+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19fc0611d0 con 0x7f19e006dc20 2024-03-13T14:52:26.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.673+0000 7f19f3400700 1 -- 172.21.15.176:0/768257059 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f19fc0611d0 con 0x7f19e006dc20 2024-03-13T14:52:26.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f19e006dc20 msgr2=0x7f19e00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:26.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f19e006dc20 0x7f19e00700e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19e8006010 tx=0x7f19e800b560 comp rx=0 tx=0).stop 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc100bb0 msgr2=0x7f19fc1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc100bb0 0x7f19fc1985e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b480 tx=0x7f19ec00b790 comp rx=0 tx=0).stop 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 shutdown_connections 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f19e006dc20 0x7f19e00700e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19fc100bb0 0x7f19fc1985e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 --2- 172.21.15.176:0/768257059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19fc101db0 0x7f19fc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 >> 172.21.15.176:0/768257059 conn(0x7f19fc0fc110 msgr2=0x7f19fc102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 shutdown_connections 2024-03-13T14:52:26.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.676+0000 7f1a01800700 1 -- 172.21.15.176:0/768257059 wait complete. 2024-03-13T14:52:26.679 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:52:26.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.852+0000 7f3bcae00700 1 -- 172.21.15.176:0/3290328494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4100bb0 msgr2=0x7f3bc4100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:26.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.852+0000 7f3bcae00700 1 --2- 172.21.15.176:0/3290328494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4100bb0 0x7f3bc4100fd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009910 tx=0x7f3bb4009c20 comp rx=0 tx=0).stop 2024-03-13T14:52:26.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.853+0000 7f3bcae00700 1 -- 172.21.15.176:0/3290328494 shutdown_connections 2024-03-13T14:52:26.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.853+0000 7f3bcae00700 1 --2- 172.21.15.176:0/3290328494 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc4101db0 0x7f3bc4102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.853+0000 7f3bcae00700 1 --2- 172.21.15.176:0/3290328494 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4100bb0 0x7f3bc4100fd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.853+0000 7f3bcae00700 1 -- 172.21.15.176:0/3290328494 >> 172.21.15.176:0/3290328494 conn(0x7f3bc40fc110 msgr2=0x7f3bc40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:26.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.853+0000 7f3bcae00700 1 -- 172.21.15.176:0/3290328494 shutdown_connections 2024-03-13T14:52:26.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.853+0000 7f3bcae00700 1 -- 172.21.15.176:0/3290328494 wait complete. 2024-03-13T14:52:26.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.854+0000 7f3bcae00700 1 Processor -- start 2024-03-13T14:52:26.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.854+0000 7f3bcae00700 1 -- start start 2024-03-13T14:52:26.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bcae00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc4100bb0 0x7f3bc41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:26.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bcae00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4101db0 0x7f3bc4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:26.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bcae00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4199150 con 0x7f3bc4101db0 2024-03-13T14:52:26.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bcae00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4199290 con 0x7f3bc4100bb0 2024-03-13T14:52:26.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4101db0 0x7f3bc4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:26.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4101db0 0x7f3bc4198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:38018/0 (socket says 172.21.15.176:38018) 2024-03-13T14:52:26.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3400700 1 -- 172.21.15.176:0/650121735 learned_addr learned my addr 172.21.15.176:0/650121735 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:52:26.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3e00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc4100bb0 0x7f3bc41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:26.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3400700 1 -- 172.21.15.176:0/650121735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc4100bb0 msgr2=0x7f3bc41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:26.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3400700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc4100bb0 0x7f3bc41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:26.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3400700 1 -- 172.21.15.176:0/650121735 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb40095f0 con 0x7f3bc4101db0 2024-03-13T14:52:26.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.855+0000 7f3bc3e00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc4100bb0 0x7f3bc41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:52:26.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.856+0000 7f3bc3400700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4101db0 0x7f3bc4198b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000b4b0 tx=0x7f3bb000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:26.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.856+0000 7f3bc0c00700 1 -- 172.21.15.176:0/650121735 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb0011800 con 0x7f3bc4101db0 2024-03-13T14:52:26.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.856+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc419dd40 con 0x7f3bc4101db0 2024-03-13T14:52:26.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.856+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc419e310 con 0x7f3bc4101db0 2024-03-13T14:52:26.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.857+0000 7f3bc0c00700 1 -- 172.21.15.176:0/650121735 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3bb0011e10 con 0x7f3bc4101db0 2024-03-13T14:52:26.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.857+0000 7f3bc0c00700 1 -- 172.21.15.176:0/650121735 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3bb000f2e0 con 0x7f3bc4101db0 2024-03-13T14:52:26.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.858+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4005320 con 0x7f3bc4101db0 2024-03-13T14:52:26.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.859+0000 7f3bc0c00700 1 -- 172.21.15.176:0/650121735 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f3bb0011960 con 0x7f3bc4101db0 2024-03-13T14:52:26.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.860+0000 7f3bc0c00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ba806dc20 0x7f3ba80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:26.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.860+0000 7f3bc0c00700 1 -- 172.21.15.176:0/650121735 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3bb008d300 con 0x7f3bc4101db0 2024-03-13T14:52:26.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.860+0000 7f3bc3e00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ba806dc20 0x7f3ba80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:26.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.861+0000 7f3bc3e00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ba806dc20 0x7f3ba80700e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb400b540 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:26.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:26.869+0000 7f3bc0c00700 1 -- 172.21.15.176:0/650121735 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f3bb005a760 con 0x7f3bc4101db0 2024-03-13T14:52:27.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.134+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ba4000bf0 con 0x7f3ba806dc20 2024-03-13T14:52:27.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.136+0000 7f3bc0c00700 1 -- 172.21.15.176:0/650121735 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f3ba4000bf0 con 0x7f3ba806dc20 2024-03-13T14:52:27.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ba806dc20 msgr2=0x7f3ba80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ba806dc20 0x7f3ba80700e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006010 tx=0x7f3bb400b540 comp rx=0 tx=0).stop 2024-03-13T14:52:27.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4101db0 msgr2=0x7f3bc4198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4101db0 0x7f3bc4198b30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000b4b0 tx=0x7f3bb000b870 comp rx=0 tx=0).stop 2024-03-13T14:52:27.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 shutdown_connections 2024-03-13T14:52:27.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ba806dc20 0x7f3ba80700e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bc4100bb0 0x7f3bc41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 --2- 172.21.15.176:0/650121735 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3bc4101db0 0x7f3bc4198b30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 >> 172.21.15.176:0/650121735 conn(0x7f3bc40fc110 msgr2=0x7f3bc4102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:27.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 shutdown_connections 2024-03-13T14:52:27.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.140+0000 7f3bcae00700 1 -- 172.21.15.176:0/650121735 wait complete. 2024-03-13T14:52:27.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 -- 172.21.15.176:0/2102486680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 msgr2=0x7fdbb0102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 --2- 172.21.15.176:0/2102486680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb0102060 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009910 tx=0x7fdba0009c20 comp rx=0 tx=0).stop 2024-03-13T14:52:27.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 -- 172.21.15.176:0/2102486680 shutdown_connections 2024-03-13T14:52:27.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 --2- 172.21.15.176:0/2102486680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb0102060 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 --2- 172.21.15.176:0/2102486680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0100a00 0x7fdbb0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 -- 172.21.15.176:0/2102486680 >> 172.21.15.176:0/2102486680 conn(0x7fdbb00fbf60 msgr2=0x7fdbb00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:27.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 -- 172.21.15.176:0/2102486680 shutdown_connections 2024-03-13T14:52:27.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.313+0000 7fdbb7200700 1 -- 172.21.15.176:0/2102486680 wait complete. 2024-03-13T14:52:27.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.314+0000 7fdbb7200700 1 Processor -- start 2024-03-13T14:52:27.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb7200700 1 -- start start 2024-03-13T14:52:27.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb7200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0100a00 0x7fdbb0196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb7200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb01966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb7200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb0196cf0 con 0x7fdbb0101c00 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb7200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb0196e30 con 0x7fdbb0100a00 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb5400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb01966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb5400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb01966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:38038/0 (socket says 172.21.15.176:38038) 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.315+0000 7fdbb5400700 1 -- 172.21.15.176:0/3438511626 learned_addr learned my addr 172.21.15.176:0/3438511626 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbb5e00700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0100a00 0x7fdbb0196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbb5400700 1 -- 172.21.15.176:0/3438511626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0100a00 msgr2=0x7fdbb0196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbb5400700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0100a00 0x7fdbb0196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbb5400700 1 -- 172.21.15.176:0/3438511626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00095f0 con 0x7fdbb0101c00 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbb5e00700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0100a00 0x7fdbb0196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbb5400700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb01966d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdba0005b40 tx=0x7fdba000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:27.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbaaa00700 1 -- 172.21.15.176:0/3438511626 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba001d070 con 0x7fdbb0101c00 2024-03-13T14:52:27.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbaaa00700 1 -- 172.21.15.176:0/3438511626 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdba0022470 con 0x7fdbb0101c00 2024-03-13T14:52:27.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbaaa00700 1 -- 172.21.15.176:0/3438511626 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba000f690 con 0x7fdbb0101c00 2024-03-13T14:52:27.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.316+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb019b880 con 0x7fdbb0101c00 2024-03-13T14:52:27.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.317+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb019bd70 con 0x7fdbb0101c00 2024-03-13T14:52:27.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.318+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb0109560 con 0x7fdbb0101c00 2024-03-13T14:52:27.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.319+0000 7fdbaaa00700 1 -- 172.21.15.176:0/3438511626 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7fdba00225e0 con 0x7fdbb0101c00 2024-03-13T14:52:27.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.320+0000 7fdbaaa00700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fdb9806dc20 0x7fdb980700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:27.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.320+0000 7fdbaaa00700 1 -- 172.21.15.176:0/3438511626 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdba008efd0 con 0x7fdbb0101c00 2024-03-13T14:52:27.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.320+0000 7fdbb5e00700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fdb9806dc20 0x7fdb980700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:27.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.321+0000 7fdbb5e00700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fdb9806dc20 0x7fdb980700e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdba4005fd0 tx=0x7fdba4005dc0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:27.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.325+0000 7fdbaaa00700 1 -- 172.21.15.176:0/3438511626 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fdba00c2f30 con 0x7fdbb0101c00 2024-03-13T14:52:27.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:27 smithi176 ceph-mon[29064]: pgmap v92: 65 pgs: 65 active+clean; 210 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 318 KiB/s rd, 371 KiB/s wr, 127 op/s 2024-03-13T14:52:27.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.606+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbb0106550 con 0x7fdb9806dc20 2024-03-13T14:52:27.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.634+0000 7fdbaaa00700 1 -- 172.21.15.176:0/3438511626 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdbb0106550 con 0x7fdb9806dc20 2024-03-13T14:52:27.621 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (30s) 20s ago 13m 13.8M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (14m) 20s ago 14m 8014k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (12m) 84s ago 12m 7868k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (14m) 20s ago 14m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (12m) 84s ago 12m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (11m) 20s ago 13m 74.5M - 9.4.7 954c08fa6188 608dd2d778f7 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (6m) 20s ago 6m 106M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (6m) 20s ago 6m 14.9M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (6m) 84s ago 6m 216M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (6m) 84s ago 6m 14.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (3m) 20s ago 15m 498M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (2m) 84s ago 12m 447M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:52:27.622 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (15m) 20s ago 15m 57.1M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (12m) 84s ago 12m 44.1M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (97s) 20s ago 14m 7134k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (90s) 84s ago 12m 2847k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (11m) 20s ago 11m 223M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (10m) 20s ago 10m 237M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (10m) 20s ago 10m 208M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (9m) 84s ago 9m 247M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (9m) 84s ago 9m 220M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (8m) 84s ago 8m 194M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:52:27.623 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (57s) 20s ago 13m 53.0M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:52:27.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.637+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fdb9806dc20 msgr2=0x7fdb980700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.637+0000 7fdbb7200700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fdb9806dc20 0x7fdb980700e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdba4005fd0 tx=0x7fdba4005dc0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.637+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 msgr2=0x7fdbb01966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.624 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.637+0000 7fdbb7200700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb01966d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fdba0005b40 tx=0x7fdba000be20 comp rx=0 tx=0).stop 2024-03-13T14:52:27.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.638+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 shutdown_connections 2024-03-13T14:52:27.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.638+0000 7fdbb7200700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fdb9806dc20 0x7fdb980700e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.638+0000 7fdbb7200700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbb0100a00 0x7fdbb0196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.638+0000 7fdbb7200700 1 --2- 172.21.15.176:0/3438511626 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdbb0101c00 0x7fdbb01966d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.638+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 >> 172.21.15.176:0/3438511626 conn(0x7fdbb00fbf60 msgr2=0x7fdbb0104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:27.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.638+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 shutdown_connections 2024-03-13T14:52:27.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.638+0000 7fdbb7200700 1 -- 172.21.15.176:0/3438511626 wait complete. 2024-03-13T14:52:27.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:27 smithi196 ceph-mon[37486]: pgmap v92: 65 pgs: 65 active+clean; 210 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 318 KiB/s rd, 371 KiB/s wr, 127 op/s 2024-03-13T14:52:27.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.797+0000 7f1b01800700 1 -- 172.21.15.176:0/3131738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc101db0 msgr2=0x7f1afc102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.797+0000 7f1b01800700 1 --2- 172.21.15.176:0/3131738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc101db0 0x7f1afc102210 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009910 tx=0x7f1ae8009c20 comp rx=0 tx=0).stop 2024-03-13T14:52:27.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.798+0000 7f1b01800700 1 -- 172.21.15.176:0/3131738064 shutdown_connections 2024-03-13T14:52:27.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.798+0000 7f1b01800700 1 --2- 172.21.15.176:0/3131738064 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc101db0 0x7f1afc102210 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.798+0000 7f1b01800700 1 --2- 172.21.15.176:0/3131738064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1afc100bb0 0x7f1afc100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.798+0000 7f1b01800700 1 -- 172.21.15.176:0/3131738064 >> 172.21.15.176:0/3131738064 conn(0x7f1afc0fc110 msgr2=0x7f1afc0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:27.785 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.798+0000 7f1b01800700 1 -- 172.21.15.176:0/3131738064 shutdown_connections 2024-03-13T14:52:27.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.798+0000 7f1b01800700 1 -- 172.21.15.176:0/3131738064 wait complete. 2024-03-13T14:52:27.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.799+0000 7f1b01800700 1 Processor -- start 2024-03-13T14:52:27.786 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.799+0000 7f1b01800700 1 -- start start 2024-03-13T14:52:27.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1b01800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc100bb0 0x7f1afc196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:27.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1b01800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1afc101db0 0x7f1afc1968a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:27.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1b01800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc196ec0 con 0x7f1afc100bb0 2024-03-13T14:52:27.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1b01800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1afc197000 con 0x7f1afc101db0 2024-03-13T14:52:27.787 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1afaa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc100bb0 0x7f1afc196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1afaa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc100bb0 0x7f1afc196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:38052/0 (socket says 172.21.15.176:38052) 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1afaa00700 1 -- 172.21.15.176:0/3891037921 learned_addr learned my addr 172.21.15.176:0/3891037921 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.800+0000 7f1afa000700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1afc101db0 0x7f1afc1968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1afaa00700 1 -- 172.21.15.176:0/3891037921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1afc101db0 msgr2=0x7f1afc1968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1afaa00700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1afc101db0 0x7f1afc1968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1afaa00700 1 -- 172.21.15.176:0/3891037921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae80095f0 con 0x7f1afc100bb0 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1afa000700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1afc101db0 0x7f1afc1968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1afaa00700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc100bb0 0x7f1afc196360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00d900 tx=0x7f1aec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1af3400700 1 -- 172.21.15.176:0/3891037921 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aec0041f0 con 0x7f1afc100bb0 2024-03-13T14:52:27.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1afc19bab0 con 0x7f1afc100bb0 2024-03-13T14:52:27.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1af3400700 1 -- 172.21.15.176:0/3891037921 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1aec004350 con 0x7f1afc100bb0 2024-03-13T14:52:27.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1afc19c000 con 0x7f1afc100bb0 2024-03-13T14:52:27.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.801+0000 7f1af3400700 1 -- 172.21.15.176:0/3891037921 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aec003da0 con 0x7f1afc100bb0 2024-03-13T14:52:27.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.804+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc005320 con 0x7f1afc100bb0 2024-03-13T14:52:27.791 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.804+0000 7f1af3400700 1 -- 172.21.15.176:0/3891037921 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f1aec009730 con 0x7f1afc100bb0 2024-03-13T14:52:27.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.805+0000 7f1af3400700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f1ae006dc20 0x7f1ae00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:27.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.805+0000 7f1af3400700 1 -- 172.21.15.176:0/3891037921 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1aec08d830 con 0x7f1afc100bb0 2024-03-13T14:52:27.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.806+0000 7f1afa000700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f1ae006dc20 0x7f1ae00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:27.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.807+0000 7f1afa000700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f1ae006dc20 0x7f1ae00700e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006010 tx=0x7f1ae800b580 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:27.798 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:27.811+0000 7f1af3400700 1 -- 172.21.15.176:0/3891037921 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f1aec05ad30 con 0x7f1afc100bb0 2024-03-13T14:52:28.177 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.189+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1adc006200 con 0x7f1afc100bb0 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.190+0000 7f1af3400700 1 -- 172.21.15.176:0/3891037921 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f1aec05a8c0 con 0x7f1afc100bb0 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:52:28.178 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:52:28.179 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:52:28.181 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.193+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f1ae006dc20 msgr2=0x7f1ae00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:28.181 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.193+0000 7f1b01800700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f1ae006dc20 0x7f1ae00700e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006010 tx=0x7f1ae800b580 comp rx=0 tx=0).stop 2024-03-13T14:52:28.181 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.193+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc100bb0 msgr2=0x7f1afc196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.193+0000 7f1b01800700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc100bb0 0x7f1afc196360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00d900 tx=0x7f1aec00dcc0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.194+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 shutdown_connections 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.194+0000 7f1b01800700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f1ae006dc20 0x7f1ae00700e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.194+0000 7f1b01800700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1afc100bb0 0x7f1afc196360 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.194+0000 7f1b01800700 1 --2- 172.21.15.176:0/3891037921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1afc101db0 0x7f1afc1968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.194+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 >> 172.21.15.176:0/3891037921 conn(0x7f1afc0fc110 msgr2=0x7f1afc104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.194+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 shutdown_connections 2024-03-13T14:52:28.182 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.194+0000 7f1b01800700 1 -- 172.21.15.176:0/3891037921 wait complete. 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.356+0000 7fd470a00700 1 -- 172.21.15.176:0/1464873721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 msgr2=0x7fd46c102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.356+0000 7fd470a00700 1 --2- 172.21.15.176:0/1464873721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c102210 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd458009910 tx=0x7fd458009c20 comp rx=0 tx=0).stop 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.357+0000 7fd470a00700 1 -- 172.21.15.176:0/1464873721 shutdown_connections 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.357+0000 7fd470a00700 1 --2- 172.21.15.176:0/1464873721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c102210 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.357+0000 7fd470a00700 1 --2- 172.21.15.176:0/1464873721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd46c100bb0 0x7fd46c100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.357+0000 7fd470a00700 1 -- 172.21.15.176:0/1464873721 >> 172.21.15.176:0/1464873721 conn(0x7fd46c0fc110 msgr2=0x7fd46c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.357+0000 7fd470a00700 1 -- 172.21.15.176:0/1464873721 shutdown_connections 2024-03-13T14:52:28.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.357+0000 7fd470a00700 1 -- 172.21.15.176:0/1464873721 wait complete. 2024-03-13T14:52:28.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.358+0000 7fd470a00700 1 Processor -- start 2024-03-13T14:52:28.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.358+0000 7fd470a00700 1 -- start start 2024-03-13T14:52:28.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.358+0000 7fd470a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd46c100bb0 0x7fd46c1926f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:28.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.358+0000 7fd470a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c192c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:28.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.358+0000 7fd470a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c18f1e0 con 0x7fd46c101db0 2024-03-13T14:52:28.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.358+0000 7fd470a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c18f350 con 0x7fd46c100bb0 2024-03-13T14:52:28.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.359+0000 7fd469400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c192c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:28.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.359+0000 7fd469400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c192c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:38070/0 (socket says 172.21.15.176:38070) 2024-03-13T14:52:28.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.359+0000 7fd469400700 1 -- 172.21.15.176:0/3429880099 learned_addr learned my addr 172.21.15.176:0/3429880099 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:52:28.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.359+0000 7fd469e00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd46c100bb0 0x7fd46c1926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:28.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.359+0000 7fd469400700 1 -- 172.21.15.176:0/3429880099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd46c100bb0 msgr2=0x7fd46c1926f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:28.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.359+0000 7fd469400700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd46c100bb0 0x7fd46c1926f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.359+0000 7fd469400700 1 -- 172.21.15.176:0/3429880099 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580095f0 con 0x7fd46c101db0 2024-03-13T14:52:28.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.360+0000 7fd469400700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c192c30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd45800b5c0 tx=0x7fd45800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:28.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.360+0000 7fd462a00700 1 -- 172.21.15.176:0/3429880099 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45801d070 con 0x7fd46c101db0 2024-03-13T14:52:28.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.360+0000 7fd462a00700 1 -- 172.21.15.176:0/3429880099 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd45800f460 con 0x7fd46c101db0 2024-03-13T14:52:28.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.360+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c18f5d0 con 0x7fd46c101db0 2024-03-13T14:52:28.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.360+0000 7fd462a00700 1 -- 172.21.15.176:0/3429880099 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd458021640 con 0x7fd46c101db0 2024-03-13T14:52:28.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.360+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c18fac0 con 0x7fd46c101db0 2024-03-13T14:52:28.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.362+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c105e90 con 0x7fd46c101db0 2024-03-13T14:52:28.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.363+0000 7fd462a00700 1 -- 172.21.15.176:0/3429880099 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7fd45800fa60 con 0x7fd46c101db0 2024-03-13T14:52:28.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.364+0000 7fd462a00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd45006dc20 0x7fd4500700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:28.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.364+0000 7fd462a00700 1 -- 172.21.15.176:0/3429880099 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd45808eb50 con 0x7fd46c101db0 2024-03-13T14:52:28.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.364+0000 7fd469e00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd45006dc20 0x7fd4500700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:28.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.365+0000 7fd469e00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd45006dc20 0x7fd4500700e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd46c101c10 tx=0x7fd45c00b410 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:28.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.370+0000 7fd462a00700 1 -- 172.21.15.176:0/3429880099 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fd45805bf30 con 0x7fd46c101db0 2024-03-13T14:52:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:28 smithi176 ceph-mon[29064]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:28 smithi176 ceph-mon[29064]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:52:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:28 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3891037921' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:28.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.672+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd46c0611d0 con 0x7fd45006dc20 2024-03-13T14:52:28.661 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.674+0000 7fd462a00700 1 -- 172.21.15.176:0/3429880099 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fd46c0611d0 con 0x7fd45006dc20 2024-03-13T14:52:28.661 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "2/2 daemons upgraded", 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading grafana daemons", 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:52:28.662 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:52:28.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd45006dc20 msgr2=0x7fd4500700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:28.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd45006dc20 0x7fd4500700e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd46c101c10 tx=0x7fd45c00b410 comp rx=0 tx=0).stop 2024-03-13T14:52:28.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 msgr2=0x7fd46c192c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:28.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c192c30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd45800b5c0 tx=0x7fd45800bd70 comp rx=0 tx=0).stop 2024-03-13T14:52:28.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 shutdown_connections 2024-03-13T14:52:28.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd45006dc20 0x7fd4500700e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd46c100bb0 0x7fd46c1926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 --2- 172.21.15.176:0/3429880099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd46c101db0 0x7fd46c192c30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:28.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 >> 172.21.15.176:0/3429880099 conn(0x7fd46c0fc110 msgr2=0x7fd46c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:28.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 shutdown_connections 2024-03-13T14:52:28.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:28.677+0000 7fd470a00700 1 -- 172.21.15.176:0/3429880099 wait complete. 2024-03-13T14:52:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:28 smithi196 ceph-mon[37486]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:28 smithi196 ceph-mon[37486]: from='client.14806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:52:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:28 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3891037921' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:29.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:29 smithi176 ceph-mon[29064]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:29.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:29 smithi176 ceph-mon[29064]: pgmap v93: 65 pgs: 65 active+clean; 210 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 317 KiB/s rd, 370 KiB/s wr, 105 op/s 2024-03-13T14:52:29.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:29 smithi196 ceph-mon[37486]: from='client.14810 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:29.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:29 smithi196 ceph-mon[37486]: pgmap v93: 65 pgs: 65 active+clean; 210 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 317 KiB/s rd, 370 KiB/s wr, 105 op/s 2024-03-13T14:52:30.579 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:30 smithi196 ceph-mon[37486]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:30.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:30 smithi176 ceph-mon[29064]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:52:31.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:31 smithi196 ceph-mon[37486]: pgmap v94: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 419 KiB/s rd, 428 KiB/s wr, 139 op/s 2024-03-13T14:52:31.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:31 smithi176 ceph-mon[29064]: pgmap v94: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 419 KiB/s rd, 428 KiB/s wr, 139 op/s 2024-03-13T14:52:33.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:33 smithi176 ceph-mon[29064]: pgmap v95: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 249 KiB/s rd, 255 KiB/s wr, 85 op/s 2024-03-13T14:52:33.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:33 smithi196 ceph-mon[37486]: pgmap v95: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 249 KiB/s rd, 255 KiB/s wr, 85 op/s 2024-03-13T14:52:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:35 smithi176 ceph-mon[29064]: pgmap v96: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 249 KiB/s rd, 255 KiB/s wr, 85 op/s 2024-03-13T14:52:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:35 smithi196 ceph-mon[37486]: pgmap v96: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 249 KiB/s rd, 255 KiB/s wr, 85 op/s 2024-03-13T14:52:36.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:36 smithi176 ceph-mon[29064]: pgmap v97: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 102 KiB/s rd, 77 KiB/s wr, 56 op/s 2024-03-13T14:52:36.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:36 smithi196 ceph-mon[37486]: pgmap v97: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 102 KiB/s rd, 77 KiB/s wr, 56 op/s 2024-03-13T14:52:38.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:38 smithi176 ceph-mon[29064]: pgmap v98: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 102 KiB/s rd, 58 KiB/s wr, 35 op/s 2024-03-13T14:52:38.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:38 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:38.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:38 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:38.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:38 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:39.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:38 smithi196 ceph-mon[37486]: pgmap v98: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 102 KiB/s rd, 58 KiB/s wr, 35 op/s 2024-03-13T14:52:39.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:38 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:39.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:38 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:39.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:38 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:41.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:41 smithi176 ceph-mon[29064]: pgmap v99: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 102 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-03-13T14:52:41.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:41 smithi196 ceph-mon[37486]: pgmap v99: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 102 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-03-13T14:52:43.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:43 smithi196 ceph-mon[37486]: pgmap v100: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:52:43.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:52:43.497 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:43 smithi176 ceph-mon[29064]: pgmap v100: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:52:43.497 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:52:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:45 smithi196 ceph-mon[37486]: pgmap v101: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:52:45.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:45 smithi176 ceph-mon[29064]: pgmap v101: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-13T14:52:46.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:46 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:46.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:46 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:46.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:46 smithi176 ceph-mon[29064]: pgmap v102: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-03-13T14:52:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:46 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:46 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:46 smithi196 ceph-mon[37486]: pgmap v102: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-03-13T14:52:48.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:48.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:48 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:48.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:48 smithi176 ceph-mon[29064]: pgmap v103: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:48.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:48.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:48 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:48.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:48 smithi196 ceph-mon[37486]: pgmap v103: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:51 smithi196 ceph-mon[37486]: pgmap v104: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:52:51.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:51 smithi176 ceph-mon[29064]: pgmap v104: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:52:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:53 smithi196 ceph-mon[37486]: pgmap v105: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:53 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:53 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:53.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:53 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:53.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:53 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:53.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:53 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:53.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:53 smithi176 ceph-mon[29064]: pgmap v105: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:53.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:53 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:53.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:53 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:53.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:53 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:53.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:53 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:53.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:53 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: Upgrade: Finalizing container_image settings 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-13T14:52:54.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-13T14:52:54.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-13T14:52:54.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-13T14:52:54.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-13T14:52:54.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-13T14:52:54.432 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:54.433 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:54.434 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:54 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: Upgrade: Finalizing container_image settings 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-13T14:52:54.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-13T14:52:54.631 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:54.632 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:54.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:54.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:54.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:52:54.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:52:54.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:52:54.633 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:54 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:55.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:55 smithi196 ceph-mon[37486]: Upgrade: Complete! 2024-03-13T14:52:55.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:55 smithi196 ceph-mon[37486]: pgmap v106: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:55.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:55 smithi176 ceph-mon[29064]: Upgrade: Complete! 2024-03-13T14:52:55.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:55 smithi176 ceph-mon[29064]: pgmap v106: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:57.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:57 smithi176 ceph-mon[29064]: pgmap v107: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:52:57.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:57 smithi196 ceph-mon[37486]: pgmap v107: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:52:58.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 -- 172.21.15.176:0/413284384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0100b80 msgr2=0x7f77f0100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:58.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 --2- 172.21.15.176:0/413284384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0100b80 0x7f77f0100fa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009910 tx=0x7f77e0009c20 comp rx=0 tx=0).stop 2024-03-13T14:52:58.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 -- 172.21.15.176:0/413284384 shutdown_connections 2024-03-13T14:52:58.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 --2- 172.21.15.176:0/413284384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f0101d80 0x7f77f01021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:58.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 --2- 172.21.15.176:0/413284384 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0100b80 0x7f77f0100fa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:58.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 -- 172.21.15.176:0/413284384 >> 172.21.15.176:0/413284384 conn(0x7f77f00fc100 msgr2=0x7f77f00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:58.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 -- 172.21.15.176:0/413284384 shutdown_connections 2024-03-13T14:52:58.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.851+0000 7f77f5600700 1 -- 172.21.15.176:0/413284384 wait complete. 2024-03-13T14:52:58.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.853+0000 7f77f5600700 1 Processor -- start 2024-03-13T14:52:58.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.853+0000 7f77f5600700 1 -- start start 2024-03-13T14:52:58.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.853+0000 7f77f5600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f0100b80 0x7f77f0074b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.853+0000 7f77f5600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0101d80 0x7f77f0073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.853+0000 7f77f5600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00736c0 con 0x7f77f0101d80 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.853+0000 7f77f5600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0073800 con 0x7f77f0100b80 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.853+0000 7f77ee000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0101d80 0x7f77f0073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.854+0000 7f77ee000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0101d80 0x7f77f0073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:41968/0 (socket says 172.21.15.176:41968) 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.854+0000 7f77ee000700 1 -- 172.21.15.176:0/2808106850 learned_addr learned my addr 172.21.15.176:0/2808106850 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.854+0000 7f77eea00700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f0100b80 0x7f77f0074b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.854+0000 7f77ee000700 1 -- 172.21.15.176:0/2808106850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f0100b80 msgr2=0x7f77f0074b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.854+0000 7f77ee000700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f0100b80 0x7f77f0074b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.854+0000 7f77ee000700 1 -- 172.21.15.176:0/2808106850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e00095f0 con 0x7f77f0101d80 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.854+0000 7f77ee000700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0101d80 0x7f77f0073180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c8a0 tx=0x7f77dc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.855+0000 7f77e7400700 1 -- 172.21.15.176:0/2808106850 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77dc00cea0 con 0x7f77f0101d80 2024-03-13T14:52:58.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.855+0000 7f77e7400700 1 -- 172.21.15.176:0/2808106850 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77dc004830 con 0x7f77f0101d80 2024-03-13T14:52:58.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.855+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f0073ae0 con 0x7f77f0101d80 2024-03-13T14:52:58.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.855+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f019e360 con 0x7f77f0101d80 2024-03-13T14:52:58.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.855+0000 7f77e7400700 1 -- 172.21.15.176:0/2808106850 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77dc0055b0 con 0x7f77f0101d80 2024-03-13T14:52:58.844 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.856+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f01096e0 con 0x7f77f0101d80 2024-03-13T14:52:58.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.857+0000 7f77e7400700 1 -- 172.21.15.176:0/2808106850 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f77dc0049a0 con 0x7f77f0101d80 2024-03-13T14:52:58.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.858+0000 7f77e7400700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f77d406dc20 0x7f77d40700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:52:58.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.858+0000 7f77e7400700 1 -- 172.21.15.176:0/2808106850 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f77dc08d500 con 0x7f77f0101d80 2024-03-13T14:52:58.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.858+0000 7f77eea00700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f77d406dc20 0x7f77d40700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:52:58.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.859+0000 7f77eea00700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f77d406dc20 0x7f77d40700e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77e0006010 tx=0x7f77e000b580 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:52:58.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:58.863+0000 7f77e7400700 1 -- 172.21.15.176:0/2808106850 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f77dc05a8e0 con 0x7f77f0101d80 2024-03-13T14:52:59.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.110+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77f01066d0 con 0x7f77d406dc20 2024-03-13T14:52:59.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:58 smithi176 ceph-mon[29064]: pgmap v108: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:59.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:59.099 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:58 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:52:59.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.112+0000 7f77e7400700 1 -- 172.21.15.176:0/2808106850 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f77f01066d0 con 0x7f77d406dc20 2024-03-13T14:52:59.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.116+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f77d406dc20 msgr2=0x7f77d40700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:59.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.116+0000 7f77f5600700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f77d406dc20 0x7f77d40700e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f77e0006010 tx=0x7f77e000b580 comp rx=0 tx=0).stop 2024-03-13T14:52:59.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.116+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0101d80 msgr2=0x7f77f0073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:52:59.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.116+0000 7f77f5600700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0101d80 0x7f77f0073180 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f77dc00c8a0 tx=0x7f77dc00cc60 comp rx=0 tx=0).stop 2024-03-13T14:52:59.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.116+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 shutdown_connections 2024-03-13T14:52:59.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.116+0000 7f77f5600700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f77d406dc20 0x7f77d40700e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:59.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.117+0000 7f77f5600700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77f0100b80 0x7f77f0074b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.117+0000 7f77f5600700 1 --2- 172.21.15.176:0/2808106850 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f77f0101d80 0x7f77f0073180 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:52:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.117+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 >> 172.21.15.176:0/2808106850 conn(0x7f77f00fc100 msgr2=0x7f77f0104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:52:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.117+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 shutdown_connections 2024-03-13T14:52:59.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:52:59.117+0000 7f77f5600700 1 -- 172.21.15.176:0/2808106850 wait complete. 2024-03-13T14:52:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:58 smithi196 ceph-mon[37486]: pgmap v108: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:52:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:52:59.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:58 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:53:00.080 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:52:59 smithi196 ceph-mon[37486]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:00.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:52:59 smithi176 ceph-mon[29064]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:01.360 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:01 smithi176 ceph-mon[29064]: pgmap v109: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:01.404 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-03-13T14:53:01.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:01 smithi196 ceph-mon[37486]: pgmap v109: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:01.852 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:03.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:03 smithi176 ceph-mon[29064]: pgmap v110: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:03.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:03 smithi196 ceph-mon[37486]: pgmap v110: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:03.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 -- 172.21.15.176:0/282022324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc10d280 msgr2=0x7f51fc10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:03.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 --2- 172.21.15.176:0/282022324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc10d280 0x7f51fc10d660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009910 tx=0x7f51ec009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:03.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 -- 172.21.15.176:0/282022324 shutdown_connections 2024-03-13T14:53:03.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 --2- 172.21.15.176:0/282022324 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc1071d0 0x7f51fc107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:03.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 --2- 172.21.15.176:0/282022324 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc10d280 0x7f51fc10d660 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:03.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 -- 172.21.15.176:0/282022324 >> 172.21.15.176:0/282022324 conn(0x7f51fc0769e0 msgr2=0x7f51fc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:03.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 -- 172.21.15.176:0/282022324 shutdown_connections 2024-03-13T14:53:03.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.002+0000 7f5201200700 1 -- 172.21.15.176:0/282022324 wait complete. 2024-03-13T14:53:03.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.003+0000 7f5201200700 1 Processor -- start 2024-03-13T14:53:03.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.003+0000 7f5201200700 1 -- start start 2024-03-13T14:53:03.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f5201200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc1071d0 0x7f51fc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:03.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f5201200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc10d280 0x7f51fc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f5201200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc19d8a0 con 0x7f51fc1071d0 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f5201200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc1a05d0 con 0x7f51fc10d280 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f51f9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc1071d0 0x7f51fc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f51f9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc1071d0 0x7f51fc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:49410/0 (socket says 172.21.15.176:49410) 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f51f9e00700 1 -- 172.21.15.176:0/3264316076 learned_addr learned my addr 172.21.15.176:0/3264316076 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f51f9400700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc10d280 0x7f51fc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f51f9e00700 1 -- 172.21.15.176:0/3264316076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc10d280 msgr2=0x7f51fc19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f51f9e00700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc10d280 0x7f51fc19d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.004+0000 7f51f9e00700 1 -- 172.21.15.176:0/3264316076 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0095f0 con 0x7f51fc1071d0 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.005+0000 7f51f9e00700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc1071d0 0x7f51fc19ccd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b5c0 tx=0x7f51ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.005+0000 7f51f2a00700 1 -- 172.21.15.176:0/3264316076 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ec01d070 con 0x7f51fc1071d0 2024-03-13T14:53:03.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.005+0000 7f51f2a00700 1 -- 172.21.15.176:0/3264316076 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51ec005e00 con 0x7f51fc1071d0 2024-03-13T14:53:03.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.005+0000 7f51f2a00700 1 -- 172.21.15.176:0/3264316076 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ec017400 con 0x7f51fc1071d0 2024-03-13T14:53:03.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.005+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc1a0850 con 0x7f51fc1071d0 2024-03-13T14:53:03.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.005+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc1a0dc0 con 0x7f51fc1071d0 2024-03-13T14:53:03.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.006+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc04ea90 con 0x7f51fc1071d0 2024-03-13T14:53:03.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.007+0000 7f51f2a00700 1 -- 172.21.15.176:0/3264316076 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f51ec00f780 con 0x7f51fc1071d0 2024-03-13T14:53:03.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.008+0000 7f51f2a00700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f51e006dc20 0x7f51e00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:03.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.008+0000 7f51f2a00700 1 -- 172.21.15.176:0/3264316076 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f51ec08ec00 con 0x7f51fc1071d0 2024-03-13T14:53:03.996 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.009+0000 7f51f9400700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f51e006dc20 0x7f51e00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:03.996 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.009+0000 7f51f9400700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f51e006dc20 0x7f51e00700e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005950 tx=0x7f51e80058e0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:04.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.013+0000 7f51f2a00700 1 -- 172.21.15.176:0/3264316076 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f51ec094020 con 0x7f51fc1071d0 2024-03-13T14:53:04.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.324+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f51fc19dff0 con 0x7f51fc1071d0 2024-03-13T14:53:04.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.325+0000 7f51f2a00700 1 -- 172.21.15.176:0/3264316076 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f51ec0585d0 con 0x7f51fc1071d0 2024-03-13T14:53:04.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f51e006dc20 msgr2=0x7f51e00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:04.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f51e006dc20 0x7f51e00700e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005950 tx=0x7f51e80058e0 comp rx=0 tx=0).stop 2024-03-13T14:53:04.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc1071d0 msgr2=0x7f51fc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:04.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc1071d0 0x7f51fc19ccd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b5c0 tx=0x7f51ec00be20 comp rx=0 tx=0).stop 2024-03-13T14:53:04.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 shutdown_connections 2024-03-13T14:53:04.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f51e006dc20 0x7f51e00700e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:04.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f51fc1071d0 0x7f51fc19ccd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:04.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 --2- 172.21.15.176:0/3264316076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f51fc10d280 0x7f51fc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:04.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 >> 172.21.15.176:0/3264316076 conn(0x7f51fc0769e0 msgr2=0x7f51fc078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:04.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 shutdown_connections 2024-03-13T14:53:04.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:04.328+0000 7f5201200700 1 -- 172.21.15.176:0/3264316076 wait complete. 2024-03-13T14:53:04.331 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:53:05.028 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-03-13T14:53:05.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:05 smithi176 ceph-mon[29064]: pgmap v111: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:05.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:05 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3264316076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:05.419 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:05.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:05 smithi196 ceph-mon[37486]: pgmap v111: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:05.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:05 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3264316076' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:07.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:07 smithi176 ceph-mon[29064]: pgmap v112: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:07.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:07 smithi196 ceph-mon[37486]: pgmap v112: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:07.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 -- 172.21.15.176:0/33427764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c102fd0 msgr2=0x7eff5c1033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:07.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 --2- 172.21.15.176:0/33427764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c102fd0 0x7eff5c1033b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7eff4c009910 tx=0x7eff4c009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:07.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 -- 172.21.15.176:0/33427764 shutdown_connections 2024-03-13T14:53:07.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 --2- 172.21.15.176:0/33427764 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c103980 0x7eff5c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:07.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 --2- 172.21.15.176:0/33427764 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c102fd0 0x7eff5c1033b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:07.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 -- 172.21.15.176:0/33427764 >> 172.21.15.176:0/33427764 conn(0x7eff5c0fe840 msgr2=0x7eff5c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:07.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 -- 172.21.15.176:0/33427764 shutdown_connections 2024-03-13T14:53:07.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.657+0000 7eff62200700 1 -- 172.21.15.176:0/33427764 wait complete. 2024-03-13T14:53:07.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.658+0000 7eff62200700 1 Processor -- start 2024-03-13T14:53:07.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.659+0000 7eff62200700 1 -- start start 2024-03-13T14:53:07.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.659+0000 7eff62200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c102fd0 0x7eff5c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:07.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.659+0000 7eff62200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c103980 0x7eff5c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:07.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.659+0000 7eff62200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c19f320 con 0x7eff5c103980 2024-03-13T14:53:07.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.659+0000 7eff62200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c198860 con 0x7eff5c102fd0 2024-03-13T14:53:07.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff53e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c103980 0x7eff5c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:07.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff53e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c103980 0x7eff5c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:49436/0 (socket says 172.21.15.176:49436) 2024-03-13T14:53:07.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff53e00700 1 -- 172.21.15.176:0/98828745 learned_addr learned my addr 172.21.15.176:0/98828745 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:07.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff5b400700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c102fd0 0x7eff5c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:07.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff53e00700 1 -- 172.21.15.176:0/98828745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c102fd0 msgr2=0x7eff5c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:07.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff53e00700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c102fd0 0x7eff5c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:07.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff53e00700 1 -- 172.21.15.176:0/98828745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c0095f0 con 0x7eff5c103980 2024-03-13T14:53:07.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.660+0000 7eff53e00700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c103980 0x7eff5c19ed20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7eff4400c960 tx=0x7eff4400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:07.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.661+0000 7eff58c00700 1 -- 172.21.15.176:0/98828745 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff44007a70 con 0x7eff5c103980 2024-03-13T14:53:07.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.661+0000 7eff58c00700 1 -- 172.21.15.176:0/98828745 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff44007bd0 con 0x7eff5c103980 2024-03-13T14:53:07.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.661+0000 7eff58c00700 1 -- 172.21.15.176:0/98828745 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff440186c0 con 0x7eff5c103980 2024-03-13T14:53:07.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.661+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c198b40 con 0x7eff5c103980 2024-03-13T14:53:07.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.661+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c199090 con 0x7eff5c103980 2024-03-13T14:53:07.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.662+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c10b370 con 0x7eff5c103980 2024-03-13T14:53:07.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.663+0000 7eff58c00700 1 -- 172.21.15.176:0/98828745 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7eff4400faa0 con 0x7eff5c103980 2024-03-13T14:53:07.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.664+0000 7eff58c00700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7eff4006dc20 0x7eff400700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:07.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.664+0000 7eff58c00700 1 -- 172.21.15.176:0/98828745 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7eff4408d7e0 con 0x7eff5c103980 2024-03-13T14:53:07.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.664+0000 7eff5b400700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7eff4006dc20 0x7eff400700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:07.652 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.665+0000 7eff5b400700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7eff4006dc20 0x7eff400700e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b5c0 tx=0x7eff4c01a040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:07.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.669+0000 7eff58c00700 1 -- 172.21.15.176:0/98828745 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7eff4405a9b0 con 0x7eff5c103980 2024-03-13T14:53:07.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.980+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7eff5c04ea90 con 0x7eff5c103980 2024-03-13T14:53:07.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.981+0000 7eff58c00700 1 -- 172.21.15.176:0/98828745 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7eff4405a7d0 con 0x7eff5c103980 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7eff4006dc20 msgr2=0x7eff400700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7eff4006dc20 0x7eff400700e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff4c00b5c0 tx=0x7eff4c01a040 comp rx=0 tx=0).stop 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c103980 msgr2=0x7eff5c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c103980 0x7eff5c19ed20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7eff4400c960 tx=0x7eff4400cd20 comp rx=0 tx=0).stop 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 shutdown_connections 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7eff4006dc20 0x7eff400700e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff5c102fd0 0x7eff5c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 --2- 172.21.15.176:0/98828745 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7eff5c103980 0x7eff5c19ed20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 >> 172.21.15.176:0/98828745 conn(0x7eff5c0fe840 msgr2=0x7eff5c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 shutdown_connections 2024-03-13T14:53:07.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:07.984+0000 7eff62200700 1 -- 172.21.15.176:0/98828745 wait complete. 2024-03-13T14:53:07.987 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)" 2024-03-13T14:53:08.326 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:08 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/98828745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:08.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:08 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/98828745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:08.703 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-03-13T14:53:09.115 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:09.464 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:09 smithi176 ceph-mon[29064]: pgmap v113: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:09 smithi196 ceph-mon[37486]: pgmap v113: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:11.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 -- 172.21.15.176:0/1660597909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c1039b0 msgr2=0x7fd74c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:11.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 --2- 172.21.15.176:0/1660597909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c1039b0 0x7fd74c107a00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd738009910 tx=0x7fd738009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:11.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 -- 172.21.15.176:0/1660597909 shutdown_connections 2024-03-13T14:53:11.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 --2- 172.21.15.176:0/1660597909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c1039b0 0x7fd74c107a00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:11.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 --2- 172.21.15.176:0/1660597909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd74c103000 0x7fd74c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:11.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 -- 172.21.15.176:0/1660597909 >> 172.21.15.176:0/1660597909 conn(0x7fd74c0fe8d0 msgr2=0x7fd74c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:11.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 -- 172.21.15.176:0/1660597909 shutdown_connections 2024-03-13T14:53:11.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.202+0000 7fd753800700 1 -- 172.21.15.176:0/1660597909 wait complete. 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.203+0000 7fd753800700 1 Processor -- start 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.204+0000 7fd753800700 1 -- start start 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.204+0000 7fd753800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c103000 0x7fd74c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.204+0000 7fd753800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd74c1039b0 0x7fd74c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.204+0000 7fd753800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c19f490 con 0x7fd74c103000 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.204+0000 7fd753800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c198940 con 0x7fd74c1039b0 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd750e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c103000 0x7fd74c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd750e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c103000 0x7fd74c19e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:49454/0 (socket says 172.21.15.176:49454) 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd750e00700 1 -- 172.21.15.176:0/3339622313 learned_addr learned my addr 172.21.15.176:0/3339622313 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:11.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd74be00700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd74c1039b0 0x7fd74c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:11.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd750e00700 1 -- 172.21.15.176:0/3339622313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd74c1039b0 msgr2=0x7fd74c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:11.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd750e00700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd74c1039b0 0x7fd74c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:11.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd750e00700 1 -- 172.21.15.176:0/3339622313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380095f0 con 0x7fd74c103000 2024-03-13T14:53:11.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.205+0000 7fd750e00700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c103000 0x7fd74c19e8c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b4b0 tx=0x7fd74000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:11.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.209+0000 7fd749600700 1 -- 172.21.15.176:0/3339622313 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd740011800 con 0x7fd74c103000 2024-03-13T14:53:11.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.209+0000 7fd749600700 1 -- 172.21.15.176:0/3339622313 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd740011e20 con 0x7fd74c103000 2024-03-13T14:53:11.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.209+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c198c20 con 0x7fd74c103000 2024-03-13T14:53:11.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.209+0000 7fd749600700 1 -- 172.21.15.176:0/3339622313 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd74000f2e0 con 0x7fd74c103000 2024-03-13T14:53:11.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.209+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c199170 con 0x7fd74c103000 2024-03-13T14:53:11.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.210+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74c10b330 con 0x7fd74c103000 2024-03-13T14:53:11.199 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.212+0000 7fd749600700 1 -- 172.21.15.176:0/3339622313 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7fd740010380 con 0x7fd74c103000 2024-03-13T14:53:11.199 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.212+0000 7fd749600700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd73406dc20 0x7fd7340700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:11.199 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.213+0000 7fd749600700 1 -- 172.21.15.176:0/3339622313 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd74008d390 con 0x7fd74c103000 2024-03-13T14:53:11.200 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.213+0000 7fd74be00700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd73406dc20 0x7fd7340700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:11.200 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.214+0000 7fd74be00700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd73406dc20 0x7fd7340700e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0ff1f0 tx=0x7fd73801a040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:11.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.221+0000 7fd749600700 1 -- 172.21.15.176:0/3339622313 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fd74005a8a0 con 0x7fd74c103000 2024-03-13T14:53:11.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:11 smithi176 ceph-mon[29064]: pgmap v114: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:11.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.527+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd74c199440 con 0x7fd74c103000 2024-03-13T14:53:11.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.528+0000 7fd749600700 1 -- 172.21.15.176:0/3339622313 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fd74005a430 con 0x7fd74c103000 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd73406dc20 msgr2=0x7fd7340700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd73406dc20 0x7fd7340700e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0ff1f0 tx=0x7fd73801a040 comp rx=0 tx=0).stop 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c103000 msgr2=0x7fd74c19e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c103000 0x7fd74c19e8c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b4b0 tx=0x7fd74000b870 comp rx=0 tx=0).stop 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 shutdown_connections 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd73406dc20 0x7fd7340700e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd74c103000 0x7fd74c19e8c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 --2- 172.21.15.176:0/3339622313 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd74c1039b0 0x7fd74c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.531+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 >> 172.21.15.176:0/3339622313 conn(0x7fd74c0fe8d0 msgr2=0x7fd74c0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.532+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 shutdown_connections 2024-03-13T14:53:11.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:11.532+0000 7fd753800700 1 -- 172.21.15.176:0/3339622313 wait complete. 2024-03-13T14:53:11.535 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:53:11.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:11 smithi196 ceph-mon[37486]: pgmap v114: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:12.290 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-03-13T14:53:12.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:12 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3339622313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:12.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:12 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3339622313' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:12.708 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:13.497 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:13 smithi176 ceph-mon[29064]: pgmap v115: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:53:13.497 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:13 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:53:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:13 smithi196 ceph-mon[37486]: pgmap v115: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:53:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:13 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:53:14.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.857+0000 7f4ddd600700 1 -- 172.21.15.176:0/1007042964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 msgr2=0x7f4dd810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:14.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.857+0000 7f4ddd600700 1 --2- 172.21.15.176:0/1007042964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd810ce40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009910 tx=0x7f4dc0009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:14.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.858+0000 7f4ddd600700 1 -- 172.21.15.176:0/1007042964 shutdown_connections 2024-03-13T14:53:14.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.858+0000 7f4ddd600700 1 --2- 172.21.15.176:0/1007042964 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd810ce40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:14.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.858+0000 7f4ddd600700 1 --2- 172.21.15.176:0/1007042964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80731c0 0x7f4dd80735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:14.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.858+0000 7f4ddd600700 1 -- 172.21.15.176:0/1007042964 >> 172.21.15.176:0/1007042964 conn(0x7f4dd80fc630 msgr2=0x7f4dd80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:14.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.858+0000 7f4ddd600700 1 -- 172.21.15.176:0/1007042964 shutdown_connections 2024-03-13T14:53:14.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.858+0000 7f4ddd600700 1 -- 172.21.15.176:0/1007042964 wait complete. 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.859+0000 7f4ddd600700 1 Processor -- start 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.859+0000 7f4ddd600700 1 -- start start 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.859+0000 7f4ddd600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80731c0 0x7f4dd8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.859+0000 7f4ddd600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.859+0000 7f4ddd600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8199530 con 0x7f4dd8073ae0 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.859+0000 7f4ddd600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd819d340 con 0x7f4dd80731c0 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.860+0000 7f4dcfe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.860+0000 7f4dcfe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd8198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:34496/0 (socket says 172.21.15.176:34496) 2024-03-13T14:53:14.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.860+0000 7f4dcfe00700 1 -- 172.21.15.176:0/54122967 learned_addr learned my addr 172.21.15.176:0/54122967 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:14.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.860+0000 7f4dd6a00700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80731c0 0x7f4dd8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:14.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.860+0000 7f4dcfe00700 1 -- 172.21.15.176:0/54122967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80731c0 msgr2=0x7f4dd8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:14.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.860+0000 7f4dcfe00700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80731c0 0x7f4dd8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:14.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.860+0000 7f4dcfe00700 1 -- 172.21.15.176:0/54122967 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00095f0 con 0x7f4dd8073ae0 2024-03-13T14:53:14.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.861+0000 7f4dcfe00700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd8198ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006010 tx=0x7f4dc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:14.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.861+0000 7f4dcf400700 1 -- 172.21.15.176:0/54122967 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc001d070 con 0x7f4dd8073ae0 2024-03-13T14:53:14.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.861+0000 7f4dcf400700 1 -- 172.21.15.176:0/54122967 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dc0004fb0 con 0x7f4dd8073ae0 2024-03-13T14:53:14.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.861+0000 7f4dcf400700 1 -- 172.21.15.176:0/54122967 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dc000f690 con 0x7f4dd8073ae0 2024-03-13T14:53:14.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.861+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd819d5c0 con 0x7f4dd8073ae0 2024-03-13T14:53:14.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.861+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd819dab0 con 0x7f4dd8073ae0 2024-03-13T14:53:14.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.862+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd810a540 con 0x7f4dd8073ae0 2024-03-13T14:53:14.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.863+0000 7f4dcf400700 1 -- 172.21.15.176:0/54122967 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f4dc0022470 con 0x7f4dd8073ae0 2024-03-13T14:53:14.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.864+0000 7f4dcf400700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4dbc06dc20 0x7f4dbc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:14.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.864+0000 7f4dcf400700 1 -- 172.21.15.176:0/54122967 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4dc008ee50 con 0x7f4dd8073ae0 2024-03-13T14:53:14.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.865+0000 7f4dd6a00700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4dbc06dc20 0x7f4dbc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:14.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.865+0000 7f4dd6a00700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4dbc06dc20 0x7f4dbc0700e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005fd0 tx=0x7f4dc800a560 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:14.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:14.869+0000 7f4dcf400700 1 -- 172.21.15.176:0/54122967 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f4dc005c350 con 0x7f4dd8073ae0 2024-03-13T14:53:15.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:15.107+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dd819ddc0 con 0x7f4dbc06dc20 2024-03-13T14:53:15.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:15 smithi176 ceph-mon[29064]: pgmap v116: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:15.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:15 smithi196 ceph-mon[37486]: pgmap v116: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:16.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:16 smithi176 ceph-mon[29064]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:16.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:16 smithi196 ceph-mon[37486]: from='client.14838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:17.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:17 smithi176 ceph-mon[29064]: pgmap v117: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:17.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:17 smithi196 ceph-mon[37486]: pgmap v117: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:17.822 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.835+0000 7f4dcf400700 1 -- 172.21.15.176:0/54122967 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f4dd819ddc0 con 0x7f4dbc06dc20 2024-03-13T14:53:17.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.838+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4dbc06dc20 msgr2=0x7f4dbc0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:17.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4dbc06dc20 0x7f4dbc0700e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8005fd0 tx=0x7f4dc800a560 comp rx=0 tx=0).stop 2024-03-13T14:53:17.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 msgr2=0x7f4dd8198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:17.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd8198ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0006010 tx=0x7f4dc000be20 comp rx=0 tx=0).stop 2024-03-13T14:53:17.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 shutdown_connections 2024-03-13T14:53:17.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4dbc06dc20 0x7f4dbc0700e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:17.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd80731c0 0x7f4dd8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:17.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 --2- 172.21.15.176:0/54122967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4dd8073ae0 0x7f4dd8198ea0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:17.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 >> 172.21.15.176:0/54122967 conn(0x7f4dd80fc630 msgr2=0x7f4dd8107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:17.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 shutdown_connections 2024-03-13T14:53:17.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:17.839+0000 7f4ddd600700 1 -- 172.21.15.176:0/54122967 wait complete. 2024-03-13T14:53:17.844 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:53:18.577 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph orch ps' 2024-03-13T14:53:18.992 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:19.472 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:19 smithi176 ceph-mon[29064]: pgmap v118: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:19.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:19 smithi196 ceph-mon[37486]: pgmap v118: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:21.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 -- 172.21.15.176:0/1120926186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068410 msgr2=0x7f76880687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:21.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 --2- 172.21.15.176:0/1120926186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068410 0x7f76880687f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f767c009910 tx=0x7f767c009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:21.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 -- 172.21.15.176:0/1120926186 shutdown_connections 2024-03-13T14:53:21.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 --2- 172.21.15.176:0/1120926186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7688068d30 0x7f768810d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:21.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 --2- 172.21.15.176:0/1120926186 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068410 0x7f76880687f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:21.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 -- 172.21.15.176:0/1120926186 >> 172.21.15.176:0/1120926186 conn(0x7f76880755a0 msgr2=0x7f76880759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:21.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 -- 172.21.15.176:0/1120926186 shutdown_connections 2024-03-13T14:53:21.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.135+0000 7f7690200700 1 -- 172.21.15.176:0/1120926186 wait complete. 2024-03-13T14:53:21.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.137+0000 7f7690200700 1 Processor -- start 2024-03-13T14:53:21.125 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.137+0000 7f7690200700 1 -- start start 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f7690200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7688068410 0x7f768819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f7690200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068d30 0x7f768819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f7690200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768819f390 con 0x7f7688068d30 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f7690200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7688198840 con 0x7f7688068410 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f768ce00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068d30 0x7f768819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f768ce00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068d30 0x7f768819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:34508/0 (socket says 172.21.15.176:34508) 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f768ce00700 1 -- 172.21.15.176:0/3522179188 learned_addr learned my addr 172.21.15.176:0/3522179188 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.138+0000 7f768d800700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7688068410 0x7f768819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:21.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f768ce00700 1 -- 172.21.15.176:0/3522179188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7688068410 msgr2=0x7f768819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:21.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f768ce00700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7688068410 0x7f768819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:21.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f768ce00700 1 -- 172.21.15.176:0/3522179188 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0095f0 con 0x7f7688068d30 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f768ce00700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068d30 0x7f768819ed00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f767400b4b0 tx=0x7f767400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f7686000700 1 -- 172.21.15.176:0/3522179188 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7674011800 con 0x7f7688068d30 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f7686000700 1 -- 172.21.15.176:0/3522179188 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7674011e20 con 0x7f7688068d30 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f7686000700 1 -- 172.21.15.176:0/3522179188 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767400f2e0 con 0x7f7688068d30 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7688198b20 con 0x7f7688068d30 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.139+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7688199070 con 0x7f7688068d30 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.141+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768810a9b0 con 0x7f7688068d30 2024-03-13T14:53:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.142+0000 7f7686000700 1 -- 172.21.15.176:0/3522179188 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f7674010380 con 0x7f7688068d30 2024-03-13T14:53:21.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.142+0000 7f7686000700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f767006dc20 0x7f76700700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:21.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.143+0000 7f7686000700 1 -- 172.21.15.176:0/3522179188 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f767408d390 con 0x7f7688068d30 2024-03-13T14:53:21.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.143+0000 7f768d800700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f767006dc20 0x7f76700700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:21.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.143+0000 7f768d800700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f767006dc20 0x7f76700700e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f767c005f50 tx=0x7f767c01a040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:21.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.148+0000 7f7686000700 1 -- 172.21.15.176:0/3522179188 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f767405a8a0 con 0x7f7688068d30 2024-03-13T14:53:21.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.378+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7688199340 con 0x7f767006dc20 2024-03-13T14:53:21.381 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.394+0000 7f7686000700 1 -- 172.21.15.176:0/3522179188 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f7688199340 con 0x7f767006dc20 2024-03-13T14:53:21.381 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:53:21.381 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (84s) 35s ago 14m 14.0M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:53:21.381 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (15m) 35s ago 15m 8070k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (13m) 2m ago 13m 7868k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (15m) 35s ago 15m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (13m) 2m ago 13m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (43s) 35s ago 14m 49.1M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (7m) 35s ago 7m 87.1M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (7m) 35s ago 7m 15.2M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (7m) 2m ago 7m 216M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (7m) 2m ago 7m 14.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (4m) 35s ago 16m 499M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (3m) 2m ago 13m 447M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (16m) 35s ago 16m 57.6M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (13m) 2m ago 13m 44.1M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:53:21.382 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (2m) 35s ago 15m 7222k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (2m) 2m ago 13m 2847k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (12m) 35s ago 12m 231M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (11m) 35s ago 11m 244M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (11m) 35s ago 11m 218M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (10m) 2m ago 10m 247M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (9m) 2m ago 9m 220M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (9m) 2m ago 9m 194M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:53:21.383 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (110s) 35s ago 14m 58.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:53:21.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f767006dc20 msgr2=0x7f76700700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:21.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f767006dc20 0x7f76700700e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f767c005f50 tx=0x7f767c01a040 comp rx=0 tx=0).stop 2024-03-13T14:53:21.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068d30 msgr2=0x7f768819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:21.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068d30 0x7f768819ed00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f767400b4b0 tx=0x7f767400b870 comp rx=0 tx=0).stop 2024-03-13T14:53:21.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 shutdown_connections 2024-03-13T14:53:21.384 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f767006dc20 0x7f76700700e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:21.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7688068410 0x7f768819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:21.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 --2- 172.21.15.176:0/3522179188 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7688068d30 0x7f768819ed00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:21.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 >> 172.21.15.176:0/3522179188 conn(0x7f76880755a0 msgr2=0x7f76880fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:21.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 shutdown_connections 2024-03-13T14:53:21.385 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:21.397+0000 7f7690200700 1 -- 172.21.15.176:0/3522179188 wait complete. 2024-03-13T14:53:21.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:21 smithi176 ceph-mon[29064]: pgmap v119: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:21.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:21 smithi196 ceph-mon[37486]: pgmap v119: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:22.119 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-13T14:53:22.120 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:53:22.120 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-03-13T14:53:22.543 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:23.498 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:23 smithi176 ceph-mon[29064]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:23.499 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:23 smithi176 ceph-mon[29064]: pgmap v120: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:23 smithi196 ceph-mon[37486]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:23 smithi196 ceph-mon[37486]: pgmap v120: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:24.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 -- 172.21.15.176:0/1950959263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f861410d280 msgr2=0x7f861410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 --2- 172.21.15.176:0/1950959263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f861410d280 0x7f861410d660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8604009910 tx=0x7f8604009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 -- 172.21.15.176:0/1950959263 shutdown_connections 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 --2- 172.21.15.176:0/1950959263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86141071d0 0x7f8614107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 --2- 172.21.15.176:0/1950959263 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f861410d280 0x7f861410d660 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 -- 172.21.15.176:0/1950959263 >> 172.21.15.176:0/1950959263 conn(0x7f86140769e0 msgr2=0x7f8614078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 -- 172.21.15.176:0/1950959263 shutdown_connections 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.651+0000 7f8619c00700 1 -- 172.21.15.176:0/1950959263 wait complete. 2024-03-13T14:53:24.639 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.653+0000 7f8619c00700 1 Processor -- start 2024-03-13T14:53:24.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.653+0000 7f8619c00700 1 -- start start 2024-03-13T14:53:24.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.653+0000 7f8619c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86141071d0 0x7f861419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:24.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8619c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f861410d280 0x7f861419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:24.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8619c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861419d8a0 con 0x7f86141071d0 2024-03-13T14:53:24.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8619c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86141a05d0 con 0x7f861410d280 2024-03-13T14:53:24.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8612a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86141071d0 0x7f861419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:24.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8612a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86141071d0 0x7f861419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59412/0 (socket says 172.21.15.176:59412) 2024-03-13T14:53:24.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8612a00700 1 -- 172.21.15.176:0/2093512304 learned_addr learned my addr 172.21.15.176:0/2093512304 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:24.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f860a000700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f861410d280 0x7f861419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:24.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8612a00700 1 -- 172.21.15.176:0/2093512304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f861410d280 msgr2=0x7f861419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:24.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8612a00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f861410d280 0x7f861419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:24.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.654+0000 7f8612a00700 1 -- 172.21.15.176:0/2093512304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040095f0 con 0x7f86141071d0 2024-03-13T14:53:24.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.655+0000 7f8612a00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86141071d0 0x7f861419ccd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f860400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:24.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.655+0000 7f860be00700 1 -- 172.21.15.176:0/2093512304 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f860401d070 con 0x7f86141071d0 2024-03-13T14:53:24.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.655+0000 7f860be00700 1 -- 172.21.15.176:0/2093512304 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8604005dc0 con 0x7f86141071d0 2024-03-13T14:53:24.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.655+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86141a0850 con 0x7f86141071d0 2024-03-13T14:53:24.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.655+0000 7f860be00700 1 -- 172.21.15.176:0/2093512304 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8604017400 con 0x7f86141071d0 2024-03-13T14:53:24.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.655+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86141a0dc0 con 0x7f86141071d0 2024-03-13T14:53:24.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.656+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861404ea90 con 0x7f86141071d0 2024-03-13T14:53:24.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.657+0000 7f860be00700 1 -- 172.21.15.176:0/2093512304 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f860400f7f0 con 0x7f86141071d0 2024-03-13T14:53:24.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.658+0000 7f860be00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f85f806dc20 0x7f85f80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:24.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.658+0000 7f860be00700 1 -- 172.21.15.176:0/2093512304 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f860408ed70 con 0x7f86141071d0 2024-03-13T14:53:24.645 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.658+0000 7f860a000700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f85f806dc20 0x7f85f80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:24.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.659+0000 7f860a000700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f85f806dc20 0x7f85f80700e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f861419e2b0 tx=0x7f85fc009040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:24.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.663+0000 7f860be00700 1 -- 172.21.15.176:0/2093512304 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f8604058620 con 0x7f86141071d0 2024-03-13T14:53:24.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.881+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f861419dff0 con 0x7f86141071d0 2024-03-13T14:53:24.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.895+0000 7f860be00700 1 -- 172.21.15.176:0/2093512304 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f860405bce0 con 0x7f86141071d0 2024-03-13T14:53:24.886 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f85f806dc20 msgr2=0x7f85f80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f85f806dc20 0x7f85f80700e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f861419e2b0 tx=0x7f85fc009040 comp rx=0 tx=0).stop 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86141071d0 msgr2=0x7f861419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86141071d0 0x7f861419ccd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f860400be20 comp rx=0 tx=0).stop 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 shutdown_connections 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f85f806dc20 0x7f85f80700e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86141071d0 0x7f861419ccd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 --2- 172.21.15.176:0/2093512304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f861410d280 0x7f861419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:24.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.898+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 >> 172.21.15.176:0/2093512304 conn(0x7f86140769e0 msgr2=0x7f8614078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:24.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.899+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 shutdown_connections 2024-03-13T14:53:24.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:24.899+0000 7f8619c00700 1 -- 172.21.15.176:0/2093512304 wait complete. 2024-03-13T14:53:25.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:25 smithi176 ceph-mon[29064]: pgmap v121: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:25.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:25 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/2093512304' entity='client.admin' 2024-03-13T14:53:25.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:25 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:53:25.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:25 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:53:25.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:25 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:53:25.573 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:25 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:25.626 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-13T14:53:25.626 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:53:25.626 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-03-13T14:53:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:25 smithi196 ceph-mon[37486]: pgmap v121: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:25 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/2093512304' entity='client.admin' 2024-03-13T14:53:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:25 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:53:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:25 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:53:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:25 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:53:25.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:25 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:26.030 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:27.599 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:27 smithi176 ceph-mon[29064]: pgmap v122: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:27.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:27 smithi196 ceph-mon[37486]: pgmap v122: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:28.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 -- 172.21.15.176:0/719738288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0105b50 msgr2=0x7f94e0068410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:28.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 --2- 172.21.15.176:0/719738288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0105b50 0x7f94e0068410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f94d000b0a0 tx=0x7f94d000b3b0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 -- 172.21.15.176:0/719738288 shutdown_connections 2024-03-13T14:53:28.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 --2- 172.21.15.176:0/719738288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94e0068950 0x7f94e0068dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 --2- 172.21.15.176:0/719738288 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0105b50 0x7f94e0068410 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 -- 172.21.15.176:0/719738288 >> 172.21.15.176:0/719738288 conn(0x7f94e00755a0 msgr2=0x7f94e00759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:28.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 -- 172.21.15.176:0/719738288 shutdown_connections 2024-03-13T14:53:28.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.027+0000 7f94e5a00700 1 -- 172.21.15.176:0/719738288 wait complete. 2024-03-13T14:53:28.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.028+0000 7f94e5a00700 1 Processor -- start 2024-03-13T14:53:28.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94e5a00700 1 -- start start 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94e5a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0068950 0x7f94e019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94e5a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94e0105b50 0x7f94e019d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94e5a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e019da90 con 0x7f94e0068950 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94e5a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e01a1820 con 0x7f94e0105b50 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94dea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0068950 0x7f94e019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94dea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0068950 0x7f94e019ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59424/0 (socket says 172.21.15.176:59424) 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.029+0000 7f94dea00700 1 -- 172.21.15.176:0/3626419296 learned_addr learned my addr 172.21.15.176:0/3626419296 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94d6000700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94e0105b50 0x7f94e019d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94dea00700 1 -- 172.21.15.176:0/3626419296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94e0105b50 msgr2=0x7f94e019d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94dea00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94e0105b50 0x7f94e019d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94dea00700 1 -- 172.21.15.176:0/3626419296 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0009d00 con 0x7f94e0068950 2024-03-13T14:53:28.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94d6000700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94e0105b50 0x7f94e019d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:53:28.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94dea00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0068950 0x7f94e019ce70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009fd0 tx=0x7f94d000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:28.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94d7e00700 1 -- 172.21.15.176:0/3626419296 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d000e040 con 0x7f94e0068950 2024-03-13T14:53:28.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94d7e00700 1 -- 172.21.15.176:0/3626419296 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94d0003de0 con 0x7f94e0068950 2024-03-13T14:53:28.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e01a1aa0 con 0x7f94e0068950 2024-03-13T14:53:28.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.030+0000 7f94d7e00700 1 -- 172.21.15.176:0/3626419296 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d001d8e0 con 0x7f94e0068950 2024-03-13T14:53:28.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.031+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e01a1f90 con 0x7f94e0068950 2024-03-13T14:53:28.019 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.032+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e01101a0 con 0x7f94e0068950 2024-03-13T14:53:28.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.033+0000 7f94d7e00700 1 -- 172.21.15.176:0/3626419296 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f94d0004040 con 0x7f94e0068950 2024-03-13T14:53:28.022 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.034+0000 7f94d7e00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f94c406dc20 0x7f94c40700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:28.022 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.034+0000 7f94d7e00700 1 -- 172.21.15.176:0/3626419296 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f94d008f0b0 con 0x7f94e0068950 2024-03-13T14:53:28.022 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.034+0000 7f94d6000700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f94c406dc20 0x7f94c40700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:28.022 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.035+0000 7f94d6000700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f94c406dc20 0x7f94c40700e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94e019e490 tx=0x7f94c8008040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:28.026 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.039+0000 7f94d7e00700 1 -- 172.21.15.176:0/3626419296 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f94d0058a00 con 0x7f94e0068950 2024-03-13T14:53:28.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.257+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f94e004ea90 con 0x7f94e0068950 2024-03-13T14:53:28.245 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.258+0000 7f94d7e00700 1 -- 172.21.15.176:0/3626419296 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7f94d005c0c0 con 0x7f94e0068950 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.261+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f94c406dc20 msgr2=0x7f94c40700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.261+0000 7f94e5a00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f94c406dc20 0x7f94c40700e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94e019e490 tx=0x7f94c8008040 comp rx=0 tx=0).stop 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.261+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0068950 msgr2=0x7f94e019ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.261+0000 7f94e5a00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0068950 0x7f94e019ce70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009fd0 tx=0x7f94d000cdd0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.262+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 shutdown_connections 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.262+0000 7f94e5a00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f94c406dc20 0x7f94c40700e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.262+0000 7f94e5a00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f94e0068950 0x7f94e019ce70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.262+0000 7f94e5a00700 1 --2- 172.21.15.176:0/3626419296 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94e0105b50 0x7f94e019d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.262+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 >> 172.21.15.176:0/3626419296 conn(0x7f94e00755a0 msgr2=0x7f94e01015b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.262+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 shutdown_connections 2024-03-13T14:53:28.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:28.262+0000 7f94e5a00700 1 -- 172.21.15.176:0/3626419296 wait complete. 2024-03-13T14:53:28.615 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:28 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:53:28.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:28 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:53:28.977 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-03-13T14:53:29.389 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:29.539 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:29 smithi176 ceph-mon[29064]: pgmap v123: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:29.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:29 smithi196 ceph-mon[37486]: pgmap v123: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:31.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 -- 172.21.15.176:0/3398372101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81009c0 msgr2=0x7f53d8100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:31.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 --2- 172.21.15.176:0/3398372101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81009c0 0x7f53d8100e40 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f53c4009910 tx=0x7f53c4009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:31.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 -- 172.21.15.176:0/3398372101 shutdown_connections 2024-03-13T14:53:31.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 --2- 172.21.15.176:0/3398372101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81009c0 0x7f53d8100e40 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:31.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 --2- 172.21.15.176:0/3398372101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d81069c0 0x7f53d8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:31.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 -- 172.21.15.176:0/3398372101 >> 172.21.15.176:0/3398372101 conn(0x7f53d80fc4d0 msgr2=0x7f53d80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:31.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 -- 172.21.15.176:0/3398372101 shutdown_connections 2024-03-13T14:53:31.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.400+0000 7f53dd600700 1 -- 172.21.15.176:0/3398372101 wait complete. 2024-03-13T14:53:31.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.402+0000 7f53dd600700 1 Processor -- start 2024-03-13T14:53:31.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.402+0000 7f53dd600700 1 -- start start 2024-03-13T14:53:31.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.402+0000 7f53dd600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d81009c0 0x7f53d819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:31.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.402+0000 7f53dd600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81069c0 0x7f53d819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:31.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53dd600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81985e0 con 0x7f53d81069c0 2024-03-13T14:53:31.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53dd600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8198750 con 0x7f53d81009c0 2024-03-13T14:53:31.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53d7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81069c0 0x7f53d819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:31.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53d7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81069c0 0x7f53d819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59448/0 (socket says 172.21.15.176:59448) 2024-03-13T14:53:31.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53d7400700 1 -- 172.21.15.176:0/2250350340 learned_addr learned my addr 172.21.15.176:0/2250350340 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:31.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53d7e00700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d81009c0 0x7f53d819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:31.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53d7400700 1 -- 172.21.15.176:0/2250350340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d81009c0 msgr2=0x7f53d819e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:31.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53d7400700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d81009c0 0x7f53d819e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:31.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.403+0000 7f53d7400700 1 -- 172.21.15.176:0/2250350340 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40095f0 con 0x7f53d81069c0 2024-03-13T14:53:31.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.404+0000 7f53d7400700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81069c0 0x7f53d819eb30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006010 tx=0x7f53c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:31.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.404+0000 7f53d4c00700 1 -- 172.21.15.176:0/2250350340 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53c401d070 con 0x7f53d81069c0 2024-03-13T14:53:31.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.404+0000 7f53d4c00700 1 -- 172.21.15.176:0/2250350340 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53c4004fb0 con 0x7f53d81069c0 2024-03-13T14:53:31.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.404+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d81989d0 con 0x7f53d81069c0 2024-03-13T14:53:31.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.404+0000 7f53d4c00700 1 -- 172.21.15.176:0/2250350340 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53c4022ea0 con 0x7f53d81069c0 2024-03-13T14:53:31.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.404+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d8198ec0 con 0x7f53d81069c0 2024-03-13T14:53:31.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.405+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d804f2e0 con 0x7f53d81069c0 2024-03-13T14:53:31.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.406+0000 7f53d4c00700 1 -- 172.21.15.176:0/2250350340 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f53c4022470 con 0x7f53d81069c0 2024-03-13T14:53:31.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.407+0000 7f53d4c00700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f53bc06dc20 0x7f53bc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:31.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.407+0000 7f53d4c00700 1 -- 172.21.15.176:0/2250350340 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f53c408ed30 con 0x7f53d81069c0 2024-03-13T14:53:31.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.407+0000 7f53d7e00700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f53bc06dc20 0x7f53bc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:31.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.408+0000 7f53d7e00700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f53bc06dc20 0x7f53bc0700e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53c800a560 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:31.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.413+0000 7f53d4c00700 1 -- 172.21.15.176:0/2250350340 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f53c4058710 con 0x7f53d81069c0 2024-03-13T14:53:31.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:31 smithi196 ceph-mon[37486]: pgmap v124: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:31.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.638+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f53d81991d0 con 0x7f53d81069c0 2024-03-13T14:53:31.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:31 smithi176 ceph-mon[29064]: pgmap v124: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:31.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.639+0000 7f53d4c00700 1 -- 172.21.15.176:0/2250350340 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7f53c40270e0 con 0x7f53d81069c0 2024-03-13T14:53:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.641+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f53bc06dc20 msgr2=0x7f53bc0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f53bc06dc20 0x7f53bc0700e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53c8005fd0 tx=0x7f53c800a560 comp rx=0 tx=0).stop 2024-03-13T14:53:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81069c0 msgr2=0x7f53d819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81069c0 0x7f53d819eb30 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f53c4006010 tx=0x7f53c400be20 comp rx=0 tx=0).stop 2024-03-13T14:53:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 shutdown_connections 2024-03-13T14:53:31.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f53bc06dc20 0x7f53bc0700e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53d81009c0 0x7f53d819e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 --2- 172.21.15.176:0/2250350340 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f53d81069c0 0x7f53d819eb30 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 >> 172.21.15.176:0/2250350340 conn(0x7f53d80fc4d0 msgr2=0x7f53d80fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 shutdown_connections 2024-03-13T14:53:31.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:31.642+0000 7f53dd600700 1 -- 172.21.15.176:0/2250350340 wait complete. 2024-03-13T14:53:32.396 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph config set global log_to_journald false --force' 2024-03-13T14:53:32.801 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:33.270 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:33 smithi176 ceph-mon[29064]: pgmap v125: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:33.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:33 smithi196 ceph-mon[37486]: pgmap v125: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 -- 172.21.15.176:0/1699814220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec1037c0 msgr2=0x7f3aec107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 --2- 172.21.15.176:0/1699814220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec1037c0 0x7f3aec107810 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009910 tx=0x7f3ad8009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 -- 172.21.15.176:0/1699814220 shutdown_connections 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 --2- 172.21.15.176:0/1699814220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec1037c0 0x7f3aec107810 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 --2- 172.21.15.176:0/1699814220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aec102e10 0x7f3aec1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 -- 172.21.15.176:0/1699814220 >> 172.21.15.176:0/1699814220 conn(0x7f3aec0fe6c0 msgr2=0x7f3aec100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 -- 172.21.15.176:0/1699814220 shutdown_connections 2024-03-13T14:53:34.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.879+0000 7f3aeb200700 1 -- 172.21.15.176:0/1699814220 wait complete. 2024-03-13T14:53:34.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.880+0000 7f3aeb200700 1 Processor -- start 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.881+0000 7f3aeb200700 1 -- start start 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.881+0000 7f3aeb200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec102e10 0x7f3aec19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.881+0000 7f3aeb200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aec1037c0 0x7f3aec19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.881+0000 7f3aeb200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec19f0c0 con 0x7f3aec102e10 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.881+0000 7f3aeb200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec198600 con 0x7f3aec1037c0 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.881+0000 7f3ae9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec102e10 0x7f3aec19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.882+0000 7f3ae9e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec102e10 0x7f3aec19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44636/0 (socket says 172.21.15.176:44636) 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.882+0000 7f3ae9e00700 1 -- 172.21.15.176:0/2283076958 learned_addr learned my addr 172.21.15.176:0/2283076958 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:34.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.882+0000 7f3ae9400700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aec1037c0 0x7f3aec19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:34.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.882+0000 7f3ae9e00700 1 -- 172.21.15.176:0/2283076958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aec1037c0 msgr2=0x7f3aec19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:34.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.882+0000 7f3ae9e00700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aec1037c0 0x7f3aec19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:34.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.882+0000 7f3ae9e00700 1 -- 172.21.15.176:0/2283076958 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80095f0 con 0x7f3aec102e10 2024-03-13T14:53:34.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.882+0000 7f3ae9e00700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec102e10 0x7f3aec19e580 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00cfd0 tx=0x7f3adc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:34.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.883+0000 7f3ae2a00700 1 -- 172.21.15.176:0/2283076958 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3adc00ad70 con 0x7f3aec102e10 2024-03-13T14:53:34.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.883+0000 7f3ae2a00700 1 -- 172.21.15.176:0/2283076958 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3adc0128e0 con 0x7f3aec102e10 2024-03-13T14:53:34.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.883+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aec1988e0 con 0x7f3aec102e10 2024-03-13T14:53:34.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.883+0000 7f3ae2a00700 1 -- 172.21.15.176:0/2283076958 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3adc011880 con 0x7f3aec102e10 2024-03-13T14:53:34.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.883+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aec198e30 con 0x7f3aec102e10 2024-03-13T14:53:34.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.884+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aec10b210 con 0x7f3aec102e10 2024-03-13T14:53:34.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.885+0000 7f3ae2a00700 1 -- 172.21.15.176:0/2283076958 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f3adc012420 con 0x7f3aec102e10 2024-03-13T14:53:34.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.886+0000 7f3ae2a00700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ad006dc20 0x7f3ad00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:34.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.886+0000 7f3ae2a00700 1 -- 172.21.15.176:0/2283076958 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3adc08d8f0 con 0x7f3aec102e10 2024-03-13T14:53:34.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.886+0000 7f3ae9400700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ad006dc20 0x7f3ad00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:34.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.887+0000 7f3ae9400700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ad006dc20 0x7f3ad00700e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8006010 tx=0x7f3ad8005960 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:34.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:34.891+0000 7f3ae2a00700 1 -- 172.21.15.176:0/2283076958 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f3adc05ad50 con 0x7f3aec102e10 2024-03-13T14:53:35.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.111+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f3aec04f2e0 con 0x7f3aec102e10 2024-03-13T14:53:35.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.112+0000 7f3ae2a00700 1 -- 172.21.15.176:0/2283076958 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7f3adc05a8e0 con 0x7f3aec102e10 2024-03-13T14:53:35.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ad006dc20 msgr2=0x7f3ad00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:35.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ad006dc20 0x7f3ad00700e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8006010 tx=0x7f3ad8005960 comp rx=0 tx=0).stop 2024-03-13T14:53:35.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec102e10 msgr2=0x7f3aec19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:35.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec102e10 0x7f3aec19e580 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00cfd0 tx=0x7f3adc00a340 comp rx=0 tx=0).stop 2024-03-13T14:53:35.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 shutdown_connections 2024-03-13T14:53:35.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f3ad006dc20 0x7f3ad00700e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:35.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3aec102e10 0x7f3aec19e580 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:35.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 --2- 172.21.15.176:0/2283076958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aec1037c0 0x7f3aec19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:35.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 >> 172.21.15.176:0/2283076958 conn(0x7f3aec0fe6c0 msgr2=0x7f3aec100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:35.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 shutdown_connections 2024-03-13T14:53:35.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:35.115+0000 7f3aeb200700 1 -- 172.21.15.176:0/2283076958 wait complete. 2024-03-13T14:53:35.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:35 smithi176 ceph-mon[29064]: pgmap v126: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:35 smithi196 ceph-mon[37486]: pgmap v126: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:35.854 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-03-13T14:53:36.251 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:37.423 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:37 smithi176 ceph-mon[29064]: pgmap v127: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:37.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:37 smithi196 ceph-mon[37486]: pgmap v127: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:38.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 -- 172.21.15.176:0/661618416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 msgr2=0x7f9538105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:38.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 --2- 172.21.15.176:0/661618416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538105e20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f9528009910 tx=0x7f9528009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 -- 172.21.15.176:0/661618416 shutdown_connections 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 --2- 172.21.15.176:0/661618416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95380ffbd0 0x7f9538100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 --2- 172.21.15.176:0/661618416 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538105e20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 -- 172.21.15.176:0/661618416 >> 172.21.15.176:0/661618416 conn(0x7f95380755a0 msgr2=0x7f95380759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 -- 172.21.15.176:0/661618416 shutdown_connections 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.286+0000 7f953fa00700 1 -- 172.21.15.176:0/661618416 wait complete. 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.287+0000 7f953fa00700 1 Processor -- start 2024-03-13T14:53:38.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.287+0000 7f953fa00700 1 -- start start 2024-03-13T14:53:38.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f953fa00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95380ffbd0 0x7f9538101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:38.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f953fa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:38.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f953fa00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538102090 con 0x7f9538105a40 2024-03-13T14:53:38.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f953fa00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95381021d0 con 0x7f95380ffbd0 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f952fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f952fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44662/0 (socket says 172.21.15.176:44662) 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f952fe00700 1 -- 172.21.15.176:0/1746705261 learned_addr learned my addr 172.21.15.176:0/1746705261 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.288+0000 7f953d000700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95380ffbd0 0x7f9538101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f952fe00700 1 -- 172.21.15.176:0/1746705261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95380ffbd0 msgr2=0x7f9538101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f952fe00700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95380ffbd0 0x7f9538101610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f952fe00700 1 -- 172.21.15.176:0/1746705261 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280095f0 con 0x7f9538105a40 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f952fe00700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538101b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f952000b4b0 tx=0x7f952000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f9536000700 1 -- 172.21.15.176:0/1746705261 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9520011800 con 0x7f9538105a40 2024-03-13T14:53:38.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f9536000700 1 -- 172.21.15.176:0/1746705261 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9520011e20 con 0x7f9538105a40 2024-03-13T14:53:38.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95381a2a10 con 0x7f9538105a40 2024-03-13T14:53:38.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f9536000700 1 -- 172.21.15.176:0/1746705261 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f952000f2e0 con 0x7f9538105a40 2024-03-13T14:53:38.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.289+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95381a2e20 con 0x7f9538105a40 2024-03-13T14:53:38.280 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.291+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9538109050 con 0x7f9538105a40 2024-03-13T14:53:38.280 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.292+0000 7f9536000700 1 -- 172.21.15.176:0/1746705261 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f9520010380 con 0x7f9538105a40 2024-03-13T14:53:38.280 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.293+0000 7f9536000700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f951c06dc20 0x7f951c0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:38.280 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.293+0000 7f9536000700 1 -- 172.21.15.176:0/1746705261 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f952008d2c0 con 0x7f9538105a40 2024-03-13T14:53:38.280 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.293+0000 7f953d000700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f951c06dc20 0x7f951c0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:38.281 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.294+0000 7f953d000700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f951c06dc20 0x7f951c0700e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9528006010 tx=0x7f952801a040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:38.284 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.298+0000 7f9536000700 1 -- 172.21.15.176:0/1746705261 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f9520056c90 con 0x7f9538105a40 2024-03-13T14:53:38.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.543+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "target": ["mon-mgr", ""]}) v1 -- 0x7f953802cd80 con 0x7f951c06dc20 2024-03-13T14:53:38.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.556+0000 7f9536000700 1 -- 172.21.15.176:0/1746705261 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f953802cd80 con 0x7f951c06dc20 2024-03-13T14:53:38.543 INFO:teuthology.orchestra.run.smithi176.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:53:38.546 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f951c06dc20 msgr2=0x7f951c0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:38.546 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f951c06dc20 0x7f951c0700e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9528006010 tx=0x7f952801a040 comp rx=0 tx=0).stop 2024-03-13T14:53:38.546 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 msgr2=0x7f9538101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:38.546 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538101b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f952000b4b0 tx=0x7f952000b870 comp rx=0 tx=0).stop 2024-03-13T14:53:38.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 shutdown_connections 2024-03-13T14:53:38.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f951c06dc20 0x7f951c0700e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:38.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95380ffbd0 0x7f9538101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:38.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 --2- 172.21.15.176:0/1746705261 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9538105a40 0x7f9538101b50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:38.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 >> 172.21.15.176:0/1746705261 conn(0x7f95380755a0 msgr2=0x7f95380686b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:38.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 shutdown_connections 2024-03-13T14:53:38.547 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:38.559+0000 7f953fa00700 1 -- 172.21.15.176:0/1746705261 wait complete. 2024-03-13T14:53:39.318 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-03-13T14:53:39.319 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T14:53:39.319 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-03-13T14:53:39.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:39 smithi176 ceph-mon[29064]: pgmap v128: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:39.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:39 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:39 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:53:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:39 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:53:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:39 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:53:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:39 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:39 smithi196 ceph-mon[37486]: pgmap v128: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:39 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:39 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:53:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:39 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:53:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:39 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:53:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:39 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:39.711 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T14:53:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:40 smithi176 ceph-mon[29064]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:40.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:40 smithi176 ceph-mon[29064]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:53:40.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:40 smithi176 ceph-mon[29064]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:53:40.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:40 smithi196 ceph-mon[37486]: from='client.14862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:40.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:40 smithi196 ceph-mon[37486]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:53:40.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:40 smithi196 ceph-mon[37486]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:9a75c81fd074f3cb252248b860d5cc88e5f3fb55 2024-03-13T14:53:41.591 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:41 smithi176 ceph-mon[29064]: pgmap v129: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:41.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:41 smithi196 ceph-mon[37486]: pgmap v129: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:42.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.537+0000 7f9865600700 1 -- 172.21.15.176:0/2082872181 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 msgr2=0x7f98601057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:42.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.537+0000 7f9865600700 1 --2- 172.21.15.176:0/2082872181 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f98601057c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9848009910 tx=0x7f9848009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:42.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.538+0000 7f9865600700 1 -- 172.21.15.176:0/2082872181 shutdown_connections 2024-03-13T14:53:42.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.538+0000 7f9865600700 1 --2- 172.21.15.176:0/2082872181 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f98601057c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.538+0000 7f9865600700 1 --2- 172.21.15.176:0/2082872181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9860100dc0 0x7f98601011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.538+0000 7f9865600700 1 -- 172.21.15.176:0/2082872181 >> 172.21.15.176:0/2082872181 conn(0x7f98600fc650 msgr2=0x7f98600fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:42.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.538+0000 7f9865600700 1 -- 172.21.15.176:0/2082872181 shutdown_connections 2024-03-13T14:53:42.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.538+0000 7f9865600700 1 -- 172.21.15.176:0/2082872181 wait complete. 2024-03-13T14:53:42.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.539+0000 7f9865600700 1 Processor -- start 2024-03-13T14:53:42.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.539+0000 7f9865600700 1 -- start start 2024-03-13T14:53:42.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.539+0000 7f9865600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9860100dc0 0x7f986019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:42.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.539+0000 7f9865600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f986019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:42.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.539+0000 7f9865600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986019f410 con 0x7f9860101770 2024-03-13T14:53:42.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.539+0000 7f9865600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601988c0 con 0x7f9860100dc0 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f9857e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f986019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f9857e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f986019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47340/0 (socket says 172.21.15.176:47340) 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f9857e00700 1 -- 172.21.15.176:0/4041445334 learned_addr learned my addr 172.21.15.176:0/4041445334 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f985ea00700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9860100dc0 0x7f986019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f9857e00700 1 -- 172.21.15.176:0/4041445334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9860100dc0 msgr2=0x7f986019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f9857e00700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9860100dc0 0x7f986019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f9857e00700 1 -- 172.21.15.176:0/4041445334 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98480095f0 con 0x7f9860101770 2024-03-13T14:53:42.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.540+0000 7f9857e00700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f986019ed80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f984800bda0 tx=0x7f984800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:42.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.541+0000 7f9857400700 1 -- 172.21.15.176:0/4041445334 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f984801d070 con 0x7f9860101770 2024-03-13T14:53:42.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.541+0000 7f9857400700 1 -- 172.21.15.176:0/4041445334 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9848004fb0 con 0x7f9860101770 2024-03-13T14:53:42.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.541+0000 7f9857400700 1 -- 172.21.15.176:0/4041445334 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f984800f460 con 0x7f9860101770 2024-03-13T14:53:42.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.541+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9860198b40 con 0x7f9860101770 2024-03-13T14:53:42.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.541+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9860199030 con 0x7f9860101770 2024-03-13T14:53:42.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.542+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9860104f40 con 0x7f9860101770 2024-03-13T14:53:42.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.543+0000 7f9857400700 1 -- 172.21.15.176:0/4041445334 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f9848022800 con 0x7f9860101770 2024-03-13T14:53:42.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.544+0000 7f9857400700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f984406dc20 0x7f98440700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:42.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.544+0000 7f9857400700 1 -- 172.21.15.176:0/4041445334 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f984808ecc0 con 0x7f9860101770 2024-03-13T14:53:42.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.544+0000 7f985ea00700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f984406dc20 0x7f98440700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:42.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.545+0000 7f985ea00700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f984406dc20 0x7f98440700e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9850006fd0 tx=0x7f9850008040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:42.537 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.551+0000 7f9857400700 1 -- 172.21.15.176:0/4041445334 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f9848094020 con 0x7f9860101770 2024-03-13T14:53:42.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.823+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f986002cd10 con 0x7f984406dc20 2024-03-13T14:53:42.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.828+0000 7f9857400700 1 -- 172.21.15.176:0/4041445334 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f986002cd10 con 0x7f984406dc20 2024-03-13T14:53:42.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f984406dc20 msgr2=0x7f98440700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:42.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f984406dc20 0x7f98440700e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9850006fd0 tx=0x7f9850008040 comp rx=0 tx=0).stop 2024-03-13T14:53:42.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 msgr2=0x7f986019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:42.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f986019ed80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f984800bda0 tx=0x7f984800be80 comp rx=0 tx=0).stop 2024-03-13T14:53:42.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 shutdown_connections 2024-03-13T14:53:42.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f984406dc20 0x7f98440700e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9860100dc0 0x7f986019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 --2- 172.21.15.176:0/4041445334 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9860101770 0x7f986019ed80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 >> 172.21.15.176:0/4041445334 conn(0x7f98600fc650 msgr2=0x7f98600fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:42.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 shutdown_connections 2024-03-13T14:53:42.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.831+0000 7f9865600700 1 -- 172.21.15.176:0/4041445334 wait complete. 2024-03-13T14:53:42.835 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:53:42.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.990+0000 7f18ca600700 1 -- 172.21.15.176:0/2288840918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4100da0 msgr2=0x7f18c4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:42.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.990+0000 7f18ca600700 1 --2- 172.21.15.176:0/2288840918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4100da0 0x7f18c4101180 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18b400b0a0 tx=0x7f18b400b3b0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.990+0000 7f18ca600700 1 -- 172.21.15.176:0/2288840918 shutdown_connections 2024-03-13T14:53:42.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.990+0000 7f18ca600700 1 --2- 172.21.15.176:0/2288840918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c4101750 0x7f18c41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.990+0000 7f18ca600700 1 --2- 172.21.15.176:0/2288840918 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4100da0 0x7f18c4101180 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.990+0000 7f18ca600700 1 -- 172.21.15.176:0/2288840918 >> 172.21.15.176:0/2288840918 conn(0x7f18c40fc630 msgr2=0x7f18c40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:42.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.990+0000 7f18ca600700 1 -- 172.21.15.176:0/2288840918 shutdown_connections 2024-03-13T14:53:42.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.991+0000 7f18ca600700 1 -- 172.21.15.176:0/2288840918 wait complete. 2024-03-13T14:53:42.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.992+0000 7f18ca600700 1 Processor -- start 2024-03-13T14:53:42.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.992+0000 7f18ca600700 1 -- start start 2024-03-13T14:53:42.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.992+0000 7f18ca600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c4100da0 0x7f18c419c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:42.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.992+0000 7f18ca600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4101750 0x7f18c419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:42.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.992+0000 7f18ca600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c419d270 con 0x7f18c4101750 2024-03-13T14:53:42.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.992+0000 7f18ca600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4196720 con 0x7f18c4100da0 2024-03-13T14:53:42.980 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.993+0000 7f18baa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4101750 0x7f18c419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:42.980 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.993+0000 7f18baa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4101750 0x7f18c419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47362/0 (socket says 172.21.15.176:47362) 2024-03-13T14:53:42.980 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.993+0000 7f18baa00700 1 -- 172.21.15.176:0/1546383899 learned_addr learned my addr 172.21.15.176:0/1546383899 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:42.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.993+0000 7f18c3400700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c4100da0 0x7f18c419c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:42.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.993+0000 7f18baa00700 1 -- 172.21.15.176:0/1546383899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c4100da0 msgr2=0x7f18c419c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:42.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.993+0000 7f18baa00700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c4100da0 0x7f18c419c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:42.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.993+0000 7f18baa00700 1 -- 172.21.15.176:0/1546383899 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b4009d00 con 0x7f18c4101750 2024-03-13T14:53:42.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.994+0000 7f18baa00700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4101750 0x7f18c419cbe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00b480 tx=0x7f18ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:42.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.994+0000 7f18c0c00700 1 -- 172.21.15.176:0/1546383899 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18ac0118c0 con 0x7f18c4101750 2024-03-13T14:53:42.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.994+0000 7f18c0c00700 1 -- 172.21.15.176:0/1546383899 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18ac010380 con 0x7f18c4101750 2024-03-13T14:53:42.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.994+0000 7f18c0c00700 1 -- 172.21.15.176:0/1546383899 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18ac00f4f0 con 0x7f18c4101750 2024-03-13T14:53:42.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.994+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c4196a00 con 0x7f18c4101750 2024-03-13T14:53:42.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.994+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c4196f50 con 0x7f18c4101750 2024-03-13T14:53:42.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.995+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c4109140 con 0x7f18c4101750 2024-03-13T14:53:42.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.997+0000 7f18c0c00700 1 -- 172.21.15.176:0/1546383899 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f18ac011a20 con 0x7f18c4101750 2024-03-13T14:53:42.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.998+0000 7f18c0c00700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f18a806dc20 0x7f18a80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:42.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.998+0000 7f18c0c00700 1 -- 172.21.15.176:0/1546383899 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f18ac08d370 con 0x7f18c4101750 2024-03-13T14:53:42.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.998+0000 7f18c3400700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f18a806dc20 0x7f18a80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:42.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:42.999+0000 7f18c3400700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f18a806dc20 0x7f18a80700e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009480 tx=0x7f18b400b7f0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:42.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.003+0000 7f18c0c00700 1 -- 172.21.15.176:0/1546383899 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f18ac05a870 con 0x7f18c4101750 2024-03-13T14:53:43.250 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.263+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18c41972a0 con 0x7f18a806dc20 2024-03-13T14:53:43.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.265+0000 7f18c0c00700 1 -- 172.21.15.176:0/1546383899 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f18c41972a0 con 0x7f18a806dc20 2024-03-13T14:53:43.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f18a806dc20 msgr2=0x7f18a80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f18a806dc20 0x7f18a80700e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009480 tx=0x7f18b400b7f0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4101750 msgr2=0x7f18c419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4101750 0x7f18c419cbe0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00b480 tx=0x7f18ac00b840 comp rx=0 tx=0).stop 2024-03-13T14:53:43.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 shutdown_connections 2024-03-13T14:53:43.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f18a806dc20 0x7f18a80700e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f18c4100da0 0x7f18c419c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 --2- 172.21.15.176:0/1546383899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f18c4101750 0x7f18c419cbe0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 >> 172.21.15.176:0/1546383899 conn(0x7f18c40fc630 msgr2=0x7f18c40fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:43.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 shutdown_connections 2024-03-13T14:53:43.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.268+0000 7f18ca600700 1 -- 172.21.15.176:0/1546383899 wait complete. 2024-03-13T14:53:43.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1430557115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4069020 msgr2=0x7fb1e4105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1430557115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4069020 0x7fb1e4105790 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009910 tx=0x7fb1d0009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:43.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1430557115 shutdown_connections 2024-03-13T14:53:43.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1430557115 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4069020 0x7fb1e4105790 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1430557115 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4068670 0x7fb1e4068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1430557115 >> 172.21.15.176:0/1430557115 conn(0x7fb1e40755a0 msgr2=0x7fb1e40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:43.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1430557115 shutdown_connections 2024-03-13T14:53:43.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.425+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1430557115 wait complete. 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.426+0000 7fb1eaa00700 1 Processor -- start 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.426+0000 7fb1eaa00700 1 -- start start 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1eaa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4068670 0x7fb1e419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1eaa00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4069020 0x7fb1e419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1eaa00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e419f3b0 con 0x7fb1e4068670 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1eaa00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4198860 con 0x7fb1e4069020 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1e3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4068670 0x7fb1e419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1e3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4068670 0x7fb1e419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47368/0 (socket says 172.21.15.176:47368) 2024-03-13T14:53:43.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1e3e00700 1 -- 172.21.15.176:0/1092499989 learned_addr learned my addr 172.21.15.176:0/1092499989 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.427+0000 7fb1e3400700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4069020 0x7fb1e419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e3e00700 1 -- 172.21.15.176:0/1092499989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4069020 msgr2=0x7fb1e419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e3e00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4069020 0x7fb1e419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e3e00700 1 -- 172.21.15.176:0/1092499989 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00095f0 con 0x7fb1e4068670 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e3400700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4069020 0x7fb1e419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e3e00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4068670 0x7fb1e419e7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c8a0 tx=0x7fb1d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e0c00700 1 -- 172.21.15.176:0/1092499989 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d400cea0 con 0x7fb1e4068670 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e0c00700 1 -- 172.21.15.176:0/1092499989 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1d4004830 con 0x7fb1e4068670 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e4198b40 con 0x7fb1e4068670 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1e0c00700 1 -- 172.21.15.176:0/1092499989 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d40055b0 con 0x7fb1e4068670 2024-03-13T14:53:43.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.428+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e4199090 con 0x7fb1e4068670 2024-03-13T14:53:43.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.430+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e404ea90 con 0x7fb1e4068670 2024-03-13T14:53:43.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.430+0000 7fb1e0c00700 1 -- 172.21.15.176:0/1092499989 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7fb1d40049a0 con 0x7fb1e4068670 2024-03-13T14:53:43.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.431+0000 7fb1e0c00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fb1c806dc20 0x7fb1c80700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:43.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.432+0000 7fb1e0c00700 1 -- 172.21.15.176:0/1092499989 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb1d408d480 con 0x7fb1e4068670 2024-03-13T14:53:43.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.432+0000 7fb1e3400700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fb1c806dc20 0x7fb1c80700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:43.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.432+0000 7fb1e3400700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fb1c806dc20 0x7fb1c80700e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b5c0 tx=0x7fb1d0005480 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:43.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.436+0000 7fb1e0c00700 1 -- 172.21.15.176:0/1092499989 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fb1d405a8e0 con 0x7fb1e4068670 2024-03-13T14:53:43.495 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:43 smithi176 ceph-mon[29064]: pgmap v130: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:43.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:53:43.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:43.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:53:43.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:43.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:43.496 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:43 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-13T14:53:43.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.670+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1e4199440 con 0x7fb1c806dc20 2024-03-13T14:53:43.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.685+0000 7fb1e0c00700 1 -- 172.21.15.176:0/1092499989 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb1e4199440 con 0x7fb1c806dc20 2024-03-13T14:53:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:53:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (106s) 58s ago 15m 14.0M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:53:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (15m) 58s ago 15m 8070k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:53:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (13m) 2m ago 13m 7868k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:53:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (15m) 58s ago 15m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:53:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (13m) 2m ago 13m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:53:43.672 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (65s) 58s ago 14m 49.1M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (7m) 58s ago 7m 87.1M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (8m) 58s ago 8m 15.2M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (7m) 2m ago 7m 216M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (7m) 2m ago 7m 14.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (4m) 58s ago 17m 499M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (3m) 2m ago 13m 447M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (17m) 58s ago 17m 57.6M 2048M 18.2.1 5be31c24972a 3e3173fcfbc8 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (13m) 2m ago 13m 44.1M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (2m) 58s ago 15m 7222k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (2m) 2m ago 13m 2847k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (12m) 58s ago 12m 231M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (12m) 58s ago 12m 244M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:53:43.673 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (11m) 58s ago 11m 218M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:53:43.674 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (10m) 2m ago 10m 247M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:53:43.674 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (10m) 2m ago 10m 220M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:53:43.674 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (9m) 2m ago 9m 194M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:53:43.674 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (2m) 58s ago 14m 58.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:53:43.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fb1c806dc20 msgr2=0x7fb1c80700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fb1c806dc20 0x7fb1c80700e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000b5c0 tx=0x7fb1d0005480 comp rx=0 tx=0).stop 2024-03-13T14:53:43.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4068670 msgr2=0x7fb1e419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4068670 0x7fb1e419e7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb1d400c8a0 tx=0x7fb1d400cc60 comp rx=0 tx=0).stop 2024-03-13T14:53:43.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 shutdown_connections 2024-03-13T14:53:43.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fb1c806dc20 0x7fb1c80700e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb1e4068670 0x7fb1e419e7e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 --2- 172.21.15.176:0/1092499989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1e4069020 0x7fb1e419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 >> 172.21.15.176:0/1092499989 conn(0x7fb1e40755a0 msgr2=0x7fb1e4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:43.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.688+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 shutdown_connections 2024-03-13T14:53:43.676 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.689+0000 7fb1eaa00700 1 -- 172.21.15.176:0/1092499989 wait complete. 2024-03-13T14:53:43.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:43 smithi196 ceph-mon[37486]: pgmap v130: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:43.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:53:43.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:43.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:53:43.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:43.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:43.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:43 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.844+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3259452862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 msgr2=0x7f6cc4101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.844+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3259452862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc4101180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009910 tx=0x7f6cb8009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.844+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3259452862 shutdown_connections 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.844+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3259452862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc4101750 0x7f6cc41057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.844+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3259452862 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc4101180 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.844+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3259452862 >> 172.21.15.176:0/3259452862 conn(0x7f6cc40fc630 msgr2=0x7f6cc40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.845+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3259452862 shutdown_connections 2024-03-13T14:53:43.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.845+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3259452862 wait complete. 2024-03-13T14:53:43.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.846+0000 7f6ccbe00700 1 Processor -- start 2024-03-13T14:53:43.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.846+0000 7f6ccbe00700 1 -- start start 2024-03-13T14:53:43.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.846+0000 7f6ccbe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:43.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.846+0000 7f6ccbe00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc4101750 0x7f6cc419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:43.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.846+0000 7f6ccbe00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc419d210 con 0x7f6cc4100da0 2024-03-13T14:53:43.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.846+0000 7f6ccbe00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc41966c0 con 0x7f6cc4101750 2024-03-13T14:53:43.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc9400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:43.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc9400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc419c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47394/0 (socket says 172.21.15.176:47394) 2024-03-13T14:53:43.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc9400700 1 -- 172.21.15.176:0/3611629249 learned_addr learned my addr 172.21.15.176:0/3611629249 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:43.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc8a00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc4101750 0x7f6cc419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:43.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc9400700 1 -- 172.21.15.176:0/3611629249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc4101750 msgr2=0x7f6cc419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:43.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc9400700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc4101750 0x7f6cc419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:43.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc9400700 1 -- 172.21.15.176:0/3611629249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb80095f0 con 0x7f6cc4100da0 2024-03-13T14:53:43.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.847+0000 7f6cc9400700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc419c640 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8006010 tx=0x7f6cb800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:43.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.848+0000 7f6cc2000700 1 -- 172.21.15.176:0/3611629249 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb801d070 con 0x7f6cc4100da0 2024-03-13T14:53:43.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.848+0000 7f6cc2000700 1 -- 172.21.15.176:0/3611629249 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cb8004fb0 con 0x7f6cc4100da0 2024-03-13T14:53:43.836 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.848+0000 7f6cc2000700 1 -- 172.21.15.176:0/3611629249 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb8022ea0 con 0x7f6cc4100da0 2024-03-13T14:53:43.836 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.848+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc4196940 con 0x7f6cc4100da0 2024-03-13T14:53:43.836 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.848+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc4196de0 con 0x7f6cc4100da0 2024-03-13T14:53:43.836 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.849+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc4109140 con 0x7f6cc4100da0 2024-03-13T14:53:43.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.850+0000 7f6cc2000700 1 -- 172.21.15.176:0/3611629249 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f6cb8022470 con 0x7f6cc4100da0 2024-03-13T14:53:43.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.851+0000 7f6cc2000700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f6cac06dc20 0x7f6cac0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:43.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.851+0000 7f6cc2000700 1 -- 172.21.15.176:0/3611629249 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6cb808ed10 con 0x7f6cc4100da0 2024-03-13T14:53:43.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.852+0000 7f6cc8a00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f6cac06dc20 0x7f6cac0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:43.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.852+0000 7f6cc8a00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f6cac06dc20 0x7f6cac0700e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0006fd0 tx=0x7f6cb0008040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:43.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:43.856+0000 7f6cc2000700 1 -- 172.21.15.176:0/3611629249 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f6cb805c240 con 0x7f6cc4100da0 2024-03-13T14:53:44.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.160+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6cc404ea90 con 0x7f6cc4100da0 2024-03-13T14:53:44.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.161+0000 7f6cc2000700 1 -- 172.21.15.176:0/3611629249 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f6cb8022720 con 0x7f6cc4100da0 2024-03-13T14:53:44.148 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:53:44.148 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:53:44.148 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2 2024-03-13T14:53:44.148 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:53:44.148 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 12, 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:53:44.149 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:53:44.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f6cac06dc20 msgr2=0x7f6cac0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f6cac06dc20 0x7f6cac0700e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0006fd0 tx=0x7f6cb0008040 comp rx=0 tx=0).stop 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 msgr2=0x7f6cc419c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc419c640 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8006010 tx=0x7f6cb800be20 comp rx=0 tx=0).stop 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 shutdown_connections 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f6cac06dc20 0x7f6cac0700e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6cc4100da0 0x7f6cc419c640 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 --2- 172.21.15.176:0/3611629249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc4101750 0x7f6cc419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.163+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 >> 172.21.15.176:0/3611629249 conn(0x7f6cc40fc630 msgr2=0x7f6cc40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.164+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 shutdown_connections 2024-03-13T14:53:44.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.164+0000 7f6ccbe00700 1 -- 172.21.15.176:0/3611629249 wait complete. 2024-03-13T14:53:44.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 -- 172.21.15.176:0/1036328105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 msgr2=0x7f4db0106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 --2- 172.21.15.176:0/1036328105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0106d50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4da4009910 tx=0x7f4da4009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:44.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 -- 172.21.15.176:0/1036328105 shutdown_connections 2024-03-13T14:53:44.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 --2- 172.21.15.176:0/1036328105 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db0100970 0x7f4db0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 --2- 172.21.15.176:0/1036328105 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0106d50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 -- 172.21.15.176:0/1036328105 >> 172.21.15.176:0/1036328105 conn(0x7f4db00fc440 msgr2=0x7f4db00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:44.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 -- 172.21.15.176:0/1036328105 shutdown_connections 2024-03-13T14:53:44.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.317+0000 7f4db7000700 1 -- 172.21.15.176:0/1036328105 wait complete. 2024-03-13T14:53:44.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.318+0000 7f4db7000700 1 Processor -- start 2024-03-13T14:53:44.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.318+0000 7f4db7000700 1 -- start start 2024-03-13T14:53:44.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db7000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db0100970 0x7f4db0196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:44.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db7000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:44.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db7000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db0197160 con 0x7f4db0106970 2024-03-13T14:53:44.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db7000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db019aef0 con 0x7f4db0100970 2024-03-13T14:53:44.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db5200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:44.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db5200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47404/0 (socket says 172.21.15.176:47404) 2024-03-13T14:53:44.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db5200700 1 -- 172.21.15.176:0/835491338 learned_addr learned my addr 172.21.15.176:0/835491338 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:44.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.319+0000 7f4db5c00700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db0100970 0x7f4db0196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4db5200700 1 -- 172.21.15.176:0/835491338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db0100970 msgr2=0x7f4db0196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4db5200700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db0100970 0x7f4db0196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4db5200700 1 -- 172.21.15.176:0/835491338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40095f0 con 0x7f4db0106970 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4db5200700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0196a80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00d8d0 tx=0x7f4d9c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4dae000700 1 -- 172.21.15.176:0/835491338 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d9c0098a0 con 0x7f4db0106970 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4dae000700 1 -- 172.21.15.176:0/835491338 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d9c010460 con 0x7f4db0106970 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db019b1d0 con 0x7f4db0106970 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4dae000700 1 -- 172.21.15.176:0/835491338 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d9c00f5f0 con 0x7f4db0106970 2024-03-13T14:53:44.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.320+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db019b720 con 0x7f4db0106970 2024-03-13T14:53:44.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.321+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db0108f70 con 0x7f4db0106970 2024-03-13T14:53:44.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.323+0000 7f4dae000700 1 -- 172.21.15.176:0/835491338 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f4d9c009a00 con 0x7f4db0106970 2024-03-13T14:53:44.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.324+0000 7f4dae000700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4d9806dc20 0x7f4d980700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:44.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.324+0000 7f4dae000700 1 -- 172.21.15.176:0/835491338 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4d9c08d640 con 0x7f4db0106970 2024-03-13T14:53:44.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.324+0000 7f4db5c00700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4d9806dc20 0x7f4d980700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:44.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.325+0000 7f4db5c00700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4d9806dc20 0x7f4d980700e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006010 tx=0x7f4da401a040 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:44.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.329+0000 7f4dae000700 1 -- 172.21.15.176:0/835491338 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f4d9c056f70 con 0x7f4db0106970 2024-03-13T14:53:44.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:44 smithi176 ceph-mon[29064]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:44.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:44 smithi176 ceph-mon[29064]: Upgrade: Target is version 19.0.0-1916-g9a75c81f (squid) 2024-03-13T14:53:44.371 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:44 smithi176 ceph-mon[29064]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a'] 2024-03-13T14:53:44.372 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:44 smithi176 ceph-mon[29064]: Upgrade: Setting container_image for all mgr 2024-03-13T14:53:44.372 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:44 smithi176 ceph-mon[29064]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:44.372 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:44 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3611629249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:44.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.590+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4db004f2e0 con 0x7f4db0106970 2024-03-13T14:53:44.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.591+0000 7f4dae000700 1 -- 172.21.15.176:0/835491338 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1856 (secure 0 0 0) 0x7f4d9c05a630 con 0x7f4db0106970 2024-03-13T14:53:44.578 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:53:44.579 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:53:44.580 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:44.581 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:44.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4d9806dc20 msgr2=0x7f4d980700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4d9806dc20 0x7f4d980700e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4da4006010 tx=0x7f4da401a040 comp rx=0 tx=0).stop 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 msgr2=0x7f4db0196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0196a80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00d8d0 tx=0x7f4d9c00dbe0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 shutdown_connections 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f4d9806dc20 0x7f4d980700e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4db0100970 0x7f4db0196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 --2- 172.21.15.176:0/835491338 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4db0106970 0x7f4db0196a80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 >> 172.21.15.176:0/835491338 conn(0x7f4db00fc440 msgr2=0x7f4db00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 shutdown_connections 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.594+0000 7f4db7000700 1 -- 172.21.15.176:0/835491338 wait complete. 2024-03-13T14:53:44.583 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:53:44.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:44 smithi196 ceph-mon[37486]: from='client.14866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:44.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:44 smithi196 ceph-mon[37486]: Upgrade: Target is version 19.0.0-1916-g9a75c81f (squid) 2024-03-13T14:53:44.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:44 smithi196 ceph-mon[37486]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a'] 2024-03-13T14:53:44.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:44 smithi196 ceph-mon[37486]: Upgrade: Setting container_image for all mgr 2024-03-13T14:53:44.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:44 smithi196 ceph-mon[37486]: from='client.14870 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:44.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:44 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3611629249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:53:44.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 -- 172.21.15.176:0/3764188732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 msgr2=0x7f9e18100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 --2- 172.21.15.176:0/3764188732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e18100e20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9e04009910 tx=0x7f9e04009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:44.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 -- 172.21.15.176:0/3764188732 shutdown_connections 2024-03-13T14:53:44.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 --2- 172.21.15.176:0/3764188732 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e18100e20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 --2- 172.21.15.176:0/3764188732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e181069a0 0x7f9e18106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 -- 172.21.15.176:0/3764188732 >> 172.21.15.176:0/3764188732 conn(0x7f9e180fc490 msgr2=0x7f9e180fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:44.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 -- 172.21.15.176:0/3764188732 shutdown_connections 2024-03-13T14:53:44.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.748+0000 7f9e17e00700 1 -- 172.21.15.176:0/3764188732 wait complete. 2024-03-13T14:53:44.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.749+0000 7f9e17e00700 1 Processor -- start 2024-03-13T14:53:44.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.749+0000 7f9e17e00700 1 -- start start 2024-03-13T14:53:44.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e17e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e1819e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:44.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e17e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e181069a0 0x7f9e1819ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:44.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e17e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1819f110 con 0x7f9e181009a0 2024-03-13T14:53:44.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e17e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181985c0 con 0x7f9e181069a0 2024-03-13T14:53:44.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e16a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e1819e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:44.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e16a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e1819e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47424/0 (socket says 172.21.15.176:47424) 2024-03-13T14:53:44.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e16a00700 1 -- 172.21.15.176:0/2388306010 learned_addr learned my addr 172.21.15.176:0/2388306010 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:44.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e16a00700 1 -- 172.21.15.176:0/2388306010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e181069a0 msgr2=0x7f9e1819ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e16000700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e181069a0 0x7f9e1819ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:44.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e16a00700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e181069a0 0x7f9e1819ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.750+0000 7f9e16a00700 1 -- 172.21.15.176:0/2388306010 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040095f0 con 0x7f9e181009a0 2024-03-13T14:53:44.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.751+0000 7f9e16a00700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e1819e540 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800d8d0 tx=0x7f9e0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:44.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.751+0000 7f9e0f400700 1 -- 172.21.15.176:0/2388306010 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e08009940 con 0x7f9e181009a0 2024-03-13T14:53:44.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.751+0000 7f9e0f400700 1 -- 172.21.15.176:0/2388306010 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e08010460 con 0x7f9e181009a0 2024-03-13T14:53:44.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.751+0000 7f9e0f400700 1 -- 172.21.15.176:0/2388306010 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e0800f5f0 con 0x7f9e181009a0 2024-03-13T14:53:44.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.751+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e181988a0 con 0x7f9e181009a0 2024-03-13T14:53:44.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.751+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e18198df0 con 0x7f9e181009a0 2024-03-13T14:53:44.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.752+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1804f2e0 con 0x7f9e181009a0 2024-03-13T14:53:44.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.753+0000 7f9e0f400700 1 -- 172.21.15.176:0/2388306010 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7f9e080105d0 con 0x7f9e181009a0 2024-03-13T14:53:44.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.753+0000 7f9e0f400700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f9dfc06dc20 0x7f9dfc0700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:44.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.753+0000 7f9e0f400700 1 -- 172.21.15.176:0/2388306010 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9e0808d7b0 con 0x7f9e181009a0 2024-03-13T14:53:44.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.753+0000 7f9e16000700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f9dfc06dc20 0x7f9dfc0700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:44.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.753+0000 7f9e16000700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f9dfc06dc20 0x7f9dfc0700e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04005fd0 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:44.747 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:44.761+0000 7f9e0f400700 1 -- 172.21.15.176:0/2388306010 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7f9e0805ac10 con 0x7f9e181009a0 2024-03-13T14:53:44.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.000+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 --> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e181992c0 con 0x7f9dfc06dc20 2024-03-13T14:53:44.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.002+0000 7f9e0f400700 1 -- 172.21.15.176:0/2388306010 <== mgr.14662 v2:172.21.15.176:6800/3527709806 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f9e181992c0 con 0x7f9dfc06dc20 2024-03-13T14:53:44.989 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:53:44.989 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:53:44.990 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:53:44.990 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:53:44.990 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:53:44.990 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:53:44.990 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:53:44.990 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "2/23 daemons upgraded", 2024-03-13T14:53:44.990 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mon daemons", 2024-03-13T14:53:44.991 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:53:44.991 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:53:44.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f9dfc06dc20 msgr2=0x7f9dfc0700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f9dfc06dc20 0x7f9dfc0700e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000c00 tx=0x7f9e04005fd0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 msgr2=0x7f9e1819e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:44.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e1819e540 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800d8d0 tx=0x7f9e0800dc90 comp rx=0 tx=0).stop 2024-03-13T14:53:44.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 shutdown_connections 2024-03-13T14:53:44.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7f9dfc06dc20 0x7f9dfc0700e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9e181009a0 0x7f9e1819e540 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 --2- 172.21.15.176:0/2388306010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e181069a0 0x7f9e1819ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:44.993 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 >> 172.21.15.176:0/2388306010 conn(0x7f9e180fc490 msgr2=0x7f9e18073020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:44.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 shutdown_connections 2024-03-13T14:53:44.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.005+0000 7f9e17e00700 1 -- 172.21.15.176:0/2388306010 wait complete. 2024-03-13T14:53:45.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 -- 172.21.15.176:0/1357216253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 msgr2=0x7fd6cc0756a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:45.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/1357216253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc0756a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc009910 tx=0x7fd6bc009c20 comp rx=0 tx=0).stop 2024-03-13T14:53:45.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 -- 172.21.15.176:0/1357216253 shutdown_connections 2024-03-13T14:53:45.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/1357216253 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc075be0 0x7fd6cc1114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:45.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/1357216253 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc0756a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:45.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 -- 172.21.15.176:0/1357216253 >> 172.21.15.176:0/1357216253 conn(0x7fd6cc0fe880 msgr2=0x7fd6cc100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:45.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 -- 172.21.15.176:0/1357216253 shutdown_connections 2024-03-13T14:53:45.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.147+0000 7fd6d2a00700 1 -- 172.21.15.176:0/1357216253 wait complete. 2024-03-13T14:53:45.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.149+0000 7fd6d2a00700 1 Processor -- start 2024-03-13T14:53:45.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.149+0000 7fd6d2a00700 1 -- start start 2024-03-13T14:53:45.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.149+0000 7fd6d2a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:45.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.149+0000 7fd6d2a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc075be0 0x7fd6cc19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:45.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6d2a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc19d810 con 0x7fd6cc0752c0 2024-03-13T14:53:45.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6d2a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc19d980 con 0x7fd6cc075be0 2024-03-13T14:53:45.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6cbe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:45.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6cbe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47438/0 (socket says 172.21.15.176:47438) 2024-03-13T14:53:45.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6cbe00700 1 -- 172.21.15.176:0/3345621086 learned_addr learned my addr 172.21.15.176:0/3345621086 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:53:45.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6cb400700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc075be0 0x7fd6cc19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:45.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6cbe00700 1 -- 172.21.15.176:0/3345621086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc075be0 msgr2=0x7fd6cc19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:45.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6cbe00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc075be0 0x7fd6cc19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:45.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.150+0000 7fd6cbe00700 1 -- 172.21.15.176:0/3345621086 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc0095f0 con 0x7fd6cc0752c0 2024-03-13T14:53:45.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.151+0000 7fd6cbe00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc19ccd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc005b40 tx=0x7fd6bc0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:45.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.151+0000 7fd6c8c00700 1 -- 172.21.15.176:0/3345621086 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6bc01d070 con 0x7fd6cc0752c0 2024-03-13T14:53:45.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.151+0000 7fd6c8c00700 1 -- 172.21.15.176:0/3345621086 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6bc00f460 con 0x7fd6cc0752c0 2024-03-13T14:53:45.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.151+0000 7fd6c8c00700 1 -- 172.21.15.176:0/3345621086 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6bc021640 con 0x7fd6cc0752c0 2024-03-13T14:53:45.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.151+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc1a0720 con 0x7fd6cc0752c0 2024-03-13T14:53:45.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.151+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc1a0c20 con 0x7fd6cc0752c0 2024-03-13T14:53:45.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.152+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc10ec60 con 0x7fd6cc0752c0 2024-03-13T14:53:45.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.153+0000 7fd6c8c00700 1 -- 172.21.15.176:0/3345621086 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 32) v1 ==== 91696+0+0 (secure 0 0 0) 0x7fd6bc00f5d0 con 0x7fd6cc0752c0 2024-03-13T14:53:45.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.155+0000 7fd6c8c00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd6b006dc20 0x7fd6b00700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:53:45.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.155+0000 7fd6c8c00700 1 -- 172.21.15.176:0/3345621086 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd6bc08eba0 con 0x7fd6cc0752c0 2024-03-13T14:53:45.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.156+0000 7fd6cb400700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd6b006dc20 0x7fd6b00700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:53:45.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.156+0000 7fd6cb400700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd6b006dc20 0x7fd6b00700e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc19e2b0 tx=0x7fd6b8005d50 comp rx=0 tx=0).ready entity=mgr.14662 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:53:45.146 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.159+0000 7fd6c8c00700 1 -- 172.21.15.176:0/3345621086 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184086 (secure 0 0 0) 0x7fd6bc058450 con 0x7fd6cc0752c0 2024-03-13T14:53:45.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:45 smithi176 ceph-mon[29064]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:45.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:45 smithi176 ceph-mon[29064]: pgmap v131: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:45.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:45 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/835491338' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:53:45.460 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.473+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd6cc04ea90 con 0x7fd6cc0752c0 2024-03-13T14:53:45.461 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.474+0000 7fd6c8c00700 1 -- 172.21.15.176:0/3345621086 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd6bc05bb10 con 0x7fd6cc0752c0 2024-03-13T14:53:45.461 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:53:45.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.476+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd6b006dc20 msgr2=0x7fd6b00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:45.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.476+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd6b006dc20 0x7fd6b00700e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc19e2b0 tx=0x7fd6b8005d50 comp rx=0 tx=0).stop 2024-03-13T14:53:45.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.476+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 msgr2=0x7fd6cc19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:53:45.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.476+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc19ccd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc005b40 tx=0x7fd6bc0056a0 comp rx=0 tx=0).stop 2024-03-13T14:53:45.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.477+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 shutdown_connections 2024-03-13T14:53:45.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.477+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:6800/3527709806,v1:172.21.15.176:6801/3527709806] conn(0x7fd6b006dc20 0x7fd6b00700e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:45.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.477+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd6cc0752c0 0x7fd6cc19ccd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:45.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.477+0000 7fd6d2a00700 1 --2- 172.21.15.176:0/3345621086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6cc075be0 0x7fd6cc19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:53:45.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.477+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 >> 172.21.15.176:0/3345621086 conn(0x7fd6cc0fe880 msgr2=0x7fd6cc100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:53:45.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.477+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 shutdown_connections 2024-03-13T14:53:45.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:53:45.477+0000 7fd6d2a00700 1 -- 172.21.15.176:0/3345621086 wait complete. 2024-03-13T14:53:45.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:45 smithi196 ceph-mon[37486]: from='client.14874 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:45.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:45 smithi196 ceph-mon[37486]: pgmap v131: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:45.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:45 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/835491338' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:53:46.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:46.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: Upgrade: Updating mon.smithi176 2024-03-13T14:53:46.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:46.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:53:46.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:53:46.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:53:46.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: Deploying daemon mon.smithi176 on smithi176 2024-03-13T14:53:46.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:46 smithi176 ceph-mon[29064]: from='client.? 172.21.15.176:0/3345621086' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:53:46.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: from='client.14886 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:53:46.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: Upgrade: Updating mon.smithi176 2024-03-13T14:53:46.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:53:46.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:53:46.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:53:46.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:53:46.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: Deploying daemon mon.smithi176 on smithi176 2024-03-13T14:53:46.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:46 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3345621086' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:53:47.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:47 smithi176 ceph-mon[29064]: pgmap v132: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:47.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:47 smithi196 ceph-mon[37486]: pgmap v132: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:53:47.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:47 smithi176 systemd[1]: Stopping Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:53:48.399 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:48 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[29060]: 2024-03-13T14:53:48.154+0000 7fd68d600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi176 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:53:48.399 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:48 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[29060]: 2024-03-13T14:53:48.154+0000 7fd68d600700 -1 mon.smithi176@0(leader) e2 *** Got Signal Terminated *** 2024-03-13T14:53:48.716 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:48 smithi176 podman[112571]: 2024-03-13 14:53:48.412320489 +0000 UTC m=+0.397304428 container died 3e3173fcfbc8bed856c14c83f765b1b53cec73ff99b5fba4ec796561f4b77dff (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.vendor=CentOS) 2024-03-13T14:53:49.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:48 smithi176 podman[112571]: 2024-03-13 14:53:48.873554013 +0000 UTC m=+0.858537942 container cleanup 3e3173fcfbc8bed856c14c83f765b1b53cec73ff99b5fba4ec796561f4b77dff (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-03-13T14:53:49.095 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:48 smithi176 bash[112571]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176 2024-03-13T14:53:49.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:49 smithi176 podman[112596]: 2024-03-13 14:53:49.107922046 +0000 UTC m=+0.683647764 container remove 3e3173fcfbc8bed856c14c83f765b1b53cec73ff99b5fba4ec796561f4b77dff (image=quay.io/ceph/ceph:v18.2.1, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-13T14:53:49.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:49 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service: Deactivated successfully. 2024-03-13T14:53:49.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:49 smithi176 systemd[1]: Stopped Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:53:49.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:49 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service: Consumed 31.930s CPU time. 2024-03-13T14:53:50.342 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 systemd[1]: Starting Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:53:50.618 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 podman[112724]: 2024-03-13 14:53:50.264676729 +0000 UTC m=+0.036139809 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:53:50.618 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 podman[112724]: 2024-03-13 14:53:50.383767172 +0000 UTC m=+0.155230288 container create 35f4ac6dca3064405c7650c6475f1f4692ccd31856e7a7c14c488c0454ead7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:53:50.618 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 podman[112724]: 2024-03-13 14:53:50.629350849 +0000 UTC m=+0.400813949 container init 35f4ac6dca3064405c7650c6475f1f4692ccd31856e7a7c14c488c0454ead7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T14:53:50.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 podman[112724]: 2024-03-13 14:53:50.640041981 +0000 UTC m=+0.411505081 container start 35f4ac6dca3064405c7650c6475f1f4692ccd31856e7a7c14c488c0454ead7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:53:50.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: set uid:gid to 167:167 (ceph:ceph) 2024-03-13T14:53:50.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev), process ceph-mon, pid 2 2024-03-13T14:53:50.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: pidfile_write: ignore empty --pid-file 2024-03-13T14:53:50.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: load: jerasure load: lrc 2024-03-13T14:53:50.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: RocksDB version: 7.9.2 2024-03-13T14:53:50.880 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Git sha 0 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Compile date 2024-03-07 04:59:45 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: DB SUMMARY 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: DB Session ID: F7L5L9OCR3S8N9MC5TY5 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: CURRENT file: CURRENT 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: IDENTITY file: IDENTITY 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1732 Bytes 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi176/store.db dir, Total Num: 1, files: 000035.sst 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi176/store.db: 000033.log size: 528827 ; 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.error_if_exists: 0 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.create_if_missing: 0 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.paranoid_checks: 1 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-13T14:53:50.881 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.env: 0x556b84b32b20 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.fs: PosixFileSystem 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.info_log: 0x556b8545d7c0 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_file_opening_threads: 16 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.statistics: (nil) 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.use_fsync: 0 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_log_file_size: 0 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.keep_log_file_num: 1000 2024-03-13T14:53:50.882 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.recycle_log_file_num: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.allow_fallocate: 1 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.allow_mmap_reads: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.allow_mmap_writes: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.use_direct_reads: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.create_missing_column_families: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.db_log_dir: 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.wal_dir: 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-13T14:53:50.883 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.advise_random_on_open: 1 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.db_write_buffer_size: 0 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.write_buffer_manager: 0x556b85431400 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.rate_limiter: (nil) 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.wal_recovery_mode: 2 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.enable_thread_tracking: 0 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.enable_pipelined_write: 0 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.unordered_write: 0 2024-03-13T14:53:50.884 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.row_cache: None 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.wal_filter: None 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.allow_ingest_behind: 0 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.two_write_queues: 0 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.manual_wal_flush: 0 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.wal_compression: 0 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.atomic_flush: 0 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-13T14:53:50.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.log_readahead_size: 0 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.best_efforts_recovery: 0 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.allow_data_in_errors: 0 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.db_host_id: __hostname__ 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-13T14:53:50.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_background_jobs: 2 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_background_compactions: -1 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_subcompactions: 1 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_total_wal_size: 0 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_open_files: -1 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bytes_per_sync: 0 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-13T14:53:50.887 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_readahead_size: 0 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_background_flushes: -1 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Compression algorithms supported: 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kZSTD supported: 0 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kXpressCompression supported: 0 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kLZ4HCCompression supported: 1 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kZlibCompression supported: 1 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kSnappyCompression supported: 1 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kLZ4Compression supported: 1 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: kBZip2Compression supported: 0 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi176/store.db/MANIFEST-000015 2024-03-13T14:53:50.888 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-13T14:53:50.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-13T14:53:50.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.merge_operator: 2024-03-13T14:53:50.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_filter: None 2024-03-13T14:53:50.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_filter_factory: None 2024-03-13T14:53:50.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.sst_partitioner_factory: None 2024-03-13T14:53:50.889 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556b8545d8e0) 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: cache_index_and_filter_blocks: 1 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: pin_top_level_index_and_filter: 1 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_type: 0 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_block_index_type: 0 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_shortening: 1 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: checksum: 4 2024-03-13T14:53:50.890 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: no_block_cache: 0 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache: 0x556b85463090 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_name: BinnedLRUCache 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_options: 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: capacity : 536870912 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: num_shard_bits : 4 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: strict_capacity_limit : 0 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: high_pri_pool_ratio: 0.000 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_cache_compressed: (nil) 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: persistent_cache: (nil) 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_size: 4096 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_size_deviation: 10 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_restart_interval: 16 2024-03-13T14:53:50.891 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: index_block_restart_interval: 1 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: metadata_block_size: 4096 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: partition_filters: 0 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: use_delta_encoding: 1 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: filter_policy: bloomfilter 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: whole_key_filtering: 1 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: verify_compression: 0 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: read_amp_bytes_per_bit: 0 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: format_version: 5 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: enable_index_compression: 1 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: block_align: 0 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_auto_readahead_size: 262144 2024-03-13T14:53:50.892 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: prepopulate_block_cache: 0 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: initial_auto_readahead_size: 8192 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: num_file_reads_for_auto_readahead: 2 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.write_buffer_size: 33554432 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_write_buffer_number: 2 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression: NoCompression 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression: Disabled 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.prefix_extractor: nullptr 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.num_levels: 7 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-13T14:53:50.893 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.level: 32767 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.strategy: 0 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:53:50.894 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.enabled: false 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.target_file_size_base: 67108864 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-13T14:53:50.895 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-13T14:53:50.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-13T14:53:50.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-13T14:53:50.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-13T14:53:50.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-13T14:53:50.896 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.arena_block_size: 1048576 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.disable_auto_compactions: 0 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-13T14:53:50.897 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.table_properties_collectors: 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.inplace_update_support: 0 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-13T14:53:50.898 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.bloom_locality: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.max_successive_merges: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.paranoid_file_checks: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.force_consistency_checks: 1 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.report_bg_io_stats: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.ttl: 2592000 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.enable_blob_files: false 2024-03-13T14:53:50.899 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.min_blob_size: 0 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.blob_file_size: 268435456 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.blob_file_starting_level: 0 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi176/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 37, last_sequence is 16512, log_number is 33,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 33 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 33 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a030fa90-416d-4be9-b1b0-c6a192d0c156 2024-03-13T14:53:50.900 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710341630697657, "job": 1, "event": "recovery_started", "wal_files": [33]} 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #33 mode 2 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710341630705029, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 38, "file_size": 514049, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16513, "largest_seqno": 17364, "table_properties": {"data_size": 510149, "index_size": 1926, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 965, "raw_key_size": 9363, "raw_average_key_size": 25, "raw_value_size": 502346, "raw_average_value_size": 1343, "num_data_blocks": 86, "num_entries": 374, "num_filter_entries": 374, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710341630, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a030fa90-416d-4be9-b1b0-c6a192d0c156", "db_session_id": "F7L5L9OCR3S8N9MC5TY5", "orig_file_number": 38, "seqno_to_time_mapping": "N/A"}} 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710341630705188, "job": 1, "event": "recovery_finished"} 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/version_set.cc:5047] Creating manifest 40 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi176/store.db/000033.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556b85554000 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: DB pointer 0x556b85540000 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** DB Stats ** 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-13T14:53:50.901 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** Compaction Stats [default] ** 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: L0 1/0 502.00 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: L6 1/0 7.99 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Sum 2/0 8.48 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-13T14:53:50.902 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** Compaction Stats [default] ** 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.6 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-13T14:53:50.903 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Cumulative compaction: 0.00 GB write, 36.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Interval compaction: 0.00 GB write, 36.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Block cache BinnedLRUCache@0x556b85463090#2 capacity: 512.00 MB usage: 3.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.03 KB,0.000196695%) IndexBlock(1,2.08 KB,0.000396371%) Misc(1,0.00 KB,0%) 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: starting mon.smithi176 rank 0 at public addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] at bind addrs [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi176 fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???) e2 preinit fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???).mds e18 new map 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???).mds e18 print_map 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: e18 2024-03-13T14:53:50.904 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: legacy client fscid: 1 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Filesystem 'cephfs' (1) 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: fs_name cephfs 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: epoch 18 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: tableserver 0 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: root 0 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: session_timeout 60 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: session_autoclose 300 2024-03-13T14:53:50.905 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_file_size 1099511627776 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_xattr_size 65536 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: required_client_features {} 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: last_failure 0 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: last_failure_osd_epoch 47 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: max_mds 1 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: in 0 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: up {0=24249} 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: failed 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: damaged 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: stopped 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: data_pools [3] 2024-03-13T14:53:50.906 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: metadata_pool 2 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: inline_data disabled 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: balancer 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: bal_rank_mask -1 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: standby_count_wanted 1 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: Standby daemons: 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout: [mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-13T14:53:50.907 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-13T14:53:50.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-03-13T14:53:50.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 ceph-mon[112742]: mon.smithi176@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-03-13T14:53:50.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 bash[112724]: 35f4ac6dca3064405c7650c6475f1f4692ccd31856e7a7c14c488c0454ead7dc 2024-03-13T14:53:50.908 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:50 smithi176 systemd[1]: Started Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: pgmap v134: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: mon.smithi176 calling monitor election 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: mon.smithi176 is new leader, mons smithi176,smithi196 in quorum (ranks 0,1) 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: monmap e2: 2 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],smithi196=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:53:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: osdmap e49: 6 total, 6 up, 6 in 2024-03-13T14:53:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: mgrmap e32: smithi176.tzfdwd(active, since 4m), standbys: smithi196.vdktbk 2024-03-13T14:53:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: overall HEALTH_OK 2024-03-13T14:53:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: from='mgr.14662 ' entity='' 2024-03-13T14:53:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:53:51 smithi196 ceph-mon[37486]: mgrmap e33: smithi176.tzfdwd(active, since 4m), standbys: smithi196.vdktbk 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: from='mgr.14662 172.21.15.176:0/1304872084' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: pgmap v134: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: mon.smithi176 calling monitor election 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: mon.smithi176 is new leader, mons smithi176,smithi196 in quorum (ranks 0,1) 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: monmap e2: 2 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],smithi196=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: osdmap e49: 6 total, 6 up, 6 in 2024-03-13T14:53:51.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: mgrmap e32: smithi176.tzfdwd(active, since 4m), standbys: smithi196.vdktbk 2024-03-13T14:53:51.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: overall HEALTH_OK 2024-03-13T14:53:51.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: from='mgr.14662 ' entity='' 2024-03-13T14:53:51.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:53:51 smithi176 ceph-mon[112742]: mgrmap e33: smithi176.tzfdwd(active, since 4m), standbys: smithi196.vdktbk 2024-03-13T14:54:00.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:00 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk restarted 2024-03-13T14:54:00.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:00 smithi196 ceph-mon[37486]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:54:00.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:00 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:54:00.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:00 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:54:00.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:00 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:54:00.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:00 smithi196 ceph-mon[37486]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:54:00.506 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:00 smithi176 ceph-mon[112742]: Standby manager daemon smithi196.vdktbk restarted 2024-03-13T14:54:00.506 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:00 smithi176 ceph-mon[112742]: Standby manager daemon smithi196.vdktbk started 2024-03-13T14:54:00.506 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:00 smithi176 ceph-mon[112742]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/crt"}]: dispatch 2024-03-13T14:54:00.506 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:00 smithi176 ceph-mon[112742]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-13T14:54:00.506 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:00 smithi176 ceph-mon[112742]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi196.vdktbk/key"}]: dispatch 2024-03-13T14:54:00.506 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:00 smithi176 ceph-mon[112742]: from='mgr.? 172.21.15.196:0/787448615' entity='mgr.smithi196.vdktbk' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: mgrmap e34: smithi176.tzfdwd(active, since 4m), standbys: smithi196.vdktbk 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: Active manager daemon smithi176.tzfdwd restarted 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: Activating manager daemon smithi176.tzfdwd 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: osdmap e50: 6 total, 6 up, 6 in 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: mgrmap e35: smithi176.tzfdwd(active, starting, since 0.0149002s), standbys: smithi196.vdktbk 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:54:01.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:54:01.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:54:01.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:01 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: mgrmap e34: smithi176.tzfdwd(active, since 4m), standbys: smithi196.vdktbk 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: Active manager daemon smithi176.tzfdwd restarted 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: Activating manager daemon smithi176.tzfdwd 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: osdmap e50: 6 total, 6 up, 6 in 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: mgrmap e35: smithi176.tzfdwd(active, starting, since 0.0149002s), standbys: smithi196.vdktbk 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T14:54:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi176.tzfdwd", "id": "smithi176.tzfdwd"}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr metadata", "who": "smithi196.vdktbk", "id": "smithi196.vdktbk"}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: Manager daemon smithi176.tzfdwd is now available 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:01.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:01.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:54:01.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/mirror_snapshot_schedule"}]: dispatch 2024-03-13T14:54:01.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:54:01.680 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:01 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi176.tzfdwd/trash_purge_schedule"}]: dispatch 2024-03-13T14:54:02.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:02 smithi176 ceph-mon[112742]: [13/Mar/2024:14:54:00] ENGINE Bus STARTING 2024-03-13T14:54:02.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:02 smithi176 ceph-mon[112742]: [13/Mar/2024:14:54:01] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:54:02.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:02 smithi176 ceph-mon[112742]: [13/Mar/2024:14:54:01] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:54:02.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:02 smithi176 ceph-mon[112742]: [13/Mar/2024:14:54:01] ENGINE Bus STARTED 2024-03-13T14:54:02.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:02 smithi176 ceph-mon[112742]: mgrmap e36: smithi176.tzfdwd(active, since 1.02603s), standbys: smithi196.vdktbk 2024-03-13T14:54:02.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:02 smithi176 ceph-mon[112742]: pgmap v3: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:02.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:02 smithi196 ceph-mon[37486]: [13/Mar/2024:14:54:00] ENGINE Bus STARTING 2024-03-13T14:54:02.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:02 smithi196 ceph-mon[37486]: [13/Mar/2024:14:54:01] ENGINE Serving on https://172.21.15.176:7150 2024-03-13T14:54:02.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:02 smithi196 ceph-mon[37486]: [13/Mar/2024:14:54:01] ENGINE Serving on http://172.21.15.176:8765 2024-03-13T14:54:02.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:02 smithi196 ceph-mon[37486]: [13/Mar/2024:14:54:01] ENGINE Bus STARTED 2024-03-13T14:54:02.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:02 smithi196 ceph-mon[37486]: mgrmap e36: smithi176.tzfdwd(active, since 1.02603s), standbys: smithi196.vdktbk 2024-03-13T14:54:02.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:02 smithi196 ceph-mon[37486]: pgmap v3: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:03.342 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:03 smithi196 ceph-mon[37486]: pgmap v4: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:03.342 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:03 smithi196 ceph-mon[37486]: mgrmap e37: smithi176.tzfdwd(active, since 2s), standbys: smithi196.vdktbk 2024-03-13T14:54:03.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:03 smithi176 ceph-mon[112742]: pgmap v4: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:03.630 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:03 smithi176 ceph-mon[112742]: mgrmap e37: smithi176.tzfdwd(active, since 2s), standbys: smithi196.vdktbk 2024-03-13T14:54:05.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:05 smithi176 ceph-mon[112742]: pgmap v5: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:05.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:05 smithi176 ceph-mon[112742]: mgrmap e38: smithi176.tzfdwd(active, since 4s), standbys: smithi196.vdktbk 2024-03-13T14:54:05.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:05 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:05.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:05 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:05.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:05 smithi196 ceph-mon[37486]: pgmap v5: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:05.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:05 smithi196 ceph-mon[37486]: mgrmap e38: smithi176.tzfdwd(active, since 4s), standbys: smithi196.vdktbk 2024-03-13T14:54:05.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:05 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:05.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:05 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:07.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:07 smithi176 ceph-mon[112742]: pgmap v6: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:07.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:07 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:07.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:07 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:07.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:07 smithi196 ceph-mon[37486]: pgmap v6: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-03-13T14:54:07.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:07 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:07.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:07 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:08.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:08 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:08.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:08 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:08.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:08 smithi176 ceph-mon[112742]: pgmap v7: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-13T14:54:08.883 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:08 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:08.883 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:08 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:08.883 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:08 smithi196 ceph-mon[37486]: pgmap v7: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 18 op/s 2024-03-13T14:54:11.002 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:10 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:11.003 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:10 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:11.003 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:10 smithi196 ceph-mon[37486]: pgmap v8: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-13T14:54:11.005 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:10 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:11.005 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:10 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:11.005 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:10 smithi176 ceph-mon[112742]: pgmap v8: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 14 op/s 2024-03-13T14:54:13.456 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:13 smithi176 ceph-mon[112742]: pgmap v9: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-13T14:54:13.456 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:13 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:13.456 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:13 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:13 smithi196 ceph-mon[37486]: pgmap v9: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-13T14:54:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:13 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:13 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:14.616 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:14 smithi176 ceph-mon[112742]: Detected new or changed devices on smithi196 2024-03-13T14:54:14.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:14 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi196 2024-03-13T14:54:15.624 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:15 smithi176 ceph-mon[112742]: pgmap v10: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-13T14:54:15.624 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:15 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:15.624 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:15 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:15.624 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:15.624 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.635+0000 7f0d0b400700 1 -- 172.21.15.176:0/2533397217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d040ffa60 msgr2=0x7f0d040ffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.635+0000 7f0d0b400700 1 --2- 172.21.15.176:0/2533397217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d040ffa60 0x7f0d040ffee0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009910 tx=0x7f0cf0009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.635+0000 7f0d0b400700 1 -- 172.21.15.176:0/2533397217 shutdown_connections 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.635+0000 7f0d0b400700 1 --2- 172.21.15.176:0/2533397217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d040ffa60 0x7f0d040ffee0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.635+0000 7f0d0b400700 1 --2- 172.21.15.176:0/2533397217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d04105a60 0x7f0d04105e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.635+0000 7f0d0b400700 1 -- 172.21.15.176:0/2533397217 >> 172.21.15.176:0/2533397217 conn(0x7f0d040fb550 msgr2=0x7f0d040fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.636+0000 7f0d0b400700 1 -- 172.21.15.176:0/2533397217 shutdown_connections 2024-03-13T14:54:15.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.636+0000 7f0d0b400700 1 -- 172.21.15.176:0/2533397217 wait complete. 2024-03-13T14:54:15.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.637+0000 7f0d0b400700 1 Processor -- start 2024-03-13T14:54:15.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d0b400700 1 -- start start 2024-03-13T14:54:15.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d040ffa60 0x7f0d0419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:15.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d0b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d04105a60 0x7f0d0419ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:15.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d0b400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0419d110 con 0x7f0d04105a60 2024-03-13T14:54:15.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d0b400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d041965c0 con 0x7f0d040ffa60 2024-03-13T14:54:15.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d03e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d04105a60 0x7f0d0419ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:15.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d03e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d04105a60 0x7f0d0419ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36906/0 (socket says 172.21.15.176:36906) 2024-03-13T14:54:15.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.638+0000 7f0d03e00700 1 -- 172.21.15.176:0/799216893 learned_addr learned my addr 172.21.15.176:0/799216893 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:15.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.639+0000 7f0d08a00700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d040ffa60 0x7f0d0419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.639+0000 7f0d03e00700 1 -- 172.21.15.176:0/799216893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d040ffa60 msgr2=0x7f0d0419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.639+0000 7f0d03e00700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d040ffa60 0x7f0d0419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.639+0000 7f0d03e00700 1 -- 172.21.15.176:0/799216893 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00095f0 con 0x7f0d04105a60 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.639+0000 7f0d03e00700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d04105a60 0x7f0d0419ca80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00092e0 tx=0x7f0cf000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.639+0000 7f0d01600700 1 -- 172.21.15.176:0/799216893 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cf001d070 con 0x7f0d04105a60 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.640+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d04196840 con 0x7f0d04105a60 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.640+0000 7f0d01600700 1 -- 172.21.15.176:0/799216893 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cf00052a0 con 0x7f0d04105a60 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.640+0000 7f0d01600700 1 -- 172.21.15.176:0/799216893 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cf0017400 con 0x7f0d04105a60 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.640+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d04196d30 con 0x7f0d04105a60 2024-03-13T14:54:15.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.641+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d04109000 con 0x7f0d04105a60 2024-03-13T14:54:15.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.642+0000 7f0d01600700 1 -- 172.21.15.176:0/799216893 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f0cf000f820 con 0x7f0d04105a60 2024-03-13T14:54:15.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.643+0000 7f0d01600700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0cec06e0d0 0x7f0cec070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:15.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.643+0000 7f0d01600700 1 -- 172.21.15.176:0/799216893 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0cf008f2a0 con 0x7f0d04105a60 2024-03-13T14:54:15.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.643+0000 7f0d08a00700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0cec06e0d0 0x7f0cec070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:15.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.644+0000 7f0d08a00700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0cec06e0d0 0x7f0cec070590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009c60 tx=0x7f0cf8009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:15.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.650+0000 7f0d01600700 1 -- 172.21.15.176:0/799216893 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f0cf005c060 con 0x7f0d04105a60 2024-03-13T14:54:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:15 smithi196 ceph-mon[37486]: pgmap v10: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-13T14:54:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:15 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:15 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:15 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:15 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:54:15.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.894+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d04197090 con 0x7f0cec06e0d0 2024-03-13T14:54:15.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.896+0000 7f0d01600700 1 -- 172.21.15.176:0/799216893 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f0d04197090 con 0x7f0cec06e0d0 2024-03-13T14:54:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0cec06e0d0 msgr2=0x7f0cec070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0cec06e0d0 0x7f0cec070590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009c60 tx=0x7f0cf8009260 comp rx=0 tx=0).stop 2024-03-13T14:54:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d04105a60 msgr2=0x7f0d0419ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d04105a60 0x7f0d0419ca80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00092e0 tx=0x7f0cf000b6c0 comp rx=0 tx=0).stop 2024-03-13T14:54:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 shutdown_connections 2024-03-13T14:54:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0cec06e0d0 0x7f0cec070590 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:15.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d040ffa60 0x7f0d0419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:15.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 --2- 172.21.15.176:0/799216893 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0d04105a60 0x7f0d0419ca80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:15.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 >> 172.21.15.176:0/799216893 conn(0x7f0d040fb550 msgr2=0x7f0d04069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:15.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 shutdown_connections 2024-03-13T14:54:15.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:15.899+0000 7f0d0b400700 1 -- 172.21.15.176:0/799216893 wait complete. 2024-03-13T14:54:15.901 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:54:16.033 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.045+0000 7ff681800700 1 -- 172.21.15.176:0/2180950571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 msgr2=0x7ff67c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.033 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.045+0000 7ff681800700 1 --2- 172.21.15.176:0/2180950571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c10ce40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff668009910 tx=0x7ff668009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:16.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.045+0000 7ff681800700 1 -- 172.21.15.176:0/2180950571 shutdown_connections 2024-03-13T14:54:16.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.045+0000 7ff681800700 1 --2- 172.21.15.176:0/2180950571 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c10ce40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.045+0000 7ff681800700 1 --2- 172.21.15.176:0/2180950571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff67c0731c0 0x7ff67c0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.045+0000 7ff681800700 1 -- 172.21.15.176:0/2180950571 >> 172.21.15.176:0/2180950571 conn(0x7ff67c0fc630 msgr2=0x7ff67c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:16.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.046+0000 7ff681800700 1 -- 172.21.15.176:0/2180950571 shutdown_connections 2024-03-13T14:54:16.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.046+0000 7ff681800700 1 -- 172.21.15.176:0/2180950571 wait complete. 2024-03-13T14:54:16.035 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff681800700 1 Processor -- start 2024-03-13T14:54:16.035 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff681800700 1 -- start start 2024-03-13T14:54:16.035 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff681800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff67c0731c0 0x7ff67c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.035 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff681800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.035 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff681800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c199550 con 0x7ff67c073ae0 2024-03-13T14:54:16.035 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff681800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c19d360 con 0x7ff67c0731c0 2024-03-13T14:54:16.036 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff67a000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.036 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff67a000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36914/0 (socket says 172.21.15.176:36914) 2024-03-13T14:54:16.036 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.047+0000 7ff67a000700 1 -- 172.21.15.176:0/817517069 learned_addr learned my addr 172.21.15.176:0/817517069 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.048+0000 7ff67aa00700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff67c0731c0 0x7ff67c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.048+0000 7ff67a000700 1 -- 172.21.15.176:0/817517069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff67c0731c0 msgr2=0x7ff67c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.048+0000 7ff67a000700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff67c0731c0 0x7ff67c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.048+0000 7ff67a000700 1 -- 172.21.15.176:0/817517069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680095f0 con 0x7ff67c073ae0 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.048+0000 7ff67a000700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c198ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff668000c00 tx=0x7ff66800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.048+0000 7ff673400700 1 -- 172.21.15.176:0/817517069 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff66801d070 con 0x7ff67c073ae0 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.048+0000 7ff673400700 1 -- 172.21.15.176:0/817517069 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff66800f460 con 0x7ff67c073ae0 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.049+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c19d5e0 con 0x7ff67c073ae0 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.049+0000 7ff673400700 1 -- 172.21.15.176:0/817517069 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff668005e20 con 0x7ff67c073ae0 2024-03-13T14:54:16.037 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.049+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c19dad0 con 0x7ff67c073ae0 2024-03-13T14:54:16.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.050+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c10a540 con 0x7ff67c073ae0 2024-03-13T14:54:16.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.051+0000 7ff673400700 1 -- 172.21.15.176:0/817517069 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7ff66800f5d0 con 0x7ff67c073ae0 2024-03-13T14:54:16.040 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.052+0000 7ff673400700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff66006e0d0 0x7ff660070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.040 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.052+0000 7ff673400700 1 -- 172.21.15.176:0/817517069 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff66808f480 con 0x7ff67c073ae0 2024-03-13T14:54:16.040 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.052+0000 7ff67aa00700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff66006e0d0 0x7ff660070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.040 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.053+0000 7ff67aa00700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff66006e0d0 0x7ff660070590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff66c004630 tx=0x7ff66c00b3c0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.058+0000 7ff673400700 1 -- 172.21.15.176:0/817517069 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7ff66805c280 con 0x7ff67c073ae0 2024-03-13T14:54:16.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.323+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff67c19dcf0 con 0x7ff66006e0d0 2024-03-13T14:54:16.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.324+0000 7ff673400700 1 -- 172.21.15.176:0/817517069 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7ff67c19dcf0 con 0x7ff66006e0d0 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff66006e0d0 msgr2=0x7ff660070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff66006e0d0 0x7ff660070590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff66c004630 tx=0x7ff66c00b3c0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 msgr2=0x7ff67c198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c198ec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff668000c00 tx=0x7ff66800bd70 comp rx=0 tx=0).stop 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 shutdown_connections 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff66006e0d0 0x7ff660070590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff67c0731c0 0x7ff67c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 --2- 172.21.15.176:0/817517069 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff67c073ae0 0x7ff67c198ec0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 >> 172.21.15.176:0/817517069 conn(0x7ff67c0fc630 msgr2=0x7ff67c107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 shutdown_connections 2024-03-13T14:54:16.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.327+0000 7ff681800700 1 -- 172.21.15.176:0/817517069 wait complete. 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.475+0000 7fa048800700 1 -- 172.21.15.176:0/489641600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040100da0 msgr2=0x7fa040101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.475+0000 7fa048800700 1 --2- 172.21.15.176:0/489641600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040100da0 0x7fa040101180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa034009910 tx=0x7fa034009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.476+0000 7fa048800700 1 -- 172.21.15.176:0/489641600 shutdown_connections 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.476+0000 7fa048800700 1 --2- 172.21.15.176:0/489641600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa040101750 0x7fa0401057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.476+0000 7fa048800700 1 --2- 172.21.15.176:0/489641600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040100da0 0x7fa040101180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.476+0000 7fa048800700 1 -- 172.21.15.176:0/489641600 >> 172.21.15.176:0/489641600 conn(0x7fa0400fc630 msgr2=0x7fa0400fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.476+0000 7fa048800700 1 -- 172.21.15.176:0/489641600 shutdown_connections 2024-03-13T14:54:16.464 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.476+0000 7fa048800700 1 -- 172.21.15.176:0/489641600 wait complete. 2024-03-13T14:54:16.465 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.477+0000 7fa048800700 1 Processor -- start 2024-03-13T14:54:16.465 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.477+0000 7fa048800700 1 -- start start 2024-03-13T14:54:16.465 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.477+0000 7fa048800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa040100da0 0x7fa040073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.465 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.477+0000 7fa048800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040101750 0x7fa0400735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.465 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.477+0000 7fa048800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa040073b10 con 0x7fa040101750 2024-03-13T14:54:16.465 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.477+0000 7fa048800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa040073c50 con 0x7fa040100da0 2024-03-13T14:54:16.466 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040101750 0x7fa0400735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.466 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040101750 0x7fa0400735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36930/0 (socket says 172.21.15.176:36930) 2024-03-13T14:54:16.466 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045400700 1 -- 172.21.15.176:0/847095659 learned_addr learned my addr 172.21.15.176:0/847095659 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:16.466 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045e00700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa040100da0 0x7fa040073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045400700 1 -- 172.21.15.176:0/847095659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa040100da0 msgr2=0x7fa040073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045400700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa040100da0 0x7fa040073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045400700 1 -- 172.21.15.176:0/847095659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0340095f0 con 0x7fa040101750 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.478+0000 7fa045400700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040101750 0x7fa0400735d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d900 tx=0x7fa03000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.479+0000 7fa03aa00700 1 -- 172.21.15.176:0/847095659 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa030009900 con 0x7fa040101750 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.479+0000 7fa03aa00700 1 -- 172.21.15.176:0/847095659 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa030010460 con 0x7fa040101750 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.479+0000 7fa03aa00700 1 -- 172.21.15.176:0/847095659 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa03000f5f0 con 0x7fa040101750 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.479+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0401a29a0 con 0x7fa040101750 2024-03-13T14:54:16.467 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.479+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0401a2d50 con 0x7fa040101750 2024-03-13T14:54:16.468 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.481+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa040109140 con 0x7fa040101750 2024-03-13T14:54:16.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.481+0000 7fa03aa00700 1 -- 172.21.15.176:0/847095659 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fa0300105d0 con 0x7fa040101750 2024-03-13T14:54:16.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.482+0000 7fa03aa00700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa02806e0d0 0x7fa028070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.482+0000 7fa03aa00700 1 -- 172.21.15.176:0/847095659 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa03008dbd0 con 0x7fa040101750 2024-03-13T14:54:16.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.483+0000 7fa045e00700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa02806e0d0 0x7fa028070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.471 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.483+0000 7fa045e00700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa02806e0d0 0x7fa028070590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa034006010 tx=0x7fa03401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:16.475 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.487+0000 7fa03aa00700 1 -- 172.21.15.176:0/847095659 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa030056de0 con 0x7fa040101750 2024-03-13T14:54:16.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:16 smithi176 ceph-mon[112742]: Detected new or changed devices on smithi176 2024-03-13T14:54:16.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:16 smithi176 ceph-mon[112742]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:54:16.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:16 smithi176 ceph-mon[112742]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:54:16.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:16.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:16 smithi196 ceph-mon[37486]: Detected new or changed devices on smithi176 2024-03-13T14:54:16.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:16 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.conf 2024-03-13T14:54:16.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:16 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.conf 2024-03-13T14:54:16.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:16 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:16.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.718+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa04002cd80 con 0x7fa02806e0d0 2024-03-13T14:54:16.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.733+0000 7fa03aa00700 1 -- 172.21.15.176:0/847095659 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa04002cd80 con 0x7fa02806e0d0 2024-03-13T14:54:16.722 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:54:16.722 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (2m) 9s ago 15m 14.3M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:54:16.722 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (16m) 9s ago 16m 8178k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:54:16.722 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (14m) 11s ago 14m 8112k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:54:16.722 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (16m) 9s ago 16m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:54:16.722 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (14m) 11s ago 14m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:54:16.722 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (98s) 9s ago 15m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:54:16.725 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (8m) 9s ago 8m 87.2M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:54:16.725 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (8m) 9s ago 8m 16.3M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:54:16.725 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (8m) 11s ago 8m 142M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (8m) 11s ago 8m 15.8M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (5m) 9s ago 17m 476M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (3m) 11s ago 14m 446M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (26s) 9s ago 17m 28.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (14m) 11s ago 14m 47.7M 2048M 18.2.1 5be31c24972a 5403fcfbde72 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (3m) 9s ago 15m 7780k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (3m) 11s ago 14m 7708k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (13m) 9s ago 13m 233M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (12m) 9s ago 12m 246M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (12m) 9s ago 12m 220M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (11m) 11s ago 11m 304M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (10m) 11s ago 10m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (10m) 11s ago 10m 228M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:54:16.726 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (2m) 9s ago 14m 63.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:54:16.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa02806e0d0 msgr2=0x7fa028070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa02806e0d0 0x7fa028070590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa034006010 tx=0x7fa03401a040 comp rx=0 tx=0).stop 2024-03-13T14:54:16.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040101750 msgr2=0x7fa0400735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040101750 0x7fa0400735d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d900 tx=0x7fa03000dc10 comp rx=0 tx=0).stop 2024-03-13T14:54:16.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 shutdown_connections 2024-03-13T14:54:16.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa02806e0d0 0x7fa028070590 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa040100da0 0x7fa040073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 --2- 172.21.15.176:0/847095659 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa040101750 0x7fa0400735d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.736+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 >> 172.21.15.176:0/847095659 conn(0x7fa0400fc630 msgr2=0x7fa0400fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:16.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.737+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 shutdown_connections 2024-03-13T14:54:16.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.737+0000 7fa048800700 1 -- 172.21.15.176:0/847095659 wait complete. 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.887+0000 7f60b6400700 1 -- 172.21.15.176:0/3237406905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00730b0 msgr2=0x7f60b0073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.887+0000 7f60b6400700 1 --2- 172.21.15.176:0/3237406905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00730b0 0x7f60b0073490 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f60a4009910 tx=0x7f60a4009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.887+0000 7f60b6400700 1 -- 172.21.15.176:0/3237406905 shutdown_connections 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.887+0000 7f60b6400700 1 --2- 172.21.15.176:0/3237406905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60b00739d0 0x7f60b010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.887+0000 7f60b6400700 1 --2- 172.21.15.176:0/3237406905 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00730b0 0x7f60b0073490 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.887+0000 7f60b6400700 1 -- 172.21.15.176:0/3237406905 >> 172.21.15.176:0/3237406905 conn(0x7f60b00fc4d0 msgr2=0x7f60b00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.888+0000 7f60b6400700 1 -- 172.21.15.176:0/3237406905 shutdown_connections 2024-03-13T14:54:16.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.888+0000 7f60b6400700 1 -- 172.21.15.176:0/3237406905 wait complete. 2024-03-13T14:54:16.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.889+0000 7f60b6400700 1 Processor -- start 2024-03-13T14:54:16.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.889+0000 7f60b6400700 1 -- start start 2024-03-13T14:54:16.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.889+0000 7f60b6400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60b00730b0 0x7f60b019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.889+0000 7f60b6400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00739d0 0x7f60b019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.889+0000 7f60b6400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b019f100 con 0x7f60b00739d0 2024-03-13T14:54:16.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.889+0000 7f60b6400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b01985b0 con 0x7f60b00730b0 2024-03-13T14:54:16.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00739d0 0x7f60b019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00739d0 0x7f60b019ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36954/0 (socket says 172.21.15.176:36954) 2024-03-13T14:54:16.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60afe00700 1 -- 172.21.15.176:0/38095686 learned_addr learned my addr 172.21.15.176:0/38095686 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:16.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60b5000700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60b00730b0 0x7f60b019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60afe00700 1 -- 172.21.15.176:0/38095686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60b00730b0 msgr2=0x7f60b019e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60afe00700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60b00730b0 0x7f60b019e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60afe00700 1 -- 172.21.15.176:0/38095686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a40095f0 con 0x7f60b00739d0 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.890+0000 7f60afe00700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00739d0 0x7f60b019ea70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d8d0 tx=0x7f609c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.891+0000 7f60ad600700 1 -- 172.21.15.176:0/38095686 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609c0098a0 con 0x7f60b00739d0 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.891+0000 7f60ad600700 1 -- 172.21.15.176:0/38095686 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f609c010460 con 0x7f60b00739d0 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.891+0000 7f60ad600700 1 -- 172.21.15.176:0/38095686 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609c00f5f0 con 0x7f60b00739d0 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.891+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b0198890 con 0x7f60b00739d0 2024-03-13T14:54:16.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.891+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b0198de0 con 0x7f60b00739d0 2024-03-13T14:54:16.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.892+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b010a440 con 0x7f60b00739d0 2024-03-13T14:54:16.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.893+0000 7f60ad600700 1 -- 172.21.15.176:0/38095686 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f609c009a00 con 0x7f60b00739d0 2024-03-13T14:54:16.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.894+0000 7f60ad600700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f609806e0d0 0x7f6098070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:16.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.894+0000 7f60ad600700 1 -- 172.21.15.176:0/38095686 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f609c08de30 con 0x7f60b00739d0 2024-03-13T14:54:16.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.894+0000 7f60b5000700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f609806e0d0 0x7f6098070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:16.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.895+0000 7f60b5000700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f609806e0d0 0x7f6098070590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60a400b5c0 tx=0x7f60a401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:16.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:16.900+0000 7f60ad600700 1 -- 172.21.15.176:0/38095686 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f609c05abf0 con 0x7f60b00739d0 2024-03-13T14:54:17.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.205+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f60b0199ac0 con 0x7f60b00739d0 2024-03-13T14:54:17.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.205+0000 7f60ad600700 1 -- 172.21.15.176:0/38095686 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7f609c05a780 con 0x7f60b00739d0 2024-03-13T14:54:17.193 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:54:17.193 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:54:17.193 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 1 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:54:17.194 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 11, 2024-03-13T14:54:17.195 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 3 2024-03-13T14:54:17.195 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:54:17.195 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f609806e0d0 msgr2=0x7f6098070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f609806e0d0 0x7f6098070590 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f60a400b5c0 tx=0x7f60a401a040 comp rx=0 tx=0).stop 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00739d0 msgr2=0x7f60b019ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00739d0 0x7f60b019ea70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d8d0 tx=0x7f609c00dbe0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 shutdown_connections 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f609806e0d0 0x7f6098070590 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60b00730b0 0x7f60b019e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.196 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 --2- 172.21.15.176:0/38095686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f60b00739d0 0x7f60b019ea70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 >> 172.21.15.176:0/38095686 conn(0x7f60b00fc4d0 msgr2=0x7f60b0107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:17.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 shutdown_connections 2024-03-13T14:54:17.197 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.208+0000 7f60b6400700 1 -- 172.21.15.176:0/38095686 wait complete. 2024-03-13T14:54:17.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.351+0000 7ff04e000700 1 -- 172.21.15.176:0/1891689974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 msgr2=0x7ff0481057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.351+0000 7ff04e000700 1 --2- 172.21.15.176:0/1891689974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff0481057a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff034009910 tx=0x7ff034009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:17.344 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.351+0000 7ff04e000700 1 -- 172.21.15.176:0/1891689974 shutdown_connections 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.351+0000 7ff04e000700 1 --2- 172.21.15.176:0/1891689974 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff0481057a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.351+0000 7ff04e000700 1 --2- 172.21.15.176:0/1891689974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff048100da0 0x7ff048101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.351+0000 7ff04e000700 1 -- 172.21.15.176:0/1891689974 >> 172.21.15.176:0/1891689974 conn(0x7ff0480fc630 msgr2=0x7ff0480fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.352+0000 7ff04e000700 1 -- 172.21.15.176:0/1891689974 shutdown_connections 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.352+0000 7ff04e000700 1 -- 172.21.15.176:0/1891689974 wait complete. 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.353+0000 7ff04e000700 1 Processor -- start 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.353+0000 7ff04e000700 1 -- start start 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.353+0000 7ff04e000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff048100da0 0x7ff04819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:17.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.353+0000 7ff04e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff04819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.353+0000 7ff04e000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04819f350 con 0x7ff048101750 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.353+0000 7ff04e000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff048198800 con 0x7ff048100da0 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff046a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff04819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff046a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff04819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36974/0 (socket says 172.21.15.176:36974) 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff046a00700 1 -- 172.21.15.176:0/1543846447 learned_addr learned my addr 172.21.15.176:0/1543846447 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff047400700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff048100da0 0x7ff04819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff046a00700 1 -- 172.21.15.176:0/1543846447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff048100da0 msgr2=0x7ff04819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff046a00700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff048100da0 0x7ff04819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff046a00700 1 -- 172.21.15.176:0/1543846447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340095f0 con 0x7ff048101750 2024-03-13T14:54:17.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.354+0000 7ff046a00700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff04819ecc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff034009fd0 tx=0x7ff03400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:17.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.355+0000 7ff03fe00700 1 -- 172.21.15.176:0/1543846447 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff03401d070 con 0x7ff048101750 2024-03-13T14:54:17.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.355+0000 7ff03fe00700 1 -- 172.21.15.176:0/1543846447 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff034022470 con 0x7ff048101750 2024-03-13T14:54:17.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.355+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff048198a80 con 0x7ff048101750 2024-03-13T14:54:17.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.355+0000 7ff03fe00700 1 -- 172.21.15.176:0/1543846447 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff034017610 con 0x7ff048101750 2024-03-13T14:54:17.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.355+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff048198f10 con 0x7ff048101750 2024-03-13T14:54:17.347 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.356+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff048104f20 con 0x7ff048101750 2024-03-13T14:54:17.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.357+0000 7ff03fe00700 1 -- 172.21.15.176:0/1543846447 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7ff0340225e0 con 0x7ff048101750 2024-03-13T14:54:17.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.358+0000 7ff03fe00700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff02c06e0d0 0x7ff02c070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:17.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.358+0000 7ff03fe00700 1 -- 172.21.15.176:0/1543846447 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff03408f5a0 con 0x7ff048101750 2024-03-13T14:54:17.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.358+0000 7ff047400700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff02c06e0d0 0x7ff02c070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:17.348 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.359+0000 7ff047400700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff02c06e0d0 0x7ff02c070590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff038009bf0 tx=0x7ff038009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:17.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.363+0000 7ff03fe00700 1 -- 172.21.15.176:0/1543846447 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7ff03405c360 con 0x7ff048101750 2024-03-13T14:54:17.609 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.621+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff04804ea90 con 0x7ff048101750 2024-03-13T14:54:17.609 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:17 smithi176 ceph-mon[112742]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:54:17.610 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:17 smithi176 ceph-mon[112742]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:54:17.610 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:17 smithi176 ceph-mon[112742]: from='client.34128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:17.610 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:17 smithi176 ceph-mon[112742]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:54:17.610 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:17 smithi176 ceph-mon[112742]: pgmap v11: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-13T14:54:17.610 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:17 smithi176 ceph-mon[112742]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:17.610 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:17 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/38095686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:54:17.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.622+0000 7ff03fe00700 1 -- 172.21.15.176:0/1543846447 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7ff03405bef0 con 0x7ff048101750 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:54:17.611 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:54:17.612 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:17.613 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff02c06e0d0 msgr2=0x7ff02c070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff02c06e0d0 0x7ff02c070590 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff038009bf0 tx=0x7ff038009260 comp rx=0 tx=0).stop 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 msgr2=0x7ff04819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff04819ecc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff034009fd0 tx=0x7ff03400f690 comp rx=0 tx=0).stop 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 shutdown_connections 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff02c06e0d0 0x7ff02c070590 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff048100da0 0x7ff04819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 --2- 172.21.15.176:0/1543846447 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff048101750 0x7ff04819ecc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 >> 172.21.15.176:0/1543846447 conn(0x7ff0480fc630 msgr2=0x7ff0480fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:17.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.626+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 shutdown_connections 2024-03-13T14:54:17.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.627+0000 7ff04e000700 1 -- 172.21.15.176:0/1543846447 wait complete. 2024-03-13T14:54:17.615 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:54:17.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:17 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:54:17.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:17 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.conf 2024-03-13T14:54:17.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:17 smithi196 ceph-mon[37486]: from='client.34128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:17.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:17 smithi196 ceph-mon[37486]: Updating smithi176:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:54:17.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:17 smithi196 ceph-mon[37486]: pgmap v11: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-13T14:54:17.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:17 smithi196 ceph-mon[37486]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:17.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:17 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/38095686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 -- 172.21.15.176:0/69843891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 msgr2=0x7f435010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 --2- 172.21.15.176:0/69843891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f435010ce40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4340009910 tx=0x7f4340009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 -- 172.21.15.176:0/69843891 shutdown_connections 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 --2- 172.21.15.176:0/69843891 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f435010ce40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 --2- 172.21.15.176:0/69843891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43500731c0 0x7f43500735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 -- 172.21.15.176:0/69843891 >> 172.21.15.176:0/69843891 conn(0x7f43500fc630 msgr2=0x7f43500fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 -- 172.21.15.176:0/69843891 shutdown_connections 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.768+0000 7f4357c00700 1 -- 172.21.15.176:0/69843891 wait complete. 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.769+0000 7f4357c00700 1 Processor -- start 2024-03-13T14:54:17.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.769+0000 7f4357c00700 1 -- start start 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4357c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43500731c0 0x7f4350198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4357c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f4350198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4357c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4350199530 con 0x7f4350073ae0 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4357c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435019d340 con 0x7f43500731c0 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4347e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f4350198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4347e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f4350198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:36996/0 (socket says 172.21.15.176:36996) 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4347e00700 1 -- 172.21.15.176:0/1139270787 learned_addr learned my addr 172.21.15.176:0/1139270787 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4355200700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43500731c0 0x7f4350198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4347e00700 1 -- 172.21.15.176:0/1139270787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43500731c0 msgr2=0x7f4350198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4347e00700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43500731c0 0x7f4350198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:17.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.770+0000 7f4347e00700 1 -- 172.21.15.176:0/1139270787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400095f0 con 0x7f4350073ae0 2024-03-13T14:54:17.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.771+0000 7f4347e00700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f4350198ea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43400092e0 tx=0x7f434000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:17.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.771+0000 7f434e000700 1 -- 172.21.15.176:0/1139270787 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f434001d070 con 0x7f4350073ae0 2024-03-13T14:54:17.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.771+0000 7f434e000700 1 -- 172.21.15.176:0/1139270787 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f434000f460 con 0x7f4350073ae0 2024-03-13T14:54:17.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.771+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f435019d5c0 con 0x7f4350073ae0 2024-03-13T14:54:17.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.771+0000 7f434e000700 1 -- 172.21.15.176:0/1139270787 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4340005e20 con 0x7f4350073ae0 2024-03-13T14:54:17.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.771+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435019dab0 con 0x7f4350073ae0 2024-03-13T14:54:17.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.772+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435010a540 con 0x7f4350073ae0 2024-03-13T14:54:17.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.773+0000 7f434e000700 1 -- 172.21.15.176:0/1139270787 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f434000f5d0 con 0x7f4350073ae0 2024-03-13T14:54:17.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.774+0000 7f434e000700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f433006e0d0 0x7f4330070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:17.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.774+0000 7f434e000700 1 -- 172.21.15.176:0/1139270787 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f434008f470 con 0x7f4350073ae0 2024-03-13T14:54:17.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.774+0000 7f4355200700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f433006e0d0 0x7f4330070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:17.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.775+0000 7f4355200700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f433006e0d0 0x7f4330070590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4338005950 tx=0x7f433800b410 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:17.767 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:17.780+0000 7f434e000700 1 -- 172.21.15.176:0/1139270787 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f434005c320 con 0x7f4350073ae0 2024-03-13T14:54:17.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.006+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f435019dcd0 con 0x7f433006e0d0 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.010+0000 7f434e000700 1 -- 172.21.15.176:0/1139270787 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f435019dcd0 con 0x7f433006e0d0 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "3/23 daemons upgraded", 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "", 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:54:17.997 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f433006e0d0 msgr2=0x7f4330070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f433006e0d0 0x7f4330070590 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4338005950 tx=0x7f433800b410 comp rx=0 tx=0).stop 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 msgr2=0x7f4350198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f4350198ea0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f43400092e0 tx=0x7f434000bd70 comp rx=0 tx=0).stop 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 shutdown_connections 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f433006e0d0 0x7f4330070590 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43500731c0 0x7f4350198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.012+0000 7f4357c00700 1 --2- 172.21.15.176:0/1139270787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4350073ae0 0x7f4350198ea0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.013+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 >> 172.21.15.176:0/1139270787 conn(0x7f43500fc630 msgr2=0x7f4350107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.013+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 shutdown_connections 2024-03-13T14:54:18.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.013+0000 7f4357c00700 1 -- 172.21.15.176:0/1139270787 wait complete. 2024-03-13T14:54:18.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.156+0000 7f8295a00700 1 -- 172.21.15.176:0/3669347244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 msgr2=0x7f8290105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:18.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.156+0000 7f8295a00700 1 --2- 172.21.15.176:0/3669347244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f8290105790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f827c009910 tx=0x7f827c009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:18.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.156+0000 7f8295a00700 1 -- 172.21.15.176:0/3669347244 shutdown_connections 2024-03-13T14:54:18.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.156+0000 7f8295a00700 1 --2- 172.21.15.176:0/3669347244 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f8290105790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.156+0000 7f8295a00700 1 --2- 172.21.15.176:0/3669347244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8290068670 0x7f8290068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.156+0000 7f8295a00700 1 -- 172.21.15.176:0/3669347244 >> 172.21.15.176:0/3669347244 conn(0x7f82900755a0 msgr2=0x7f82900759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:18.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.157+0000 7f8295a00700 1 -- 172.21.15.176:0/3669347244 shutdown_connections 2024-03-13T14:54:18.144 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.157+0000 7f8295a00700 1 -- 172.21.15.176:0/3669347244 wait complete. 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.157+0000 7f8295a00700 1 Processor -- start 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f8295a00700 1 -- start start 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f8295a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8290068670 0x7f829019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f8295a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f829019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f8295a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019d220 con 0x7f8290069020 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f8295a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82901966d0 con 0x7f8290068670 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f828e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f829019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f828e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f829019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37018/0 (socket says 172.21.15.176:37018) 2024-03-13T14:54:18.145 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.158+0000 7f828e000700 1 -- 172.21.15.176:0/3276321128 learned_addr learned my addr 172.21.15.176:0/3276321128 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:18.146 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f828ea00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8290068670 0x7f829019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:18.146 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f828e000700 1 -- 172.21.15.176:0/3276321128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8290068670 msgr2=0x7f829019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:18.146 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f828e000700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8290068670 0x7f829019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.146 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f828e000700 1 -- 172.21.15.176:0/3276321128 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0095f0 con 0x7f8290069020 2024-03-13T14:54:18.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f828e000700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f829019cb90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f827c005f50 tx=0x7f827c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:18.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f8287400700 1 -- 172.21.15.176:0/3276321128 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f827c01d070 con 0x7f8290069020 2024-03-13T14:54:18.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f8287400700 1 -- 172.21.15.176:0/3276321128 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f827c00f460 con 0x7f8290069020 2024-03-13T14:54:18.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f8287400700 1 -- 172.21.15.176:0/3276321128 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f827c017630 con 0x7f8290069020 2024-03-13T14:54:18.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8290196950 con 0x7f8290069020 2024-03-13T14:54:18.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.159+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8290196e40 con 0x7f8290069020 2024-03-13T14:54:18.147 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.161+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82901090e0 con 0x7f8290069020 2024-03-13T14:54:18.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.162+0000 7f8287400700 1 -- 172.21.15.176:0/3276321128 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f827c00f5d0 con 0x7f8290069020 2024-03-13T14:54:18.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.163+0000 7f8287400700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f827406e0d0 0x7f8274070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:18.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.163+0000 7f8287400700 1 -- 172.21.15.176:0/3276321128 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f827c08f480 con 0x7f8290069020 2024-03-13T14:54:18.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.163+0000 7f828ea00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f827406e0d0 0x7f8274070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:18.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.164+0000 7f828ea00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f827406e0d0 0x7f8274070590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8280006fd0 tx=0x7f8280008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:18.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.168+0000 7f8287400700 1 -- 172.21.15.176:0/3276321128 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f827c05c2e0 con 0x7f8290069020 2024-03-13T14:54:18.466 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.479+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f829004ea90 con 0x7f8290069020 2024-03-13T14:54:18.466 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.479+0000 7f8287400700 1 -- 172.21.15.176:0/3276321128 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f827c05be70 con 0x7f8290069020 2024-03-13T14:54:18.467 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:54:18.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f827406e0d0 msgr2=0x7f8274070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:18.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f827406e0d0 0x7f8274070590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8280006fd0 tx=0x7f8280008040 comp rx=0 tx=0).stop 2024-03-13T14:54:18.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 msgr2=0x7f829019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:18.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f829019cb90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f827c005f50 tx=0x7f827c00be20 comp rx=0 tx=0).stop 2024-03-13T14:54:18.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 shutdown_connections 2024-03-13T14:54:18.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f827406e0d0 0x7f8274070590 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8290068670 0x7f829019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 --2- 172.21.15.176:0/3276321128 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8290069020 0x7f829019cb90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:18.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 >> 172.21.15.176:0/3276321128 conn(0x7f82900755a0 msgr2=0x7f82900fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:18.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.482+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 shutdown_connections 2024-03-13T14:54:18.470 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:18.483+0000 7f8295a00700 1 -- 172.21.15.176:0/3276321128 wait complete. 2024-03-13T14:54:18.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:54:18.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:18.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1543846447' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:54:18.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: Updating smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: Updating smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/config/ceph.client.admin.keyring 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/1543846447' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:18.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:18.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:54:18.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:18 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-03-13T14:54:19.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:19 smithi196 ceph-mon[37486]: from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:19.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:19 smithi196 ceph-mon[37486]: pgmap v12: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-13T14:54:19.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:19 smithi196 ceph-mon[37486]: from='client.? 172.21.15.176:0/3276321128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:54:19.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:19 smithi176 ceph-mon[112742]: from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:19.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:19 smithi176 ceph-mon[112742]: pgmap v12: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 0 B/s wr, 12 op/s 2024-03-13T14:54:19.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:19 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3276321128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:54:21.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:21 smithi196 ceph-mon[37486]: pgmap v13: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:21 smithi196 ceph-mon[37486]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:21 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:54:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:21 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:54:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:21 smithi196 ceph-mon[37486]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:21.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:21 smithi176 ceph-mon[112742]: pgmap v13: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:21.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:21 smithi176 ceph-mon[112742]: from='mgr.24551 ' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:21.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:54:21.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:54:21.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:22.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:22 smithi176 ceph-mon[112742]: Upgrade: Updating mon.smithi196 2024-03-13T14:54:22.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:22 smithi176 ceph-mon[112742]: Deploying daemon mon.smithi196 on smithi196 2024-03-13T14:54:22.753 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:22 smithi196 ceph-mon[37486]: Upgrade: Updating mon.smithi196 2024-03-13T14:54:22.754 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:22 smithi196 ceph-mon[37486]: Deploying daemon mon.smithi196 on smithi196 2024-03-13T14:54:23.624 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:23 smithi196 systemd[1]: Stopping Ceph mon.smithi196 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:54:23.624 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:23 smithi196 ceph-mon[37486]: pgmap v14: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:23.778 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:23 smithi176 ceph-mon[112742]: pgmap v14: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:23.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:23 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196[37479]: 2024-03-13T14:54:23.635+0000 7f3e17000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi196 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:54:23.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:23 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196[37479]: 2024-03-13T14:54:23.635+0000 7f3e17000700 -1 mon.smithi196@1(peon) e2 *** Got Signal Terminated *** 2024-03-13T14:54:23.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:23 smithi196 podman[86036]: 2024-03-13 14:54:23.70881179 +0000 UTC m=+0.187734582 container died 5403fcfbde7282ff12b09239365ff198d264b965e97fc08ce52e69f99fd0f02c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-13T14:54:24.266 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:24 smithi196 podman[86036]: 2024-03-13 14:54:24.095981356 +0000 UTC m=+0.574904165 container cleanup 5403fcfbde7282ff12b09239365ff198d264b965e97fc08ce52e69f99fd0f02c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222) 2024-03-13T14:54:24.266 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:24 smithi196 bash[86036]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196 2024-03-13T14:54:24.596 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:24 smithi196 podman[86050]: 2024-03-13 14:54:24.388994968 +0000 UTC m=+0.672877731 container remove 5403fcfbde7282ff12b09239365ff198d264b965e97fc08ce52e69f99fd0f02c (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, org.label-schema.schema-version=1.0) 2024-03-13T14:54:24.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:24 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi196.service: Deactivated successfully. 2024-03-13T14:54:24.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:24 smithi196 systemd[1]: Stopped Ceph mon.smithi196 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:54:24.930 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:24 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi196.service: Consumed 18.353s CPU time. 2024-03-13T14:54:25.397 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 systemd[1]: Starting Ceph mon.smithi196 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:54:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 podman[86175]: 2024-03-13 14:54:25.316415049 +0000 UTC m=+0.032486709 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:54:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 podman[86175]: 2024-03-13 14:54:25.431770809 +0000 UTC m=+0.147842508 container create 3f7f3fad03733d993fa7623f7c7228f3c39607a4f671d4834051974a382afa5a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 podman[86175]: 2024-03-13 14:54:25.752218925 +0000 UTC m=+0.468290627 container init 3f7f3fad03733d993fa7623f7c7228f3c39607a4f671d4834051974a382afa5a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 podman[86175]: 2024-03-13 14:54:25.762575901 +0000 UTC m=+0.478647630 container start 3f7f3fad03733d993fa7623f7c7228f3c39607a4f671d4834051974a382afa5a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: set uid:gid to 167:167 (ceph:ceph) 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev), process ceph-mon, pid 2 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: pidfile_write: ignore empty --pid-file 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: load: jerasure load: lrc 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: RocksDB version: 7.9.2 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Git sha 0 2024-03-13T14:54:26.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Compile date 2024-03-07 04:59:45 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: DB SUMMARY 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: DB Session ID: VYRNS048S3DTH9IML6DN 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: CURRENT file: CURRENT 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: IDENTITY file: IDENTITY 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1626 Bytes 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi196/store.db dir, Total Num: 1, files: 000030.sst 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi196/store.db: 000028.log size: 3847894 ; 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.error_if_exists: 0 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.create_if_missing: 0 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.paranoid_checks: 1 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.env: 0x5632becfcb20 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.fs: PosixFileSystem 2024-03-13T14:54:26.180 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.info_log: 0x5632c06437c0 2024-03-13T14:54:26.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_file_opening_threads: 16 2024-03-13T14:54:26.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.statistics: (nil) 2024-03-13T14:54:26.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.use_fsync: 0 2024-03-13T14:54:26.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_log_file_size: 0 2024-03-13T14:54:26.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.keep_log_file_num: 1000 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.recycle_log_file_num: 0 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.allow_fallocate: 1 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.allow_mmap_reads: 0 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.allow_mmap_writes: 0 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.use_direct_reads: 0 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.create_missing_column_families: 0 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.db_log_dir: 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.wal_dir: 2024-03-13T14:54:26.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.advise_random_on_open: 1 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.db_write_buffer_size: 0 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.write_buffer_manager: 0x5632c0617400 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.rate_limiter: (nil) 2024-03-13T14:54:26.183 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.wal_recovery_mode: 2 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.enable_thread_tracking: 0 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.enable_pipelined_write: 0 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.unordered_write: 0 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.row_cache: None 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.wal_filter: None 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.allow_ingest_behind: 0 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.two_write_queues: 0 2024-03-13T14:54:26.184 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.manual_wal_flush: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.wal_compression: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.atomic_flush: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.log_readahead_size: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.best_efforts_recovery: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.allow_data_in_errors: 0 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.db_host_id: __hostname__ 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_background_jobs: 2 2024-03-13T14:54:26.185 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_background_compactions: -1 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_subcompactions: 1 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_total_wal_size: 0 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_open_files: -1 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bytes_per_sync: 0 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-13T14:54:26.186 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_readahead_size: 0 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_background_flushes: -1 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Compression algorithms supported: 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kZSTD supported: 0 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kXpressCompression supported: 0 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kLZ4HCCompression supported: 1 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kZlibCompression supported: 1 2024-03-13T14:54:26.187 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kSnappyCompression supported: 1 2024-03-13T14:54:26.188 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kLZ4Compression supported: 1 2024-03-13T14:54:26.188 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: kBZip2Compression supported: 0 2024-03-13T14:54:26.188 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-13T14:54:26.188 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi196/store.db/MANIFEST-000010 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.merge_operator: 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_filter: None 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_filter_factory: None 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.sst_partitioner_factory: None 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5632c06438e0) 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: cache_index_and_filter_blocks: 1 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-13T14:54:26.189 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: pin_top_level_index_and_filter: 1 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: index_type: 0 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: data_block_index_type: 0 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: index_shortening: 1 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: checksum: 4 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: no_block_cache: 0 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache: 0x5632c0649090 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache_name: BinnedLRUCache 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache_options: 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: capacity : 536870912 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: num_shard_bits : 4 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: strict_capacity_limit : 0 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: high_pri_pool_ratio: 0.000 2024-03-13T14:54:26.190 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_cache_compressed: (nil) 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: persistent_cache: (nil) 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_size: 4096 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_size_deviation: 10 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_restart_interval: 16 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: index_block_restart_interval: 1 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: metadata_block_size: 4096 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: partition_filters: 0 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: use_delta_encoding: 1 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: filter_policy: bloomfilter 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: whole_key_filtering: 1 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: verify_compression: 0 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: read_amp_bytes_per_bit: 0 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: format_version: 5 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: enable_index_compression: 1 2024-03-13T14:54:26.191 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: block_align: 0 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: max_auto_readahead_size: 262144 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: prepopulate_block_cache: 0 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: initial_auto_readahead_size: 8192 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: num_file_reads_for_auto_readahead: 2 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.write_buffer_size: 33554432 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_write_buffer_number: 2 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression: NoCompression 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression: Disabled 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.prefix_extractor: nullptr 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.num_levels: 7 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-13T14:54:26.192 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.level: 32767 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.strategy: 0 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-13T14:54:26.193 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.enabled: false 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.target_file_size_base: 67108864 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-13T14:54:26.194 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.arena_block_size: 1048576 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.disable_auto_compactions: 0 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-13T14:54:26.195 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.table_properties_collectors: 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.inplace_update_support: 0 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.bloom_locality: 0 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.max_successive_merges: 0 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-13T14:54:26.196 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.paranoid_file_checks: 0 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.force_consistency_checks: 1 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.report_bg_io_stats: 0 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.ttl: 2592000 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.enable_blob_files: false 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.min_blob_size: 0 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.blob_file_size: 268435456 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-13T14:54:26.197 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.blob_file_starting_level: 0 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi196/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 32, last_sequence is 16796, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 28 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9d11ef0e-c5f8-4649-b9e1-4277fbe3bcb9 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710341665818377, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710341665833899, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2439170, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 16802, "largest_seqno": 18323, "table_properties": {"data_size": 2432901, "index_size": 3650, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1605, "raw_key_size": 15198, "raw_average_key_size": 24, "raw_value_size": 2419685, "raw_average_value_size": 3877, "num_data_blocks": 165, "num_entries": 624, "num_filter_entries": 624, "num_deletions": 6, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1710341665, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9d11ef0e-c5f8-4649-b9e1-4277fbe3bcb9", "db_session_id": "VYRNS048S3DTH9IML6DN", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: EVENT_LOG_v1 {"time_micros": 1710341665834181, "job": 1, "event": "recovery_finished"} 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi196/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5632c073a000 2024-03-13T14:54:26.198 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: DB pointer 0x5632c0726000 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** DB Stats ** 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-13T14:54:26.199 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: L0 1/0 2.33 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 508.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: L6 1/0 7.99 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Sum 2/0 10.32 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 508.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 508.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** Compaction Stats [default] ** 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 508.1 0.00 0.00 1 0.005 0 0 0.0 0.0 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-03-13T14:54:26.200 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Cumulative compaction: 0.00 GB write, 101.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Interval compaction: 0.00 GB write, 101.04 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Block cache BinnedLRUCache@0x5632c0649090#2 capacity: 512.00 MB usage: 5.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.78 KB,0.000339746%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: starting mon.smithi196 rank 1 at public addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] at bind addrs [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi196 fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???) e2 preinit fsid f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???).mds e18 new map 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???).mds e18 print_map 2024-03-13T14:54:26.201 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: e18 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: legacy client fscid: 1 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Filesystem 'cephfs' (1) 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: fs_name cephfs 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: epoch 18 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: tableserver 0 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: root 0 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: session_timeout 60 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: session_autoclose 300 2024-03-13T14:54:26.202 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: max_file_size 1099511627776 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: max_xattr_size 65536 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: required_client_features {} 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: last_failure 0 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: last_failure_osd_epoch 47 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: max_mds 1 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: in 0 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: up {0=24249} 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: failed 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: damaged 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: stopped 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: data_pools [3] 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: metadata_pool 2 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: inline_data disabled 2024-03-13T14:54:26.203 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: balancer 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: bal_rank_mask -1 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: standby_count_wanted 1 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: [mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: [mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: Standby daemons: 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: [mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout: [mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-03-13T14:54:26.204 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 ceph-mon[86193]: mon.smithi196@-1(???).paxosservice(auth 1..25) refresh upgraded, format 0 -> 3 2024-03-13T14:54:26.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 bash[86175]: 3f7f3fad03733d993fa7623f7c7228f3c39607a4f671d4834051974a382afa5a 2024-03-13T14:54:26.205 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:25 smithi196 systemd[1]: Started Ceph mon.smithi196 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:54:28.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: mon.smithi176 calling monitor election 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: mon.smithi196 calling monitor election 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: mon.smithi176 is new leader, mons smithi176,smithi196 in quorum (ranks 0,1) 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: monmap e3: 2 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],smithi196=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: osdmap e50: 6 total, 6 up, 6 in 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: mgrmap e38: smithi176.tzfdwd(active, since 27s), standbys: smithi196.vdktbk 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: overall HEALTH_OK 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:28.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi176"}]: dispatch 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: mon.smithi176 calling monitor election 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: mon.smithi196 calling monitor election 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mon metadata", "id": "smithi196"}]: dispatch 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: mon.smithi176 is new leader, mons smithi176,smithi196 in quorum (ranks 0,1) 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: monmap e3: 2 mons at {smithi176=[v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0],smithi196=[v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: osdmap e50: 6 total, 6 up, 6 in 2024-03-13T14:54:28.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: mgrmap e38: smithi176.tzfdwd(active, since 27s), standbys: smithi196.vdktbk 2024-03-13T14:54:28.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: overall HEALTH_OK 2024-03-13T14:54:28.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:28.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:28.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:54:29.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:29 smithi196 ceph-mon[86193]: pgmap v17: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:29.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:29 smithi176 ceph-mon[112742]: pgmap v17: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:30.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:30.837 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:31.542 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:31 smithi196 ceph-mon[86193]: pgmap v18: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:31.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:31 smithi176 ceph-mon[112742]: pgmap v18: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:33.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:32 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:33.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:32 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:33.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:32 smithi176 ceph-mon[112742]: pgmap v19: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:33.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:32 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:33.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:32 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:33.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:32 smithi196 ceph-mon[86193]: pgmap v19: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:35.108 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:34 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:35.109 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:34 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:35.109 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:34 smithi196 ceph-mon[86193]: pgmap v20: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:54:35.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:34 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:35.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:34 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:35.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:34 smithi176 ceph-mon[112742]: pgmap v20: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:54:37.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:37 smithi176 ceph-mon[112742]: pgmap v21: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:37.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:37 smithi196 ceph-mon[86193]: pgmap v21: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:39.503 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:39 smithi196 ceph-mon[86193]: pgmap v22: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:39.503 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:39.503 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:39.503 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:39.503 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:54:39.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:39 smithi176 ceph-mon[112742]: pgmap v22: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:39.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:54:40.621 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:40 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:40.621 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:40 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:54:40.621 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:40 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:54:40.621 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:40 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:40 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:40 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:54:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:40 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:54:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:40 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:41.505 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:41 smithi176 ceph-mon[112742]: Reconfiguring mon.smithi176 (monmap changed)... 2024-03-13T14:54:41.505 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:41 smithi176 ceph-mon[112742]: Reconfiguring daemon mon.smithi176 on smithi176 2024-03-13T14:54:41.505 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:41 smithi176 ceph-mon[112742]: pgmap v23: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T14:54:41.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:41 smithi196 ceph-mon[86193]: Reconfiguring mon.smithi176 (monmap changed)... 2024-03-13T14:54:41.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:41 smithi196 ceph-mon[86193]: Reconfiguring daemon mon.smithi176 on smithi176 2024-03-13T14:54:41.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:41 smithi196 ceph-mon[86193]: pgmap v23: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T14:54:42.885 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:42.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:42.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: Reconfiguring mgr.smithi176.tzfdwd (monmap changed)... 2024-03-13T14:54:42.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:54:42.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:54:42.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:42.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: Reconfiguring daemon mgr.smithi176.tzfdwd on smithi176 2024-03-13T14:54:42.886 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:42 smithi176 ceph-mon[112742]: pgmap v24: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: Reconfiguring mgr.smithi176.tzfdwd (monmap changed)... 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi176.tzfdwd", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: Reconfiguring daemon mgr.smithi176.tzfdwd on smithi176 2024-03-13T14:54:43.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:42 smithi196 ceph-mon[86193]: pgmap v24: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: Reconfiguring ceph-exporter.smithi176 (monmap changed)... 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: Unable to update caps for client.ceph-exporter.smithi176 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi176"}]: dispatch 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: Reconfiguring daemon ceph-exporter.smithi176 on smithi176 2024-03-13T14:54:45.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:44 smithi176 ceph-mon[112742]: pgmap v25: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T14:54:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: Reconfiguring ceph-exporter.smithi176 (monmap changed)... 2024-03-13T14:54:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:54:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:54:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-13T14:54:45.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: Unable to update caps for client.ceph-exporter.smithi176 2024-03-13T14:54:45.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi176"}]: dispatch 2024-03-13T14:54:45.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:45.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: Reconfiguring daemon ceph-exporter.smithi176 on smithi176 2024-03-13T14:54:45.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:44 smithi196 ceph-mon[86193]: pgmap v25: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T14:54:46.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:46.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:46.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:54:46.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:47.192 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:47.192 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:47 smithi176 ceph-mon[112742]: Reconfiguring crash.smithi176 (monmap changed)... 2024-03-13T14:54:47.192 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:54:47.192 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:47.193 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:47 smithi176 ceph-mon[112742]: Reconfiguring daemon crash.smithi176 on smithi176 2024-03-13T14:54:47.193 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:47 smithi176 ceph-mon[112742]: pgmap v26: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T14:54:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:47.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:47 smithi196 ceph-mon[86193]: Reconfiguring crash.smithi176 (monmap changed)... 2024-03-13T14:54:47.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:54:47.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:47.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:47 smithi196 ceph-mon[86193]: Reconfiguring daemon crash.smithi176 on smithi176 2024-03-13T14:54:47.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:47 smithi196 ceph-mon[86193]: pgmap v26: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T14:54:48.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.647+0000 7f155c400700 1 -- 172.21.15.176:0/341959439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 msgr2=0x7f15541079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:48.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.647+0000 7f155c400700 1 --2- 172.21.15.176:0/341959439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f15541079d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1540009910 tx=0x7f1540009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:48.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.648+0000 7f155c400700 1 -- 172.21.15.176:0/341959439 shutdown_connections 2024-03-13T14:54:48.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.648+0000 7f155c400700 1 --2- 172.21.15.176:0/341959439 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f15541079d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:48.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.648+0000 7f155c400700 1 --2- 172.21.15.176:0/341959439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1554102fd0 0x7f15541033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:48.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.648+0000 7f155c400700 1 -- 172.21.15.176:0/341959439 >> 172.21.15.176:0/341959439 conn(0x7f15540fe840 msgr2=0x7f1554100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:48.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.648+0000 7f155c400700 1 -- 172.21.15.176:0/341959439 shutdown_connections 2024-03-13T14:54:48.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.648+0000 7f155c400700 1 -- 172.21.15.176:0/341959439 wait complete. 2024-03-13T14:54:48.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.649+0000 7f155c400700 1 Processor -- start 2024-03-13T14:54:48.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.650+0000 7f155c400700 1 -- start start 2024-03-13T14:54:48.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.650+0000 7f155c400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1554102fd0 0x7f155419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:48.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.650+0000 7f155c400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f155419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:48.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.650+0000 7f155c400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155419f3a0 con 0x7f1554103980 2024-03-13T14:54:48.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.650+0000 7f155c400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554198850 con 0x7f1554102fd0 2024-03-13T14:54:48.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.651+0000 7f1559000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f155419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:48.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.651+0000 7f1559000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f155419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42726/0 (socket says 172.21.15.176:42726) 2024-03-13T14:54:48.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.651+0000 7f1559000700 1 -- 172.21.15.176:0/607265249 learned_addr learned my addr 172.21.15.176:0/607265249 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:48.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.651+0000 7f1559a00700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1554102fd0 0x7f155419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:48.641 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.651+0000 7f1559000700 1 -- 172.21.15.176:0/607265249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1554102fd0 msgr2=0x7f155419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.651+0000 7f1559000700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1554102fd0 0x7f155419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.651+0000 7f1559000700 1 -- 172.21.15.176:0/607265249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15400095f0 con 0x7f1554103980 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.652+0000 7f1559000700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f155419ed10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f154000bd70 tx=0x7f154000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.652+0000 7f1552000700 1 -- 172.21.15.176:0/607265249 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154001d070 con 0x7f1554103980 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.652+0000 7f1552000700 1 -- 172.21.15.176:0/607265249 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1540005dc0 con 0x7f1554103980 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.652+0000 7f1552000700 1 -- 172.21.15.176:0/607265249 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1540017400 con 0x7f1554103980 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.653+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1554198ad0 con 0x7f1554103980 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.653+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1554198fc0 con 0x7f1554103980 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.654+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155410b3d0 con 0x7f1554103980 2024-03-13T14:54:48.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.655+0000 7f1552000700 1 -- 172.21.15.176:0/607265249 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f154000f7f0 con 0x7f1554103980 2024-03-13T14:54:48.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.656+0000 7f1552000700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f153c06e0d0 0x7f153c070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:48.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.657+0000 7f1552000700 1 -- 172.21.15.176:0/607265249 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f154008f3c0 con 0x7f1554103980 2024-03-13T14:54:48.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.657+0000 7f1559a00700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f153c06e0d0 0x7f153c070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:48.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.658+0000 7f1559a00700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f153c06e0d0 0x7f153c070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1548005950 tx=0x7f15480058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:48.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.662+0000 7f1552000700 1 -- 172.21.15.176:0/607265249 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f154005c220 con 0x7f1554103980 2024-03-13T14:54:48.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.917+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15541992f0 con 0x7f153c06e0d0 2024-03-13T14:54:48.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.920+0000 7f1552000700 1 -- 172.21.15.176:0/607265249 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f15541992f0 con 0x7f153c06e0d0 2024-03-13T14:54:48.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f153c06e0d0 msgr2=0x7f153c070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:48.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f153c06e0d0 0x7f153c070590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1548005950 tx=0x7f15480058e0 comp rx=0 tx=0).stop 2024-03-13T14:54:48.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 msgr2=0x7f155419ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:48.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f155419ed10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f154000bd70 tx=0x7f154000be50 comp rx=0 tx=0).stop 2024-03-13T14:54:48.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 shutdown_connections 2024-03-13T14:54:48.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f153c06e0d0 0x7f153c070590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:48.909 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1554102fd0 0x7f155419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:48.910 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 --2- 172.21.15.176:0/607265249 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1554103980 0x7f155419ed10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:48.910 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.922+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 >> 172.21.15.176:0/607265249 conn(0x7f15540fe840 msgr2=0x7f1554100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:48.910 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.923+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 shutdown_connections 2024-03-13T14:54:48.910 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:48.923+0000 7f155c400700 1 -- 172.21.15.176:0/607265249 wait complete. 2024-03-13T14:54:48.925 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:54:49.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.080+0000 7efffd400700 1 -- 172.21.15.176:0/607782130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff8103000 msgr2=0x7efff81033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.080+0000 7efffd400700 1 --2- 172.21.15.176:0/607782130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff8103000 0x7efff81033e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7effe8009910 tx=0x7effe8009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:49.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.080+0000 7efffd400700 1 -- 172.21.15.176:0/607782130 shutdown_connections 2024-03-13T14:54:49.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.080+0000 7efffd400700 1 --2- 172.21.15.176:0/607782130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff81039b0 0x7efff8107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.080+0000 7efffd400700 1 --2- 172.21.15.176:0/607782130 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff8103000 0x7efff81033e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.080+0000 7efffd400700 1 -- 172.21.15.176:0/607782130 >> 172.21.15.176:0/607782130 conn(0x7efff80fe8d0 msgr2=0x7efff8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:49.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.081+0000 7efffd400700 1 -- 172.21.15.176:0/607782130 shutdown_connections 2024-03-13T14:54:49.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.081+0000 7efffd400700 1 -- 172.21.15.176:0/607782130 wait complete. 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efffd400700 1 Processor -- start 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efffd400700 1 -- start start 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efffd400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8103000 0x7efff819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efffd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff81039b0 0x7efff819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efffd400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff819f4d0 con 0x7efff81039b0 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efffd400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff8198980 con 0x7efff8103000 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efff5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff81039b0 0x7efff819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.082+0000 7efff5e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff81039b0 0x7efff819ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42742/0 (socket says 172.21.15.176:42742) 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.083+0000 7efff5e00700 1 -- 172.21.15.176:0/826796156 learned_addr learned my addr 172.21.15.176:0/826796156 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.083+0000 7efff6800700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8103000 0x7efff819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.083+0000 7efff5e00700 1 -- 172.21.15.176:0/826796156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8103000 msgr2=0x7efff819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.083+0000 7efff5e00700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8103000 0x7efff819e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.083+0000 7efff5e00700 1 -- 172.21.15.176:0/826796156 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80095f0 con 0x7efff81039b0 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.083+0000 7efff5e00700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff81039b0 0x7efff819ee40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7effe400b480 tx=0x7effe400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.083+0000 7effef400700 1 -- 172.21.15.176:0/826796156 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effe40118c0 con 0x7efff81039b0 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.084+0000 7effef400700 1 -- 172.21.15.176:0/826796156 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effe4010380 con 0x7efff81039b0 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.084+0000 7effef400700 1 -- 172.21.15.176:0/826796156 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effe400f4f0 con 0x7efff81039b0 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.084+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff8198c60 con 0x7efff81039b0 2024-03-13T14:54:49.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.084+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff81991b0 con 0x7efff81039b0 2024-03-13T14:54:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.085+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff810b3a0 con 0x7efff81039b0 2024-03-13T14:54:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.086+0000 7effef400700 1 -- 172.21.15.176:0/826796156 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7effe40104f0 con 0x7efff81039b0 2024-03-13T14:54:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.087+0000 7effef400700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7effdc06e0d0 0x7effdc070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.087+0000 7effef400700 1 -- 172.21.15.176:0/826796156 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7effe408da00 con 0x7efff81039b0 2024-03-13T14:54:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.087+0000 7efff6800700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7effdc06e0d0 0x7effdc070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.088+0000 7efff6800700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7effdc06e0d0 0x7effdc070590 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7effe8006010 tx=0x7effe801a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:49.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.091+0000 7effef400700 1 -- 172.21.15.176:0/826796156 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7effe405a970 con 0x7efff81039b0 2024-03-13T14:54:49.172 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:49.172 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:49.172 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:49 smithi176 ceph-mon[112742]: Reconfiguring osd.0 (monmap changed)... 2024-03-13T14:54:49.172 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-13T14:54:49.172 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:49.172 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:49 smithi176 ceph-mon[112742]: Reconfiguring daemon osd.0 on smithi176 2024-03-13T14:54:49.172 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:49 smithi176 ceph-mon[112742]: pgmap v27: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:49.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.345+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efff81994f0 con 0x7effdc06e0d0 2024-03-13T14:54:49.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.346+0000 7effef400700 1 -- 172.21.15.176:0/826796156 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7efff81994f0 con 0x7effdc06e0d0 2024-03-13T14:54:49.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7effdc06e0d0 msgr2=0x7effdc070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7effdc06e0d0 0x7effdc070590 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7effe8006010 tx=0x7effe801a040 comp rx=0 tx=0).stop 2024-03-13T14:54:49.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff81039b0 msgr2=0x7efff819ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff81039b0 0x7efff819ee40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7effe400b480 tx=0x7effe400b840 comp rx=0 tx=0).stop 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 shutdown_connections 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7effdc06e0d0 0x7effdc070590 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efff8103000 0x7efff819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 --2- 172.21.15.176:0/826796156 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efff81039b0 0x7efff819ee40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.350+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 >> 172.21.15.176:0/826796156 conn(0x7efff80fe8d0 msgr2=0x7efff80fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.351+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 shutdown_connections 2024-03-13T14:54:49.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.351+0000 7efffd400700 1 -- 172.21.15.176:0/826796156 wait complete. 2024-03-13T14:54:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:49 smithi196 ceph-mon[86193]: Reconfiguring osd.0 (monmap changed)... 2024-03-13T14:54:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-13T14:54:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:49 smithi196 ceph-mon[86193]: Reconfiguring daemon osd.0 on smithi176 2024-03-13T14:54:49.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:49 smithi196 ceph-mon[86193]: pgmap v27: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:49.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.510+0000 7fa94ca00700 1 -- 172.21.15.176:0/2780618348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa948106f60 msgr2=0x7fa9481073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.510+0000 7fa94ca00700 1 --2- 172.21.15.176:0/2780618348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa948106f60 0x7fa9481073e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa930009910 tx=0x7fa930009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:49.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.511+0000 7fa94ca00700 1 -- 172.21.15.176:0/2780618348 shutdown_connections 2024-03-13T14:54:49.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.511+0000 7fa94ca00700 1 --2- 172.21.15.176:0/2780618348 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa948106f60 0x7fa9481073e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.511+0000 7fa94ca00700 1 --2- 172.21.15.176:0/2780618348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa94810d010 0x7fa94810d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.511+0000 7fa94ca00700 1 -- 172.21.15.176:0/2780618348 >> 172.21.15.176:0/2780618348 conn(0x7fa948076760 msgr2=0x7fa948078b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:49.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.511+0000 7fa94ca00700 1 -- 172.21.15.176:0/2780618348 shutdown_connections 2024-03-13T14:54:49.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.511+0000 7fa94ca00700 1 -- 172.21.15.176:0/2780618348 wait complete. 2024-03-13T14:54:49.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.512+0000 7fa94ca00700 1 Processor -- start 2024-03-13T14:54:49.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.512+0000 7fa94ca00700 1 -- start start 2024-03-13T14:54:49.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.512+0000 7fa94ca00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948106f60 0x7fa9481a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.512+0000 7fa94ca00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa94810d010 0x7fa9481a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.512+0000 7fa94ca00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9481a3480 con 0x7fa94810d010 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.512+0000 7fa94ca00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94819c930 con 0x7fa948106f60 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.513+0000 7fa93fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa94810d010 0x7fa9481a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.513+0000 7fa93fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa94810d010 0x7fa9481a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42760/0 (socket says 172.21.15.176:42760) 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.513+0000 7fa93fe00700 1 -- 172.21.15.176:0/1448901127 learned_addr learned my addr 172.21.15.176:0/1448901127 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.513+0000 7fa946800700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948106f60 0x7fa9481a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.513+0000 7fa93fe00700 1 -- 172.21.15.176:0/1448901127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948106f60 msgr2=0x7fa9481a28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.513+0000 7fa93fe00700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948106f60 0x7fa9481a28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.513+0000 7fa93fe00700 1 -- 172.21.15.176:0/1448901127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300095f0 con 0x7fa94810d010 2024-03-13T14:54:49.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.514+0000 7fa93fe00700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa94810d010 0x7fa9481a2df0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa930006010 tx=0x7fa93000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:49.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.514+0000 7fa93f400700 1 -- 172.21.15.176:0/1448901127 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa93001d070 con 0x7fa94810d010 2024-03-13T14:54:49.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.514+0000 7fa93f400700 1 -- 172.21.15.176:0/1448901127 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa93000f460 con 0x7fa94810d010 2024-03-13T14:54:49.503 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.514+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94819cbb0 con 0x7fa94810d010 2024-03-13T14:54:49.503 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.514+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94819d0a0 con 0x7fa94810d010 2024-03-13T14:54:49.503 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.514+0000 7fa93f400700 1 -- 172.21.15.176:0/1448901127 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa930017630 con 0x7fa94810d010 2024-03-13T14:54:49.503 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.515+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94810f4a0 con 0x7fa94810d010 2024-03-13T14:54:49.504 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.516+0000 7fa93f400700 1 -- 172.21.15.176:0/1448901127 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fa93000f5d0 con 0x7fa94810d010 2024-03-13T14:54:49.504 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.517+0000 7fa93f400700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa92c06e0e0 0x7fa92c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.504 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.517+0000 7fa93f400700 1 -- 172.21.15.176:0/1448901127 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa93008f4b0 con 0x7fa94810d010 2024-03-13T14:54:49.505 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.517+0000 7fa946800700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa92c06e0e0 0x7fa92c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.505 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.518+0000 7fa946800700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa92c06e0e0 0x7fa92c0705a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa938005950 tx=0x7fa9380058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:49.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.522+0000 7fa93f400700 1 -- 172.21.15.176:0/1448901127 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa93005c1f0 con 0x7fa94810d010 2024-03-13T14:54:49.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.756+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa94819d3d0 con 0x7fa92c06e0e0 2024-03-13T14:54:49.769 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.782+0000 7fa93f400700 1 -- 172.21.15.176:0/1448901127 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa94819d3d0 con 0x7fa92c06e0e0 2024-03-13T14:54:49.770 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:54:49.770 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (2m) 42s ago 16m 14.3M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:54:49.770 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (16m) 42s ago 16m 8178k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:54:49.770 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (15m) 18s ago 15m 8144k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:54:49.770 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (16m) 42s ago 16m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (15m) 18s ago 15m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (2m) 42s ago 15m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (8m) 42s ago 8m 87.2M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (9m) 42s ago 9m 16.3M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (9m) 18s ago 9m 141M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (8m) 18s ago 8m 16.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (5m) 42s ago 18m 476M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (4m) 18s ago 14m 448M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (59s) 42s ago 18m 28.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (24s) 18s ago 14m 19.6M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (3m) 42s ago 16m 7780k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:54:49.771 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (3m) 18s ago 14m 8150k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:54:49.772 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (13m) 42s ago 13m 233M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:54:49.772 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (13m) 42s ago 13m 246M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:54:49.772 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (12m) 42s ago 12m 220M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:54:49.772 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (12m) 18s ago 12m 305M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:54:49.772 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (11m) 18s ago 11m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:54:49.772 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (10m) 18s ago 10m 229M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:54:49.772 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (3m) 42s ago 15m 63.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:54:49.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.785+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa92c06e0e0 msgr2=0x7fa92c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.785+0000 7fa94ca00700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa92c06e0e0 0x7fa92c0705a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa938005950 tx=0x7fa9380058e0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.785+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa94810d010 msgr2=0x7fa9481a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.785+0000 7fa94ca00700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa94810d010 0x7fa9481a2df0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa930006010 tx=0x7fa93000be20 comp rx=0 tx=0).stop 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.786+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 shutdown_connections 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.786+0000 7fa94ca00700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa92c06e0e0 0x7fa92c0705a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.786+0000 7fa94ca00700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa948106f60 0x7fa9481a28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.786+0000 7fa94ca00700 1 --2- 172.21.15.176:0/1448901127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa94810d010 0x7fa9481a2df0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.786+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 >> 172.21.15.176:0/1448901127 conn(0x7fa948076760 msgr2=0x7fa9480780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.786+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 shutdown_connections 2024-03-13T14:54:49.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.786+0000 7fa94ca00700 1 -- 172.21.15.176:0/1448901127 wait complete. 2024-03-13T14:54:49.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 -- 172.21.15.176:0/1563829672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 msgr2=0x7f512c107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 --2- 172.21.15.176:0/1563829672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c107810 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5118009910 tx=0x7f5118009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:49.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 -- 172.21.15.176:0/1563829672 shutdown_connections 2024-03-13T14:54:49.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 --2- 172.21.15.176:0/1563829672 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c107810 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 --2- 172.21.15.176:0/1563829672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102e10 0x7f512c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 -- 172.21.15.176:0/1563829672 >> 172.21.15.176:0/1563829672 conn(0x7f512c0fe6c0 msgr2=0x7f512c100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:49.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 -- 172.21.15.176:0/1563829672 shutdown_connections 2024-03-13T14:54:49.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.944+0000 7f512be00700 1 -- 172.21.15.176:0/1563829672 wait complete. 2024-03-13T14:54:49.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.945+0000 7f512be00700 1 Processor -- start 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512be00700 1 -- start start 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512be00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102e10 0x7f512c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512be00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c19f0c0 con 0x7f512c1037c0 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512be00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c198600 con 0x7f512c102e10 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512a000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512a000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42788/0 (socket says 172.21.15.176:42788) 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.946+0000 7f512a000700 1 -- 172.21.15.176:0/1269675276 learned_addr learned my addr 172.21.15.176:0/1269675276 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.947+0000 7f512aa00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102e10 0x7f512c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.947+0000 7f512a000700 1 -- 172.21.15.176:0/1269675276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102e10 msgr2=0x7f512c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:49.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.947+0000 7f512a000700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102e10 0x7f512c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:49.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.947+0000 7f512a000700 1 -- 172.21.15.176:0/1269675276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51180095f0 con 0x7f512c1037c0 2024-03-13T14:54:49.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.947+0000 7f512a000700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c19eac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f511800bda0 tx=0x7f511800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:49.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.947+0000 7f5130a00700 1 -- 172.21.15.176:0/1269675276 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511801d070 con 0x7f512c1037c0 2024-03-13T14:54:49.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.947+0000 7f5130a00700 1 -- 172.21.15.176:0/1269675276 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5118004fb0 con 0x7f512c1037c0 2024-03-13T14:54:49.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.948+0000 7f5130a00700 1 -- 172.21.15.176:0/1269675276 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511800f460 con 0x7f512c1037c0 2024-03-13T14:54:49.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.948+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c198880 con 0x7f512c1037c0 2024-03-13T14:54:49.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.948+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c198d70 con 0x7f512c1037c0 2024-03-13T14:54:49.937 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.949+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c10b210 con 0x7f512c1037c0 2024-03-13T14:54:49.937 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.950+0000 7f5130a00700 1 -- 172.21.15.176:0/1269675276 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f5118022800 con 0x7f512c1037c0 2024-03-13T14:54:49.937 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.951+0000 7f5130a00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f511006e0d0 0x7f5110070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:49.938 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.951+0000 7f5130a00700 1 -- 172.21.15.176:0/1269675276 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f511808f440 con 0x7f512c1037c0 2024-03-13T14:54:49.938 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.951+0000 7f512aa00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f511006e0d0 0x7f5110070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:49.938 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.951+0000 7f512aa00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f511006e0d0 0x7f5110070590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f511c005ea0 tx=0x7f511c005e30 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:49.942 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:49.955+0000 7f5130a00700 1 -- 172.21.15.176:0/1269675276 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f511805c3d0 con 0x7f512c1037c0 2024-03-13T14:54:50.238 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.252+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f512c1990a0 con 0x7f512c1037c0 2024-03-13T14:54:50.239 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.253+0000 7f5130a00700 1 -- 172.21.15.176:0/1269675276 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f5118058030 con 0x7f512c1037c0 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:50.240 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:54:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:54:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:54:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:54:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-13T14:54:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T14:54:50.241 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:54:50.241 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:54:50.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.255+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f511006e0d0 msgr2=0x7f5110070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.255+0000 7f512be00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f511006e0d0 0x7f5110070590 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f511c005ea0 tx=0x7f511c005e30 comp rx=0 tx=0).stop 2024-03-13T14:54:50.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.255+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 msgr2=0x7f512c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.255+0000 7f512be00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c19eac0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f511800bda0 tx=0x7f511800be80 comp rx=0 tx=0).stop 2024-03-13T14:54:50.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.256+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 shutdown_connections 2024-03-13T14:54:50.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.256+0000 7f512be00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f511006e0d0 0x7f5110070590 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.256+0000 7f512be00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f512c102e10 0x7f512c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.256+0000 7f512be00700 1 --2- 172.21.15.176:0/1269675276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f512c1037c0 0x7f512c19eac0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.256+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 >> 172.21.15.176:0/1269675276 conn(0x7f512c0fe6c0 msgr2=0x7f512c100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:50.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.256+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 shutdown_connections 2024-03-13T14:54:50.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.256+0000 7f512be00700 1 -- 172.21.15.176:0/1269675276 wait complete. 2024-03-13T14:54:50.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:50 smithi176 ceph-mon[112742]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:50.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:50 smithi176 ceph-mon[112742]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:50.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 -- 172.21.15.176:0/645160676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674073ae0 msgr2=0x7fe67410ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 --2- 172.21.15.176:0/645160676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674073ae0 0x7fe67410ce40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe660009910 tx=0x7fe660009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:50.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 -- 172.21.15.176:0/645160676 shutdown_connections 2024-03-13T14:54:50.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 --2- 172.21.15.176:0/645160676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674073ae0 0x7fe67410ce40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 --2- 172.21.15.176:0/645160676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6740731c0 0x7fe6740735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 -- 172.21.15.176:0/645160676 >> 172.21.15.176:0/645160676 conn(0x7fe6740fc630 msgr2=0x7fe6740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:50.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 -- 172.21.15.176:0/645160676 shutdown_connections 2024-03-13T14:54:50.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.437+0000 7fe67a400700 1 -- 172.21.15.176:0/645160676 wait complete. 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.438+0000 7fe67a400700 1 Processor -- start 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe67a400700 1 -- start start 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe67a400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6740731c0 0x7fe674198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe67a400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674073ae0 0x7fe674198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe67a400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674199530 con 0x7fe6740731c0 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe67a400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67419d340 con 0x7fe674073ae0 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe673400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6740731c0 0x7fe674198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:50.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe673400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6740731c0 0x7fe674198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42802/0 (socket says 172.21.15.176:42802) 2024-03-13T14:54:50.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.439+0000 7fe673400700 1 -- 172.21.15.176:0/4033126614 learned_addr learned my addr 172.21.15.176:0/4033126614 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:50.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe672a00700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674073ae0 0x7fe674198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:50.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:50 smithi196 ceph-mon[86193]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:50.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:50 smithi196 ceph-mon[86193]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:50.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe673400700 1 -- 172.21.15.176:0/4033126614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674073ae0 msgr2=0x7fe674198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe673400700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674073ae0 0x7fe674198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe673400700 1 -- 172.21.15.176:0/4033126614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600095f0 con 0x7fe6740731c0 2024-03-13T14:54:50.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe672a00700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674073ae0 0x7fe674198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:54:50.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe673400700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6740731c0 0x7fe674198960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe66400d8d0 tx=0x7fe66400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:50.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe66be00700 1 -- 172.21.15.176:0/4033126614 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe664009940 con 0x7fe6740731c0 2024-03-13T14:54:50.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe66be00700 1 -- 172.21.15.176:0/4033126614 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe664010460 con 0x7fe6740731c0 2024-03-13T14:54:50.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe67419d620 con 0x7fe6740731c0 2024-03-13T14:54:50.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe66be00700 1 -- 172.21.15.176:0/4033126614 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66400f5f0 con 0x7fe6740731c0 2024-03-13T14:54:50.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.440+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67419db70 con 0x7fe6740731c0 2024-03-13T14:54:50.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.442+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67410a540 con 0x7fe6740731c0 2024-03-13T14:54:50.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.443+0000 7fe66be00700 1 -- 172.21.15.176:0/4033126614 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fe6640105d0 con 0x7fe6740731c0 2024-03-13T14:54:50.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.443+0000 7fe66be00700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806e0d0 0x7fe658070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:50.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.444+0000 7fe66be00700 1 -- 172.21.15.176:0/4033126614 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe66408de80 con 0x7fe6740731c0 2024-03-13T14:54:50.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.444+0000 7fe672a00700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806e0d0 0x7fe658070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:50.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.444+0000 7fe672a00700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806e0d0 0x7fe658070590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe674199f80 tx=0x7fe66000b560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:50.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.448+0000 7fe66be00700 1 -- 172.21.15.176:0/4033126614 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fe66405ace0 con 0x7fe6740731c0 2024-03-13T14:54:50.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.741+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe67404ea90 con 0x7fe6740731c0 2024-03-13T14:54:50.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.741+0000 7fe66be00700 1 -- 172.21.15.176:0/4033126614 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fe664016770 con 0x7fe6740731c0 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:54:50.729 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:54:50.730 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:50.731 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:54:50.732 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:54:50.732 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:50.732 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:54:50.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806e0d0 msgr2=0x7fe658070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806e0d0 0x7fe658070590 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe674199f80 tx=0x7fe66000b560 comp rx=0 tx=0).stop 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6740731c0 msgr2=0x7fe674198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6740731c0 0x7fe674198960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe66400d8d0 tx=0x7fe66400dc90 comp rx=0 tx=0).stop 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 shutdown_connections 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806e0d0 0x7fe658070590 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe6740731c0 0x7fe674198960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 --2- 172.21.15.176:0/4033126614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674073ae0 0x7fe674198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 >> 172.21.15.176:0/4033126614 conn(0x7fe6740fc630 msgr2=0x7fe674107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 shutdown_connections 2024-03-13T14:54:50.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.744+0000 7fe67a400700 1 -- 172.21.15.176:0/4033126614 wait complete. 2024-03-13T14:54:50.735 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:54:50.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 -- 172.21.15.176:0/1047153235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 msgr2=0x7f0e4010cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 --2- 172.21.15.176:0/1047153235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4010cec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009910 tx=0x7f0e28009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:50.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 -- 172.21.15.176:0/1047153235 shutdown_connections 2024-03-13T14:54:50.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 --2- 172.21.15.176:0/1047153235 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4010cec0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 --2- 172.21.15.176:0/1047153235 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40073160 0x7f0e40073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 -- 172.21.15.176:0/1047153235 >> 172.21.15.176:0/1047153235 conn(0x7f0e400fc650 msgr2=0x7f0e400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 -- 172.21.15.176:0/1047153235 shutdown_connections 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.901+0000 7f0e44e00700 1 -- 172.21.15.176:0/1047153235 wait complete. 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.903+0000 7f0e44e00700 1 Processor -- start 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.903+0000 7f0e44e00700 1 -- start start 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.903+0000 7f0e44e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40073160 0x7f0e4019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.903+0000 7f0e44e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.903+0000 7f0e44e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4019f2c0 con 0x7f0e40073a80 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.903+0000 7f0e44e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e40198800 con 0x7f0e40073160 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e37e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4019ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42814/0 (socket says 172.21.15.176:42814) 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e37e00700 1 -- 172.21.15.176:0/4265148227 learned_addr learned my addr 172.21.15.176:0/4265148227 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:50.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e3de00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40073160 0x7f0e4019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:50.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e37e00700 1 -- 172.21.15.176:0/4265148227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40073160 msgr2=0x7f0e4019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:50.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e37e00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40073160 0x7f0e4019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:50.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e37e00700 1 -- 172.21.15.176:0/4265148227 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e280095f0 con 0x7f0e40073a80 2024-03-13T14:54:50.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.904+0000 7f0e37e00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4019ecc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e280092e0 tx=0x7f0e2800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:50.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.905+0000 7f0e36a00700 1 -- 172.21.15.176:0/4265148227 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e2801d070 con 0x7f0e40073a80 2024-03-13T14:54:50.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.905+0000 7f0e36a00700 1 -- 172.21.15.176:0/4265148227 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e2800f460 con 0x7f0e40073a80 2024-03-13T14:54:50.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.905+0000 7f0e36a00700 1 -- 172.21.15.176:0/4265148227 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e28017630 con 0x7f0e40073a80 2024-03-13T14:54:50.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.905+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e40198a80 con 0x7f0e40073a80 2024-03-13T14:54:50.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.905+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e40198f70 con 0x7f0e40073a80 2024-03-13T14:54:50.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.906+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4010a5c0 con 0x7f0e40073a80 2024-03-13T14:54:50.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.907+0000 7f0e36a00700 1 -- 172.21.15.176:0/4265148227 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f0e2800f5d0 con 0x7f0e40073a80 2024-03-13T14:54:50.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.908+0000 7f0e36a00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e2406e0d0 0x7f0e24070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:50.896 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.908+0000 7f0e36a00700 1 -- 172.21.15.176:0/4265148227 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0e2808f4a0 con 0x7f0e40073a80 2024-03-13T14:54:50.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.909+0000 7f0e3de00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e2406e0d0 0x7f0e24070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:50.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.909+0000 7f0e3de00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e2406e0d0 0x7f0e24070590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0e30005ea0 tx=0x7f0e30005e30 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:50.900 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:50.913+0000 7f0e36a00700 1 -- 172.21.15.176:0/4265148227 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f0e2805c300 con 0x7f0e40073a80 2024-03-13T14:54:51.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.177+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e401991e0 con 0x7f0e2406e0d0 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.178+0000 7f0e36a00700 1 -- 172.21.15.176:0/4265148227 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f0e401991e0 con 0x7f0e2406e0d0 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:54:51.165 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:54:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:54:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "4/23 daemons upgraded", 2024-03-13T14:54:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mon daemons", 2024-03-13T14:54:51.166 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:54:51.166 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e2406e0d0 msgr2=0x7f0e24070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e2406e0d0 0x7f0e24070590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0e30005ea0 tx=0x7f0e30005e30 comp rx=0 tx=0).stop 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 msgr2=0x7f0e4019ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4019ecc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e280092e0 tx=0x7f0e2800be00 comp rx=0 tx=0).stop 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 shutdown_connections 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e2406e0d0 0x7f0e24070590 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e40073160 0x7f0e4019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 --2- 172.21.15.176:0/4265148227 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e40073a80 0x7f0e4019ecc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 >> 172.21.15.176:0/4265148227 conn(0x7f0e400fc650 msgr2=0x7f0e40107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:51.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 shutdown_connections 2024-03-13T14:54:51.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.182+0000 7f0e44e00700 1 -- 172.21.15.176:0/4265148227 wait complete. 2024-03-13T14:54:51.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 -- 172.21.15.176:0/3005226327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7120103000 msgr2=0x7f71201033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:51.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 --2- 172.21.15.176:0/3005226327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7120103000 0x7f71201033e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7110009910 tx=0x7f7110009c20 comp rx=0 tx=0).stop 2024-03-13T14:54:51.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 -- 172.21.15.176:0/3005226327 shutdown_connections 2024-03-13T14:54:51.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 --2- 172.21.15.176:0/3005226327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71201039b0 0x7f7120107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 --2- 172.21.15.176:0/3005226327 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7120103000 0x7f71201033e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 -- 172.21.15.176:0/3005226327 >> 172.21.15.176:0/3005226327 conn(0x7f71200fe8d0 msgr2=0x7f7120100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:51.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 -- 172.21.15.176:0/3005226327 shutdown_connections 2024-03-13T14:54:51.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.341+0000 7f7125e00700 1 -- 172.21.15.176:0/3005226327 wait complete. 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1269675276' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: pgmap v28: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:51.329 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:51 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/4033126614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:54:51.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.342+0000 7f7125e00700 1 Processor -- start 2024-03-13T14:54:51.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.342+0000 7f7125e00700 1 -- start start 2024-03-13T14:54:51.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f7125e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7120103000 0x7f712019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:51.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f7125e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71201039b0 0x7f712019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:51.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f7125e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712019f4d0 con 0x7f71201039b0 2024-03-13T14:54:51.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f7125e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7120198980 con 0x7f7120103000 2024-03-13T14:54:51.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f711e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71201039b0 0x7f712019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:51.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f711e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71201039b0 0x7f712019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42826/0 (socket says 172.21.15.176:42826) 2024-03-13T14:54:51.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f711e000700 1 -- 172.21.15.176:0/2801907034 learned_addr learned my addr 172.21.15.176:0/2801907034 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:54:51.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f711ea00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7120103000 0x7f712019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:51.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.343+0000 7f711e000700 1 -- 172.21.15.176:0/2801907034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7120103000 msgr2=0x7f712019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:51.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f711e000700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7120103000 0x7f712019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f711e000700 1 -- 172.21.15.176:0/2801907034 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100095f0 con 0x7f71201039b0 2024-03-13T14:54:51.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f711e000700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71201039b0 0x7f712019ee40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f710c00b480 tx=0x7f710c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:51.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f7124a00700 1 -- 172.21.15.176:0/2801907034 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f710c0118c0 con 0x7f71201039b0 2024-03-13T14:54:51.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f7124a00700 1 -- 172.21.15.176:0/2801907034 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f710c010380 con 0x7f71201039b0 2024-03-13T14:54:51.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7120198c60 con 0x7f71201039b0 2024-03-13T14:54:51.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f7124a00700 1 -- 172.21.15.176:0/2801907034 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f710c00f4f0 con 0x7f71201039b0 2024-03-13T14:54:51.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.344+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71201991b0 con 0x7f71201039b0 2024-03-13T14:54:51.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.345+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712010b3a0 con 0x7f71201039b0 2024-03-13T14:54:51.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.346+0000 7f7124a00700 1 -- 172.21.15.176:0/2801907034 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f710c0104f0 con 0x7f71201039b0 2024-03-13T14:54:51.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.347+0000 7f7124a00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f710406e0d0 0x7f7104070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:54:51.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.347+0000 7f7124a00700 1 -- 172.21.15.176:0/2801907034 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f710c08daa0 con 0x7f71201039b0 2024-03-13T14:54:51.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.347+0000 7f711ea00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f710406e0d0 0x7f7104070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:54:51.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.348+0000 7f711ea00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f710406e0d0 0x7f7104070590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7110000c00 tx=0x7f7110005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:54:51.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.352+0000 7f7124a00700 1 -- 172.21.15.176:0/2801907034 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f710c05a970 con 0x7f71201039b0 2024-03-13T14:54:51.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1269675276' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:54:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: pgmap v28: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-13T14:54:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:51.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:51 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/4033126614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:54:51.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.669+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7120199480 con 0x7f71201039b0 2024-03-13T14:54:51.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.669+0000 7f7124a00700 1 -- 172.21.15.176:0/2801907034 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f710c05a500 con 0x7f71201039b0 2024-03-13T14:54:51.656 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:54:51.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.672+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f710406e0d0 msgr2=0x7f7104070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.672+0000 7f7125e00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f710406e0d0 0x7f7104070590 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7110000c00 tx=0x7f7110005fd0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.672+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71201039b0 msgr2=0x7f712019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.672+0000 7f7125e00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71201039b0 0x7f712019ee40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f710c00b480 tx=0x7f710c00b840 comp rx=0 tx=0).stop 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.673+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 shutdown_connections 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.673+0000 7f7125e00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f710406e0d0 0x7f7104070590 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.673+0000 7f7125e00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7120103000 0x7f712019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.673+0000 7f7125e00700 1 --2- 172.21.15.176:0/2801907034 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f71201039b0 0x7f712019ee40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.673+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 >> 172.21.15.176:0/2801907034 conn(0x7f71200fe8d0 msgr2=0x7f71200fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.673+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 shutdown_connections 2024-03-13T14:54:51.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:54:51.673+0000 7f7125e00700 1 -- 172.21.15.176:0/2801907034 wait complete. 2024-03-13T14:54:52.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:52 smithi176 ceph-mon[112742]: Reconfiguring osd.1 (monmap changed)... 2024-03-13T14:54:52.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:52 smithi176 ceph-mon[112742]: Reconfiguring daemon osd.1 on smithi176 2024-03-13T14:54:52.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:52 smithi176 ceph-mon[112742]: from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:52.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:52 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2801907034' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:54:52.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:52 smithi196 ceph-mon[86193]: Reconfiguring osd.1 (monmap changed)... 2024-03-13T14:54:52.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:52 smithi196 ceph-mon[86193]: Reconfiguring daemon osd.1 on smithi176 2024-03-13T14:54:52.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:52 smithi196 ceph-mon[86193]: from='client.34176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:54:52.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:52 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2801907034' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:54:53.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:53 smithi176 ceph-mon[112742]: pgmap v29: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:53.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:53.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:53.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-13T14:54:53.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:53.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:53 smithi196 ceph-mon[86193]: pgmap v29: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:53.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:53.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-13T14:54:53.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:54.192 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:54 smithi176 ceph-mon[112742]: Reconfiguring osd.2 (monmap changed)... 2024-03-13T14:54:54.193 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:54 smithi176 ceph-mon[112742]: Reconfiguring daemon osd.2 on smithi176 2024-03-13T14:54:54.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:54 smithi196 ceph-mon[86193]: Reconfiguring osd.2 (monmap changed)... 2024-03-13T14:54:54.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:54 smithi196 ceph-mon[86193]: Reconfiguring daemon osd.2 on smithi176 2024-03-13T14:54:55.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:55 smithi196 ceph-mon[86193]: pgmap v30: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:55.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:55 smithi176 ceph-mon[112742]: pgmap v30: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:56.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:56 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:56.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:56 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:56.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:56 smithi176 ceph-mon[112742]: Reconfiguring mds.cephfs.smithi176.ldskqt (monmap changed)... 2024-03-13T14:54:56.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:56 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:54:56.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:56 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:56.347 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:56 smithi176 ceph-mon[112742]: Reconfiguring daemon mds.cephfs.smithi176.ldskqt on smithi176 2024-03-13T14:54:56.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:56 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:56 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:56 smithi196 ceph-mon[86193]: Reconfiguring mds.cephfs.smithi176.ldskqt (monmap changed)... 2024-03-13T14:54:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:56 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:54:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:56 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:56.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:56 smithi196 ceph-mon[86193]: Reconfiguring daemon mds.cephfs.smithi176.ldskqt on smithi176 2024-03-13T14:54:57.173 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:57 smithi176 ceph-mon[112742]: pgmap v31: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:57.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:57 smithi196 ceph-mon[86193]: pgmap v31: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:54:58.519 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:58.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:58.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:58 smithi176 ceph-mon[112742]: Reconfiguring mds.cephfs.smithi176.hjrgov (monmap changed)... 2024-03-13T14:54:58.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:54:58.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:58.520 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:58 smithi176 ceph-mon[112742]: Reconfiguring daemon mds.cephfs.smithi176.hjrgov on smithi176 2024-03-13T14:54:58.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:58.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:54:58.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:58 smithi196 ceph-mon[86193]: Reconfiguring mds.cephfs.smithi176.hjrgov (monmap changed)... 2024-03-13T14:54:58.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:54:58.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:54:58.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:58 smithi196 ceph-mon[86193]: Reconfiguring daemon mds.cephfs.smithi176.hjrgov on smithi176 2024-03-13T14:54:59.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:54:59 smithi176 ceph-mon[112742]: pgmap v32: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:54:59.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:54:59 smithi196 ceph-mon[86193]: pgmap v32: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: Reconfiguring ceph-exporter.smithi196 (monmap changed)... 2024-03-13T14:55:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:55:00.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:55:00.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-13T14:55:00.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: Unable to update caps for client.ceph-exporter.smithi196 2024-03-13T14:55:00.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi196"}]: dispatch 2024-03-13T14:55:00.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:00.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: Reconfiguring daemon ceph-exporter.smithi196 on smithi196 2024-03-13T14:55:00.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: Reconfiguring ceph-exporter.smithi196 (monmap changed)... 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: Unable to update caps for client.ceph-exporter.smithi196 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi196"}]: dispatch 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: Reconfiguring daemon ceph-exporter.smithi196 on smithi196 2024-03-13T14:55:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:01.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:01 smithi196 ceph-mon[86193]: pgmap v33: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:01 smithi176 ceph-mon[112742]: pgmap v33: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:02.812 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:02 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:02.812 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:02 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:02.812 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:02 smithi196 ceph-mon[86193]: Reconfiguring crash.smithi196 (monmap changed)... 2024-03-13T14:55:02.813 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:02 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:55:02.813 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:02 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:02.813 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:02 smithi196 ceph-mon[86193]: Reconfiguring daemon crash.smithi196 on smithi196 2024-03-13T14:55:02.813 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:02 smithi196 ceph-mon[86193]: pgmap v34: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:02.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:02 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:02.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:02 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:02.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:02 smithi176 ceph-mon[112742]: Reconfiguring crash.smithi196 (monmap changed)... 2024-03-13T14:55:02.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:02 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:55:02.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:02 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:02.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:02 smithi176 ceph-mon[112742]: Reconfiguring daemon crash.smithi196 on smithi196 2024-03-13T14:55:02.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:02 smithi176 ceph-mon[112742]: pgmap v34: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:04.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:04.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:04.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: Reconfiguring mgr.smithi196.vdktbk (monmap changed)... 2024-03-13T14:55:04.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:55:04.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:55:04.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:04.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: Reconfiguring daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:55:04.879 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:04 smithi176 ceph-mon[112742]: pgmap v35: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:04.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:04.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:04.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: Reconfiguring mgr.smithi196.vdktbk (monmap changed)... 2024-03-13T14:55:04.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi196.vdktbk", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-13T14:55:04.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-13T14:55:04.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:04.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: Reconfiguring daemon mgr.smithi196.vdktbk on smithi196 2024-03-13T14:55:04.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:04 smithi196 ceph-mon[86193]: pgmap v35: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:07.274 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:07.275 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:07.275 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: Reconfiguring mon.smithi196 (monmap changed)... 2024-03-13T14:55:07.275 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:55:07.275 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:55:07.276 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:07.276 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: Reconfiguring daemon mon.smithi196 on smithi196 2024-03-13T14:55:07.276 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:07 smithi196 ceph-mon[86193]: pgmap v36: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:07.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: Reconfiguring mon.smithi196 (monmap changed)... 2024-03-13T14:55:07.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-13T14:55:07.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-13T14:55:07.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:07.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: Reconfiguring daemon mon.smithi196 on smithi196 2024-03-13T14:55:07.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:07 smithi176 ceph-mon[112742]: pgmap v36: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:09.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:09 smithi176 ceph-mon[112742]: pgmap v37: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:09 smithi176 ceph-mon[112742]: Reconfiguring osd.3 (monmap changed)... 2024-03-13T14:55:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-13T14:55:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:09.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:09 smithi176 ceph-mon[112742]: Reconfiguring daemon osd.3 on smithi196 2024-03-13T14:55:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:09 smithi196 ceph-mon[86193]: pgmap v37: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:09 smithi196 ceph-mon[86193]: Reconfiguring osd.3 (monmap changed)... 2024-03-13T14:55:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-13T14:55:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:09.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:09 smithi196 ceph-mon[86193]: Reconfiguring daemon osd.3 on smithi196 2024-03-13T14:55:11.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:11 smithi176 ceph-mon[112742]: pgmap v38: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:11.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:11.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:11.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-13T14:55:11.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:11.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:11 smithi196 ceph-mon[86193]: pgmap v38: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:11.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:11.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:11.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-13T14:55:11.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:12.530 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:12 smithi176 ceph-mon[112742]: Reconfiguring osd.4 (monmap changed)... 2024-03-13T14:55:12.530 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:12 smithi176 ceph-mon[112742]: Reconfiguring daemon osd.4 on smithi196 2024-03-13T14:55:12.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:12 smithi196 ceph-mon[86193]: Reconfiguring osd.4 (monmap changed)... 2024-03-13T14:55:12.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:12 smithi196 ceph-mon[86193]: Reconfiguring daemon osd.4 on smithi196 2024-03-13T14:55:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:13 smithi176 ceph-mon[112742]: pgmap v39: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:13.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-13T14:55:13.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:13 smithi196 ceph-mon[86193]: pgmap v39: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-13T14:55:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:14.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:14 smithi196 ceph-mon[86193]: Reconfiguring osd.5 (monmap changed)... 2024-03-13T14:55:14.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:14 smithi196 ceph-mon[86193]: Reconfiguring daemon osd.5 on smithi196 2024-03-13T14:55:14.782 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:14 smithi176 ceph-mon[112742]: Reconfiguring osd.5 (monmap changed)... 2024-03-13T14:55:14.782 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:14 smithi176 ceph-mon[112742]: Reconfiguring daemon osd.5 on smithi196 2024-03-13T14:55:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:15 smithi196 ceph-mon[86193]: pgmap v40: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:55:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:15.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:15 smithi176 ceph-mon[112742]: pgmap v40: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:55:15.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:16.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:16.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:16.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:16 smithi176 ceph-mon[112742]: Reconfiguring mds.cephfs.smithi196.tcitrk (monmap changed)... 2024-03-13T14:55:16.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:55:16.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:16.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:16 smithi176 ceph-mon[112742]: Reconfiguring daemon mds.cephfs.smithi196.tcitrk on smithi196 2024-03-13T14:55:16.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:16 smithi176 ceph-mon[112742]: pgmap v41: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:16.917 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:16.918 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:16.918 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:16 smithi196 ceph-mon[86193]: Reconfiguring mds.cephfs.smithi196.tcitrk (monmap changed)... 2024-03-13T14:55:16.918 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:55:16.918 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:16.918 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:16 smithi196 ceph-mon[86193]: Reconfiguring daemon mds.cephfs.smithi196.tcitrk on smithi196 2024-03-13T14:55:16.918 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:16 smithi196 ceph-mon[86193]: pgmap v41: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:19.021 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:19.021 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:19.021 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:18 smithi196 ceph-mon[86193]: Reconfiguring mds.cephfs.smithi196.wabhcu (monmap changed)... 2024-03-13T14:55:19.021 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:55:19.021 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:19.021 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:18 smithi196 ceph-mon[86193]: Reconfiguring daemon mds.cephfs.smithi196.wabhcu on smithi196 2024-03-13T14:55:19.021 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:18 smithi196 ceph-mon[86193]: pgmap v42: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:19.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:19.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:18 smithi176 ceph-mon[112742]: Reconfiguring mds.cephfs.smithi196.wabhcu (monmap changed)... 2024-03-13T14:55:19.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:55:19.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:19.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:18 smithi176 ceph-mon[112742]: Reconfiguring daemon mds.cephfs.smithi196.wabhcu on smithi196 2024-03-13T14:55:19.129 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:18 smithi176 ceph-mon[112742]: pgmap v42: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:21.511 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: pgmap v43: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all mon 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]: dispatch 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]': finished 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi196"}]: dispatch 2024-03-13T14:55:21.512 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi196"}]': finished 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: pgmap v43: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all mon 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]: dispatch 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi176"}]': finished 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi196"}]: dispatch 2024-03-13T14:55:21.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi196"}]': finished 2024-03-13T14:55:21.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.844+0000 7f438d800700 1 -- 172.21.15.176:0/805700300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 msgr2=0x7f4388103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.844+0000 7f438d800700 1 --2- 172.21.15.176:0/805700300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f4388103380 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4378009910 tx=0x7f4378009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.844+0000 7f438d800700 1 -- 172.21.15.176:0/805700300 shutdown_connections 2024-03-13T14:55:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.844+0000 7f438d800700 1 --2- 172.21.15.176:0/805700300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388103950 0x7f43881079a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.844+0000 7f438d800700 1 --2- 172.21.15.176:0/805700300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f4388103380 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.844+0000 7f438d800700 1 -- 172.21.15.176:0/805700300 >> 172.21.15.176:0/805700300 conn(0x7f43880fe810 msgr2=0x7f4388100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.845+0000 7f438d800700 1 -- 172.21.15.176:0/805700300 shutdown_connections 2024-03-13T14:55:21.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.845+0000 7f438d800700 1 -- 172.21.15.176:0/805700300 wait complete. 2024-03-13T14:55:21.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.846+0000 7f438d800700 1 Processor -- start 2024-03-13T14:55:21.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.846+0000 7f438d800700 1 -- start start 2024-03-13T14:55:21.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.846+0000 7f438d800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f438819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:21.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f438d800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388103950 0x7f438819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:21.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f438d800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438819f2e0 con 0x7f4388102fa0 2024-03-13T14:55:21.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f438d800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4388198790 con 0x7f4388103950 2024-03-13T14:55:21.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f438819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:21.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f438819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50322/0 (socket says 172.21.15.176:50322) 2024-03-13T14:55:21.834 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386a00700 1 -- 172.21.15.176:0/2663959910 learned_addr learned my addr 172.21.15.176:0/2663959910 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386000700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388103950 0x7f438819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386a00700 1 -- 172.21.15.176:0/2663959910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388103950 msgr2=0x7f438819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386a00700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388103950 0x7f438819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386a00700 1 -- 172.21.15.176:0/2663959910 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43780095f0 con 0x7f4388102fa0 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.847+0000 7f4386000700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388103950 0x7f438819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.848+0000 7f4386a00700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f438819e710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4378006010 tx=0x7f437800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.848+0000 7f437f400700 1 -- 172.21.15.176:0/2663959910 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f437801d070 con 0x7f4388102fa0 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.848+0000 7f437f400700 1 -- 172.21.15.176:0/2663959910 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f437800f460 con 0x7f4388102fa0 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.848+0000 7f437f400700 1 -- 172.21.15.176:0/2663959910 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4378017630 con 0x7f4388102fa0 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.848+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4388198a10 con 0x7f4388102fa0 2024-03-13T14:55:21.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.848+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4388198f00 con 0x7f4388102fa0 2024-03-13T14:55:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.849+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438810b340 con 0x7f4388102fa0 2024-03-13T14:55:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.851+0000 7f437f400700 1 -- 172.21.15.176:0/2663959910 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f437800f5d0 con 0x7f4388102fa0 2024-03-13T14:55:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.851+0000 7f437f400700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f436c06e0e0 0x7f436c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.852+0000 7f437f400700 1 -- 172.21.15.176:0/2663959910 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f437808f4a0 con 0x7f4388102fa0 2024-03-13T14:55:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.852+0000 7f4386000700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f436c06e0e0 0x7f436c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.852+0000 7f4386000700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f436c06e0e0 0x7f436c0705a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4374005fd0 tx=0x7f4374005ee0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:21.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:21.857+0000 7f437f400700 1 -- 172.21.15.176:0/2663959910 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f437805c1e0 con 0x7f4388102fa0 2024-03-13T14:55:22.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.103+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43881992a0 con 0x7f436c06e0e0 2024-03-13T14:55:22.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.106+0000 7f437f400700 1 -- 172.21.15.176:0/2663959910 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f43881992a0 con 0x7f436c06e0e0 2024-03-13T14:55:22.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f436c06e0e0 msgr2=0x7f436c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f436c06e0e0 0x7f436c0705a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4374005fd0 tx=0x7f4374005ee0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 msgr2=0x7f438819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f438819e710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4378006010 tx=0x7f437800be20 comp rx=0 tx=0).stop 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 shutdown_connections 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f436c06e0e0 0x7f436c0705a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4388102fa0 0x7f438819e710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 --2- 172.21.15.176:0/2663959910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4388103950 0x7f438819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 >> 172.21.15.176:0/2663959910 conn(0x7f43880fe810 msgr2=0x7f43880fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 shutdown_connections 2024-03-13T14:55:22.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.109+0000 7f438d800700 1 -- 172.21.15.176:0/2663959910 wait complete. 2024-03-13T14:55:22.115 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:55:22.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 -- 172.21.15.176:0/1990028223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 msgr2=0x7f02001079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 --2- 172.21.15.176:0/1990028223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f02001079d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009910 tx=0x7f01ec009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:22.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 -- 172.21.15.176:0/1990028223 shutdown_connections 2024-03-13T14:55:22.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 --2- 172.21.15.176:0/1990028223 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f02001079d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 --2- 172.21.15.176:0/1990028223 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0200102fd0 0x7f02001033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 -- 172.21.15.176:0/1990028223 >> 172.21.15.176:0/1990028223 conn(0x7f02000fe840 msgr2=0x7f0200100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:22.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 -- 172.21.15.176:0/1990028223 shutdown_connections 2024-03-13T14:55:22.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.271+0000 7f0207a00700 1 -- 172.21.15.176:0/1990028223 wait complete. 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.272+0000 7f0207a00700 1 Processor -- start 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.272+0000 7f0207a00700 1 -- start start 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f0207a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0200102fd0 0x7f020019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f0207a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f020019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f0207a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020019f3a0 con 0x7f0200103980 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f0207a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0200198850 con 0x7f0200102fd0 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f01ffe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f020019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f01ffe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f020019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37462/0 (socket says 172.21.15.176:37462) 2024-03-13T14:55:22.260 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f01ffe00700 1 -- 172.21.15.176:0/690505592 learned_addr learned my addr 172.21.15.176:0/690505592 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:22.261 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f0205000700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0200102fd0 0x7f020019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:22.261 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.273+0000 7f01ffe00700 1 -- 172.21.15.176:0/690505592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0200102fd0 msgr2=0x7f020019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f01ffe00700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0200102fd0 0x7f020019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f01ffe00700 1 -- 172.21.15.176:0/690505592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0095f0 con 0x7f0200103980 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f01ffe00700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f020019ed10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00bda0 tx=0x7f01ec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f01fd600700 1 -- 172.21.15.176:0/690505592 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01ec01d070 con 0x7f0200103980 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f01fd600700 1 -- 172.21.15.176:0/690505592 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01ec004fb0 con 0x7f0200103980 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f01fd600700 1 -- 172.21.15.176:0/690505592 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01ec00f460 con 0x7f0200103980 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0200198ad0 con 0x7f0200103980 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.274+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0200198f70 con 0x7f0200103980 2024-03-13T14:55:22.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.276+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020010b3d0 con 0x7f0200103980 2024-03-13T14:55:22.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.277+0000 7f01fd600700 1 -- 172.21.15.176:0/690505592 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f01ec022800 con 0x7f0200103980 2024-03-13T14:55:22.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.277+0000 7f01fd600700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f01e806e0d0 0x7f01e8070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:22.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.278+0000 7f01fd600700 1 -- 172.21.15.176:0/690505592 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f01ec08f5a0 con 0x7f0200103980 2024-03-13T14:55:22.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.278+0000 7f0205000700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f01e806e0d0 0x7f01e8070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:22.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.278+0000 7f0205000700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f01e806e0d0 0x7f01e8070590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005fd0 tx=0x7f01f400a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:22.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.282+0000 7f01fd600700 1 -- 172.21.15.176:0/690505592 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f01ec05c4b0 con 0x7f0200103980 2024-03-13T14:55:22.520 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.533+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f020002cd80 con 0x7f01e806e0d0 2024-03-13T14:55:22.521 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.535+0000 7f01fd600700 1 -- 172.21.15.176:0/690505592 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f020002cd80 con 0x7f01e806e0d0 2024-03-13T14:55:22.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.537+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f01e806e0d0 msgr2=0x7f01e8070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.537+0000 7f0207a00700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f01e806e0d0 0x7f01e8070590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01f4005fd0 tx=0x7f01f400a560 comp rx=0 tx=0).stop 2024-03-13T14:55:22.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.537+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 msgr2=0x7f020019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.537+0000 7f0207a00700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f020019ed10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00bda0 tx=0x7f01ec00be80 comp rx=0 tx=0).stop 2024-03-13T14:55:22.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.538+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 shutdown_connections 2024-03-13T14:55:22.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.538+0000 7f0207a00700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f01e806e0d0 0x7f01e8070590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.538+0000 7f0207a00700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0200102fd0 0x7f020019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.538+0000 7f0207a00700 1 --2- 172.21.15.176:0/690505592 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0200103980 0x7f020019ed10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.538+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 >> 172.21.15.176:0/690505592 conn(0x7f02000fe840 msgr2=0x7f0200100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:22.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.538+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 shutdown_connections 2024-03-13T14:55:22.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.538+0000 7f0207a00700 1 -- 172.21.15.176:0/690505592 wait complete. 2024-03-13T14:55:22.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.691+0000 7f7ee5800700 1 -- 172.21.15.176:0/444248680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 msgr2=0x7f7ee0106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.691+0000 7f7ee5800700 1 --2- 172.21.15.176:0/444248680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee0106da0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009910 tx=0x7f7ed0009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:22.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.692+0000 7f7ee5800700 1 -- 172.21.15.176:0/444248680 shutdown_connections 2024-03-13T14:55:22.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.692+0000 7f7ee5800700 1 --2- 172.21.15.176:0/444248680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee01009c0 0x7f7ee0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.692+0000 7f7ee5800700 1 --2- 172.21.15.176:0/444248680 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee0106da0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.692+0000 7f7ee5800700 1 -- 172.21.15.176:0/444248680 >> 172.21.15.176:0/444248680 conn(0x7f7ee00fc4b0 msgr2=0x7f7ee00fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:22.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.692+0000 7f7ee5800700 1 -- 172.21.15.176:0/444248680 shutdown_connections 2024-03-13T14:55:22.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.692+0000 7f7ee5800700 1 -- 172.21.15.176:0/444248680 wait complete. 2024-03-13T14:55:22.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.693+0000 7f7ee5800700 1 Processor -- start 2024-03-13T14:55:22.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.693+0000 7f7ee5800700 1 -- start start 2024-03-13T14:55:22.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.693+0000 7f7ee5800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee01009c0 0x7f7ee0073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:22.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.693+0000 7f7ee5800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee00735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:22.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.693+0000 7f7ee5800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0073ba0 con 0x7f7ee01069c0 2024-03-13T14:55:22.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.693+0000 7f7ee5800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0073d10 con 0x7f7ee01009c0 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edf400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee00735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edf400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee00735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37486/0 (socket says 172.21.15.176:37486) 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edf400700 1 -- 172.21.15.176:0/446873315 learned_addr learned my addr 172.21.15.176:0/446873315 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edfe00700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee01009c0 0x7f7ee0073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edf400700 1 -- 172.21.15.176:0/446873315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee01009c0 msgr2=0x7f7ee0073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edf400700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee01009c0 0x7f7ee0073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edf400700 1 -- 172.21.15.176:0/446873315 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed00095f0 con 0x7f7ee01069c0 2024-03-13T14:55:22.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.694+0000 7f7edf400700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee00735d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00b4b0 tx=0x7f7ecc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:22.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.695+0000 7f7edcc00700 1 -- 172.21.15.176:0/446873315 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ecc011800 con 0x7f7ee01069c0 2024-03-13T14:55:22.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.695+0000 7f7edcc00700 1 -- 172.21.15.176:0/446873315 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ecc011e20 con 0x7f7ee01069c0 2024-03-13T14:55:22.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.695+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee01a28b0 con 0x7f7ee01069c0 2024-03-13T14:55:22.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.695+0000 7f7edcc00700 1 -- 172.21.15.176:0/446873315 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ecc00f2e0 con 0x7f7ee01069c0 2024-03-13T14:55:22.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.695+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee01a2d20 con 0x7f7ee01069c0 2024-03-13T14:55:22.682 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.696+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee0108fc0 con 0x7f7ee01069c0 2024-03-13T14:55:22.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.697+0000 7f7edcc00700 1 -- 172.21.15.176:0/446873315 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f7ecc010380 con 0x7f7ee01069c0 2024-03-13T14:55:22.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.698+0000 7f7edcc00700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ec406e0d0 0x7f7ec4070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:22.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.698+0000 7f7edcc00700 1 -- 172.21.15.176:0/446873315 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7ecc08d990 con 0x7f7ee01069c0 2024-03-13T14:55:22.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.699+0000 7f7edfe00700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ec406e0d0 0x7f7ec4070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:22.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.699+0000 7f7edfe00700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ec406e0d0 0x7f7ec4070590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0006010 tx=0x7f7ed000b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:22.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.703+0000 7f7edcc00700 1 -- 172.21.15.176:0/446873315 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f7ecc056e80 con 0x7f7ee01069c0 2024-03-13T14:55:22.919 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.932+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ee00611d0 con 0x7f7ec406e0d0 2024-03-13T14:55:22.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.944+0000 7f7edcc00700 1 -- 172.21.15.176:0/446873315 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ee00611d0 con 0x7f7ec406e0d0 2024-03-13T14:55:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:55:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (3m) 75s ago 16m 14.3M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:55:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (17m) 75s ago 17m 8178k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:55:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (15m) 51s ago 15m 8144k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:55:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (17m) 75s ago 17m 6660k - 18.2.1 5be31c24972a 21ed0e096a05 2024-03-13T14:55:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (15m) 51s ago 15m 6660k - 18.2.1 5be31c24972a a4c8653353c2 2024-03-13T14:55:22.932 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (2m) 75s ago 16m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (9m) 75s ago 9m 87.2M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (9m) 75s ago 9m 16.3M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (9m) 51s ago 9m 141M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (9m) 51s ago 9m 16.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (6m) 75s ago 18m 476M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (5m) 51s ago 15m 448M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (92s) 75s ago 18m 28.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (57s) 51s ago 15m 19.6M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (4m) 75s ago 17m 7780k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (4m) 51s ago 15m 8150k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (14m) 75s ago 14m 233M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (13m) 75s ago 13m 246M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:55:22.933 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (13m) 75s ago 13m 220M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:55:22.934 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (12m) 51s ago 12m 305M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:55:22.934 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (12m) 51s ago 11m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:55:22.934 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (11m) 51s ago 11m 229M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:55:22.934 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (3m) 75s ago 16m 63.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:55:22.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.947+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ec406e0d0 msgr2=0x7f7ec4070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.947+0000 7f7ee5800700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ec406e0d0 0x7f7ec4070590 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0006010 tx=0x7f7ed000b540 comp rx=0 tx=0).stop 2024-03-13T14:55:22.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 msgr2=0x7f7ee00735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:22.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee00735d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00b4b0 tx=0x7f7ecc00b870 comp rx=0 tx=0).stop 2024-03-13T14:55:22.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 shutdown_connections 2024-03-13T14:55:22.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ec406e0d0 0x7f7ec4070590 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee01009c0 0x7f7ee0073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 --2- 172.21.15.176:0/446873315 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ee01069c0 0x7f7ee00735d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:22.935 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 >> 172.21.15.176:0/446873315 conn(0x7f7ee00fc4b0 msgr2=0x7f7ee00fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:22.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 shutdown_connections 2024-03-13T14:55:22.936 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:22.948+0000 7f7ee5800700 1 -- 172.21.15.176:0/446873315 wait complete. 2024-03-13T14:55:23.094 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.107+0000 7fbf2a800700 1 -- 172.21.15.176:0/1937233328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24101750 msgr2=0x7fbf241057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.094 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.107+0000 7fbf2a800700 1 --2- 172.21.15.176:0/1937233328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24101750 0x7fbf241057a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009910 tx=0x7fbf10009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:23.094 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.107+0000 7fbf2a800700 1 -- 172.21.15.176:0/1937233328 shutdown_connections 2024-03-13T14:55:23.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.107+0000 7fbf2a800700 1 --2- 172.21.15.176:0/1937233328 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24101750 0x7fbf241057a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.107+0000 7fbf2a800700 1 --2- 172.21.15.176:0/1937233328 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24100da0 0x7fbf24101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.107+0000 7fbf2a800700 1 -- 172.21.15.176:0/1937233328 >> 172.21.15.176:0/1937233328 conn(0x7fbf240fc630 msgr2=0x7fbf240fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:23.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.108+0000 7fbf2a800700 1 -- 172.21.15.176:0/1937233328 shutdown_connections 2024-03-13T14:55:23.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.108+0000 7fbf2a800700 1 -- 172.21.15.176:0/1937233328 wait complete. 2024-03-13T14:55:23.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.109+0000 7fbf2a800700 1 Processor -- start 2024-03-13T14:55:23.095 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.109+0000 7fbf2a800700 1 -- start start 2024-03-13T14:55:23.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.109+0000 7fbf2a800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24100da0 0x7fbf2419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:23.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.109+0000 7fbf2a800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101750 0x7fbf2419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:23.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.109+0000 7fbf2a800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2419d0d0 con 0x7fbf24100da0 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.109+0000 7fbf2a800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24196580 con 0x7fbf24101750 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf23400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24100da0 0x7fbf2419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf23400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24100da0 0x7fbf2419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37502/0 (socket says 172.21.15.176:37502) 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf23400700 1 -- 172.21.15.176:0/2989129678 learned_addr learned my addr 172.21.15.176:0/2989129678 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf23400700 1 -- 172.21.15.176:0/2989129678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101750 msgr2=0x7fbf2419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf22a00700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101750 0x7fbf2419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf23400700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101750 0x7fbf2419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf23400700 1 -- 172.21.15.176:0/2989129678 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100095f0 con 0x7fbf24100da0 2024-03-13T14:55:23.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.110+0000 7fbf23400700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24100da0 0x7fbf2419c500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d8d0 tx=0x7fbf1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:23.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.111+0000 7fbf1be00700 1 -- 172.21.15.176:0/2989129678 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf14009940 con 0x7fbf24100da0 2024-03-13T14:55:23.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.111+0000 7fbf1be00700 1 -- 172.21.15.176:0/2989129678 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf14010460 con 0x7fbf24100da0 2024-03-13T14:55:23.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.111+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf24196860 con 0x7fbf24100da0 2024-03-13T14:55:23.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.111+0000 7fbf1be00700 1 -- 172.21.15.176:0/2989129678 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf1400f5f0 con 0x7fbf24100da0 2024-03-13T14:55:23.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.111+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf24196db0 con 0x7fbf24100da0 2024-03-13T14:55:23.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.112+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf241091a0 con 0x7fbf24100da0 2024-03-13T14:55:23.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.113+0000 7fbf1be00700 1 -- 172.21.15.176:0/2989129678 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7fbf140105d0 con 0x7fbf24100da0 2024-03-13T14:55:23.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.114+0000 7fbf1be00700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbf0806e0e0 0x7fbf080705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:23.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.114+0000 7fbf1be00700 1 -- 172.21.15.176:0/2989129678 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbf1408df20 con 0x7fbf24100da0 2024-03-13T14:55:23.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.114+0000 7fbf22a00700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbf0806e0e0 0x7fbf080705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:23.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.115+0000 7fbf22a00700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbf0806e0e0 0x7fbf080705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbf100059d0 tx=0x7fbf10005960 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:23.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.119+0000 7fbf1be00700 1 -- 172.21.15.176:0/2989129678 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fbf1405ace0 con 0x7fbf24100da0 2024-03-13T14:55:23.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.466+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbf241970f0 con 0x7fbf24100da0 2024-03-13T14:55:23.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.467+0000 7fbf1be00700 1 -- 172.21.15.176:0/2989129678 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fbf1405a870 con 0x7fbf24100da0 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:55:23.455 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:55:23.456 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:23.456 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:55:23.456 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-13T14:55:23.456 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T14:55:23.456 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:55:23.456 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:55:23.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbf0806e0e0 msgr2=0x7fbf080705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbf0806e0e0 0x7fbf080705a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbf100059d0 tx=0x7fbf10005960 comp rx=0 tx=0).stop 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24100da0 msgr2=0x7fbf2419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24100da0 0x7fbf2419c500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400d8d0 tx=0x7fbf1400dc90 comp rx=0 tx=0).stop 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 shutdown_connections 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbf0806e0e0 0x7fbf080705a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbf24100da0 0x7fbf2419c500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 --2- 172.21.15.176:0/2989129678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf24101750 0x7fbf2419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 >> 172.21.15.176:0/2989129678 conn(0x7fbf240fc630 msgr2=0x7fbf240fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:23.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.470+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 shutdown_connections 2024-03-13T14:55:23.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.471+0000 7fbf2a800700 1 -- 172.21.15.176:0/2989129678 wait complete. 2024-03-13T14:55:23.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 -- 172.21.15.176:0/4282375276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068410 msgr2=0x7f77500687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 --2- 172.21.15.176:0/4282375276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068410 0x7f77500687f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7744009910 tx=0x7f7744009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:23.613 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:23 smithi176 ceph-mon[112742]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:23.613 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:23 smithi176 ceph-mon[112742]: pgmap v44: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:23.613 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:23 smithi176 ceph-mon[112742]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:23.613 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:23.614 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:55:23.614 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:23.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 -- 172.21.15.176:0/4282375276 shutdown_connections 2024-03-13T14:55:23.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 --2- 172.21.15.176:0/4282375276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7750068d30 0x7f775010d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 --2- 172.21.15.176:0/4282375276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068410 0x7f77500687f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 -- 172.21.15.176:0/4282375276 >> 172.21.15.176:0/4282375276 conn(0x7f77500755a0 msgr2=0x7f77500759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:23.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 -- 172.21.15.176:0/4282375276 shutdown_connections 2024-03-13T14:55:23.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.626+0000 7f7757400700 1 -- 172.21.15.176:0/4282375276 wait complete. 2024-03-13T14:55:23.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.627+0000 7f7757400700 1 Processor -- start 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.627+0000 7f7757400700 1 -- start start 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.628+0000 7f7757400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7750068410 0x7f7750101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.628+0000 7f7757400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068d30 0x7f77501023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.628+0000 7f7757400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750106040 con 0x7f7750068d30 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.628+0000 7f7757400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750102930 con 0x7f7750068410 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.628+0000 7f774fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068d30 0x7f77501023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.628+0000 7f774fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068d30 0x7f77501023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37520/0 (socket says 172.21.15.176:37520) 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.628+0000 7f774fe00700 1 -- 172.21.15.176:0/2490228600 learned_addr learned my addr 172.21.15.176:0/2490228600 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f7754a00700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7750068410 0x7f7750101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f774fe00700 1 -- 172.21.15.176:0/2490228600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7750068410 msgr2=0x7f7750101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f774fe00700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7750068410 0x7f7750101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f774fe00700 1 -- 172.21.15.176:0/2490228600 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440095f0 con 0x7f7750068d30 2024-03-13T14:55:23.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f7754a00700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7750068410 0x7f7750101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:55:23.617 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f774fe00700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068d30 0x7f77501023f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c8a0 tx=0x7f773c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:23.617 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f774d600700 1 -- 172.21.15.176:0/2490228600 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f773c00cea0 con 0x7f7750068d30 2024-03-13T14:55:23.617 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750102c10 con 0x7f7750068d30 2024-03-13T14:55:23.617 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f774d600700 1 -- 172.21.15.176:0/2490228600 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f773c004830 con 0x7f7750068d30 2024-03-13T14:55:23.618 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.629+0000 7f774d600700 1 -- 172.21.15.176:0/2490228600 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f773c0055b0 con 0x7f7750068d30 2024-03-13T14:55:23.618 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.630+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77501a7110 con 0x7f7750068d30 2024-03-13T14:55:23.618 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.631+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775010a9b0 con 0x7f7750068d30 2024-03-13T14:55:23.619 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.632+0000 7f774d600700 1 -- 172.21.15.176:0/2490228600 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f773c0049a0 con 0x7f7750068d30 2024-03-13T14:55:23.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.634+0000 7f774d600700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f773806e0d0 0x7f7738070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:23.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.634+0000 7f774d600700 1 -- 172.21.15.176:0/2490228600 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f773c08dff0 con 0x7f7750068d30 2024-03-13T14:55:23.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.634+0000 7f7754a00700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f773806e0d0 0x7f7738070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:23.621 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.635+0000 7f7754a00700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f773806e0d0 0x7f7738070590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7744006010 tx=0x7f774400b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:23.623 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.637+0000 7f774d600700 1 -- 172.21.15.176:0/2490228600 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f773c057360 con 0x7f7750068d30 2024-03-13T14:55:23.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:23 smithi196 ceph-mon[86193]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:23 smithi196 ceph-mon[86193]: pgmap v44: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:23 smithi196 ceph-mon[86193]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi176", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:55:23.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:23.898 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.911+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f775004ea90 con 0x7f7750068d30 2024-03-13T14:55:23.899 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.913+0000 7f774d600700 1 -- 172.21.15.176:0/2490228600 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f773c01d070 con 0x7f7750068d30 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:55:23.900 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:55:23.901 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:55:23.902 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:23.903 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:23.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f773806e0d0 msgr2=0x7f7738070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f773806e0d0 0x7f7738070590 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7744006010 tx=0x7f774400b540 comp rx=0 tx=0).stop 2024-03-13T14:55:23.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068d30 msgr2=0x7f77501023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:23.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068d30 0x7f77501023f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f773c00c8a0 tx=0x7f773c00cc60 comp rx=0 tx=0).stop 2024-03-13T14:55:23.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 shutdown_connections 2024-03-13T14:55:23.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f773806e0d0 0x7f7738070590 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.906 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7750068410 0x7f7750101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 --2- 172.21.15.176:0/2490228600 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7750068d30 0x7f77501023f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:23.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 >> 172.21.15.176:0/2490228600 conn(0x7f77500755a0 msgr2=0x7f77500fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:23.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 shutdown_connections 2024-03-13T14:55:23.907 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:23.917+0000 7f7757400700 1 -- 172.21.15.176:0/2490228600 wait complete. 2024-03-13T14:55:23.907 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:55:24.057 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.070+0000 7f6053e00700 1 -- 172.21.15.176:0/231909653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 msgr2=0x7f604c1033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.057 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.070+0000 7f6053e00700 1 --2- 172.21.15.176:0/231909653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c1033b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6040009910 tx=0x7f6040009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:24.057 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.070+0000 7f6053e00700 1 -- 172.21.15.176:0/231909653 shutdown_connections 2024-03-13T14:55:24.058 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.070+0000 7f6053e00700 1 --2- 172.21.15.176:0/231909653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604c103980 0x7f604c1079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.058 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.070+0000 7f6053e00700 1 --2- 172.21.15.176:0/231909653 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c1033b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.058 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.070+0000 7f6053e00700 1 -- 172.21.15.176:0/231909653 >> 172.21.15.176:0/231909653 conn(0x7f604c0fe840 msgr2=0x7f604c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:24.058 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.071+0000 7f6053e00700 1 -- 172.21.15.176:0/231909653 shutdown_connections 2024-03-13T14:55:24.058 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.071+0000 7f6053e00700 1 -- 172.21.15.176:0/231909653 wait complete. 2024-03-13T14:55:24.059 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6053e00700 1 Processor -- start 2024-03-13T14:55:24.059 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6053e00700 1 -- start start 2024-03-13T14:55:24.059 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6053e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:24.059 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6053e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604c103980 0x7f604c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:24.059 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6053e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c19f3b0 con 0x7f604c102fd0 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6053e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c198860 con 0x7f604c103980 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6051400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6051400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37538/0 (socket says 172.21.15.176:37538) 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.072+0000 7f6051400700 1 -- 172.21.15.176:0/1114866421 learned_addr learned my addr 172.21.15.176:0/1114866421 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f6050a00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604c103980 0x7f604c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f6051400700 1 -- 172.21.15.176:0/1114866421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604c103980 msgr2=0x7f604c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f6051400700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604c103980 0x7f604c19ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f6051400700 1 -- 172.21.15.176:0/1114866421 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400095f0 con 0x7f604c102fd0 2024-03-13T14:55:24.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f6051400700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c19e7e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f604000b5c0 tx=0x7f604000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:24.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f604a000700 1 -- 172.21.15.176:0/1114866421 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f604001d070 con 0x7f604c102fd0 2024-03-13T14:55:24.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f604a000700 1 -- 172.21.15.176:0/1114866421 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6040005e00 con 0x7f604c102fd0 2024-03-13T14:55:24.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f604a000700 1 -- 172.21.15.176:0/1114866421 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6040017400 con 0x7f604c102fd0 2024-03-13T14:55:24.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.073+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604c198ae0 con 0x7f604c102fd0 2024-03-13T14:55:24.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.074+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604c198fd0 con 0x7f604c102fd0 2024-03-13T14:55:24.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.075+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f604c10b370 con 0x7f604c102fd0 2024-03-13T14:55:24.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.076+0000 7f604a000700 1 -- 172.21.15.176:0/1114866421 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f604000f460 con 0x7f604c102fd0 2024-03-13T14:55:24.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.076+0000 7f604a000700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f603406e0d0 0x7f6034070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:24.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.077+0000 7f604a000700 1 -- 172.21.15.176:0/1114866421 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f604008f3b0 con 0x7f604c102fd0 2024-03-13T14:55:24.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.077+0000 7f6050a00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f603406e0d0 0x7f6034070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:24.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.077+0000 7f6050a00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f603406e0d0 0x7f6034070590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6038005950 tx=0x7f60380058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:24.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.081+0000 7f604a000700 1 -- 172.21.15.176:0/1114866421 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f604005c210 con 0x7f604c102fd0 2024-03-13T14:55:24.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.339+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f604c199240 con 0x7f603406e0d0 2024-03-13T14:55:24.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.342+0000 7f604a000700 1 -- 172.21.15.176:0/1114866421 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f604c199240 con 0x7f603406e0d0 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "4/23 daemons upgraded", 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading crash daemons", 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:55:24.329 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f603406e0d0 msgr2=0x7f6034070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f603406e0d0 0x7f6034070590 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6038005950 tx=0x7f60380058e0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 msgr2=0x7f604c19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c19e7e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f604000b5c0 tx=0x7f604000be20 comp rx=0 tx=0).stop 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 shutdown_connections 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f603406e0d0 0x7f6034070590 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f604c102fd0 0x7f604c19e7e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 --2- 172.21.15.176:0/1114866421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f604c103980 0x7f604c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.345+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 >> 172.21.15.176:0/1114866421 conn(0x7f604c0fe840 msgr2=0x7f604c0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.346+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 shutdown_connections 2024-03-13T14:55:24.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.346+0000 7f6053e00700 1 -- 172.21.15.176:0/1114866421 wait complete. 2024-03-13T14:55:24.478 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:24 smithi176 ceph-mon[112742]: Upgrade: Updating crash.smithi176 (1/2) 2024-03-13T14:55:24.478 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:24 smithi176 ceph-mon[112742]: Deploying daemon crash.smithi176 on smithi176 2024-03-13T14:55:24.478 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:24 smithi176 ceph-mon[112742]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:24.478 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:24 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2989129678' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:24.478 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:24 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2490228600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:55:24.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 -- 172.21.15.176:0/2318511517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 msgr2=0x7f98a010d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 --2- 172.21.15.176:0/2318511517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a010d080 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f988c009910 tx=0x7f988c009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:24.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 -- 172.21.15.176:0/2318511517 shutdown_connections 2024-03-13T14:55:24.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 --2- 172.21.15.176:0/2318511517 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a010d080 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 --2- 172.21.15.176:0/2318511517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98a0068410 0x7f98a00687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 -- 172.21.15.176:0/2318511517 >> 172.21.15.176:0/2318511517 conn(0x7f98a0075330 msgr2=0x7f98a0075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:24.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 -- 172.21.15.176:0/2318511517 shutdown_connections 2024-03-13T14:55:24.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.502+0000 7f98a4e00700 1 -- 172.21.15.176:0/2318511517 wait complete. 2024-03-13T14:55:24.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.503+0000 7f98a4e00700 1 Processor -- start 2024-03-13T14:55:24.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.503+0000 7f98a4e00700 1 -- start start 2024-03-13T14:55:24.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f98a4e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98a0068410 0x7f98a019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:24.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f98a4e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a019eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:24.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f98a4e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a019f0f0 con 0x7f98a0068d30 2024-03-13T14:55:24.491 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f98a4e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0198630 con 0x7f98a0068410 2024-03-13T14:55:24.491 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f989ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a019eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:24.491 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f989ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a019eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37556/0 (socket says 172.21.15.176:37556) 2024-03-13T14:55:24.491 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f989ea00700 1 -- 172.21.15.176:0/3229862044 learned_addr learned my addr 172.21.15.176:0/3229862044 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f989f400700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98a0068410 0x7f98a019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f989ea00700 1 -- 172.21.15.176:0/3229862044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98a0068410 msgr2=0x7f98a019e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f989ea00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98a0068410 0x7f98a019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.504+0000 7f989ea00700 1 -- 172.21.15.176:0/3229862044 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c0095f0 con 0x7f98a0068d30 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.505+0000 7f989ea00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a019eaf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f988c006010 tx=0x7f988c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.505+0000 7f9897e00700 1 -- 172.21.15.176:0/3229862044 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f988c01d070 con 0x7f98a0068d30 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.505+0000 7f9897e00700 1 -- 172.21.15.176:0/3229862044 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f988c004fb0 con 0x7f98a0068d30 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.505+0000 7f9897e00700 1 -- 172.21.15.176:0/3229862044 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f988c022ea0 con 0x7f98a0068d30 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.505+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a0198910 con 0x7f98a0068d30 2024-03-13T14:55:24.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.505+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a0198e60 con 0x7f98a0068d30 2024-03-13T14:55:24.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.506+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a010a7f0 con 0x7f98a0068d30 2024-03-13T14:55:24.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.507+0000 7f9897e00700 1 -- 172.21.15.176:0/3229862044 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 38) v1 ==== 91945+0+0 (secure 0 0 0) 0x7f988c022470 con 0x7f98a0068d30 2024-03-13T14:55:24.494 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.508+0000 7f9897e00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f988406e0e0 0x7f98840705a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:24.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.508+0000 7f9897e00700 1 -- 172.21.15.176:0/3229862044 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f988c08f410 con 0x7f98a0068d30 2024-03-13T14:55:24.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.508+0000 7f989f400700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f988406e0e0 0x7f98840705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:24.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.509+0000 7f989f400700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f988406e0e0 0x7f98840705a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98a00768b0 tx=0x7f9890005d70 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:24.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.513+0000 7f9897e00700 1 -- 172.21.15.176:0/3229862044 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f988c05c320 con 0x7f98a0068d30 2024-03-13T14:55:24.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:24 smithi196 ceph-mon[86193]: Upgrade: Updating crash.smithi176 (1/2) 2024-03-13T14:55:24.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:24 smithi196 ceph-mon[86193]: Deploying daemon crash.smithi176 on smithi176 2024-03-13T14:55:24.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:24 smithi196 ceph-mon[86193]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:24.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:24 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2989129678' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:24.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:24 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2490228600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:55:24.822 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:55:24.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.833+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f98a004f2e0 con 0x7f98a0068d30 2024-03-13T14:55:24.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.834+0000 7f9897e00700 1 -- 172.21.15.176:0/3229862044 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f988c027020 con 0x7f98a0068d30 2024-03-13T14:55:24.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.837+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f988406e0e0 msgr2=0x7f98840705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.837+0000 7f98a4e00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f988406e0e0 0x7f98840705a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f98a00768b0 tx=0x7f9890005d70 comp rx=0 tx=0).stop 2024-03-13T14:55:24.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 msgr2=0x7f98a019eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a019eaf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f988c006010 tx=0x7f988c00be20 comp rx=0 tx=0).stop 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 shutdown_connections 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f988406e0e0 0x7f98840705a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98a0068410 0x7f98a019e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 --2- 172.21.15.176:0/3229862044 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98a0068d30 0x7f98a019eaf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 >> 172.21.15.176:0/3229862044 conn(0x7f98a0075330 msgr2=0x7f98a00fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 shutdown_connections 2024-03-13T14:55:24.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:24.838+0000 7f98a4e00700 1 -- 172.21.15.176:0/3229862044 wait complete. 2024-03-13T14:55:25.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:25 smithi176 ceph-mon[112742]: pgmap v45: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:25.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:25 smithi176 ceph-mon[112742]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:25.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:25 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3229862044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:55:25.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:25 smithi196 ceph-mon[86193]: pgmap v45: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:25 smithi196 ceph-mon[86193]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:25.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:25 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3229862044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:55:26.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:26 smithi176 ceph-mon[112742]: pgmap v46: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:27.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:26 smithi196 ceph-mon[86193]: pgmap v46: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:29.005 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:29.005 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:29.005 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:28 smithi196 ceph-mon[86193]: pgmap v47: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:29.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:29.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:29.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:28 smithi176 ceph-mon[112742]: pgmap v47: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:31.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:30 smithi196 ceph-mon[86193]: Upgrade: Updating crash.smithi196 (2/2) 2024-03-13T14:55:31.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:31.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:55:31.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:31.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:30 smithi196 ceph-mon[86193]: Deploying daemon crash.smithi196 on smithi196 2024-03-13T14:55:31.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:30 smithi196 ceph-mon[86193]: pgmap v48: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:31.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:31.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:30 smithi176 ceph-mon[112742]: Upgrade: Updating crash.smithi196 (2/2) 2024-03-13T14:55:31.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:31.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi196", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-03-13T14:55:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:30 smithi176 ceph-mon[112742]: Deploying daemon crash.smithi196 on smithi196 2024-03-13T14:55:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:30 smithi176 ceph-mon[112742]: pgmap v48: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:31.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:33.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:33 smithi176 ceph-mon[112742]: pgmap v49: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:33.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:33 smithi176 ceph-mon[112742]: mgrmap e39: smithi176.tzfdwd(active, since 92s), standbys: smithi196.vdktbk 2024-03-13T14:55:33.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:33 smithi196 ceph-mon[86193]: pgmap v49: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:33.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:33 smithi196 ceph-mon[86193]: mgrmap e39: smithi176.tzfdwd(active, since 92s), standbys: smithi196.vdktbk 2024-03-13T14:55:35.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:35 smithi176 ceph-mon[112742]: pgmap v50: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-13T14:55:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:35 smithi196 ceph-mon[86193]: pgmap v50: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-13T14:55:36.684 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:36.684 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:36.684 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:55:36.711 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:36.711 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:36.711 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:55:37.790 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:37 smithi196 ceph-mon[86193]: pgmap v51: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:37.817 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:37 smithi176 ceph-mon[112742]: pgmap v51: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:39.791 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:39 smithi176 ceph-mon[112742]: pgmap v52: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:39.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:39 smithi196 ceph-mon[86193]: pgmap v52: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:41.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:41 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:41.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:41 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:41.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:41 smithi176 ceph-mon[112742]: pgmap v53: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-13T14:55:41.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:41 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:41.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:41 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:41.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:41 smithi196 ceph-mon[86193]: pgmap v53: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-03-13T14:55:43.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:43 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:43.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:43 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:43.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:43 smithi176 ceph-mon[112742]: pgmap v54: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:43.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:43 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:43.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:43 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:43.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:43 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:43.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:43 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:43.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:43 smithi196 ceph-mon[86193]: pgmap v54: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-13T14:55:43.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:43 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:43.179 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:43 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:45.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:45 smithi176 ceph-mon[112742]: pgmap v55: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:45.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:45.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:45 smithi196 ceph-mon[86193]: pgmap v55: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:45.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:45.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:46.586 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:46.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:55:47.476 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:47 smithi176 ceph-mon[112742]: pgmap v56: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:47.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:47 smithi196 ceph-mon[86193]: pgmap v56: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:48 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:48 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:48 smithi176 ceph-mon[112742]: pgmap v57: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:48 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:48 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:48 smithi196 ceph-mon[86193]: pgmap v57: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:51.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:51.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:55:51.571 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: pgmap v58: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]: dispatch 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]': finished 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi196"}]: dispatch 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi196"}]': finished 2024-03-13T14:55:51.572 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: pgmap v58: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:51.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]: dispatch 2024-03-13T14:55:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi176"}]': finished 2024-03-13T14:55:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi196"}]: dispatch 2024-03-13T14:55:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi196"}]': finished 2024-03-13T14:55:51.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-13T14:55:52.622 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:52 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all crash 2024-03-13T14:55:52.622 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:52 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-13T14:55:52.622 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:52 smithi176 ceph-mon[112742]: Upgrade: osd.0 is safe to restart 2024-03-13T14:55:52.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:52 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all crash 2024-03-13T14:55:52.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:52 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-03-13T14:55:52.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:52 smithi196 ceph-mon[86193]: Upgrade: osd.0 is safe to restart 2024-03-13T14:55:53.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:53 smithi176 ceph-mon[112742]: pgmap v59: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:53.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:53.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-13T14:55:53.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:53.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:53 smithi196 ceph-mon[86193]: pgmap v59: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:53.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:55:53.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-13T14:55:53.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:55:54.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:54 smithi176 ceph-mon[112742]: Upgrade: Updating osd.0 2024-03-13T14:55:54.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:54 smithi176 ceph-mon[112742]: Deploying daemon osd.0 on smithi176 2024-03-13T14:55:54.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:54 smithi196 ceph-mon[86193]: Upgrade: Updating osd.0 2024-03-13T14:55:54.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:54 smithi196 ceph-mon[86193]: Deploying daemon osd.0 on smithi176 2024-03-13T14:55:54.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.000+0000 7f022ca00700 1 -- 172.21.15.176:0/509169004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 msgr2=0x7f022810ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:54.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.000+0000 7f022ca00700 1 --2- 172.21.15.176:0/509169004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f022810ce40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0214009910 tx=0x7f0214009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:54.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.000+0000 7f022ca00700 1 -- 172.21.15.176:0/509169004 shutdown_connections 2024-03-13T14:55:54.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.000+0000 7f022ca00700 1 --2- 172.21.15.176:0/509169004 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f022810ce40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:54.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.000+0000 7f022ca00700 1 --2- 172.21.15.176:0/509169004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02280731c0 0x7f02280735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:54.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.000+0000 7f022ca00700 1 -- 172.21.15.176:0/509169004 >> 172.21.15.176:0/509169004 conn(0x7f02280fc630 msgr2=0x7f02280fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:54.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.001+0000 7f022ca00700 1 -- 172.21.15.176:0/509169004 shutdown_connections 2024-03-13T14:55:54.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.001+0000 7f022ca00700 1 -- 172.21.15.176:0/509169004 wait complete. 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.002+0000 7f022ca00700 1 Processor -- start 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.002+0000 7f022ca00700 1 -- start start 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.002+0000 7f022ca00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02280731c0 0x7f0228198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.002+0000 7f022ca00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f0228198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.002+0000 7f022ca00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0228199530 con 0x7f0228073ae0 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.002+0000 7f022ca00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022819d340 con 0x7f02280731c0 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f0228198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f0228198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42516/0 (socket says 172.21.15.176:42516) 2024-03-13T14:55:54.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225400700 1 -- 172.21.15.176:0/3213226049 learned_addr learned my addr 172.21.15.176:0/3213226049 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:54.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225e00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02280731c0 0x7f0228198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:54.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225400700 1 -- 172.21.15.176:0/3213226049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02280731c0 msgr2=0x7f0228198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225400700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02280731c0 0x7f0228198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225400700 1 -- 172.21.15.176:0/3213226049 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140095f0 con 0x7f0228073ae0 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.003+0000 7f0225400700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f0228198ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02140092e0 tx=0x7f021400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.004+0000 7f021ea00700 1 -- 172.21.15.176:0/3213226049 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f021401d070 con 0x7f0228073ae0 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.004+0000 7f021ea00700 1 -- 172.21.15.176:0/3213226049 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f021400f460 con 0x7f0228073ae0 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.004+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022819d5c0 con 0x7f0228073ae0 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.004+0000 7f021ea00700 1 -- 172.21.15.176:0/3213226049 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0214017630 con 0x7f0228073ae0 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.004+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022819dab0 con 0x7f0228073ae0 2024-03-13T14:55:54.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.005+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022810a540 con 0x7f0228073ae0 2024-03-13T14:55:54.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.008+0000 7f021ea00700 1 -- 172.21.15.176:0/3213226049 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f021400f5d0 con 0x7f0228073ae0 2024-03-13T14:55:54.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.009+0000 7f021ea00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f020c06dfa0 0x7f020c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:54.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.009+0000 7f021ea00700 1 -- 172.21.15.176:0/3213226049 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f021408f4b0 con 0x7f0228073ae0 2024-03-13T14:55:54.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.009+0000 7f0225e00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f020c06dfa0 0x7f020c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:54.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.010+0000 7f0225e00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f020c06dfa0 0x7f020c070460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0218005fd0 tx=0x7f021800a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:54.999 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.013+0000 7f021ea00700 1 -- 172.21.15.176:0/3213226049 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f021405c5d0 con 0x7f0228073ae0 2024-03-13T14:55:55.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.266+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f022819dd90 con 0x7f020c06dfa0 2024-03-13T14:55:55.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.268+0000 7f021ea00700 1 -- 172.21.15.176:0/3213226049 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f022819dd90 con 0x7f020c06dfa0 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f020c06dfa0 msgr2=0x7f020c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f020c06dfa0 0x7f020c070460 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0218005fd0 tx=0x7f021800a560 comp rx=0 tx=0).stop 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 msgr2=0x7f0228198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f0228198ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f02140092e0 tx=0x7f021400be00 comp rx=0 tx=0).stop 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 shutdown_connections 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f020c06dfa0 0x7f020c070460 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f02280731c0 0x7f0228198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 --2- 172.21.15.176:0/3213226049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0228073ae0 0x7f0228198ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.271+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 >> 172.21.15.176:0/3213226049 conn(0x7f02280fc630 msgr2=0x7f0228107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:55.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.272+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 shutdown_connections 2024-03-13T14:55:55.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.272+0000 7f022ca00700 1 -- 172.21.15.176:0/3213226049 wait complete. 2024-03-13T14:55:55.274 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:55:55.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.433+0000 7f2d9e600700 1 -- 172.21.15.176:0/3330405721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 msgr2=0x7f2d98068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.433+0000 7f2d9e600700 1 --2- 172.21.15.176:0/3330405721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d98068a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009910 tx=0x7f2d88009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:55.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.434+0000 7f2d9e600700 1 -- 172.21.15.176:0/3330405721 shutdown_connections 2024-03-13T14:55:55.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.434+0000 7f2d9e600700 1 --2- 172.21.15.176:0/3330405721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d98069020 0x7f2d98105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.434+0000 7f2d9e600700 1 --2- 172.21.15.176:0/3330405721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d98068a50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.434+0000 7f2d9e600700 1 -- 172.21.15.176:0/3330405721 >> 172.21.15.176:0/3330405721 conn(0x7f2d980755a0 msgr2=0x7f2d980759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:55.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.434+0000 7f2d9e600700 1 -- 172.21.15.176:0/3330405721 shutdown_connections 2024-03-13T14:55:55.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.434+0000 7f2d9e600700 1 -- 172.21.15.176:0/3330405721 wait complete. 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.435+0000 7f2d9e600700 1 Processor -- start 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.435+0000 7f2d9e600700 1 -- start start 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.435+0000 7f2d9e600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d9819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.435+0000 7f2d9e600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d98069020 0x7f2d9819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.435+0000 7f2d9e600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9819f2e0 con 0x7f2d98068670 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.435+0000 7f2d9e600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d98198790 con 0x7f2d98069020 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.436+0000 7f2d97400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d9819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.436+0000 7f2d97400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d9819e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42528/0 (socket says 172.21.15.176:42528) 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.436+0000 7f2d97400700 1 -- 172.21.15.176:0/1266094022 learned_addr learned my addr 172.21.15.176:0/1266094022 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.436+0000 7f2d96a00700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d98069020 0x7f2d9819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.436+0000 7f2d97400700 1 -- 172.21.15.176:0/1266094022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d98069020 msgr2=0x7f2d9819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.436+0000 7f2d97400700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d98069020 0x7f2d9819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.436+0000 7f2d97400700 1 -- 172.21.15.176:0/1266094022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880095f0 con 0x7f2d98068670 2024-03-13T14:55:55.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.437+0000 7f2d97400700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d9819e710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005e00 tx=0x7f2d8800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:55.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.437+0000 7f2d8fe00700 1 -- 172.21.15.176:0/1266094022 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8801d070 con 0x7f2d98068670 2024-03-13T14:55:55.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.437+0000 7f2d8fe00700 1 -- 172.21.15.176:0/1266094022 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d88004fb0 con 0x7f2d98068670 2024-03-13T14:55:55.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.437+0000 7f2d8fe00700 1 -- 172.21.15.176:0/1266094022 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8800f690 con 0x7f2d98068670 2024-03-13T14:55:55.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.437+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d98198a10 con 0x7f2d98068670 2024-03-13T14:55:55.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.437+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d98198f00 con 0x7f2d98068670 2024-03-13T14:55:55.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.439+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9804ea90 con 0x7f2d98068670 2024-03-13T14:55:55.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.439+0000 7f2d8fe00700 1 -- 172.21.15.176:0/1266094022 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2d88022470 con 0x7f2d98068670 2024-03-13T14:55:55.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.441+0000 7f2d8fe00700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2d7c06dfa0 0x7f2d7c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:55.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.441+0000 7f2d8fe00700 1 -- 172.21.15.176:0/1266094022 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2d8808f6d0 con 0x7f2d98068670 2024-03-13T14:55:55.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.441+0000 7f2d96a00700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2d7c06dfa0 0x7f2d7c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:55.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.442+0000 7f2d96a00700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2d7c06dfa0 0x7f2d7c070460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d98199c00 tx=0x7f2d84005cf0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:55.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.446+0000 7f2d8fe00700 1 -- 172.21.15.176:0/1266094022 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f2d8805c630 con 0x7f2d98068670 2024-03-13T14:55:55.548 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:55 smithi176 ceph-mon[112742]: pgmap v60: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:55.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:55 smithi196 ceph-mon[86193]: pgmap v60: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:55.702 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.715+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d98199320 con 0x7f2d7c06dfa0 2024-03-13T14:55:55.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.717+0000 7f2d8fe00700 1 -- 172.21.15.176:0/1266094022 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f2d98199320 con 0x7f2d7c06dfa0 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2d7c06dfa0 msgr2=0x7f2d7c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2d7c06dfa0 0x7f2d7c070460 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d98199c00 tx=0x7f2d84005cf0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 msgr2=0x7f2d9819e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d9819e710 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005e00 tx=0x7f2d8800be20 comp rx=0 tx=0).stop 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 shutdown_connections 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2d7c06dfa0 0x7f2d7c070460 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2d98068670 0x7f2d9819e710 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 --2- 172.21.15.176:0/1266094022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d98069020 0x7f2d9819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 >> 172.21.15.176:0/1266094022 conn(0x7f2d980755a0 msgr2=0x7f2d98102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 shutdown_connections 2024-03-13T14:55:55.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.720+0000 7f2d9e600700 1 -- 172.21.15.176:0/1266094022 wait complete. 2024-03-13T14:55:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.882+0000 7fe679c00700 1 -- 172.21.15.176:0/2254673485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 msgr2=0x7fe6741057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.882+0000 7fe679c00700 1 --2- 172.21.15.176:0/2254673485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6741057a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe660009910 tx=0x7fe660009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.882+0000 7fe679c00700 1 -- 172.21.15.176:0/2254673485 shutdown_connections 2024-03-13T14:55:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.882+0000 7fe679c00700 1 --2- 172.21.15.176:0/2254673485 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6741057a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.882+0000 7fe679c00700 1 --2- 172.21.15.176:0/2254673485 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674100da0 0x7fe674101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.882+0000 7fe679c00700 1 -- 172.21.15.176:0/2254673485 >> 172.21.15.176:0/2254673485 conn(0x7fe6740fc630 msgr2=0x7fe6740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.883+0000 7fe679c00700 1 -- 172.21.15.176:0/2254673485 shutdown_connections 2024-03-13T14:55:55.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.883+0000 7fe679c00700 1 -- 172.21.15.176:0/2254673485 wait complete. 2024-03-13T14:55:55.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe679c00700 1 Processor -- start 2024-03-13T14:55:55.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe679c00700 1 -- start start 2024-03-13T14:55:55.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe679c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674100da0 0x7fe674073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:55.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe679c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6740735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:55.871 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe679c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674073b10 con 0x7fe674101750 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe679c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe674073c50 con 0x7fe674100da0 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe672000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6740735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe672000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6740735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42552/0 (socket says 172.21.15.176:42552) 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.884+0000 7fe672000700 1 -- 172.21.15.176:0/4033853466 learned_addr learned my addr 172.21.15.176:0/4033853466 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe672a00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674100da0 0x7fe674073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe672000700 1 -- 172.21.15.176:0/4033853466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674100da0 msgr2=0x7fe674073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe672000700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674100da0 0x7fe674073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe672000700 1 -- 172.21.15.176:0/4033853466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600095f0 con 0x7fe674101750 2024-03-13T14:55:55.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe672000700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6740735d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe66000bda0 tx=0x7fe66000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:55.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe66b400700 1 -- 172.21.15.176:0/4033853466 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66001d070 con 0x7fe674101750 2024-03-13T14:55:55.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe66b400700 1 -- 172.21.15.176:0/4033853466 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe660004fb0 con 0x7fe674101750 2024-03-13T14:55:55.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.885+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe674073ed0 con 0x7fe674101750 2024-03-13T14:55:55.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.886+0000 7fe66b400700 1 -- 172.21.15.176:0/4033853466 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe66000f460 con 0x7fe674101750 2024-03-13T14:55:55.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.886+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6741a2c30 con 0x7fe674101750 2024-03-13T14:55:55.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.887+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6741091a0 con 0x7fe674101750 2024-03-13T14:55:55.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.888+0000 7fe66b400700 1 -- 172.21.15.176:0/4033853466 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe660022800 con 0x7fe674101750 2024-03-13T14:55:55.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.889+0000 7fe66b400700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806dfa0 0x7fe658070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:55.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.890+0000 7fe66b400700 1 -- 172.21.15.176:0/4033853466 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe66008f160 con 0x7fe674101750 2024-03-13T14:55:55.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.890+0000 7fe672a00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806dfa0 0x7fe658070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:55.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.890+0000 7fe672a00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806dfa0 0x7fe658070460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe664005fd0 tx=0x7fe66400a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:55.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:55.894+0000 7fe66b400700 1 -- 172.21.15.176:0/4033853466 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fe660058880 con 0x7fe674101750 2024-03-13T14:55:56.112 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.125+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6740611d0 con 0x7fe65806dfa0 2024-03-13T14:55:56.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.137+0000 7fe66b400700 1 -- 172.21.15.176:0/4033853466 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe6740611d0 con 0x7fe65806dfa0 2024-03-13T14:55:56.123 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:55:56.123 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (3m) 13s ago 17m 14.2M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:55:56.123 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (17m) 13s ago 17m 8304k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (16m) 16s ago 16m 8238k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (28s) 13s ago 17m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (21s) 16s ago 16m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (3m) 13s ago 16m 73.7M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (10m) 13s ago 10m 87.6M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (10m) 13s ago 10m 17.2M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (10m) 16s ago 10m 141M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (10m) 16s ago 10m 16.9M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (6m) 13s ago 19m 498M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (5m) 16s ago 15m 449M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (2m) 13s ago 19m 43.8M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (90s) 16s ago 15m 36.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (5m) 13s ago 17m 7503k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (4m) 16s ago 16m 7910k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:55:56.124 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (14m) 13s ago 14m 233M 1217M 18.2.1 5be31c24972a 01f3ab7b58e3 2024-03-13T14:55:56.125 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (14m) 13s ago 14m 246M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:55:56.125 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (13m) 13s ago 13m 220M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:55:56.125 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (13m) 16s ago 13m 306M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:55:56.125 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (12m) 16s ago 12m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:55:56.125 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (11m) 16s ago 11m 230M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:55:56.125 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (4m) 13s ago 16m 66.9M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:55:56.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.139+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806dfa0 msgr2=0x7fe658070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.139+0000 7fe679c00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806dfa0 0x7fe658070460 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe664005fd0 tx=0x7fe66400a560 comp rx=0 tx=0).stop 2024-03-13T14:55:56.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.139+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 msgr2=0x7fe6740735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.139+0000 7fe679c00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6740735d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe66000bda0 tx=0x7fe66000be80 comp rx=0 tx=0).stop 2024-03-13T14:55:56.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.140+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 shutdown_connections 2024-03-13T14:55:56.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.140+0000 7fe679c00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe65806dfa0 0x7fe658070460 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.140+0000 7fe679c00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe674100da0 0x7fe674073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.140+0000 7fe679c00700 1 --2- 172.21.15.176:0/4033853466 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe674101750 0x7fe6740735d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.140+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 >> 172.21.15.176:0/4033853466 conn(0x7fe6740fc630 msgr2=0x7fe6740fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:56.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.140+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 shutdown_connections 2024-03-13T14:55:56.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.140+0000 7fe679c00700 1 -- 172.21.15.176:0/4033853466 wait complete. 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 -- 172.21.15.176:0/3816776695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 msgr2=0x7f4c04073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 --2- 172.21.15.176:0/3816776695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c04073540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009910 tx=0x7f4bf4009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 -- 172.21.15.176:0/3816776695 shutdown_connections 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 --2- 172.21.15.176:0/3816776695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c04073a80 0x7f4c0410cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 --2- 172.21.15.176:0/3816776695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c04073540 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 -- 172.21.15.176:0/3816776695 >> 172.21.15.176:0/3816776695 conn(0x7f4c040fc4f0 msgr2=0x7f4c040fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 -- 172.21.15.176:0/3816776695 shutdown_connections 2024-03-13T14:55:56.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.286+0000 7f4c03200700 1 -- 172.21.15.176:0/3816776695 wait complete. 2024-03-13T14:55:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.287+0000 7f4c03200700 1 Processor -- start 2024-03-13T14:55:56.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c03200700 1 -- start start 2024-03-13T14:55:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c03200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c0419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c03200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c04073a80 0x7f4c0419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c03200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0419f140 con 0x7f4c04073160 2024-03-13T14:55:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c03200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c041985f0 con 0x7f4c04073a80 2024-03-13T14:55:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c01e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c0419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c01e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c0419e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42562/0 (socket says 172.21.15.176:42562) 2024-03-13T14:55:56.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.288+0000 7f4c01e00700 1 -- 172.21.15.176:0/3947798935 learned_addr learned my addr 172.21.15.176:0/3947798935 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4c01400700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c04073a80 0x7f4c0419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4c01e00700 1 -- 172.21.15.176:0/3947798935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c04073a80 msgr2=0x7f4c0419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4c01e00700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c04073a80 0x7f4c0419eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4c01e00700 1 -- 172.21.15.176:0/3947798935 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf40095f0 con 0x7f4c04073160 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4c01e00700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c0419e570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4bfaa00700 1 -- 172.21.15.176:0/3947798935 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bf401d070 con 0x7f4c04073160 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4bfaa00700 1 -- 172.21.15.176:0/3947798935 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bf400f460 con 0x7f4c04073160 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c04198870 con 0x7f4c04073160 2024-03-13T14:55:56.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4bfaa00700 1 -- 172.21.15.176:0/3947798935 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bf4005e20 con 0x7f4c04073160 2024-03-13T14:55:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.289+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c04198d60 con 0x7f4c04073160 2024-03-13T14:55:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.290+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be4005320 con 0x7f4c04073160 2024-03-13T14:55:56.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.291+0000 7f4bfaa00700 1 -- 172.21.15.176:0/3947798935 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4bf400fad0 con 0x7f4c04073160 2024-03-13T14:55:56.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.292+0000 7f4bfaa00700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4be806dfa0 0x7f4be8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:56.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.292+0000 7f4bfaa00700 1 -- 172.21.15.176:0/3947798935 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4bf408f120 con 0x7f4c04073160 2024-03-13T14:55:56.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.293+0000 7f4c01400700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4be806dfa0 0x7f4be8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:56.279 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.293+0000 7f4c01400700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4be806dfa0 0x7f4be8070460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0005950 tx=0x7f4bf00058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:56.283 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.297+0000 7f4bfaa00700 1 -- 172.21.15.176:0/3947798935 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4bf405c140 con 0x7f4c04073160 2024-03-13T14:55:56.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.600+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4be4006200 con 0x7f4c04073160 2024-03-13T14:55:56.587 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:56 smithi176 ceph-mon[112742]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:56.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.601+0000 7f4bfaa00700 1 -- 172.21.15.176:0/3947798935 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f4bf405bcd0 con 0x7f4c04073160 2024-03-13T14:55:56.587 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:55:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:55:56.587 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 10, 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:55:56.588 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:55:56.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4be806dfa0 msgr2=0x7f4be8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4be806dfa0 0x7f4be8070460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0005950 tx=0x7f4bf00058e0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 msgr2=0x7f4c0419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c0419e570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf400be20 comp rx=0 tx=0).stop 2024-03-13T14:55:56.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 shutdown_connections 2024-03-13T14:55:56.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4be806dfa0 0x7f4be8070460 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4c04073160 0x7f4c0419e570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 --2- 172.21.15.176:0/3947798935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c04073a80 0x7f4c0419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 >> 172.21.15.176:0/3947798935 conn(0x7f4c040fc4f0 msgr2=0x7f4c04107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:56.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 shutdown_connections 2024-03-13T14:55:56.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.604+0000 7f4c03200700 1 -- 172.21.15.176:0/3947798935 wait complete. 2024-03-13T14:55:56.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:56 smithi196 ceph-mon[86193]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 -- 172.21.15.176:0/1809479685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 msgr2=0x7fc3600735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 --2- 172.21.15.176:0/1809479685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc3600735a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc350009910 tx=0x7fc350009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 -- 172.21.15.176:0/1809479685 shutdown_connections 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 --2- 172.21.15.176:0/1809479685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc360073ae0 0x7fc36010ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 --2- 172.21.15.176:0/1809479685 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc3600735a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 -- 172.21.15.176:0/1809479685 >> 172.21.15.176:0/1809479685 conn(0x7fc3600fc630 msgr2=0x7fc3600fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 -- 172.21.15.176:0/1809479685 shutdown_connections 2024-03-13T14:55:56.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.756+0000 7fc366000700 1 -- 172.21.15.176:0/1809479685 wait complete. 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.757+0000 7fc366000700 1 Processor -- start 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc366000700 1 -- start start 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc366000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc360198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc366000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc360073ae0 0x7fc360198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc366000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc360199520 con 0x7fc3600731c0 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc366000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36019d330 con 0x7fc360073ae0 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc35f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc360198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc35f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc360198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42592/0 (socket says 172.21.15.176:42592) 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.758+0000 7fc35f400700 1 -- 172.21.15.176:0/1838590248 learned_addr learned my addr 172.21.15.176:0/1838590248 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc35f400700 1 -- 172.21.15.176:0/1838590248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc360073ae0 msgr2=0x7fc360198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc35f400700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc360073ae0 0x7fc360198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:56.745 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc35f400700 1 -- 172.21.15.176:0/1838590248 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500095f0 con 0x7fc3600731c0 2024-03-13T14:55:56.746 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc35f400700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc360198950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3500092e0 tx=0x7fc35000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:56.746 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc35cc00700 1 -- 172.21.15.176:0/1838590248 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35001d070 con 0x7fc3600731c0 2024-03-13T14:55:56.746 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc35cc00700 1 -- 172.21.15.176:0/1838590248 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc35000f460 con 0x7fc3600731c0 2024-03-13T14:55:56.746 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36019d5b0 con 0x7fc3600731c0 2024-03-13T14:55:56.747 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc35cc00700 1 -- 172.21.15.176:0/1838590248 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc350005e20 con 0x7fc3600731c0 2024-03-13T14:55:56.747 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.759+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36019daa0 con 0x7fc3600731c0 2024-03-13T14:55:56.747 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.760+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36010a540 con 0x7fc3600731c0 2024-03-13T14:55:56.747 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.761+0000 7fc35cc00700 1 -- 172.21.15.176:0/1838590248 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc35000f5d0 con 0x7fc3600731c0 2024-03-13T14:55:56.748 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.762+0000 7fc35cc00700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc34406dfa0 0x7fc344070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:56.749 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.762+0000 7fc35cc00700 1 -- 172.21.15.176:0/1838590248 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc35008f170 con 0x7fc3600731c0 2024-03-13T14:55:56.749 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.762+0000 7fc357e00700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc34406dfa0 0x7fc344070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:56.749 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.763+0000 7fc357e00700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc34406dfa0 0x7fc344070460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc348005950 tx=0x7fc3480058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:56.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:56.767+0000 7fc35cc00700 1 -- 172.21.15.176:0/1838590248 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc35005c2b0 con 0x7fc3600731c0 2024-03-13T14:55:57.055 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.069+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc36004ea90 con 0x7fc3600731c0 2024-03-13T14:55:57.056 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.070+0000 7fc35cc00700 1 -- 172.21.15.176:0/1838590248 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fc35005be40 con 0x7fc3600731c0 2024-03-13T14:55:57.056 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:55:57.057 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:55:57.058 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:57.059 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:55:57.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc34406dfa0 msgr2=0x7fc344070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.060 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc34406dfa0 0x7fc344070460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc348005950 tx=0x7fc3480058e0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 msgr2=0x7fc360198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc360198950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc3500092e0 tx=0x7fc35000be20 comp rx=0 tx=0).stop 2024-03-13T14:55:57.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 shutdown_connections 2024-03-13T14:55:57.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc34406dfa0 0x7fc344070460 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc3600731c0 0x7fc360198950 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 --2- 172.21.15.176:0/1838590248 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc360073ae0 0x7fc360198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.074+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 >> 172.21.15.176:0/1838590248 conn(0x7fc3600fc630 msgr2=0x7fc360107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:57.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.075+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 shutdown_connections 2024-03-13T14:55:57.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.075+0000 7fc366000700 1 -- 172.21.15.176:0/1838590248 wait complete. 2024-03-13T14:55:57.062 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 -- 172.21.15.176:0/933385567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef010d280 msgr2=0x7f4ef010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 --2- 172.21.15.176:0/933385567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef010d280 0x7f4ef010d660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009910 tx=0x7f4ee0009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 -- 172.21.15.176:0/933385567 shutdown_connections 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 --2- 172.21.15.176:0/933385567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef01071d0 0x7f4ef0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 --2- 172.21.15.176:0/933385567 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef010d280 0x7f4ef010d660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 -- 172.21.15.176:0/933385567 >> 172.21.15.176:0/933385567 conn(0x7f4ef0076990 msgr2=0x7f4ef0078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 -- 172.21.15.176:0/933385567 shutdown_connections 2024-03-13T14:55:57.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.232+0000 7f4ef6600700 1 -- 172.21.15.176:0/933385567 wait complete. 2024-03-13T14:55:57.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.233+0000 7f4ef6600700 1 Processor -- start 2024-03-13T14:55:57.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.233+0000 7f4ef6600700 1 -- start start 2024-03-13T14:55:57.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4ef6600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef01071d0 0x7f4ef019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:57.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4ef6600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef010d280 0x7f4ef019d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:57.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4ef6600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef019d9f0 con 0x7f4ef01071d0 2024-03-13T14:55:57.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4ef6600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef01a1780 con 0x7f4ef010d280 2024-03-13T14:55:57.221 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4eef400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef01071d0 0x7f4ef019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:57.221 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4eef400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef01071d0 0x7f4ef019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42610/0 (socket says 172.21.15.176:42610) 2024-03-13T14:55:57.221 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4eef400700 1 -- 172.21.15.176:0/611562272 learned_addr learned my addr 172.21.15.176:0/611562272 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4eef400700 1 -- 172.21.15.176:0/611562272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef010d280 msgr2=0x7f4ef019d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4eeea00700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef010d280 0x7f4ef019d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4eef400700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef010d280 0x7f4ef019d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.234+0000 7f4eef400700 1 -- 172.21.15.176:0/611562272 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00095f0 con 0x7f4ef01071d0 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.235+0000 7f4eef400700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef01071d0 0x7f4ef019cdd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006010 tx=0x7f4ee000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.235+0000 7f4ee7e00700 1 -- 172.21.15.176:0/611562272 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee001d070 con 0x7f4ef01071d0 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.235+0000 7f4ee7e00700 1 -- 172.21.15.176:0/611562272 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ee0004fb0 con 0x7f4ef01071d0 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.235+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef01a1a00 con 0x7f4ef01071d0 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.235+0000 7f4ee7e00700 1 -- 172.21.15.176:0/611562272 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee000f690 con 0x7f4ef01071d0 2024-03-13T14:55:57.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.235+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef01a1f70 con 0x7f4ef01071d0 2024-03-13T14:55:57.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.236+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef004ea90 con 0x7f4ef01071d0 2024-03-13T14:55:57.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.237+0000 7f4ee7e00700 1 -- 172.21.15.176:0/611562272 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4ee00228a0 con 0x7f4ef01071d0 2024-03-13T14:55:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.238+0000 7f4ee7e00700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4ed406dfa0 0x7f4ed4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:57.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.238+0000 7f4ee7e00700 1 -- 172.21.15.176:0/611562272 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4ee008f5b0 con 0x7f4ef01071d0 2024-03-13T14:55:57.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.238+0000 7f4eeea00700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4ed406dfa0 0x7f4ed4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:57.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.239+0000 7f4eeea00700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4ed406dfa0 0x7f4ed4070460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4edc006fd0 tx=0x7f4edc008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:57.229 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.243+0000 7f4ee7e00700 1 -- 172.21.15.176:0/611562272 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4ee0058a80 con 0x7f4ef01071d0 2024-03-13T14:55:57.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.505+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ef01a2440 con 0x7f4ed406dfa0 2024-03-13T14:55:57.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-mon[112742]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:57.493 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-mon[112742]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:57.494 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-mon[112742]: pgmap v61: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:57.494 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3947798935' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:57.494 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1838590248' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.508+0000 7f4ee7e00700 1 -- 172.21.15.176:0/611562272 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4ef01a2440 con 0x7f4ed406dfa0 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:55:57.495 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "6/23 daemons upgraded", 2024-03-13T14:55:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading osd daemons", 2024-03-13T14:55:57.496 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:55:57.496 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:55:57.497 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4ed406dfa0 msgr2=0x7f4ed4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4ed406dfa0 0x7f4ed4070460 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4edc006fd0 tx=0x7f4edc008040 comp rx=0 tx=0).stop 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef01071d0 msgr2=0x7f4ef019cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef01071d0 0x7f4ef019cdd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0006010 tx=0x7f4ee000be20 comp rx=0 tx=0).stop 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 shutdown_connections 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4ed406dfa0 0x7f4ed4070460 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ef01071d0 0x7f4ef019cdd0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 --2- 172.21.15.176:0/611562272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef010d280 0x7f4ef019d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.511+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 >> 172.21.15.176:0/611562272 conn(0x7f4ef0076990 msgr2=0x7f4ef0078430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:57.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.512+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 shutdown_connections 2024-03-13T14:55:57.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.512+0000 7f4ef6600700 1 -- 172.21.15.176:0/611562272 wait complete. 2024-03-13T14:55:57.613 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:57 smithi196 ceph-mon[86193]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:57.613 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:57 smithi196 ceph-mon[86193]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:57.613 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:57 smithi196 ceph-mon[86193]: pgmap v61: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:55:57.613 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:57 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3947798935' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:55:57.613 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:57 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1838590248' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:55:57.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.667+0000 7f32f9000700 1 -- 172.21.15.176:0/3357933785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068d30 msgr2=0x7f32f410d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.667+0000 7f32f9000700 1 --2- 172.21.15.176:0/3357933785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068d30 0x7f32f410d240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009910 tx=0x7f32e0009c20 comp rx=0 tx=0).stop 2024-03-13T14:55:57.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.668+0000 7f32f9000700 1 -- 172.21.15.176:0/3357933785 shutdown_connections 2024-03-13T14:55:57.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.668+0000 7f32f9000700 1 --2- 172.21.15.176:0/3357933785 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068d30 0x7f32f410d240 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.668+0000 7f32f9000700 1 --2- 172.21.15.176:0/3357933785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32f4068410 0x7f32f40687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.668+0000 7f32f9000700 1 -- 172.21.15.176:0/3357933785 >> 172.21.15.176:0/3357933785 conn(0x7f32f40755a0 msgr2=0x7f32f40759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:57.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.668+0000 7f32f9000700 1 -- 172.21.15.176:0/3357933785 shutdown_connections 2024-03-13T14:55:57.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.668+0000 7f32f9000700 1 -- 172.21.15.176:0/3357933785 wait complete. 2024-03-13T14:55:57.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.669+0000 7f32f9000700 1 Processor -- start 2024-03-13T14:55:57.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.669+0000 7f32f9000700 1 -- start start 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.669+0000 7f32f9000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068410 0x7f32f419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.669+0000 7f32f9000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32f4068d30 0x7f32f419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.669+0000 7f32f9000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f419f380 con 0x7f32f4068410 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.669+0000 7f32f9000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4198830 con 0x7f32f4068d30 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068410 0x7f32f419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068410 0x7f32f419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:42634/0 (socket says 172.21.15.176:42634) 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1e00700 1 -- 172.21.15.176:0/4227233973 learned_addr learned my addr 172.21.15.176:0/4227233973 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1400700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32f4068d30 0x7f32f419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1e00700 1 -- 172.21.15.176:0/4227233973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32f4068d30 msgr2=0x7f32f419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1e00700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32f4068d30 0x7f32f419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1e00700 1 -- 172.21.15.176:0/4227233973 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00095f0 con 0x7f32f4068410 2024-03-13T14:55:57.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.670+0000 7f32f1e00700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068410 0x7f32f419e7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32e400b480 tx=0x7f32e400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:57.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.671+0000 7f32eaa00700 1 -- 172.21.15.176:0/4227233973 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32e40118c0 con 0x7f32f4068410 2024-03-13T14:55:57.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.671+0000 7f32eaa00700 1 -- 172.21.15.176:0/4227233973 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32e4010380 con 0x7f32f4068410 2024-03-13T14:55:57.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.671+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f4198b10 con 0x7f32f4068410 2024-03-13T14:55:57.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.671+0000 7f32eaa00700 1 -- 172.21.15.176:0/4227233973 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32e400f4f0 con 0x7f32f4068410 2024-03-13T14:55:57.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.672+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f4199060 con 0x7f32f4068410 2024-03-13T14:55:57.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.673+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f410a9b0 con 0x7f32f4068410 2024-03-13T14:55:57.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.674+0000 7f32eaa00700 1 -- 172.21.15.176:0/4227233973 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f32e40104f0 con 0x7f32f4068410 2024-03-13T14:55:57.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.676+0000 7f32eaa00700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f32d806dfa0 0x7f32d8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:55:57.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.676+0000 7f32eaa00700 1 -- 172.21.15.176:0/4227233973 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f32e408d670 con 0x7f32f4068410 2024-03-13T14:55:57.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.676+0000 7f32f1400700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f32d806dfa0 0x7f32d8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:55:57.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.677+0000 7f32f1400700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f32d806dfa0 0x7f32d8070460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f32e0000c00 tx=0x7f32e0005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:55:57.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.684+0000 7f32eaa00700 1 -- 172.21.15.176:0/4227233973 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f32e405a670 con 0x7f32f4068410 2024-03-13T14:55:57.877 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:57 smithi176 systemd[1]: Stopping Ceph osd.0 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:55:57.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.993+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f32f404ea90 con 0x7f32f4068410 2024-03-13T14:55:57.980 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.994+0000 7f32eaa00700 1 -- 172.21.15.176:0/4227233973 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f32e405a200 con 0x7f32f4068410 2024-03-13T14:55:57.980 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:55:57.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.996+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f32d806dfa0 msgr2=0x7f32d8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.996+0000 7f32f9000700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f32d806dfa0 0x7f32d8070460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f32e0000c00 tx=0x7f32e0005fd0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.996+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068410 msgr2=0x7f32f419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:55:57.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.996+0000 7f32f9000700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068410 0x7f32f419e7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f32e400b480 tx=0x7f32e400b840 comp rx=0 tx=0).stop 2024-03-13T14:55:57.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.997+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 shutdown_connections 2024-03-13T14:55:57.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.997+0000 7f32f9000700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f32d806dfa0 0x7f32d8070460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.997+0000 7f32f9000700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f32f4068410 0x7f32f419e7b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.997+0000 7f32f9000700 1 --2- 172.21.15.176:0/4227233973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32f4068d30 0x7f32f419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:55:57.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.997+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 >> 172.21.15.176:0/4227233973 conn(0x7f32f40755a0 msgr2=0x7f32f40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:55:57.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.997+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 shutdown_connections 2024-03-13T14:55:57.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:55:57.997+0000 7f32f9000700 1 -- 172.21.15.176:0/4227233973 wait complete. 2024-03-13T14:55:58.331 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[47463]: 2024-03-13T14:55:57.948+0000 7f08a8400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:55:58.331 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[47463]: 2024-03-13T14:55:57.948+0000 7f08a8400700 -1 osd.0 50 *** Got signal Terminated *** 2024-03-13T14:55:58.331 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:57 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[47463]: 2024-03-13T14:55:57.949+0000 7f08a8400700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T14:55:58.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:58 smithi176 ceph-mon[112742]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:58.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:58 smithi176 ceph-mon[112742]: osd.0 marked itself down and dead 2024-03-13T14:55:58.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:58 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/4227233973' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:55:58.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:58 smithi196 ceph-mon[86193]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:55:58.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:58 smithi196 ceph-mon[86193]: osd.0 marked itself down and dead 2024-03-13T14:55:58.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:58 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/4227233973' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:55:59.262 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:58 smithi176 podman[124587]: 2024-03-13 14:55:58.978659827 +0000 UTC m=+1.171095857 container died 01f3ab7b58e326b5b9edc618db6179c9b0e4f7407a741678ac6bdcb53c3d5d91 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-03-13T14:55:59.623 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:59 smithi176 ceph-mon[112742]: pgmap v62: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:59.624 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:59 smithi176 ceph-mon[112742]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:55:59.624 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:55:59 smithi176 ceph-mon[112742]: osdmap e51: 6 total, 5 up, 6 in 2024-03-13T14:55:59.624 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:59 smithi176 podman[124587]: 2024-03-13 14:55:59.394172147 +0000 UTC m=+1.586608160 container cleanup 01f3ab7b58e326b5b9edc618db6179c9b0e4f7407a741678ac6bdcb53c3d5d91 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1) 2024-03-13T14:55:59.624 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:59 smithi176 bash[124587]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0 2024-03-13T14:55:59.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:59 smithi196 ceph-mon[86193]: pgmap v62: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:55:59.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:59 smithi196 ceph-mon[86193]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:55:59.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:55:59 smithi196 ceph-mon[86193]: osdmap e51: 6 total, 5 up, 6 in 2024-03-13T14:55:59.877 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:55:59 smithi176 podman[124608]: 2024-03-13 14:55:59.637065811 +0000 UTC m=+0.653117972 container remove 01f3ab7b58e326b5b9edc618db6179c9b0e4f7407a741678ac6bdcb53c3d5d91 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-13T14:56:00.628 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:00 smithi176 podman[124702]: 2024-03-13 14:56:00.339108243 +0000 UTC m=+0.033659017 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:56:00.628 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:00 smithi176 podman[124702]: 2024-03-13 14:56:00.455076618 +0000 UTC m=+0.149627413 container create 5455ec797129950ee8197d40153e9239666398e586662572ae42fb45e3fba95a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-13T14:56:00.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: osdmap e52: 6 total, 5 up, 6 in 2024-03-13T14:56:00.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]: dispatch 2024-03-13T14:56:00.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [3, 5]}]: dispatch 2024-03-13T14:56:00.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 0]}]: dispatch 2024-03-13T14:56:00.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 2]}]: dispatch 2024-03-13T14:56:00.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]: dispatch 2024-03-13T14:56:00.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [1, 5]}]: dispatch 2024-03-13T14:56:00.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]: dispatch 2024-03-13T14:56:00.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 5]}]: dispatch 2024-03-13T14:56:00.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-03-13T14:56:00.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: osdmap e52: 6 total, 5 up, 6 in 2024-03-13T14:56:00.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [3, 5]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 0]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 2]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [1, 5]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 5]}]: dispatch 2024-03-13T14:56:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]: dispatch 2024-03-13T14:56:01.127 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:00 smithi176 podman[124702]: 2024-03-13 14:56:00.751855604 +0000 UTC m=+0.446406455 container init 5455ec797129950ee8197d40153e9239666398e586662572ae42fb45e3fba95a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T14:56:01.127 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:00 smithi176 podman[124702]: 2024-03-13 14:56:00.76309739 +0000 UTC m=+0.457648163 container start 5455ec797129950ee8197d40153e9239666398e586662572ae42fb45e3fba95a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:56:01.127 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:00 smithi176 podman[124702]: 2024-03-13 14:56:00.89969751 +0000 UTC m=+0.594248331 container attach 5455ec797129950ee8197d40153e9239666398e586662572ae42fb45e3fba95a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True) 2024-03-13T14:56:01.544 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:01 smithi176 podman[124702]: 2024-03-13 14:56:01.210387989 +0000 UTC m=+0.904938818 container died 5455ec797129950ee8197d40153e9239666398e586662572ae42fb45e3fba95a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:56:01.545 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: pgmap v65: 65 pgs: 9 stale+active+clean, 56 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-13T14:56:01.545 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [3, 5]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 0]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 2]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [1, 5]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 5]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: osdmap e53: 6 total, 5 up, 6 in 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:01.546 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:01 smithi176 ceph-mon[112742]: osdmap e54: 6 total, 5 up, 6 in 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: pgmap v65: 65 pgs: 9 stale+active+clean, 56 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [3, 2]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [3, 5]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [3, 0]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [1, 2]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [4, 2]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [1, 5]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.16", "id": [3, 0]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [3, 5]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 0]}]': finished 2024-03-13T14:56:01.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: osdmap e53: 6 total, 5 up, 6 in 2024-03-13T14:56:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:01.679 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:01 smithi196 ceph-mon[86193]: osdmap e54: 6 total, 5 up, 6 in 2024-03-13T14:56:02.217 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:01 smithi176 podman[124727]: 2024-03-13 14:56:01.928369942 +0000 UTC m=+0.701012788 container remove 5455ec797129950ee8197d40153e9239666398e586662572ae42fb45e3fba95a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5) 2024-03-13T14:56:02.217 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:01 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.0.service: Deactivated successfully. 2024-03-13T14:56:02.218 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:01 smithi176 systemd[1]: Stopped Ceph osd.0 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:56:02.218 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:01 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.0.service: Consumed 1min 57.073s CPU time. 2024-03-13T14:56:02.627 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:02 smithi176 systemd[1]: Starting Ceph osd.0 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:56:02.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:02 smithi196 ceph-mon[86193]: osdmap e55: 6 total, 5 up, 6 in 2024-03-13T14:56:02.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:02 smithi196 ceph-mon[86193]: pgmap v69: 65 pgs: 9 remapped+peering, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 37/225 objects degraded (16.444%) 2024-03-13T14:56:03.061 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:02 smithi176 ceph-mon[112742]: osdmap e55: 6 total, 5 up, 6 in 2024-03-13T14:56:03.062 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:02 smithi176 ceph-mon[112742]: pgmap v69: 65 pgs: 9 remapped+peering, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 37/225 objects degraded (16.444%) 2024-03-13T14:56:03.062 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:02 smithi176 podman[124827]: 2024-03-13 14:56:02.617864186 +0000 UTC m=+0.034418145 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:56:03.062 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:02 smithi176 podman[124827]: 2024-03-13 14:56:02.746323308 +0000 UTC m=+0.162877357 container create 22319f5ef0a6ccdcb2b075a84282026731b33af160790ab39eb2aa1ab29c0bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-13T14:56:03.062 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:03 smithi176 podman[124827]: 2024-03-13 14:56:03.075390892 +0000 UTC m=+0.491944863 container init 22319f5ef0a6ccdcb2b075a84282026731b33af160790ab39eb2aa1ab29c0bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-13T14:56:03.377 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:03 smithi176 podman[124827]: 2024-03-13 14:56:03.087595776 +0000 UTC m=+0.504149741 container start 22319f5ef0a6ccdcb2b075a84282026731b33af160790ab39eb2aa1ab29c0bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-13T14:56:03.377 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:03 smithi176 podman[124827]: 2024-03-13 14:56:03.174056591 +0000 UTC m=+0.590610559 container attach 22319f5ef0a6ccdcb2b075a84282026731b33af160790ab39eb2aa1ab29c0bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate[124841]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate[124841]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 bash[124827]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 bash[124827]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 bash[124827]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 bash[124827]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 bash[124827]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 bash[124827]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 bash[124827]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate[124841]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate[124841]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate[124841]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-03-13T14:56:04.035 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate[124841]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-13T14:56:04.036 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate[124841]: --> ceph-volume raw activate successful for osd ID: 0 2024-03-13T14:56:04.036 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:03 smithi176 ceph-mon[112742]: Health check failed: Degraded data redundancy: 37/225 objects degraded (16.444%), 16 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:04.036 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:03 smithi176 ceph-mon[112742]: osdmap e56: 6 total, 5 up, 6 in 2024-03-13T14:56:04.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:03 smithi196 ceph-mon[86193]: Health check failed: Degraded data redundancy: 37/225 objects degraded (16.444%), 16 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:04.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:03 smithi196 ceph-mon[86193]: osdmap e56: 6 total, 5 up, 6 in 2024-03-13T14:56:04.376 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 podman[124827]: 2024-03-13 14:56:04.049094023 +0000 UTC m=+1.465647984 container died 22319f5ef0a6ccdcb2b075a84282026731b33af160790ab39eb2aa1ab29c0bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True) 2024-03-13T14:56:05.112 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-mon[112742]: osdmap e57: 6 total, 5 up, 6 in 2024-03-13T14:56:05.112 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:04 smithi176 ceph-mon[112742]: pgmap v72: 65 pgs: 9 remapped+peering, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 37/225 objects degraded (16.444%) 2024-03-13T14:56:05.112 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:04 smithi176 podman[124977]: 2024-03-13 14:56:04.778450943 +0000 UTC m=+0.710700166 container remove 22319f5ef0a6ccdcb2b075a84282026731b33af160790ab39eb2aa1ab29c0bd3 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, io.buildah.version=1.33.5) 2024-03-13T14:56:05.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:04 smithi196 ceph-mon[86193]: osdmap e57: 6 total, 5 up, 6 in 2024-03-13T14:56:05.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:04 smithi196 ceph-mon[86193]: pgmap v72: 65 pgs: 9 remapped+peering, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 37/225 objects degraded (16.444%) 2024-03-13T14:56:05.377 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:05 smithi176 podman[125036]: 2024-03-13 14:56:05.032559244 +0000 UTC m=+0.032783490 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:56:05.377 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:05 smithi176 podman[125036]: 2024-03-13 14:56:05.204575638 +0000 UTC m=+0.204799921 container create 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:56:05.808 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:05 smithi176 podman[125036]: 2024-03-13 14:56:05.457684532 +0000 UTC m=+0.457908848 container init 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222) 2024-03-13T14:56:05.808 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:05 smithi176 podman[125036]: 2024-03-13 14:56:05.468179937 +0000 UTC m=+0.468404240 container start 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:56:05.808 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:05 smithi176 ceph-osd[125055]: -- 172.21.15.176:0/3302564360 <== mon.0 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ac1f7a1680 con 0x55ac1f76b000 2024-03-13T14:56:05.808 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:05 smithi176 bash[125036]: 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 2024-03-13T14:56:05.808 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:05 smithi176 systemd[1]: Started Ceph osd.0 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:56:06.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:05 smithi176 ceph-mon[112742]: osdmap e58: 6 total, 5 up, 6 in 2024-03-13T14:56:06.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:05 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:06.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:05 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:06.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:05 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:56:06.127 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:06 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[125051]: 2024-03-13T14:56:06.067+0000 7f4ad190e7c0 -1 Falling back to public interface 2024-03-13T14:56:06.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:05 smithi196 ceph-mon[86193]: osdmap e58: 6 total, 5 up, 6 in 2024-03-13T14:56:06.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:05 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:06.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:05 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:06.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:05 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:56:07.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:06 smithi176 ceph-mon[112742]: osdmap e59: 6 total, 5 up, 6 in 2024-03-13T14:56:07.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:06 smithi176 ceph-mon[112742]: pgmap v75: 65 pgs: 1 active+undersized+remapped, 8 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 3.0 MiB/s, 728 objects/s recovering 2024-03-13T14:56:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:06 smithi196 ceph-mon[86193]: osdmap e59: 6 total, 5 up, 6 in 2024-03-13T14:56:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:06 smithi196 ceph-mon[86193]: pgmap v75: 65 pgs: 1 active+undersized+remapped, 8 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 3.0 MiB/s, 728 objects/s recovering 2024-03-13T14:56:08.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:07 smithi176 ceph-mon[112742]: osdmap e60: 6 total, 5 up, 6 in 2024-03-13T14:56:08.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:07 smithi196 ceph-mon[86193]: osdmap e60: 6 total, 5 up, 6 in 2024-03-13T14:56:09.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:08 smithi176 ceph-mon[112742]: osdmap e61: 6 total, 5 up, 6 in 2024-03-13T14:56:09.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:08 smithi176 ceph-mon[112742]: pgmap v78: 65 pgs: 1 active+undersized+remapped, 8 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 3.0 MiB/s, 728 objects/s recovering 2024-03-13T14:56:09.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:08 smithi196 ceph-mon[86193]: osdmap e61: 6 total, 5 up, 6 in 2024-03-13T14:56:09.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:08 smithi196 ceph-mon[86193]: pgmap v78: 65 pgs: 1 active+undersized+remapped, 8 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 3.0 MiB/s, 728 objects/s recovering 2024-03-13T14:56:11.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:11 smithi176 ceph-mon[112742]: pgmap v79: 65 pgs: 1 active+undersized+remapped, 8 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 926 B/s rd, 1 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 2.2 MiB/s, 527 objects/s recovering 2024-03-13T14:56:11.627 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:11 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[125051]: 2024-03-13T14:56:11.300+0000 7f4ad190e7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-03-13T14:56:11.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:11 smithi196 ceph-mon[86193]: pgmap v79: 65 pgs: 1 active+undersized+remapped, 8 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 926 B/s rd, 1 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 2.2 MiB/s, 527 objects/s recovering 2024-03-13T14:56:12.047 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:11 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[125051]: 2024-03-13T14:56:11.907+0000 7f4ad190e7c0 -1 osd.0 50 log_to_monitors true 2024-03-13T14:56:12.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:12 smithi176 ceph-mon[112742]: from='osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-13T14:56:12.627 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 14:56:12 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[125051]: 2024-03-13T14:56:12.417+0000 7f4ac0c00700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T14:56:12.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:12 smithi196 ceph-mon[86193]: from='osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-13T14:56:13.484 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:13 smithi176 ceph-mon[112742]: pgmap v80: 65 pgs: 9 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 1.8 MiB/s, 447 objects/s recovering 2024-03-13T14:56:13.485 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:13 smithi176 ceph-mon[112742]: from='osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-13T14:56:13.485 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:13 smithi176 ceph-mon[112742]: osdmap e62: 6 total, 5 up, 6 in 2024-03-13T14:56:13.485 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:13 smithi176 ceph-mon[112742]: from='osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:56:13.485 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:13.485 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:13 smithi196 ceph-mon[86193]: pgmap v80: 65 pgs: 9 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%); 1.8 MiB/s, 447 objects/s recovering 2024-03-13T14:56:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:13 smithi196 ceph-mon[86193]: from='osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-13T14:56:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:13 smithi196 ceph-mon[86193]: osdmap e62: 6 total, 5 up, 6 in 2024-03-13T14:56:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:13 smithi196 ceph-mon[86193]: from='osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:56:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:13.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:14.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:14 smithi196 ceph-mon[86193]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:56:14.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:14 smithi196 ceph-mon[86193]: osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948] boot 2024-03-13T14:56:14.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:14 smithi196 ceph-mon[86193]: osdmap e63: 6 total, 6 up, 6 in 2024-03-13T14:56:14.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:14 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:56:14.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:14 smithi196 ceph-mon[86193]: osdmap e64: 6 total, 6 up, 6 in 2024-03-13T14:56:14.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:14 smithi176 ceph-mon[112742]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:56:14.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:14 smithi176 ceph-mon[112742]: osd.0 [v2:172.21.15.176:6802/468040948,v1:172.21.15.176:6803/468040948] boot 2024-03-13T14:56:14.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:14 smithi176 ceph-mon[112742]: osdmap e63: 6 total, 6 up, 6 in 2024-03-13T14:56:14.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:14 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-13T14:56:14.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:14 smithi176 ceph-mon[112742]: osdmap e64: 6 total, 6 up, 6 in 2024-03-13T14:56:15.715 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:15 smithi176 ceph-mon[112742]: pgmap v83: 65 pgs: 9 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 788 B/s rd, 1 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%) 2024-03-13T14:56:15.715 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:15.715 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:15.715 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:15.715 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:15.716 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:15 smithi176 ceph-mon[112742]: osdmap e65: 6 total, 6 up, 6 in 2024-03-13T14:56:15.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:15 smithi196 ceph-mon[86193]: pgmap v83: 65 pgs: 9 active+clean+remapped, 16 active+undersized+degraded, 12 active+undersized, 28 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 788 B/s rd, 1 op/s; 37/225 objects degraded (16.444%); 8/225 objects misplaced (3.556%) 2024-03-13T14:56:15.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:15.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:15.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:15.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:15.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:15 smithi196 ceph-mon[86193]: osdmap e65: 6 total, 6 up, 6 in 2024-03-13T14:56:17.032 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:16 smithi196 ceph-mon[86193]: osdmap e66: 6 total, 6 up, 6 in 2024-03-13T14:56:17.032 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:16 smithi196 ceph-mon[86193]: pgmap v87: 65 pgs: 53 active+clean, 1 active+recovering+undersized+remapped, 3 active+undersized, 1 peering, 3 active+clean+remapped, 3 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 689/225 objects degraded (306.222%); 4.1 MiB/s, 63 objects/s recovering 2024-03-13T14:56:17.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:16 smithi176 ceph-mon[112742]: osdmap e66: 6 total, 6 up, 6 in 2024-03-13T14:56:17.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:16 smithi176 ceph-mon[112742]: pgmap v87: 65 pgs: 53 active+clean, 1 active+recovering+undersized+remapped, 3 active+undersized, 1 peering, 3 active+clean+remapped, 3 active+undersized+degraded, 1 active+recovery_wait+undersized+degraded+remapped; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 689/225 objects degraded (306.222%); 4.1 MiB/s, 63 objects/s recovering 2024-03-13T14:56:18.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:17 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 689/225 objects degraded (306.222%), 4 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:18.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:17 smithi176 ceph-mon[112742]: osdmap e67: 6 total, 6 up, 6 in 2024-03-13T14:56:18.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:17 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 689/225 objects degraded (306.222%), 4 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:18.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:17 smithi196 ceph-mon[86193]: osdmap e67: 6 total, 6 up, 6 in 2024-03-13T14:56:19.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:18 smithi176 ceph-mon[112742]: osdmap e68: 6 total, 6 up, 6 in 2024-03-13T14:56:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:18 smithi176 ceph-mon[112742]: pgmap v90: 65 pgs: 62 active+clean, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+undersized+degraded+remapped; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 683/225 objects degraded (303.556%); 4.1 MiB/s, 0 objects/s recovering 2024-03-13T14:56:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:18 smithi196 ceph-mon[86193]: osdmap e68: 6 total, 6 up, 6 in 2024-03-13T14:56:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:18 smithi196 ceph-mon[86193]: pgmap v90: 65 pgs: 62 active+clean, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+undersized+degraded+remapped; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 683/225 objects degraded (303.556%); 4.1 MiB/s, 0 objects/s recovering 2024-03-13T14:56:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:19 smithi176 ceph-mon[112742]: osdmap e69: 6 total, 6 up, 6 in 2024-03-13T14:56:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:19 smithi196 ceph-mon[86193]: osdmap e69: 6 total, 6 up, 6 in 2024-03-13T14:56:21.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:20 smithi176 ceph-mon[112742]: osdmap e70: 6 total, 6 up, 6 in 2024-03-13T14:56:21.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:20 smithi176 ceph-mon[112742]: pgmap v93: 65 pgs: 62 active+clean, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+undersized+degraded+remapped; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 683/225 objects degraded (303.556%) 2024-03-13T14:56:21.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:21.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:21.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:56:21.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:56:21.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:21.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:20 smithi196 ceph-mon[86193]: osdmap e70: 6 total, 6 up, 6 in 2024-03-13T14:56:21.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:20 smithi196 ceph-mon[86193]: pgmap v93: 65 pgs: 62 active+clean, 1 active+recovering+undersized+remapped, 1 peering, 1 active+recovery_wait+undersized+degraded+remapped; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 683/225 objects degraded (303.556%) 2024-03-13T14:56:21.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:21.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:21.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:56:21.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:56:21.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:22.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: osdmap e71: 6 total, 6 up, 6 in 2024-03-13T14:56:22.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:56:22.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:22.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:22.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:22.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:22.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:22.128 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:21 smithi176 ceph-mon[112742]: Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-03-13T14:56:22.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: osdmap e71: 6 total, 6 up, 6 in 2024-03-13T14:56:22.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:56:22.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:22.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:22.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:22.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:22.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:22.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:21 smithi196 ceph-mon[86193]: Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-03-13T14:56:23.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:22 smithi176 ceph-mon[112742]: pgmap v95: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 257 objects/s recovering 2024-03-13T14:56:23.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:22 smithi196 ceph-mon[86193]: pgmap v95: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s; 0 B/s, 257 objects/s recovering 2024-03-13T14:56:24.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:23 smithi176 ceph-mon[112742]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 683/225 objects degraded (303.556%), 1 pg degraded) 2024-03-13T14:56:24.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:23 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T14:56:24.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:23 smithi196 ceph-mon[86193]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 683/225 objects degraded (303.556%), 1 pg degraded) 2024-03-13T14:56:24.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:23 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T14:56:25.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:24 smithi176 ceph-mon[112742]: pgmap v96: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 194 objects/s recovering 2024-03-13T14:56:25.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:24 smithi196 ceph-mon[86193]: pgmap v96: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 194 objects/s recovering 2024-03-13T14:56:27.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:27 smithi196 ceph-mon[86193]: pgmap v97: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 155 objects/s recovering 2024-03-13T14:56:27.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:27 smithi176 ceph-mon[112742]: pgmap v97: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 155 objects/s recovering 2024-03-13T14:56:28.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 -- 172.21.15.176:0/3388344138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151010d280 msgr2=0x7f151010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 --2- 172.21.15.176:0/3388344138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151010d280 0x7f151010d660 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1504009910 tx=0x7f1504009c20 comp rx=0 tx=0).stop 2024-03-13T14:56:28.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 -- 172.21.15.176:0/3388344138 shutdown_connections 2024-03-13T14:56:28.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 --2- 172.21.15.176:0/3388344138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15101071d0 0x7f1510107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 --2- 172.21.15.176:0/3388344138 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f151010d280 0x7f151010d660 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 -- 172.21.15.176:0/3388344138 >> 172.21.15.176:0/3388344138 conn(0x7f15100769e0 msgr2=0x7f1510078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:28.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 -- 172.21.15.176:0/3388344138 shutdown_connections 2024-03-13T14:56:28.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.166+0000 7f1518800700 1 -- 172.21.15.176:0/3388344138 wait complete. 2024-03-13T14:56:28.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.167+0000 7f1518800700 1 Processor -- start 2024-03-13T14:56:28.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.167+0000 7f1518800700 1 -- start start 2024-03-13T14:56:28.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.167+0000 7f1518800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15101071d0 0x7f151019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.167+0000 7f1518800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f151010d280 0x7f151019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.167+0000 7f1518800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151019d8a0 con 0x7f15101071d0 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.167+0000 7f1518800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15101a05d0 con 0x7f151010d280 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15101071d0 0x7f151019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15101071d0 0x7f151019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60348/0 (socket says 172.21.15.176:60348) 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515e00700 1 -- 172.21.15.176:0/441686184 learned_addr learned my addr 172.21.15.176:0/441686184 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515400700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f151010d280 0x7f151019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515e00700 1 -- 172.21.15.176:0/441686184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f151010d280 msgr2=0x7f151019d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515e00700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f151010d280 0x7f151019d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515e00700 1 -- 172.21.15.176:0/441686184 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15040095f0 con 0x7f15101071d0 2024-03-13T14:56:28.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.168+0000 7f1515e00700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15101071d0 0x7f151019ccd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f150400b5c0 tx=0x7f150400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:28.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.169+0000 7f150aa00700 1 -- 172.21.15.176:0/441686184 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f150401d070 con 0x7f15101071d0 2024-03-13T14:56:28.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.169+0000 7f150aa00700 1 -- 172.21.15.176:0/441686184 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f150400f460 con 0x7f15101071d0 2024-03-13T14:56:28.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.169+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15101a0850 con 0x7f15101071d0 2024-03-13T14:56:28.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.169+0000 7f150aa00700 1 -- 172.21.15.176:0/441686184 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1504017610 con 0x7f15101071d0 2024-03-13T14:56:28.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.169+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15101a0dc0 con 0x7f15101071d0 2024-03-13T14:56:28.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.170+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151004ea90 con 0x7f15101071d0 2024-03-13T14:56:28.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.171+0000 7f150aa00700 1 -- 172.21.15.176:0/441686184 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f150400f5d0 con 0x7f15101071d0 2024-03-13T14:56:28.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.172+0000 7f150aa00700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f14f806dfa0 0x7f14f8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.172+0000 7f150aa00700 1 -- 172.21.15.176:0/441686184 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f150408f430 con 0x7f15101071d0 2024-03-13T14:56:28.159 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.172+0000 7f1515400700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f14f806dfa0 0x7f14f8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.159 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.173+0000 7f1515400700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f14f806dfa0 0x7f14f8070460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1500005fd0 tx=0x7f150000a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:28.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.177+0000 7f150aa00700 1 -- 172.21.15.176:0/441686184 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f15040587f0 con 0x7f15101071d0 2024-03-13T14:56:28.406 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.420+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f15101a1150 con 0x7f14f806dfa0 2024-03-13T14:56:28.409 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.423+0000 7f150aa00700 1 -- 172.21.15.176:0/441686184 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f15101a1150 con 0x7f14f806dfa0 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.425+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f14f806dfa0 msgr2=0x7f14f8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.425+0000 7f1518800700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f14f806dfa0 0x7f14f8070460 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1500005fd0 tx=0x7f150000a560 comp rx=0 tx=0).stop 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.425+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15101071d0 msgr2=0x7f151019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.425+0000 7f1518800700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15101071d0 0x7f151019ccd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f150400b5c0 tx=0x7f150400be00 comp rx=0 tx=0).stop 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.426+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 shutdown_connections 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.426+0000 7f1518800700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f14f806dfa0 0x7f14f8070460 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.426+0000 7f1518800700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f15101071d0 0x7f151019ccd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.426+0000 7f1518800700 1 --2- 172.21.15.176:0/441686184 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f151010d280 0x7f151019d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.426+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 >> 172.21.15.176:0/441686184 conn(0x7f15100769e0 msgr2=0x7f1510078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:28.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.426+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 shutdown_connections 2024-03-13T14:56:28.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.426+0000 7f1518800700 1 -- 172.21.15.176:0/441686184 wait complete. 2024-03-13T14:56:28.427 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:56:28.568 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.581+0000 7f065b400700 1 -- 172.21.15.176:0/3098815032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06541039b0 msgr2=0x7f0654107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.568 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.581+0000 7f065b400700 1 --2- 172.21.15.176:0/3098815032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06541039b0 0x7f0654107a00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0640009910 tx=0x7f0640009c20 comp rx=0 tx=0).stop 2024-03-13T14:56:28.568 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.582+0000 7f065b400700 1 -- 172.21.15.176:0/3098815032 shutdown_connections 2024-03-13T14:56:28.568 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.582+0000 7f065b400700 1 --2- 172.21.15.176:0/3098815032 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f06541039b0 0x7f0654107a00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.568 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.582+0000 7f065b400700 1 --2- 172.21.15.176:0/3098815032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0654103000 0x7f06541033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.582+0000 7f065b400700 1 -- 172.21.15.176:0/3098815032 >> 172.21.15.176:0/3098815032 conn(0x7f06540fe8d0 msgr2=0x7f0654100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:28.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.582+0000 7f065b400700 1 -- 172.21.15.176:0/3098815032 shutdown_connections 2024-03-13T14:56:28.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.582+0000 7f065b400700 1 -- 172.21.15.176:0/3098815032 wait complete. 2024-03-13T14:56:28.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.583+0000 7f065b400700 1 Processor -- start 2024-03-13T14:56:28.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.583+0000 7f065b400700 1 -- start start 2024-03-13T14:56:28.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f065b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0654103000 0x7f065419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f065b400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541039b0 0x7f065419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f065b400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065419f300 con 0x7f0654103000 2024-03-13T14:56:28.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f065b400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0654198840 con 0x7f06541039b0 2024-03-13T14:56:28.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f0658a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0654103000 0x7f065419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f0658a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0654103000 0x7f065419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60368/0 (socket says 172.21.15.176:60368) 2024-03-13T14:56:28.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f0658a00700 1 -- 172.21.15.176:0/2884268903 learned_addr learned my addr 172.21.15.176:0/2884268903 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:56:28.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f0653e00700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541039b0 0x7f065419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f0658a00700 1 -- 172.21.15.176:0/2884268903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541039b0 msgr2=0x7f065419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f0658a00700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541039b0 0x7f065419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.584+0000 7f0658a00700 1 -- 172.21.15.176:0/2884268903 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400095f0 con 0x7f0654103000 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.585+0000 7f0658a00700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0654103000 0x7f065419e7c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f064800c930 tx=0x7f064800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.585+0000 7f0651600700 1 -- 172.21.15.176:0/2884268903 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0648007ad0 con 0x7f0654103000 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.585+0000 7f0651600700 1 -- 172.21.15.176:0/2884268903 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0648007c30 con 0x7f0654103000 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.585+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0654198b20 con 0x7f0654103000 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.585+0000 7f0651600700 1 -- 172.21.15.176:0/2884268903 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0648018610 con 0x7f0654103000 2024-03-13T14:56:28.572 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.585+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0654199070 con 0x7f0654103000 2024-03-13T14:56:28.573 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.587+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065410b400 con 0x7f0654103000 2024-03-13T14:56:28.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.588+0000 7f0651600700 1 -- 172.21.15.176:0/2884268903 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f064800f450 con 0x7f0654103000 2024-03-13T14:56:28.575 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.588+0000 7f0651600700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f063c06dfa0 0x7f063c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.575 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.589+0000 7f0651600700 1 -- 172.21.15.176:0/2884268903 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f064808dff0 con 0x7f0654103000 2024-03-13T14:56:28.575 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.589+0000 7f0653e00700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f063c06dfa0 0x7f063c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.590+0000 7f0653e00700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f063c06dfa0 0x7f063c070460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06540ff0a0 tx=0x7f064001a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:28.579 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.594+0000 7f0651600700 1 -- 172.21.15.176:0/2884268903 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f064805af00 con 0x7f0654103000 2024-03-13T14:56:28.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.837+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06541992f0 con 0x7f063c06dfa0 2024-03-13T14:56:28.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.839+0000 7f0651600700 1 -- 172.21.15.176:0/2884268903 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f06541992f0 con 0x7f063c06dfa0 2024-03-13T14:56:28.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.842+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f063c06dfa0 msgr2=0x7f063c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.842+0000 7f065b400700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f063c06dfa0 0x7f063c070460 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06540ff0a0 tx=0x7f064001a040 comp rx=0 tx=0).stop 2024-03-13T14:56:28.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.842+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0654103000 msgr2=0x7f065419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.842+0000 7f065b400700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0654103000 0x7f065419e7c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f064800c930 tx=0x7f064800ccf0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.843+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 shutdown_connections 2024-03-13T14:56:28.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.843+0000 7f065b400700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f063c06dfa0 0x7f063c070460 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.843+0000 7f065b400700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0654103000 0x7f065419e7c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.843+0000 7f065b400700 1 --2- 172.21.15.176:0/2884268903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06541039b0 0x7f065419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.843+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 >> 172.21.15.176:0/2884268903 conn(0x7f06540fe8d0 msgr2=0x7f0654100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:28.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.843+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 shutdown_connections 2024-03-13T14:56:28.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.843+0000 7f065b400700 1 -- 172.21.15.176:0/2884268903 wait complete. 2024-03-13T14:56:28.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 -- 172.21.15.176:0/2571995127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 msgr2=0x7fb94c107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.981 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 --2- 172.21.15.176:0/2571995127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c107810 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb938009910 tx=0x7fb938009c20 comp rx=0 tx=0).stop 2024-03-13T14:56:28.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 -- 172.21.15.176:0/2571995127 shutdown_connections 2024-03-13T14:56:28.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 --2- 172.21.15.176:0/2571995127 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c107810 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 --2- 172.21.15.176:0/2571995127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb94c102e10 0x7fb94c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 -- 172.21.15.176:0/2571995127 >> 172.21.15.176:0/2571995127 conn(0x7fb94c0fe6c0 msgr2=0x7fb94c100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:28.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 -- 172.21.15.176:0/2571995127 shutdown_connections 2024-03-13T14:56:28.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.995+0000 7fb951a00700 1 -- 172.21.15.176:0/2571995127 wait complete. 2024-03-13T14:56:28.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.996+0000 7fb951a00700 1 Processor -- start 2024-03-13T14:56:28.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.996+0000 7fb951a00700 1 -- start start 2024-03-13T14:56:28.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.996+0000 7fb951a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb94c102e10 0x7fb94c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.996+0000 7fb951a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.996+0000 7fb951a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c19f260 con 0x7fb94c1037c0 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.996+0000 7fb951a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c198710 con 0x7fb94c102e10 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c19ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60384/0 (socket says 172.21.15.176:60384) 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94b400700 1 -- 172.21.15.176:0/2859703370 learned_addr learned my addr 172.21.15.176:0/2859703370 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94b400700 1 -- 172.21.15.176:0/2859703370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb94c102e10 msgr2=0x7fb94c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94be00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb94c102e10 0x7fb94c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94b400700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb94c102e10 0x7fb94c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94b400700 1 -- 172.21.15.176:0/2859703370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380095f0 con 0x7fb94c1037c0 2024-03-13T14:56:28.984 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.997+0000 7fb94b400700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c19ebd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb938005f50 tx=0x7fb93800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:28.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.998+0000 7fb948c00700 1 -- 172.21.15.176:0/2859703370 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb93801d070 con 0x7fb94c1037c0 2024-03-13T14:56:28.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.998+0000 7fb948c00700 1 -- 172.21.15.176:0/2859703370 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb93800f460 con 0x7fb94c1037c0 2024-03-13T14:56:28.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.998+0000 7fb948c00700 1 -- 172.21.15.176:0/2859703370 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb938017630 con 0x7fb94c1037c0 2024-03-13T14:56:28.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.998+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c198990 con 0x7fb94c1037c0 2024-03-13T14:56:28.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.998+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c198e80 con 0x7fb94c1037c0 2024-03-13T14:56:28.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:28.999+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c10b160 con 0x7fb94c1037c0 2024-03-13T14:56:28.986 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.000+0000 7fb948c00700 1 -- 172.21.15.176:0/2859703370 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb93800f5d0 con 0x7fb94c1037c0 2024-03-13T14:56:28.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.001+0000 7fb948c00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb93006dfa0 0x7fb930070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:28.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.001+0000 7fb948c00700 1 -- 172.21.15.176:0/2859703370 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fb93808f3c0 con 0x7fb94c1037c0 2024-03-13T14:56:28.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.001+0000 7fb94be00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb93006dfa0 0x7fb930070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:28.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.002+0000 7fb94be00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb93006dfa0 0x7fb930070460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb93c006fd0 tx=0x7fb93c008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:28.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.005+0000 7fb948c00700 1 -- 172.21.15.176:0/2859703370 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fb93805c290 con 0x7fb94c1037c0 2024-03-13T14:56:29.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.221+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb94c1991b0 con 0x7fb93006dfa0 2024-03-13T14:56:29.238 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.252+0000 7fb948c00700 1 -- 172.21.15.176:0/2859703370 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb94c1991b0 con 0x7fb93006dfa0 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (4m) 16s ago 17m 14.2M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (18m) 16s ago 18m 8342k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (16m) 49s ago 16m 8238k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (62s) 16s ago 18m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (54s) 49s ago 16m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (3m) 16s ago 17m 73.9M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (10m) 16s ago 10m 87.6M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (10m) 16s ago 10m 17.4M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (10m) 49s ago 10m 141M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (10m) 49s ago 10m 16.9M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:56:29.324 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (7m) 16s ago 19m 501M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (6m) 49s ago 16m 449M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (2m) 16s ago 19m 44.8M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (2m) 49s ago 16m 36.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (5m) 16s ago 18m 7658k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (5m) 49s ago 16m 7910k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (24s) 16s ago 15m 11.5M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (14m) 16s ago 14m 301M 1217M 18.2.1 5be31c24972a 2dbe414075db 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (14m) 16s ago 14m 235M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (13m) 49s ago 13m 306M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (13m) 49s ago 13m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (12m) 49s ago 12m 230M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:56:29.325 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (4m) 16s ago 17m 67.4M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.254+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb93006dfa0 msgr2=0x7fb930070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.254+0000 7fb951a00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb93006dfa0 0x7fb930070460 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb93c006fd0 tx=0x7fb93c008040 comp rx=0 tx=0).stop 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.254+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 msgr2=0x7fb94c19ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.254+0000 7fb951a00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c19ebd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb938005f50 tx=0x7fb93800be20 comp rx=0 tx=0).stop 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.255+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 shutdown_connections 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.255+0000 7fb951a00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb93006dfa0 0x7fb930070460 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.255+0000 7fb951a00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb94c102e10 0x7fb94c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.326 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.255+0000 7fb951a00700 1 --2- 172.21.15.176:0/2859703370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb94c1037c0 0x7fb94c19ebd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.255+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 >> 172.21.15.176:0/2859703370 conn(0x7fb94c0fe6c0 msgr2=0x7fb94c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:29.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.255+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 shutdown_connections 2024-03-13T14:56:29.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.255+0000 7fb951a00700 1 -- 172.21.15.176:0/2859703370 wait complete. 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 -- 172.21.15.176:0/787603245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440739d0 msgr2=0x7f4b4410cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 --2- 172.21.15.176:0/787603245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440739d0 0x7f4b4410cd40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009910 tx=0x7f4b30009c20 comp rx=0 tx=0).stop 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 -- 172.21.15.176:0/787603245 shutdown_connections 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 --2- 172.21.15.176:0/787603245 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440739d0 0x7f4b4410cd40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 --2- 172.21.15.176:0/787603245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440730b0 0x7f4b44073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 -- 172.21.15.176:0/787603245 >> 172.21.15.176:0/787603245 conn(0x7f4b440fc4d0 msgr2=0x7f4b440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 -- 172.21.15.176:0/787603245 shutdown_connections 2024-03-13T14:56:29.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.404+0000 7f4b4b000700 1 -- 172.21.15.176:0/787603245 wait complete. 2024-03-13T14:56:29.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.405+0000 7f4b4b000700 1 Processor -- start 2024-03-13T14:56:29.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.405+0000 7f4b4b000700 1 -- start start 2024-03-13T14:56:29.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b4b000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440730b0 0x7f4b4419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:29.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b4b000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440739d0 0x7f4b4419ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:29.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b4b000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4419f0f0 con 0x7f4b440730b0 2024-03-13T14:56:29.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b4b000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b441985a0 con 0x7f4b440739d0 2024-03-13T14:56:29.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b49c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440730b0 0x7f4b4419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:29.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b49c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440730b0 0x7f4b4419e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60404/0 (socket says 172.21.15.176:60404) 2024-03-13T14:56:29.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b49c00700 1 -- 172.21.15.176:0/882275274 learned_addr learned my addr 172.21.15.176:0/882275274 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b49200700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440739d0 0x7f4b4419ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b49c00700 1 -- 172.21.15.176:0/882275274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440739d0 msgr2=0x7f4b4419ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b49c00700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440739d0 0x7f4b4419ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.406+0000 7f4b49c00700 1 -- 172.21.15.176:0/882275274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300095f0 con 0x7f4b440730b0 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.407+0000 7f4b49200700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440739d0 0x7f4b4419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.407+0000 7f4b49c00700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440730b0 0x7f4b4419e520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d8d0 tx=0x7f4b3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.407+0000 7f4b42000700 1 -- 172.21.15.176:0/882275274 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b38009940 con 0x7f4b440730b0 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.407+0000 7f4b42000700 1 -- 172.21.15.176:0/882275274 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b38010460 con 0x7f4b440730b0 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.407+0000 7f4b42000700 1 -- 172.21.15.176:0/882275274 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b3800f5f0 con 0x7f4b440730b0 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.407+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b44198880 con 0x7f4b440730b0 2024-03-13T14:56:29.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.407+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b44198dd0 con 0x7f4b440730b0 2024-03-13T14:56:29.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.408+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4410a440 con 0x7f4b440730b0 2024-03-13T14:56:29.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.409+0000 7f4b42000700 1 -- 172.21.15.176:0/882275274 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4b380105d0 con 0x7f4b440730b0 2024-03-13T14:56:29.396 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.410+0000 7f4b42000700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b2c06dfa0 0x7f4b2c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:29.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.410+0000 7f4b42000700 1 -- 172.21.15.176:0/882275274 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f4b3808dd10 con 0x7f4b440730b0 2024-03-13T14:56:29.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.410+0000 7f4b49200700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b2c06dfa0 0x7f4b2c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:29.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.411+0000 7f4b49200700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b2c06dfa0 0x7f4b2c070460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b5c0 tx=0x7f4b30005960 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:29.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.415+0000 7f4b42000700 1 -- 172.21.15.176:0/882275274 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4b3805ac80 con 0x7f4b440730b0 2024-03-13T14:56:29.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:29 smithi176 ceph-mon[112742]: pgmap v98: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 137 objects/s recovering 2024-03-13T14:56:29.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:29 smithi176 ceph-mon[112742]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:29.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:29 smithi196 ceph-mon[86193]: pgmap v98: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 0 B/s, 137 objects/s recovering 2024-03-13T14:56:29.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:29 smithi196 ceph-mon[86193]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:29.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.707+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b441990c0 con 0x7f4b440730b0 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.708+0000 7f4b42000700 1 -- 172.21.15.176:0/882275274 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f4b3805a810 con 0x7f4b440730b0 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:56:29.694 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 1 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 9, 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 5 2024-03-13T14:56:29.695 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:56:29.696 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:56:29.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b2c06dfa0 msgr2=0x7f4b2c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b2c06dfa0 0x7f4b2c070460 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000b5c0 tx=0x7f4b30005960 comp rx=0 tx=0).stop 2024-03-13T14:56:29.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440730b0 msgr2=0x7f4b4419e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440730b0 0x7f4b4419e520 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800d8d0 tx=0x7f4b3800dc90 comp rx=0 tx=0).stop 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 shutdown_connections 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b2c06dfa0 0x7f4b2c070460 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b440730b0 0x7f4b4419e520 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 --2- 172.21.15.176:0/882275274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b440739d0 0x7f4b4419ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 >> 172.21.15.176:0/882275274 conn(0x7f4b440fc4d0 msgr2=0x7f4b44107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.711+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 shutdown_connections 2024-03-13T14:56:29.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.712+0000 7f4b4b000700 1 -- 172.21.15.176:0/882275274 wait complete. 2024-03-13T14:56:29.845 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 -- 172.21.15.176:0/4105631634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de81071d0 msgr2=0x7f5de8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 --2- 172.21.15.176:0/4105631634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de81071d0 0x7f5de8107650 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009910 tx=0x7f5dd4009c20 comp rx=0 tx=0).stop 2024-03-13T14:56:29.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 -- 172.21.15.176:0/4105631634 shutdown_connections 2024-03-13T14:56:29.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 --2- 172.21.15.176:0/4105631634 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de81071d0 0x7f5de8107650 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 --2- 172.21.15.176:0/4105631634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de810d280 0x7f5de810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 -- 172.21.15.176:0/4105631634 >> 172.21.15.176:0/4105631634 conn(0x7f5de80769e0 msgr2=0x7f5de8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:29.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 -- 172.21.15.176:0/4105631634 shutdown_connections 2024-03-13T14:56:29.846 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.859+0000 7f5decc00700 1 -- 172.21.15.176:0/4105631634 wait complete. 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.860+0000 7f5decc00700 1 Processor -- start 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.860+0000 7f5decc00700 1 -- start start 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5decc00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de81071d0 0x7f5de819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5decc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de810d280 0x7f5de819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5decc00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de819d880 con 0x7f5de810d280 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5decc00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de81a05b0 con 0x7f5de81071d0 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5de5400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de810d280 0x7f5de819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:29.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5de5400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de810d280 0x7f5de819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60422/0 (socket says 172.21.15.176:60422) 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5de5400700 1 -- 172.21.15.176:0/2911718904 learned_addr learned my addr 172.21.15.176:0/2911718904 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.861+0000 7f5de5e00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de81071d0 0x7f5de819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5de5400700 1 -- 172.21.15.176:0/2911718904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de81071d0 msgr2=0x7f5de819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5de5400700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de81071d0 0x7f5de819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5de5400700 1 -- 172.21.15.176:0/2911718904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40095f0 con 0x7f5de810d280 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5de5400700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de810d280 0x7f5de819d1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4006010 tx=0x7f5dd400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5ddea00700 1 -- 172.21.15.176:0/2911718904 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dd401d070 con 0x7f5de810d280 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5ddea00700 1 -- 172.21.15.176:0/2911718904 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dd4004fb0 con 0x7f5de810d280 2024-03-13T14:56:29.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5ddea00700 1 -- 172.21.15.176:0/2911718904 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dd400f690 con 0x7f5de810d280 2024-03-13T14:56:29.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.862+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de81a0830 con 0x7f5de810d280 2024-03-13T14:56:29.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.863+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de81a0da0 con 0x7f5de810d280 2024-03-13T14:56:29.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.864+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de810f6c0 con 0x7f5de810d280 2024-03-13T14:56:29.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.864+0000 7f5ddea00700 1 -- 172.21.15.176:0/2911718904 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5dd40228a0 con 0x7f5de810d280 2024-03-13T14:56:29.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.865+0000 7f5ddea00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5dcc06dfa0 0x7f5dcc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:29.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.865+0000 7f5ddea00700 1 -- 172.21.15.176:0/2911718904 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f5dd408f680 con 0x7f5de810d280 2024-03-13T14:56:29.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.866+0000 7f5de5e00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5dcc06dfa0 0x7f5dcc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:29.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.866+0000 7f5de5e00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5dcc06dfa0 0x7f5dcc070460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005fd0 tx=0x7f5dd800a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:29.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:29.870+0000 7f5ddea00700 1 -- 172.21.15.176:0/2911718904 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f5dd4058a20 con 0x7f5de810d280 2024-03-13T14:56:30.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.121+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5de81a1010 con 0x7f5de810d280 2024-03-13T14:56:30.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.122+0000 7f5ddea00700 1 -- 172.21.15.176:0/2911718904 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f5dd4027770 con 0x7f5de810d280 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:56:30.109 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:56:30.110 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:56:30.111 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5dcc06dfa0 msgr2=0x7f5dcc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5dcc06dfa0 0x7f5dcc070460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8005fd0 tx=0x7f5dd800a560 comp rx=0 tx=0).stop 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de810d280 msgr2=0x7f5de819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de810d280 0x7f5de819d1f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4006010 tx=0x7f5dd400be20 comp rx=0 tx=0).stop 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 shutdown_connections 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5dcc06dfa0 0x7f5dcc070460 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5de81071d0 0x7f5de819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 --2- 172.21.15.176:0/2911718904 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5de810d280 0x7f5de819d1f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.126+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 >> 172.21.15.176:0/2911718904 conn(0x7f5de80769e0 msgr2=0x7f5de8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:30.114 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.127+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 shutdown_connections 2024-03-13T14:56:30.114 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.127+0000 7f5decc00700 1 -- 172.21.15.176:0/2911718904 wait complete. 2024-03-13T14:56:30.114 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:56:30.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 -- 172.21.15.176:0/2498782020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c1039b0 msgr2=0x7f744c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 --2- 172.21.15.176:0/2498782020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c1039b0 0x7f744c107a00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7438009910 tx=0x7f7438009c20 comp rx=0 tx=0).stop 2024-03-13T14:56:30.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 -- 172.21.15.176:0/2498782020 shutdown_connections 2024-03-13T14:56:30.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 --2- 172.21.15.176:0/2498782020 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c1039b0 0x7f744c107a00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 --2- 172.21.15.176:0/2498782020 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c103000 0x7f744c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 -- 172.21.15.176:0/2498782020 >> 172.21.15.176:0/2498782020 conn(0x7f744c0fe8d0 msgr2=0x7f744c100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:30.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 -- 172.21.15.176:0/2498782020 shutdown_connections 2024-03-13T14:56:30.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.283+0000 7f7453200700 1 -- 172.21.15.176:0/2498782020 wait complete. 2024-03-13T14:56:30.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.285+0000 7f7453200700 1 Processor -- start 2024-03-13T14:56:30.271 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.285+0000 7f7453200700 1 -- start start 2024-03-13T14:56:30.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.285+0000 7f7453200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c103000 0x7f744c19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:30.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.285+0000 7f7453200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c1039b0 0x7f744c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:30.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.285+0000 7f7453200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c19f300 con 0x7f744c103000 2024-03-13T14:56:30.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.285+0000 7f7453200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c198840 con 0x7f744c1039b0 2024-03-13T14:56:30.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.286+0000 7f744be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c103000 0x7f744c19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:30.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.286+0000 7f744be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c103000 0x7f744c19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60448/0 (socket says 172.21.15.176:60448) 2024-03-13T14:56:30.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.286+0000 7f744be00700 1 -- 172.21.15.176:0/3734004746 learned_addr learned my addr 172.21.15.176:0/3734004746 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:56:30.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.286+0000 7f744be00700 1 -- 172.21.15.176:0/3734004746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c1039b0 msgr2=0x7f744c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.286+0000 7f744be00700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c1039b0 0x7f744c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.286+0000 7f744be00700 1 -- 172.21.15.176:0/3734004746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380095f0 con 0x7f744c103000 2024-03-13T14:56:30.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.286+0000 7f744be00700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c103000 0x7f744c19e7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c930 tx=0x7f743c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:30.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.287+0000 7f7448c00700 1 -- 172.21.15.176:0/3734004746 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743c007ad0 con 0x7f744c103000 2024-03-13T14:56:30.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.287+0000 7f7448c00700 1 -- 172.21.15.176:0/3734004746 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f743c007c30 con 0x7f744c103000 2024-03-13T14:56:30.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.287+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c198b20 con 0x7f744c103000 2024-03-13T14:56:30.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.287+0000 7f7448c00700 1 -- 172.21.15.176:0/3734004746 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743c018610 con 0x7f744c103000 2024-03-13T14:56:30.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.287+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c199070 con 0x7f744c103000 2024-03-13T14:56:30.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.288+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c10b400 con 0x7f744c103000 2024-03-13T14:56:30.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.289+0000 7f7448c00700 1 -- 172.21.15.176:0/3734004746 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f743c00f450 con 0x7f744c103000 2024-03-13T14:56:30.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.290+0000 7f7448c00700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f743006dfa0 0x7f7430070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:30.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.290+0000 7f7448c00700 1 -- 172.21.15.176:0/3734004746 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f743c08dff0 con 0x7f744c103000 2024-03-13T14:56:30.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.290+0000 7f744b400700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f743006dfa0 0x7f7430070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:30.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.291+0000 7f744b400700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f743006dfa0 0x7f7430070460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743800b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:30.281 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.295+0000 7f7448c00700 1 -- 172.21.15.176:0/3734004746 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f743c05af00 con 0x7f744c103000 2024-03-13T14:56:30.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.539+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f744c1992f0 con 0x7f743006dfa0 2024-03-13T14:56:30.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.541+0000 7f7448c00700 1 -- 172.21.15.176:0/3734004746 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f744c1992f0 con 0x7f743006dfa0 2024-03-13T14:56:30.527 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:56:30.527 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "7/23 daemons upgraded", 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading osd daemons", 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:56:30.528 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:56:30.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f743006dfa0 msgr2=0x7f7430070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f743006dfa0 0x7f7430070460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7438006010 tx=0x7f743800b540 comp rx=0 tx=0).stop 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c103000 msgr2=0x7f744c19e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c103000 0x7f744c19e7c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f743c00c930 tx=0x7f743c00ccf0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 shutdown_connections 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f743006dfa0 0x7f7430070460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f744c103000 0x7f744c19e7c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 --2- 172.21.15.176:0/3734004746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f744c1039b0 0x7f744c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 >> 172.21.15.176:0/3734004746 conn(0x7f744c0fe8d0 msgr2=0x7f744c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 shutdown_connections 2024-03-13T14:56:30.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.544+0000 7f7453200700 1 -- 172.21.15.176:0/3734004746 wait complete. 2024-03-13T14:56:30.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:30 smithi196 ceph-mon[86193]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:30.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:30 smithi196 ceph-mon[86193]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:30.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:30 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/882275274' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:30.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:30 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2911718904' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:56:30.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:30.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:30.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:30.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.694+0000 7f594a800700 1 -- 172.21.15.176:0/2170925889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944068670 msgr2=0x7f5944068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.694+0000 7f594a800700 1 --2- 172.21.15.176:0/2170925889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944068670 0x7f5944068a50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5934009910 tx=0x7f5934009c20 comp rx=0 tx=0).stop 2024-03-13T14:56:30.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.694+0000 7f594a800700 1 -- 172.21.15.176:0/2170925889 shutdown_connections 2024-03-13T14:56:30.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.694+0000 7f594a800700 1 --2- 172.21.15.176:0/2170925889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5944069020 0x7f5944105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.694+0000 7f594a800700 1 --2- 172.21.15.176:0/2170925889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944068670 0x7f5944068a50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.694+0000 7f594a800700 1 -- 172.21.15.176:0/2170925889 >> 172.21.15.176:0/2170925889 conn(0x7f59440755a0 msgr2=0x7f59440759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:30.681 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:30 smithi176 ceph-mon[112742]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:30.681 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:30 smithi176 ceph-mon[112742]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:30.681 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:30 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/882275274' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:56:30.682 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:30 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2911718904' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:56:30.682 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:30.682 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:30.682 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:30.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.695+0000 7f594a800700 1 -- 172.21.15.176:0/2170925889 shutdown_connections 2024-03-13T14:56:30.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.695+0000 7f594a800700 1 -- 172.21.15.176:0/2170925889 wait complete. 2024-03-13T14:56:30.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.696+0000 7f594a800700 1 Processor -- start 2024-03-13T14:56:30.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.696+0000 7f594a800700 1 -- start start 2024-03-13T14:56:30.683 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.696+0000 7f594a800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5944068670 0x7f594419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.696+0000 7f594a800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944069020 0x7f594419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.696+0000 7f594a800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594419d290 con 0x7f5944069020 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.696+0000 7f594a800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5944196740 con 0x7f5944068670 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5942a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944069020 0x7f594419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5942a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944069020 0x7f594419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60456/0 (socket says 172.21.15.176:60456) 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5942a00700 1 -- 172.21.15.176:0/1090331987 learned_addr learned my addr 172.21.15.176:0/1090331987 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5943400700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5944068670 0x7f594419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5942a00700 1 -- 172.21.15.176:0/1090331987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5944068670 msgr2=0x7f594419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5942a00700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5944068670 0x7f594419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.684 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5942a00700 1 -- 172.21.15.176:0/1090331987 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59340095f0 con 0x7f5944069020 2024-03-13T14:56:30.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.697+0000 7f5942a00700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944069020 0x7f594419cc00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f593000b480 tx=0x7f593000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:30.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.698+0000 7f593be00700 1 -- 172.21.15.176:0/1090331987 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59300118c0 con 0x7f5944069020 2024-03-13T14:56:30.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.698+0000 7f593be00700 1 -- 172.21.15.176:0/1090331987 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5930010380 con 0x7f5944069020 2024-03-13T14:56:30.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.698+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5944196a20 con 0x7f5944069020 2024-03-13T14:56:30.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.698+0000 7f593be00700 1 -- 172.21.15.176:0/1090331987 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f593000f4f0 con 0x7f5944069020 2024-03-13T14:56:30.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.698+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5944196f70 con 0x7f5944069020 2024-03-13T14:56:30.686 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.699+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944109130 con 0x7f5944069020 2024-03-13T14:56:30.686 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.700+0000 7f593be00700 1 -- 172.21.15.176:0/1090331987 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f59300104f0 con 0x7f5944069020 2024-03-13T14:56:30.687 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.701+0000 7f593be00700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f592806dfa0 0x7f5928070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:56:30.687 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.701+0000 7f593be00700 1 -- 172.21.15.176:0/1090331987 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f593008d9a0 con 0x7f5944069020 2024-03-13T14:56:30.687 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.701+0000 7f5943400700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f592806dfa0 0x7f5928070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:56:30.688 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.702+0000 7f5943400700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f592806dfa0 0x7f5928070460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5934006010 tx=0x7f593401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:56:30.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.706+0000 7f593be00700 1 -- 172.21.15.176:0/1090331987 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f593005a910 con 0x7f5944069020 2024-03-13T14:56:30.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.992+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f594404ea90 con 0x7f5944069020 2024-03-13T14:56:30.979 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.993+0000 7f593be00700 1 -- 172.21.15.176:0/1090331987 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f593005a4a0 con 0x7f5944069020 2024-03-13T14:56:30.979 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:56:30.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.995+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f592806dfa0 msgr2=0x7f5928070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.982 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.995+0000 7f594a800700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f592806dfa0 0x7f5928070460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5934006010 tx=0x7f593401a040 comp rx=0 tx=0).stop 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.995+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944069020 msgr2=0x7f594419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.995+0000 7f594a800700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944069020 0x7f594419cc00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f593000b480 tx=0x7f593000b840 comp rx=0 tx=0).stop 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.996+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 shutdown_connections 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.996+0000 7f594a800700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f592806dfa0 0x7f5928070460 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.996+0000 7f594a800700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5944068670 0x7f594419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.996+0000 7f594a800700 1 --2- 172.21.15.176:0/1090331987 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5944069020 0x7f594419cc00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.996+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 >> 172.21.15.176:0/1090331987 conn(0x7f59440755a0 msgr2=0x7f59440fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.996+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 shutdown_connections 2024-03-13T14:56:30.983 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:56:30.996+0000 7f594a800700 1 -- 172.21.15.176:0/1090331987 wait complete. 2024-03-13T14:56:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:31 smithi196 ceph-mon[86193]: pgmap v99: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 0 B/s, 116 objects/s recovering 2024-03-13T14:56:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:31 smithi196 ceph-mon[86193]: from='client.34264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:31 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1090331987' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:56:31.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:31 smithi176 ceph-mon[112742]: pgmap v99: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 0 B/s, 116 objects/s recovering 2024-03-13T14:56:31.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:31 smithi176 ceph-mon[112742]: from='client.34264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:56:31.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:31 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1090331987' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:56:33.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:33 smithi196 ceph-mon[86193]: pgmap v100: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 101 objects/s recovering 2024-03-13T14:56:33.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:33 smithi176 ceph-mon[112742]: pgmap v100: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 0 B/s, 101 objects/s recovering 2024-03-13T14:56:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:35 smithi196 ceph-mon[86193]: pgmap v101: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:56:35.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:35 smithi176 ceph-mon[112742]: pgmap v101: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:56:37.007 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:37.007 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:36 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:37.007 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:36 smithi176 ceph-mon[112742]: Upgrade: osd.1 is safe to restart 2024-03-13T14:56:37.007 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:36 smithi176 ceph-mon[112742]: pgmap v102: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:56:37.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:37.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:36 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-03-13T14:56:37.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:36 smithi196 ceph-mon[86193]: Upgrade: osd.1 is safe to restart 2024-03-13T14:56:37.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:36 smithi196 ceph-mon[86193]: pgmap v102: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:56:39.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:39 smithi176 ceph-mon[112742]: Upgrade: Updating osd.1 2024-03-13T14:56:39.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:39.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-13T14:56:39.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:56:39.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:39 smithi176 ceph-mon[112742]: Deploying daemon osd.1 on smithi176 2024-03-13T14:56:39.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:39 smithi176 ceph-mon[112742]: pgmap v103: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:56:39.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:39 smithi196 ceph-mon[86193]: Upgrade: Updating osd.1 2024-03-13T14:56:39.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:39.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-13T14:56:39.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:56:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:39 smithi196 ceph-mon[86193]: Deploying daemon osd.1 on smithi176 2024-03-13T14:56:39.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:39 smithi196 ceph-mon[86193]: pgmap v103: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:56:41.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:41 smithi176 ceph-mon[112742]: pgmap v104: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:56:41.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:41 smithi196 ceph-mon[86193]: pgmap v104: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:56:43.377 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:42 smithi176 systemd[1]: Stopping Ceph osd.1 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:56:43.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:43 smithi196 ceph-mon[86193]: pgmap v105: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:56:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:43 smithi176 ceph-mon[112742]: pgmap v105: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:56:43.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:43 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[54924]: 2024-03-13T14:56:43.404+0000 7f742e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:56:43.878 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:43 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[54924]: 2024-03-13T14:56:43.404+0000 7f742e000700 -1 osd.1 71 *** Got signal Terminated *** 2024-03-13T14:56:43.878 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:43 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[54924]: 2024-03-13T14:56:43.404+0000 7f742e000700 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T14:56:44.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:44 smithi196 ceph-mon[86193]: osd.1 marked itself down and dead 2024-03-13T14:56:44.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:44 smithi176 ceph-mon[112742]: osd.1 marked itself down and dead 2024-03-13T14:56:45.180 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:44 smithi176 podman[130015]: 2024-03-13 14:56:44.905329525 +0000 UTC m=+1.644254829 container died 2dbe414075db4d3da169d0b7a34541aa0286b806c0516705d2b4636d4d98fbd3 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-13T14:56:45.582 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:45 smithi176 ceph-mon[112742]: pgmap v106: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:56:45.582 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:45 smithi176 ceph-mon[112742]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:56:45.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:45 smithi176 ceph-mon[112742]: osdmap e72: 6 total, 5 up, 6 in 2024-03-13T14:56:45.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:45.583 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:45.583 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:45 smithi176 podman[130015]: 2024-03-13 14:56:45.393870473 +0000 UTC m=+2.132795790 container cleanup 2dbe414075db4d3da169d0b7a34541aa0286b806c0516705d2b4636d4d98fbd3 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, org.label-schema.build-date=20240222, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-13T14:56:45.583 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:45 smithi176 bash[130015]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1 2024-03-13T14:56:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:45 smithi196 ceph-mon[86193]: pgmap v106: 65 pgs: 65 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:56:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:45 smithi196 ceph-mon[86193]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:56:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:45 smithi196 ceph-mon[86193]: osdmap e72: 6 total, 5 up, 6 in 2024-03-13T14:56:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:56:45.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:45 smithi176 podman[130034]: 2024-03-13 14:56:45.595455012 +0000 UTC m=+0.678719743 container remove 2dbe414075db4d3da169d0b7a34541aa0286b806c0516705d2b4636d4d98fbd3 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD) 2024-03-13T14:56:46.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:46 smithi196 ceph-mon[86193]: osdmap e73: 6 total, 5 up, 6 in 2024-03-13T14:56:46.703 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:46 smithi176 ceph-mon[112742]: osdmap e73: 6 total, 5 up, 6 in 2024-03-13T14:56:46.704 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:46 smithi176 podman[130121]: 2024-03-13 14:56:46.298604171 +0000 UTC m=+0.034534856 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:56:46.704 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:46 smithi176 podman[130121]: 2024-03-13 14:56:46.421732248 +0000 UTC m=+0.157662969 container create 050e2703d82a661be881e6a8a25eb6037bae0eb17bb3cafc012a894cd4ad608a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-13T14:56:46.704 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:46 smithi176 podman[130121]: 2024-03-13 14:56:46.717510507 +0000 UTC m=+0.453441238 container init 050e2703d82a661be881e6a8a25eb6037bae0eb17bb3cafc012a894cd4ad608a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-13T14:56:47.047 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:46 smithi176 podman[130121]: 2024-03-13 14:56:46.72882599 +0000 UTC m=+0.464756724 container start 050e2703d82a661be881e6a8a25eb6037bae0eb17bb3cafc012a894cd4ad608a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T14:56:47.047 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:46 smithi176 podman[130121]: 2024-03-13 14:56:46.841236613 +0000 UTC m=+0.577167354 container attach 050e2703d82a661be881e6a8a25eb6037bae0eb17bb3cafc012a894cd4ad608a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-13T14:56:47.377 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:47 smithi176 podman[130121]: 2024-03-13 14:56:47.170295431 +0000 UTC m=+0.906226159 container died 050e2703d82a661be881e6a8a25eb6037bae0eb17bb3cafc012a894cd4ad608a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-13T14:56:47.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:47 smithi196 ceph-mon[86193]: pgmap v109: 65 pgs: 18 active+undersized, 11 active+undersized+degraded, 3 stale+active+clean, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 27/225 objects degraded (12.000%) 2024-03-13T14:56:47.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:47 smithi196 ceph-mon[86193]: Health check failed: Degraded data redundancy: 27/225 objects degraded (12.000%), 11 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:47.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:47 smithi176 ceph-mon[112742]: pgmap v109: 65 pgs: 18 active+undersized, 11 active+undersized+degraded, 3 stale+active+clean, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 27/225 objects degraded (12.000%) 2024-03-13T14:56:47.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:47 smithi176 ceph-mon[112742]: Health check failed: Degraded data redundancy: 27/225 objects degraded (12.000%), 11 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:48.377 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:48 smithi176 podman[130146]: 2024-03-13 14:56:48.053748722 +0000 UTC m=+0.866137795 container remove 050e2703d82a661be881e6a8a25eb6037bae0eb17bb3cafc012a894cd4ad608a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-13T14:56:48.378 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:48 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.1.service: Deactivated successfully. 2024-03-13T14:56:48.378 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:48 smithi176 systemd[1]: Stopped Ceph osd.1 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:56:48.378 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:48 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.1.service: Consumed 2min 11.857s CPU time. 2024-03-13T14:56:48.798 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:48 smithi176 systemd[1]: Starting Ceph osd.1 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:56:49.127 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:48 smithi176 podman[130247]: 2024-03-13 14:56:48.720217898 +0000 UTC m=+0.036804563 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:56:49.127 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:48 smithi176 podman[130247]: 2024-03-13 14:56:48.837894429 +0000 UTC m=+0.154481112 container create b9f2d8bd995f0d64b0a43fece5b2151ef1e6238bc683840d0dc72040b597d7e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:56:49.430 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:49 smithi176 podman[130247]: 2024-03-13 14:56:49.142760694 +0000 UTC m=+0.459347424 container init b9f2d8bd995f0d64b0a43fece5b2151ef1e6238bc683840d0dc72040b597d7e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-13T14:56:49.430 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:49 smithi176 podman[130247]: 2024-03-13 14:56:49.154224774 +0000 UTC m=+0.470811480 container start b9f2d8bd995f0d64b0a43fece5b2151ef1e6238bc683840d0dc72040b597d7e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:56:49.430 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:49 smithi176 podman[130247]: 2024-03-13 14:56:49.240704473 +0000 UTC m=+0.557291189 container attach b9f2d8bd995f0d64b0a43fece5b2151ef1e6238bc683840d0dc72040b597d7e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, ceph=True) 2024-03-13T14:56:49.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:49 smithi176 ceph-mon[112742]: pgmap v110: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:49 smithi196 ceph-mon[86193]: pgmap v110: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:50.338 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate[130262]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-13T14:56:50.338 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate[130262]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 bash[130247]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 bash[130247]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 bash[130247]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 bash[130247]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 bash[130247]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 bash[130247]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 bash[130247]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate[130262]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate[130262]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate[130262]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate[130262]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate[130262]: --> ceph-volume raw activate successful for osd ID: 1 2024-03-13T14:56:50.339 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 podman[130247]: 2024-03-13 14:56:50.064643519 +0000 UTC m=+1.381230208 container died b9f2d8bd995f0d64b0a43fece5b2151ef1e6238bc683840d0dc72040b597d7e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-13T14:56:51.120 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:50 smithi176 podman[130402]: 2024-03-13 14:56:50.777792986 +0000 UTC m=+0.693763364 container remove b9f2d8bd995f0d64b0a43fece5b2151ef1e6238bc683840d0dc72040b597d7e4 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-activate, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-13T14:56:51.377 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:51 smithi176 podman[130465]: 2024-03-13 14:56:51.041534366 +0000 UTC m=+0.039610992 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:56:51.377 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:51 smithi176 podman[130465]: 2024-03-13 14:56:51.203744224 +0000 UTC m=+0.201820875 container create 6ac5d28d6884134ff7a1e40b9b59de6e9e096f7766591e2eb45809493f08a79e (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-03-13T14:56:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:51 smithi176 ceph-mon[112742]: pgmap v111: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:51.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:51 smithi176 podman[130465]: 2024-03-13 14:56:51.483137387 +0000 UTC m=+0.481214078 container init 6ac5d28d6884134ff7a1e40b9b59de6e9e096f7766591e2eb45809493f08a79e (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T14:56:51.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:51 smithi176 podman[130465]: 2024-03-13 14:56:51.494204816 +0000 UTC m=+0.492281524 container start 6ac5d28d6884134ff7a1e40b9b59de6e9e096f7766591e2eb45809493f08a79e (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-13T14:56:51.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:51 smithi176 ceph-osd[130484]: -- 172.21.15.176:0/1866595970 <== mon.0 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d082879680 con 0x55d082842c00 2024-03-13T14:56:51.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:51 smithi176 bash[130465]: 6ac5d28d6884134ff7a1e40b9b59de6e9e096f7766591e2eb45809493f08a79e 2024-03-13T14:56:51.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:51 smithi176 systemd[1]: Started Ceph osd.1 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:56:51.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:51 smithi196 ceph-mon[86193]: pgmap v111: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:52.376 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:52 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[130480]: 2024-03-13T14:56:52.065+0000 7f4e9dd3f7c0 -1 Falling back to public interface 2024-03-13T14:56:53.014 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:53.014 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:53.014 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:56:53.015 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:52 smithi176 ceph-mon[112742]: pgmap v112: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:53.074 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:53.074 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:53.074 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:56:53.075 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:52 smithi196 ceph-mon[86193]: pgmap v112: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:55.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:55 smithi196 ceph-mon[86193]: pgmap v113: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:55.861 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:55 smithi176 ceph-mon[112742]: pgmap v113: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:56.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:56 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 28/225 objects degraded (12.444%), 12 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:56.719 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:56 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 28/225 objects degraded (12.444%), 12 pgs degraded (PG_DEGRADED) 2024-03-13T14:56:57.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:57 smithi176 ceph-mon[112742]: pgmap v114: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:57.627 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:57 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[130480]: 2024-03-13T14:56:57.251+0000 7f4e9dd3f7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-03-13T14:56:57.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:57 smithi196 ceph-mon[86193]: pgmap v114: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:58.127 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:57 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[130480]: 2024-03-13T14:56:57.853+0000 7f4e9dd3f7c0 -1 osd.1 71 log_to_monitors true 2024-03-13T14:56:58.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:58 smithi196 ceph-mon[86193]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-13T14:56:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:58 smithi176 ceph-mon[112742]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-13T14:56:59.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:59 smithi196 ceph-mon[86193]: pgmap v115: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:59.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:59 smithi196 ceph-mon[86193]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-13T14:56:59.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:59 smithi196 ceph-mon[86193]: osdmap e74: 6 total, 5 up, 6 in 2024-03-13T14:56:59.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:59 smithi196 ceph-mon[86193]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:56:59.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:59 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:59.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:56:59 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:59.877 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 14:56:59 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[130480]: 2024-03-13T14:56:59.481+0000 7f4e88000700 -1 osd.1 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T14:56:59.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:59 smithi176 ceph-mon[112742]: pgmap v115: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:56:59.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:59 smithi176 ceph-mon[112742]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-13T14:56:59.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:59 smithi176 ceph-mon[112742]: osdmap e74: 6 total, 5 up, 6 in 2024-03-13T14:56:59.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:59 smithi176 ceph-mon[112742]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:56:59.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:59 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:56:59.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:56:59 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:00.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:00 smithi176 ceph-mon[112742]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' 2024-03-13T14:57:00.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:00.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:00 smithi196 ceph-mon[86193]: from='osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375]' entity='osd.1' 2024-03-13T14:57:00.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:00.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 -- 172.21.15.176:0/326096792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04100da0 msgr2=0x7f8f04101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 --2- 172.21.15.176:0/326096792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04100da0 0x7f8f04101180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009910 tx=0x7f8ef8009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 -- 172.21.15.176:0/326096792 shutdown_connections 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 --2- 172.21.15.176:0/326096792 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f04101750 0x7f8f041057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 --2- 172.21.15.176:0/326096792 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04100da0 0x7f8f04101180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 -- 172.21.15.176:0/326096792 >> 172.21.15.176:0/326096792 conn(0x7f8f040fc630 msgr2=0x7f8f040fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 -- 172.21.15.176:0/326096792 shutdown_connections 2024-03-13T14:57:01.150 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.163+0000 7f8f0b800700 1 -- 172.21.15.176:0/326096792 wait complete. 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.164+0000 7f8f0b800700 1 Processor -- start 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f0b800700 1 -- start start 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f0b800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f04100da0 0x7f8f0419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f0b800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04101750 0x7f8f0419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f0b800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0419f330 con 0x7f8f04101750 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f0b800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f041988c0 con 0x7f8f04100da0 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f03e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04101750 0x7f8f0419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f03e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04101750 0x7f8f0419ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:52622/0 (socket says 172.21.15.176:52622) 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.165+0000 7f8f03e00700 1 -- 172.21.15.176:0/64152892 learned_addr learned my addr 172.21.15.176:0/64152892 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:01.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f08e00700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f04100da0 0x7f8f0419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f03e00700 1 -- 172.21.15.176:0/64152892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f04100da0 msgr2=0x7f8f0419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f03e00700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f04100da0 0x7f8f0419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f03e00700 1 -- 172.21.15.176:0/64152892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80095f0 con 0x7f8f04101750 2024-03-13T14:57:01.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f03e00700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04101750 0x7f8f0419ed10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b480 tx=0x7f8ef000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f01600700 1 -- 172.21.15.176:0/64152892 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ef00118c0 con 0x7f8f04101750 2024-03-13T14:57:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f01600700 1 -- 172.21.15.176:0/64152892 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ef0010380 con 0x7f8f04101750 2024-03-13T14:57:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f01600700 1 -- 172.21.15.176:0/64152892 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ef000f4f0 con 0x7f8f04101750 2024-03-13T14:57:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.166+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f04198ba0 con 0x7f8f04101750 2024-03-13T14:57:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.167+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f041990f0 con 0x7f8f04101750 2024-03-13T14:57:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.168+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f04104f20 con 0x7f8f04101750 2024-03-13T14:57:01.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.169+0000 7f8f01600700 1 -- 172.21.15.176:0/64152892 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8ef00104f0 con 0x7f8f04101750 2024-03-13T14:57:01.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.170+0000 7f8f01600700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f8eec06dfa0 0x7f8eec070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:01.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.170+0000 7f8f01600700 1 -- 172.21.15.176:0/64152892 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f8ef008d9a0 con 0x7f8f04101750 2024-03-13T14:57:01.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.170+0000 7f8f08e00700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f8eec06dfa0 0x7f8eec070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:01.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.171+0000 7f8f08e00700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f8eec06dfa0 0x7f8eec070460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006010 tx=0x7f8ef801a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:01.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.175+0000 7f8f01600700 1 -- 172.21.15.176:0/64152892 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f8ef005a910 con 0x7f8f04101750 2024-03-13T14:57:01.405 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.418+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f041994e0 con 0x7f8eec06dfa0 2024-03-13T14:57:01.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.421+0000 7f8f01600700 1 -- 172.21.15.176:0/64152892 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8f041994e0 con 0x7f8eec06dfa0 2024-03-13T14:57:01.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f8eec06dfa0 msgr2=0x7f8eec070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f8eec06dfa0 0x7f8eec070460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8006010 tx=0x7f8ef801a040 comp rx=0 tx=0).stop 2024-03-13T14:57:01.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04101750 msgr2=0x7f8f0419ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04101750 0x7f8f0419ed10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8ef000b480 tx=0x7f8ef000b840 comp rx=0 tx=0).stop 2024-03-13T14:57:01.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 shutdown_connections 2024-03-13T14:57:01.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f8eec06dfa0 0x7f8eec070460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f04100da0 0x7f8f0419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 --2- 172.21.15.176:0/64152892 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f8f04101750 0x7f8f0419ed10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 >> 172.21.15.176:0/64152892 conn(0x7f8f040fc630 msgr2=0x7f8f040fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:01.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 shutdown_connections 2024-03-13T14:57:01.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.424+0000 7f8f0b800700 1 -- 172.21.15.176:0/64152892 wait complete. 2024-03-13T14:57:01.427 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:57:01.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 -- 172.21.15.176:0/3322104647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 msgr2=0x7f062010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 --2- 172.21.15.176:0/3322104647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f062010ce40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f060c009910 tx=0x7f060c009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 -- 172.21.15.176:0/3322104647 shutdown_connections 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 --2- 172.21.15.176:0/3322104647 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f062010ce40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 --2- 172.21.15.176:0/3322104647 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06200731c0 0x7f06200735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 -- 172.21.15.176:0/3322104647 >> 172.21.15.176:0/3322104647 conn(0x7f06200fc630 msgr2=0x7f06200fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 -- 172.21.15.176:0/3322104647 shutdown_connections 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.598+0000 7f0625600700 1 -- 172.21.15.176:0/3322104647 wait complete. 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.599+0000 7f0625600700 1 Processor -- start 2024-03-13T14:57:01.585 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f0625600700 1 -- start start 2024-03-13T14:57:01.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f0625600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06200731c0 0x7f0620198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:01.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f0625600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f0620198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:01.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f0625600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0620199530 con 0x7f0620073ae0 2024-03-13T14:57:01.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f0625600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062019d340 con 0x7f06200731c0 2024-03-13T14:57:01.586 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f061e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f0620198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:01.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f061e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f0620198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:52644/0 (socket says 172.21.15.176:52644) 2024-03-13T14:57:01.587 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.600+0000 7f061e000700 1 -- 172.21.15.176:0/3545155097 learned_addr learned my addr 172.21.15.176:0/3545155097 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f061ea00700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06200731c0 0x7f0620198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f061e000700 1 -- 172.21.15.176:0/3545155097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06200731c0 msgr2=0x7f0620198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f061e000700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06200731c0 0x7f0620198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f061e000700 1 -- 172.21.15.176:0/3545155097 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0095f0 con 0x7f0620073ae0 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f061e000700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f0620198ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f060c0092e0 tx=0x7f060c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f0617400700 1 -- 172.21.15.176:0/3545155097 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c01d070 con 0x7f0620073ae0 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062019d5c0 con 0x7f0620073ae0 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f0617400700 1 -- 172.21.15.176:0/3545155097 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f060c00f460 con 0x7f0620073ae0 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.601+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062019dab0 con 0x7f0620073ae0 2024-03-13T14:57:01.588 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.602+0000 7f0617400700 1 -- 172.21.15.176:0/3545155097 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c005e20 con 0x7f0620073ae0 2024-03-13T14:57:01.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.603+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062010a540 con 0x7f0620073ae0 2024-03-13T14:57:01.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.604+0000 7f0617400700 1 -- 172.21.15.176:0/3545155097 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f060c00f5d0 con 0x7f0620073ae0 2024-03-13T14:57:01.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.605+0000 7f0617400700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f060406dfa0 0x7f0604070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:01.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.605+0000 7f0617400700 1 -- 172.21.15.176:0/3545155097 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f060c08f3d0 con 0x7f0620073ae0 2024-03-13T14:57:01.591 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.605+0000 7f061ea00700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f060406dfa0 0x7f0604070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:01.592 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.606+0000 7f061ea00700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f060406dfa0 0x7f0604070460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0610005950 tx=0x7f06100058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:01.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.610+0000 7f0617400700 1 -- 172.21.15.176:0/3545155097 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f060c05c2e0 con 0x7f0620073ae0 2024-03-13T14:57:01.829 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:01 smithi196 ceph-mon[86193]: pgmap v117: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:57:01.829 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:01 smithi196 ceph-mon[86193]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:57:01.829 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:01 smithi196 ceph-mon[86193]: osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375] boot 2024-03-13T14:57:01.829 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:01 smithi196 ceph-mon[86193]: osdmap e75: 6 total, 6 up, 6 in 2024-03-13T14:57:01.829 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:57:01.830 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:01.830 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:01.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.860+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f062019dde0 con 0x7f060406dfa0 2024-03-13T14:57:01.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:01 smithi176 ceph-mon[112742]: pgmap v117: 65 pgs: 20 active+undersized, 12 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 28/225 objects degraded (12.444%) 2024-03-13T14:57:01.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:01 smithi176 ceph-mon[112742]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:57:01.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:01 smithi176 ceph-mon[112742]: osd.1 [v2:172.21.15.176:6810/2654405375,v1:172.21.15.176:6811/2654405375] boot 2024-03-13T14:57:01.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:01 smithi176 ceph-mon[112742]: osdmap e75: 6 total, 6 up, 6 in 2024-03-13T14:57:01.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-13T14:57:01.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:01.847 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:01.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.863+0000 7f0617400700 1 -- 172.21.15.176:0/3545155097 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f062019dde0 con 0x7f060406dfa0 2024-03-13T14:57:01.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f060406dfa0 msgr2=0x7f0604070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f060406dfa0 0x7f0604070460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0610005950 tx=0x7f06100058e0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 msgr2=0x7f0620198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:01.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f0620198ea0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f060c0092e0 tx=0x7f060c00be20 comp rx=0 tx=0).stop 2024-03-13T14:57:01.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 shutdown_connections 2024-03-13T14:57:01.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f060406dfa0 0x7f0604070460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f06200731c0 0x7f0620198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 --2- 172.21.15.176:0/3545155097 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0620073ae0 0x7f0620198ea0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:01.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 >> 172.21.15.176:0/3545155097 conn(0x7f06200fc630 msgr2=0x7f0620107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:01.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 shutdown_connections 2024-03-13T14:57:01.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:01.866+0000 7f0625600700 1 -- 172.21.15.176:0/3545155097 wait complete. 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 -- 172.21.15.176:0/2440425080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c100970 msgr2=0x7f455c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 --2- 172.21.15.176:0/2440425080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c100970 0x7f455c100df0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4544009910 tx=0x7f4544009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 -- 172.21.15.176:0/2440425080 shutdown_connections 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 --2- 172.21.15.176:0/2440425080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c100970 0x7f455c100df0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 --2- 172.21.15.176:0/2440425080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c106970 0x7f455c106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 -- 172.21.15.176:0/2440425080 >> 172.21.15.176:0/2440425080 conn(0x7f455c0fc440 msgr2=0x7f455c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 -- 172.21.15.176:0/2440425080 shutdown_connections 2024-03-13T14:57:02.007 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.020+0000 7f455be00700 1 -- 172.21.15.176:0/2440425080 wait complete. 2024-03-13T14:57:02.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.021+0000 7f455be00700 1 Processor -- start 2024-03-13T14:57:02.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f455be00700 1 -- start start 2024-03-13T14:57:02.009 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f455be00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c100970 0x7f455c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f455be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c106970 0x7f455c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f455be00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c073b10 con 0x7f455c106970 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f455be00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c073c50 con 0x7f455c100970 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f4552000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c106970 0x7f455c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f4552000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c106970 0x7f455c0735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59130/0 (socket says 172.21.15.176:59130) 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.022+0000 7f4552000700 1 -- 172.21.15.176:0/3863271284 learned_addr learned my addr 172.21.15.176:0/3863271284 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f455aa00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c100970 0x7f455c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f4552000700 1 -- 172.21.15.176:0/3863271284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c100970 msgr2=0x7f455c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f4552000700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c100970 0x7f455c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f4552000700 1 -- 172.21.15.176:0/3863271284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440095f0 con 0x7f455c106970 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f4552000700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c106970 0x7f455c0735d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45440092e0 tx=0x7f454400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f4553e00700 1 -- 172.21.15.176:0/3863271284 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f454401d070 con 0x7f455c106970 2024-03-13T14:57:02.010 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f4553e00700 1 -- 172.21.15.176:0/3863271284 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4544005e00 con 0x7f455c106970 2024-03-13T14:57:02.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f4553e00700 1 -- 172.21.15.176:0/3863271284 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4544017400 con 0x7f455c106970 2024-03-13T14:57:02.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.023+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c073ed0 con 0x7f455c106970 2024-03-13T14:57:02.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.024+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c1a2a50 con 0x7f455c106970 2024-03-13T14:57:02.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.024+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455c108f00 con 0x7f455c106970 2024-03-13T14:57:02.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.025+0000 7f4553e00700 1 -- 172.21.15.176:0/3863271284 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f454400f460 con 0x7f455c106970 2024-03-13T14:57:02.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.026+0000 7f4553e00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f454006dfa0 0x7f4540070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.026+0000 7f4553e00700 1 -- 172.21.15.176:0/3863271284 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f454408f2a0 con 0x7f455c106970 2024-03-13T14:57:02.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.027+0000 7f455aa00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f454006dfa0 0x7f4540070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.027+0000 7f455aa00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f454006dfa0 0x7f4540070460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f454c005950 tx=0x7f454c0058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:02.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.031+0000 7f4553e00700 1 -- 172.21.15.176:0/3863271284 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f45440586e0 con 0x7f455c106970 2024-03-13T14:57:02.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.263+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f455c02cd50 con 0x7f454006dfa0 2024-03-13T14:57:02.261 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.275+0000 7f4553e00700 1 -- 172.21.15.176:0/3863271284 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f455c02cd50 con 0x7f454006dfa0 2024-03-13T14:57:02.261 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:57:02.261 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (5m) 3s ago 18m 14.6M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:57:02.261 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (18m) 3s ago 18m 8627k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:57:02.261 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (17m) 82s ago 17m 8238k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (95s) 3s ago 18m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (87s) 82s ago 17m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (4m) 3s ago 18m 74.5M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (11m) 3s ago 11m 87.6M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (11m) 3s ago 11m 17.7M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (11m) 82s ago 11m 141M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (11m) 82s ago 11m 16.9M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (7m) 3s ago 20m 503M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (6m) 82s ago 17m 449M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (3m) 3s ago 20m 46.1M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (2m) 82s ago 16m 36.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (6m) 3s ago 18m 7726k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (6m) 82s ago 17m 7910k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:57:02.262 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (57s) 3s ago 16m 121M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T14:57:02.263 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (11s) 3s ago 15m 12.6M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T14:57:02.263 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (14m) 3s ago 14m 234M 1217M 18.2.1 5be31c24972a f7b8474aa165 2024-03-13T14:57:02.263 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (14m) 82s ago 14m 306M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:57:02.263 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (13m) 82s ago 13m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:57:02.263 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (13m) 82s ago 13m 230M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:57:02.263 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (5m) 3s ago 17m 68.2M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.277+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f454006dfa0 msgr2=0x7f4540070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.277+0000 7f455be00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f454006dfa0 0x7f4540070460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f454c005950 tx=0x7f454c0058e0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.277+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c106970 msgr2=0x7f455c0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.277+0000 7f455be00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c106970 0x7f455c0735d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f45440092e0 tx=0x7f454400be20 comp rx=0 tx=0).stop 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.278+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 shutdown_connections 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.278+0000 7f455be00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f454006dfa0 0x7f4540070460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.278+0000 7f455be00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f455c100970 0x7f455c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.278+0000 7f455be00700 1 --2- 172.21.15.176:0/3863271284 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f455c106970 0x7f455c0735d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.278+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 >> 172.21.15.176:0/3863271284 conn(0x7f455c0fc440 msgr2=0x7f455c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.278+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 shutdown_connections 2024-03-13T14:57:02.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.278+0000 7f455be00700 1 -- 172.21.15.176:0/3863271284 wait complete. 2024-03-13T14:57:02.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.425+0000 7f6910400700 1 -- 172.21.15.176:0/4077128854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 msgr2=0x7f69081057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.425+0000 7f6910400700 1 --2- 172.21.15.176:0/4077128854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f69081057a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009910 tx=0x7f68f4009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:02.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.426+0000 7f6910400700 1 -- 172.21.15.176:0/4077128854 shutdown_connections 2024-03-13T14:57:02.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.426+0000 7f6910400700 1 --2- 172.21.15.176:0/4077128854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f69081057a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.426+0000 7f6910400700 1 --2- 172.21.15.176:0/4077128854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908100da0 0x7f6908101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.426+0000 7f6910400700 1 -- 172.21.15.176:0/4077128854 >> 172.21.15.176:0/4077128854 conn(0x7f69080fc630 msgr2=0x7f69080fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:02.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.426+0000 7f6910400700 1 -- 172.21.15.176:0/4077128854 shutdown_connections 2024-03-13T14:57:02.413 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.426+0000 7f6910400700 1 -- 172.21.15.176:0/4077128854 wait complete. 2024-03-13T14:57:02.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.427+0000 7f6910400700 1 Processor -- start 2024-03-13T14:57:02.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f6910400700 1 -- start start 2024-03-13T14:57:02.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f6910400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908100da0 0x7f690819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f6910400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f690819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f6910400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690819d270 con 0x7f6908101750 2024-03-13T14:57:02.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f6910400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6908196720 con 0x7f6908100da0 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f690d000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f690819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f690d000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f690819cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59150/0 (socket says 172.21.15.176:59150) 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.428+0000 7f690d000700 1 -- 172.21.15.176:0/2245576509 learned_addr learned my addr 172.21.15.176:0/2245576509 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f690da00700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908100da0 0x7f690819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f690d000700 1 -- 172.21.15.176:0/2245576509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908100da0 msgr2=0x7f690819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f690d000700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908100da0 0x7f690819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f690d000700 1 -- 172.21.15.176:0/2245576509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40095f0 con 0x7f6908101750 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f690d000700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f690819cbe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68f40092e0 tx=0x7f68f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f6906000700 1 -- 172.21.15.176:0/2245576509 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f401d070 con 0x7f6908101750 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f6906000700 1 -- 172.21.15.176:0/2245576509 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68f400f460 con 0x7f6908101750 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.429+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69081969a0 con 0x7f6908101750 2024-03-13T14:57:02.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.430+0000 7f6906000700 1 -- 172.21.15.176:0/2245576509 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68f4021640 con 0x7f6908101750 2024-03-13T14:57:02.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.430+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6908196e90 con 0x7f6908101750 2024-03-13T14:57:02.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.431+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69081090f0 con 0x7f6908101750 2024-03-13T14:57:02.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.432+0000 7f6906000700 1 -- 172.21.15.176:0/2245576509 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f68f400f5d0 con 0x7f6908101750 2024-03-13T14:57:02.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.433+0000 7f6906000700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68f006dfa0 0x7f68f0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.433+0000 7f6906000700 1 -- 172.21.15.176:0/2245576509 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f68f408f0a0 con 0x7f6908101750 2024-03-13T14:57:02.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.433+0000 7f690da00700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68f006dfa0 0x7f68f0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.434+0000 7f690da00700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68f006dfa0 0x7f68f0070460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68fc006fd0 tx=0x7f68fc008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:02.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.439+0000 7f6906000700 1 -- 172.21.15.176:0/2245576509 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f68f405bf70 con 0x7f6908101750 2024-03-13T14:57:02.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.742+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f690804ea90 con 0x7f6908101750 2024-03-13T14:57:02.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.743+0000 7f6906000700 1 -- 172.21.15.176:0/2245576509 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f68f405bb00 con 0x7f6908101750 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:02.730 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 8, 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:57:02.731 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:57:02.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68f006dfa0 msgr2=0x7f68f0070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68f006dfa0 0x7f68f0070460 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f68fc006fd0 tx=0x7f68fc008040 comp rx=0 tx=0).stop 2024-03-13T14:57:02.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 msgr2=0x7f690819cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f690819cbe0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f68f40092e0 tx=0x7f68f400be20 comp rx=0 tx=0).stop 2024-03-13T14:57:02.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 shutdown_connections 2024-03-13T14:57:02.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68f006dfa0 0x7f68f0070460 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6908100da0 0x7f690819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 --2- 172.21.15.176:0/2245576509 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6908101750 0x7f690819cbe0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 >> 172.21.15.176:0/2245576509 conn(0x7f69080fc630 msgr2=0x7f69080fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:02.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 shutdown_connections 2024-03-13T14:57:02.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.747+0000 7f6910400700 1 -- 172.21.15.176:0/2245576509 wait complete. 2024-03-13T14:57:02.857 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:02 smithi196 ceph-mon[86193]: from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:02.857 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:02 smithi196 ceph-mon[86193]: osdmap e76: 6 total, 6 up, 6 in 2024-03-13T14:57:02.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:02 smithi176 ceph-mon[112742]: from='client.34276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:02.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:02 smithi176 ceph-mon[112742]: osdmap e76: 6 total, 6 up, 6 in 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 -- 172.21.15.176:0/2173362217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 msgr2=0x7fed54101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 --2- 172.21.15.176:0/2173362217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed54101180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed48009910 tx=0x7fed48009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 -- 172.21.15.176:0/2173362217 shutdown_connections 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 --2- 172.21.15.176:0/2173362217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed54101750 0x7fed541057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 --2- 172.21.15.176:0/2173362217 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed54101180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 -- 172.21.15.176:0/2173362217 >> 172.21.15.176:0/2173362217 conn(0x7fed540fc630 msgr2=0x7fed540fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 -- 172.21.15.176:0/2173362217 shutdown_connections 2024-03-13T14:57:02.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.898+0000 7fed5c000700 1 -- 172.21.15.176:0/2173362217 wait complete. 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.899+0000 7fed5c000700 1 Processor -- start 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed5c000700 1 -- start start 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed5c000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed5419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed5c000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed54101750 0x7fed5419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed5c000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed5419f420 con 0x7fed54100da0 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed5c000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed541988d0 con 0x7fed54101750 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed59600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed5419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed59600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed5419e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59174/0 (socket says 172.21.15.176:59174) 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.900+0000 7fed59600700 1 -- 172.21.15.176:0/1956654341 learned_addr learned my addr 172.21.15.176:0/1956654341 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed59600700 1 -- 172.21.15.176:0/1956654341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed54101750 msgr2=0x7fed5419ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed59600700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed54101750 0x7fed5419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed59600700 1 -- 172.21.15.176:0/1956654341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed480095f0 con 0x7fed54100da0 2024-03-13T14:57:02.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed59600700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed5419e850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed4800b5c0 tx=0x7fed4800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:02.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed52000700 1 -- 172.21.15.176:0/1956654341 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed4801d070 con 0x7fed54100da0 2024-03-13T14:57:02.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed52000700 1 -- 172.21.15.176:0/1956654341 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed4800f460 con 0x7fed54100da0 2024-03-13T14:57:02.888 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed54198b50 con 0x7fed54100da0 2024-03-13T14:57:02.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed52000700 1 -- 172.21.15.176:0/1956654341 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed48017610 con 0x7fed54100da0 2024-03-13T14:57:02.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.901+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed54199040 con 0x7fed54100da0 2024-03-13T14:57:02.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.902+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed54104f20 con 0x7fed54100da0 2024-03-13T14:57:02.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.903+0000 7fed52000700 1 -- 172.21.15.176:0/1956654341 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fed4800f5d0 con 0x7fed54100da0 2024-03-13T14:57:02.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.904+0000 7fed52000700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fed3c06dfa0 0x7fed3c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:02.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.904+0000 7fed52000700 1 -- 172.21.15.176:0/1956654341 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fed4808f410 con 0x7fed54100da0 2024-03-13T14:57:02.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.904+0000 7fed58c00700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fed3c06dfa0 0x7fed3c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:02.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.905+0000 7fed58c00700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fed3c06dfa0 0x7fed3c070460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed540fcd70 tx=0x7fed40006d20 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:02.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:02.909+0000 7fed52000700 1 -- 172.21.15.176:0/1956654341 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fed4805c2e0 con 0x7fed54100da0 2024-03-13T14:57:03.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.178+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fed54199370 con 0x7fed54100da0 2024-03-13T14:57:03.165 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.179+0000 7fed52000700 1 -- 172.21.15.176:0/1956654341 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fed4805be70 con 0x7fed54100da0 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:57:03.166 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:57:03.167 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:57:03.168 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:03.169 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:03.169 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:03.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fed3c06dfa0 msgr2=0x7fed3c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fed3c06dfa0 0x7fed3c070460 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fed540fcd70 tx=0x7fed40006d20 comp rx=0 tx=0).stop 2024-03-13T14:57:03.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 msgr2=0x7fed5419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed5419e850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fed4800b5c0 tx=0x7fed4800be00 comp rx=0 tx=0).stop 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 shutdown_connections 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fed3c06dfa0 0x7fed3c070460 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fed54100da0 0x7fed5419e850 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 --2- 172.21.15.176:0/1956654341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed54101750 0x7fed5419ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.182+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 >> 172.21.15.176:0/1956654341 conn(0x7fed540fc630 msgr2=0x7fed540fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.183+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 shutdown_connections 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.183+0000 7fed5c000700 1 -- 172.21.15.176:0/1956654341 wait complete. 2024-03-13T14:57:03.170 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:57:03.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.323+0000 7fdb83c00700 1 -- 172.21.15.176:0/2031079652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c073ae0 msgr2=0x7fdb7c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.323+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2031079652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c073ae0 0x7fdb7c10ce40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009910 tx=0x7fdb68009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:03.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.323+0000 7fdb83c00700 1 -- 172.21.15.176:0/2031079652 shutdown_connections 2024-03-13T14:57:03.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.323+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2031079652 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c073ae0 0x7fdb7c10ce40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.323+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2031079652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c0731c0 0x7fdb7c0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.323+0000 7fdb83c00700 1 -- 172.21.15.176:0/2031079652 >> 172.21.15.176:0/2031079652 conn(0x7fdb7c0fc630 msgr2=0x7fdb7c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:03.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.323+0000 7fdb83c00700 1 -- 172.21.15.176:0/2031079652 shutdown_connections 2024-03-13T14:57:03.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.324+0000 7fdb83c00700 1 -- 172.21.15.176:0/2031079652 wait complete. 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb83c00700 1 Processor -- start 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb83c00700 1 -- start start 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb83c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c0731c0 0x7fdb7c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb83c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c073ae0 0x7fdb7c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb83c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c199530 con 0x7fdb7c0731c0 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb83c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c19d340 con 0x7fdb7c073ae0 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb81200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c0731c0 0x7fdb7c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb81200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c0731c0 0x7fdb7c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59192/0 (socket says 172.21.15.176:59192) 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.325+0000 7fdb81200700 1 -- 172.21.15.176:0/2881977433 learned_addr learned my addr 172.21.15.176:0/2881977433 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb7be00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c073ae0 0x7fdb7c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb81200700 1 -- 172.21.15.176:0/2881977433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c073ae0 msgr2=0x7fdb7c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb81200700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c073ae0 0x7fdb7c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.313 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb81200700 1 -- 172.21.15.176:0/2881977433 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb680095f0 con 0x7fdb7c0731c0 2024-03-13T14:57:03.313 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb81200700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c0731c0 0x7fdb7c198960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c8a0 tx=0x7fdb7000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:03.313 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb79600700 1 -- 172.21.15.176:0/2881977433 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7000cea0 con 0x7fdb7c0731c0 2024-03-13T14:57:03.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb79600700 1 -- 172.21.15.176:0/2881977433 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb70004830 con 0x7fdb7c0731c0 2024-03-13T14:57:03.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c19d620 con 0x7fdb7c0731c0 2024-03-13T14:57:03.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.326+0000 7fdb79600700 1 -- 172.21.15.176:0/2881977433 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb700055b0 con 0x7fdb7c0731c0 2024-03-13T14:57:03.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.327+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c19db70 con 0x7fdb7c0731c0 2024-03-13T14:57:03.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.328+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb7c10a540 con 0x7fdb7c0731c0 2024-03-13T14:57:03.314 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.329+0000 7fdb79600700 1 -- 172.21.15.176:0/2881977433 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fdb700049a0 con 0x7fdb7c0731c0 2024-03-13T14:57:03.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.330+0000 7fdb79600700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdb6406dfa0 0x7fdb64070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:03.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.330+0000 7fdb79600700 1 -- 172.21.15.176:0/2881977433 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fdb7008dac0 con 0x7fdb7c0731c0 2024-03-13T14:57:03.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.330+0000 7fdb7be00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdb6406dfa0 0x7fdb64070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:03.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.331+0000 7fdb7be00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdb6406dfa0 0x7fdb64070460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c199f80 tx=0x7fdb6801a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:03.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.335+0000 7fdb79600700 1 -- 172.21.15.176:0/2881977433 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fdb7005a9b0 con 0x7fdb7c0731c0 2024-03-13T14:57:03.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.578+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb7c19ddf0 con 0x7fdb6406dfa0 2024-03-13T14:57:03.566 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.581+0000 7fdb79600700 1 -- 172.21.15.176:0/2881977433 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdb7c19ddf0 con 0x7fdb6406dfa0 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:57:03.567 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "8/23 daemons upgraded", 2024-03-13T14:57:03.568 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading osd daemons", 2024-03-13T14:57:03.568 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:57:03.568 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:57:03.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.583+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdb6406dfa0 msgr2=0x7fdb64070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.583+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdb6406dfa0 0x7fdb64070460 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c199f80 tx=0x7fdb6801a040 comp rx=0 tx=0).stop 2024-03-13T14:57:03.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.583+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c0731c0 msgr2=0x7fdb7c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.583+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c0731c0 0x7fdb7c198960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000c8a0 tx=0x7fdb7000cc60 comp rx=0 tx=0).stop 2024-03-13T14:57:03.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.584+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 shutdown_connections 2024-03-13T14:57:03.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.584+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdb6406dfa0 0x7fdb64070460 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.584+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdb7c0731c0 0x7fdb7c198960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.584+0000 7fdb83c00700 1 --2- 172.21.15.176:0/2881977433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdb7c073ae0 0x7fdb7c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.584+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 >> 172.21.15.176:0/2881977433 conn(0x7fdb7c0fc630 msgr2=0x7fdb7c107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:03.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.584+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 shutdown_connections 2024-03-13T14:57:03.571 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.584+0000 7fdb83c00700 1 -- 172.21.15.176:0/2881977433 wait complete. 2024-03-13T14:57:03.722 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.735+0000 7fe7cc600700 1 -- 172.21.15.176:0/1578368158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4101750 msgr2=0x7fe7c41057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.722 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.735+0000 7fe7cc600700 1 --2- 172.21.15.176:0/1578368158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4101750 0x7fe7c41057a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009910 tx=0x7fe7b0009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:03.722 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.736+0000 7fe7cc600700 1 -- 172.21.15.176:0/1578368158 shutdown_connections 2024-03-13T14:57:03.722 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.736+0000 7fe7cc600700 1 --2- 172.21.15.176:0/1578368158 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4101750 0x7fe7c41057a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.722 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.736+0000 7fe7cc600700 1 --2- 172.21.15.176:0/1578368158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7c4100da0 0x7fe7c4101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.736+0000 7fe7cc600700 1 -- 172.21.15.176:0/1578368158 >> 172.21.15.176:0/1578368158 conn(0x7fe7c40fc630 msgr2=0x7fe7c40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:03.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.736+0000 7fe7cc600700 1 -- 172.21.15.176:0/1578368158 shutdown_connections 2024-03-13T14:57:03.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.736+0000 7fe7cc600700 1 -- 172.21.15.176:0/1578368158 wait complete. 2024-03-13T14:57:03.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.737+0000 7fe7cc600700 1 Processor -- start 2024-03-13T14:57:03.723 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.737+0000 7fe7cc600700 1 -- start start 2024-03-13T14:57:03.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7cc600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4100da0 0x7fe7c4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:03.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7cc600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7c4101750 0x7fe7c4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:03.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7cc600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c4199530 con 0x7fe7c4100da0 2024-03-13T14:57:03.724 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7cc600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c419d340 con 0x7fe7c4101750 2024-03-13T14:57:03.725 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7c9c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4100da0 0x7fe7c4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:03.725 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7c9c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4100da0 0x7fe7c4198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:59202/0 (socket says 172.21.15.176:59202) 2024-03-13T14:57:03.725 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7c9c00700 1 -- 172.21.15.176:0/2868673686 learned_addr learned my addr 172.21.15.176:0/2868673686 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:03.725 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7c9200700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7c4101750 0x7fe7c4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:03.725 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7c9c00700 1 -- 172.21.15.176:0/2868673686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7c4101750 msgr2=0x7fe7c4198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7c9c00700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7c4101750 0x7fe7c4198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.738+0000 7fe7c9c00700 1 -- 172.21.15.176:0/2868673686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00095f0 con 0x7fe7c4100da0 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.739+0000 7fe7c9c00700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4100da0 0x7fe7c4198960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c8f0 tx=0x7fe7b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.739+0000 7fe7c2000700 1 -- 172.21.15.176:0/2868673686 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7b8007a70 con 0x7fe7c4100da0 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.739+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c419d620 con 0x7fe7c4100da0 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.739+0000 7fe7c2000700 1 -- 172.21.15.176:0/2868673686 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7b8004cf0 con 0x7fe7c4100da0 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.739+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c419db70 con 0x7fe7c4100da0 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.739+0000 7fe7c2000700 1 -- 172.21.15.176:0/2868673686 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7b80055b0 con 0x7fe7c4100da0 2024-03-13T14:57:03.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.740+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c4105630 con 0x7fe7c4100da0 2024-03-13T14:57:03.727 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.741+0000 7fe7c2000700 1 -- 172.21.15.176:0/2868673686 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe7b8004750 con 0x7fe7c4100da0 2024-03-13T14:57:03.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.742+0000 7fe7c2000700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe7ac06dfa0 0x7fe7ac070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:03.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.742+0000 7fe7c2000700 1 -- 172.21.15.176:0/2868673686 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fe7b8013070 con 0x7fe7c4100da0 2024-03-13T14:57:03.728 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.742+0000 7fe7c9200700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe7ac06dfa0 0x7fe7ac070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:03.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.743+0000 7fe7c9200700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe7ac06dfa0 0x7fe7ac070460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b5c0 tx=0x7fe7b0005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:03.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:03.747+0000 7fe7c2000700 1 -- 172.21.15.176:0/2868673686 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fe7b805ab60 con 0x7fe7c4100da0 2024-03-13T14:57:03.761 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:03 smithi176 ceph-mon[112742]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:03.761 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:03 smithi176 ceph-mon[112742]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:03.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:03 smithi176 ceph-mon[112742]: pgmap v120: 65 pgs: 20 peering, 6 active+undersized, 6 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 15/225 objects degraded (6.667%) 2024-03-13T14:57:03.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:03 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:03.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:03 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2245576509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:03.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:03 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1956654341' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:57:03.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:03 smithi196 ceph-mon[86193]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:03.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:03 smithi196 ceph-mon[86193]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:03.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:03 smithi196 ceph-mon[86193]: pgmap v120: 65 pgs: 20 peering, 6 active+undersized, 6 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 15/225 objects degraded (6.667%) 2024-03-13T14:57:03.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:03 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:03.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:03 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2245576509' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:03.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:03 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1956654341' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:57:04.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.047+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe7c419de40 con 0x7fe7c4100da0 2024-03-13T14:57:04.034 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.048+0000 7fe7c2000700 1 -- 172.21.15.176:0/2868673686 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+512 (secure 0 0 0) 0x7fe7b8004e60 con 0x7fe7c4100da0 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_WARN Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-03-13T14:57:04.035 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe7ac06dfa0 msgr2=0x7fe7ac070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe7ac06dfa0 0x7fe7ac070460 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b5c0 tx=0x7fe7b0005fd0 comp rx=0 tx=0).stop 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4100da0 msgr2=0x7fe7c4198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4100da0 0x7fe7c4198960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800c8f0 tx=0x7fe7b800ccb0 comp rx=0 tx=0).stop 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 shutdown_connections 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe7ac06dfa0 0x7fe7ac070460 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe7c4100da0 0x7fe7c4198960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 --2- 172.21.15.176:0/2868673686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7c4101750 0x7fe7c4198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.051+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 >> 172.21.15.176:0/2868673686 conn(0x7fe7c40fc630 msgr2=0x7fe7c4102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.052+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 shutdown_connections 2024-03-13T14:57:04.038 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:04.052+0000 7fe7cc600700 1 -- 172.21.15.176:0/2868673686 wait complete. 2024-03-13T14:57:04.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:04 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2868673686' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:57:04.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:04 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2868673686' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:57:05.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:05 smithi176 ceph-mon[112742]: from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:05.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:05 smithi176 ceph-mon[112742]: pgmap v121: 65 pgs: 20 peering, 45 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:57:05.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:05 smithi176 ceph-mon[112742]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded) 2024-03-13T14:57:05.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:05 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T14:57:05.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:05 smithi196 ceph-mon[86193]: from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:05.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:05 smithi196 ceph-mon[86193]: pgmap v121: 65 pgs: 20 peering, 45 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:57:05.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:05 smithi196 ceph-mon[86193]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 15/225 objects degraded (6.667%), 6 pgs degraded) 2024-03-13T14:57:05.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:05 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T14:57:07.761 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: pgmap v122: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-13T14:57:07.761 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:07.761 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:07.762 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-13T14:57:07.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: pgmap v122: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-13T14:57:07.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:07.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:07.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-13T14:57:08.795 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:08 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-13T14:57:08.795 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:08 smithi176 ceph-mon[112742]: Upgrade: osd.2 is safe to restart 2024-03-13T14:57:08.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:08 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-03-13T14:57:08.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:08 smithi196 ceph-mon[86193]: Upgrade: osd.2 is safe to restart 2024-03-13T14:57:09.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:09 smithi176 ceph-mon[112742]: pgmap v123: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:09.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:09.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-13T14:57:09.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:09 smithi196 ceph-mon[86193]: pgmap v123: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-13T14:57:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:10.774 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:10 smithi176 ceph-mon[112742]: Upgrade: Updating osd.2 2024-03-13T14:57:10.774 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:10 smithi176 ceph-mon[112742]: Deploying daemon osd.2 on smithi176 2024-03-13T14:57:10.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:10 smithi196 ceph-mon[86193]: Upgrade: Updating osd.2 2024-03-13T14:57:10.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:10 smithi196 ceph-mon[86193]: Deploying daemon osd.2 on smithi176 2024-03-13T14:57:11.855 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:11 smithi176 ceph-mon[112742]: pgmap v124: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-13T14:57:11.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:11 smithi196 ceph-mon[86193]: pgmap v124: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-13T14:57:12.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:12 smithi176 ceph-mon[112742]: pgmap v125: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 942 B/s rd, 1 op/s 2024-03-13T14:57:12.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:12 smithi196 ceph-mon[86193]: pgmap v125: 65 pgs: 65 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 942 B/s rd, 1 op/s 2024-03-13T14:57:14.127 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:13 smithi176 systemd[1]: Stopping Ceph osd.2 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:57:14.513 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:14 smithi196 ceph-mon[86193]: osd.2 marked itself down and dead 2024-03-13T14:57:14.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:14 smithi176 ceph-mon[112742]: osd.2 marked itself down and dead 2024-03-13T14:57:14.627 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:14 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[62580]: 2024-03-13T14:57:14.195+0000 7f541a600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:57:14.627 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:14 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[62580]: 2024-03-13T14:57:14.195+0000 7f541a600700 -1 osd.2 76 *** Got signal Terminated *** 2024-03-13T14:57:14.627 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:14 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[62580]: 2024-03-13T14:57:14.195+0000 7f541a600700 -1 osd.2 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T14:57:15.033 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:14 smithi176 podman[135582]: 2024-03-13 14:57:14.741708899 +0000 UTC m=+0.684068197 container died f7b8474aa165d0cdcb83d5095f620f2c57129c1f078a14e42ae75ce3ef8cbd20 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.29.1, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1) 2024-03-13T14:57:15.291 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:15 smithi176 podman[135582]: 2024-03-13 14:57:15.267800842 +0000 UTC m=+1.210160202 container cleanup f7b8474aa165d0cdcb83d5095f620f2c57129c1f078a14e42ae75ce3ef8cbd20 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-13T14:57:15.291 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:15 smithi176 bash[135582]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2 2024-03-13T14:57:15.292 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:15 smithi176 ceph-mon[112742]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:57:15.292 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:15 smithi176 ceph-mon[112742]: osdmap e77: 6 total, 5 up, 6 in 2024-03-13T14:57:15.292 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:15 smithi176 ceph-mon[112742]: pgmap v127: 65 pgs: 10 stale+active+clean, 55 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-13T14:57:15.292 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:15 smithi176 ceph-mon[112742]: osdmap e78: 6 total, 5 up, 6 in 2024-03-13T14:57:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:15 smithi196 ceph-mon[86193]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:57:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:15 smithi196 ceph-mon[86193]: osdmap e77: 6 total, 5 up, 6 in 2024-03-13T14:57:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:15 smithi196 ceph-mon[86193]: pgmap v127: 65 pgs: 10 stale+active+clean, 55 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-13T14:57:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:15 smithi196 ceph-mon[86193]: osdmap e78: 6 total, 5 up, 6 in 2024-03-13T14:57:15.876 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:15 smithi176 podman[135602]: 2024-03-13 14:57:15.527655795 +0000 UTC m=+0.770479659 container remove f7b8474aa165d0cdcb83d5095f620f2c57129c1f078a14e42ae75ce3ef8cbd20 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1) 2024-03-13T14:57:16.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:16.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:16.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:16.628 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:16 smithi176 podman[135689]: 2024-03-13 14:57:16.260194653 +0000 UTC m=+0.032774930 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:57:16.628 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:16 smithi176 podman[135689]: 2024-03-13 14:57:16.379042689 +0000 UTC m=+0.151623003 container create f01a9bab3ffdb01ec0e3d499382a2f29418e7343b0fb3beede168cb922bdde5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-13T14:57:16.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:16.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:16.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:17.108 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:16 smithi176 podman[135689]: 2024-03-13 14:57:16.67731008 +0000 UTC m=+0.449890401 container init f01a9bab3ffdb01ec0e3d499382a2f29418e7343b0fb3beede168cb922bdde5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-13T14:57:17.108 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:16 smithi176 podman[135689]: 2024-03-13 14:57:16.688648666 +0000 UTC m=+0.461228987 container start f01a9bab3ffdb01ec0e3d499382a2f29418e7343b0fb3beede168cb922bdde5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:57:17.108 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:16 smithi176 podman[135689]: 2024-03-13 14:57:16.79138788 +0000 UTC m=+0.563968181 container attach f01a9bab3ffdb01ec0e3d499382a2f29418e7343b0fb3beede168cb922bdde5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2) 2024-03-13T14:57:17.377 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:17 smithi176 podman[135689]: 2024-03-13 14:57:17.123522436 +0000 UTC m=+0.896102753 container died f01a9bab3ffdb01ec0e3d499382a2f29418e7343b0fb3beede168cb922bdde5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-13T14:57:17.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:17 smithi196 ceph-mon[86193]: pgmap v129: 65 pgs: 7 active+undersized, 6 active+undersized+degraded, 7 stale+active+clean, 45 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 11/225 objects degraded (4.889%) 2024-03-13T14:57:17.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:17 smithi196 ceph-mon[86193]: Health check failed: Degraded data redundancy: 11/225 objects degraded (4.889%), 6 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:17.726 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:17 smithi176 ceph-mon[112742]: pgmap v129: 65 pgs: 7 active+undersized, 6 active+undersized+degraded, 7 stale+active+clean, 45 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 11/225 objects degraded (4.889%) 2024-03-13T14:57:17.727 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:17 smithi176 ceph-mon[112742]: Health check failed: Degraded data redundancy: 11/225 objects degraded (4.889%), 6 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:17.980 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:17 smithi176 podman[135715]: 2024-03-13 14:57:17.969009494 +0000 UTC m=+0.825552068 container remove f01a9bab3ffdb01ec0e3d499382a2f29418e7343b0fb3beede168cb922bdde5f (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-03-13T14:57:18.361 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:17 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.2.service: Deactivated successfully. 2024-03-13T14:57:18.362 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:17 smithi176 systemd[1]: Stopped Ceph osd.2 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:57:18.362 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:17 smithi176 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.2.service: Consumed 1min 55.216s CPU time. 2024-03-13T14:57:18.626 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:18 smithi176 systemd[1]: Starting Ceph osd.2 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:57:18.977 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:18 smithi176 podman[135819]: 2024-03-13 14:57:18.625688782 +0000 UTC m=+0.031463052 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:57:18.977 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:18 smithi176 podman[135819]: 2024-03-13 14:57:18.745271818 +0000 UTC m=+0.151046125 container create effdf0a392f6994c6c5f5618119aabba8c37e6893c8c533bcc9eead675a880a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:57:18.977 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:18 smithi176 podman[135819]: 2024-03-13 14:57:18.990759824 +0000 UTC m=+0.396534137 container init effdf0a392f6994c6c5f5618119aabba8c37e6893c8c533bcc9eead675a880a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, ceph=True) 2024-03-13T14:57:19.377 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 podman[135819]: 2024-03-13 14:57:19.001811866 +0000 UTC m=+0.407586203 container start effdf0a392f6994c6c5f5618119aabba8c37e6893c8c533bcc9eead675a880a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-03-13T14:57:19.377 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 podman[135819]: 2024-03-13 14:57:19.080841786 +0000 UTC m=+0.486616100 container attach effdf0a392f6994c6c5f5618119aabba8c37e6893c8c533bcc9eead675a880a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-13T14:57:19.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:19 smithi196 ceph-mon[86193]: pgmap v130: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 4 stale+active+clean, 39 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 19/225 objects degraded (8.444%) 2024-03-13T14:57:19.842 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-mon[112742]: pgmap v130: 65 pgs: 12 active+undersized, 10 active+undersized+degraded, 4 stale+active+clean, 39 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 19/225 objects degraded (8.444%) 2024-03-13T14:57:20.128 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate[135833]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate[135833]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 bash[135819]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 bash[135819]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 bash[135819]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 bash[135819]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 bash[135819]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 bash[135819]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 bash[135819]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate[135833]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate[135833]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate[135833]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate[135833]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate[135833]: --> ceph-volume raw activate successful for osd ID: 2 2024-03-13T14:57:20.129 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:19 smithi176 podman[135819]: 2024-03-13 14:57:19.915625089 +0000 UTC m=+1.321399403 container died effdf0a392f6994c6c5f5618119aabba8c37e6893c8c533bcc9eead675a880a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:57:20.876 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:20 smithi176 podman[135963]: 2024-03-13 14:57:20.584507225 +0000 UTC m=+0.650138479 container remove effdf0a392f6994c6c5f5618119aabba8c37e6893c8c533bcc9eead675a880a9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T14:57:21.216 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:20 smithi176 podman[136027]: 2024-03-13 14:57:20.851939723 +0000 UTC m=+0.035634634 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:57:21.216 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:20 smithi176 podman[136027]: 2024-03-13 14:57:20.968866871 +0000 UTC m=+0.152561817 container create 56889da43f2b88a34f545c9413178c87454bcaa8e82322ef66e8ee59625f0a77 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:57:21.217 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:21 smithi176 podman[136027]: 2024-03-13 14:57:21.230568077 +0000 UTC m=+0.414263033 container init 56889da43f2b88a34f545c9413178c87454bcaa8e82322ef66e8ee59625f0a77 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-13T14:57:21.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:21 smithi176 ceph-mon[112742]: pgmap v131: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:21.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:21.627 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:21 smithi176 podman[136027]: 2024-03-13 14:57:21.241425373 +0000 UTC m=+0.425120293 container start 56889da43f2b88a34f545c9413178c87454bcaa8e82322ef66e8ee59625f0a77 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-03-13T14:57:21.627 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:21 smithi176 ceph-osd[136046]: -- 172.21.15.176:0/1462499828 <== mon.0 v2:172.21.15.176:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55771108d680 con 0x557711056c00 2024-03-13T14:57:21.627 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:21 smithi176 bash[136027]: 56889da43f2b88a34f545c9413178c87454bcaa8e82322ef66e8ee59625f0a77 2024-03-13T14:57:21.627 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:21 smithi176 systemd[1]: Started Ceph osd.2 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:57:21.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:21 smithi196 ceph-mon[86193]: pgmap v131: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:21.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:22.126 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:21 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[136042]: 2024-03-13T14:57:21.808+0000 7f7aa2e897c0 -1 Falling back to public interface 2024-03-13T14:57:22.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:22 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:22.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:22 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:57:22.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:22 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:22.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:22 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:57:23.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:23 smithi196 ceph-mon[86193]: pgmap v132: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:23.792 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:23 smithi176 ceph-mon[112742]: pgmap v132: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:25.736 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:25 smithi176 ceph-mon[112742]: pgmap v133: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:25.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:25 smithi196 ceph-mon[86193]: pgmap v133: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:26.695 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:26 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 30/225 objects degraded (13.333%), 13 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:26.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:26 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 30/225 objects degraded (13.333%), 13 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:27.127 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:26 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[136042]: 2024-03-13T14:57:26.937+0000 7f7aa2e897c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-03-13T14:57:27.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:27 smithi176 ceph-mon[112742]: pgmap v134: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:27.877 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:27 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[136042]: 2024-03-13T14:57:27.490+0000 7f7aa2e897c0 -1 osd.2 76 log_to_monitors true 2024-03-13T14:57:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:27 smithi196 ceph-mon[86193]: pgmap v134: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:28.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:28 smithi176 ceph-mon[112742]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-13T14:57:28.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:28 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:28.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:28 smithi196 ceph-mon[86193]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-13T14:57:28.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:28 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:29.876 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 14:57:29 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[136042]: 2024-03-13T14:57:29.511+0000 7f7a8d200700 -1 osd.2 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T14:57:29.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:29 smithi176 ceph-mon[112742]: pgmap v135: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:29.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:29 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:29.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:29 smithi176 ceph-mon[112742]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-13T14:57:29.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:29 smithi176 ceph-mon[112742]: osdmap e79: 6 total, 5 up, 6 in 2024-03-13T14:57:29.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:29 smithi176 ceph-mon[112742]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:57:29.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:29 smithi196 ceph-mon[86193]: pgmap v135: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:29.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:29 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:29.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:29 smithi196 ceph-mon[86193]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-13T14:57:29.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:29 smithi196 ceph-mon[86193]: osdmap e79: 6 total, 5 up, 6 in 2024-03-13T14:57:29.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:29 smithi196 ceph-mon[86193]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi176", "root=default"]}]: dispatch 2024-03-13T14:57:30.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:30 smithi176 ceph-mon[112742]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' 2024-03-13T14:57:30.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:30.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:30.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:30 smithi196 ceph-mon[86193]: from='osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103]' entity='osd.2' 2024-03-13T14:57:30.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:30.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:31.757 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: pgmap v137: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:31.757 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:57:31.757 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103] boot 2024-03-13T14:57:31.757 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: osdmap e80: 6 total, 6 up, 6 in 2024-03-13T14:57:31.757 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:57:31.757 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:31.757 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:31.758 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:31 smithi176 ceph-mon[112742]: osdmap e81: 6 total, 6 up, 6 in 2024-03-13T14:57:31.830 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: pgmap v137: 65 pgs: 17 active+undersized, 13 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 30/225 objects degraded (13.333%) 2024-03-13T14:57:31.831 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:57:31.831 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: osd.2 [v2:172.21.15.176:6818/2651949103,v1:172.21.15.176:6819/2651949103] boot 2024-03-13T14:57:31.831 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: osdmap e80: 6 total, 6 up, 6 in 2024-03-13T14:57:31.831 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-13T14:57:31.831 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:31.831 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:31.831 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:31 smithi196 ceph-mon[86193]: osdmap e81: 6 total, 6 up, 6 in 2024-03-13T14:57:32.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:32 smithi176 ceph-mon[112742]: pgmap v140: 65 pgs: 10 peering, 11 active+undersized, 9 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 21/225 objects degraded (9.333%) 2024-03-13T14:57:32.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:32 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 21/225 objects degraded (9.333%), 9 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:32.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:32 smithi196 ceph-mon[86193]: pgmap v140: 65 pgs: 10 peering, 11 active+undersized, 9 active+undersized+degraded, 35 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 21/225 objects degraded (9.333%) 2024-03-13T14:57:32.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:32 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 21/225 objects degraded (9.333%), 9 pgs degraded (PG_DEGRADED) 2024-03-13T14:57:34.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.218+0000 7fe9d4400700 1 -- 172.21.15.176:0/1209750365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc102620 msgr2=0x7fe9cc10ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.218+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1209750365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc102620 0x7fe9cc10ab10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009910 tx=0x7fe9b8009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:34.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.219+0000 7fe9d4400700 1 -- 172.21.15.176:0/1209750365 shutdown_connections 2024-03-13T14:57:34.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.219+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1209750365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc102620 0x7fe9cc10ab10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.219+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1209750365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc101d00 0x7fe9cc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.219+0000 7fe9d4400700 1 -- 172.21.15.176:0/1209750365 >> 172.21.15.176:0/1209750365 conn(0x7fe9cc0fb570 msgr2=0x7fe9cc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:34.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.219+0000 7fe9d4400700 1 -- 172.21.15.176:0/1209750365 shutdown_connections 2024-03-13T14:57:34.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.219+0000 7fe9d4400700 1 -- 172.21.15.176:0/1209750365 wait complete. 2024-03-13T14:57:34.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.220+0000 7fe9d4400700 1 Processor -- start 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.220+0000 7fe9d4400700 1 -- start start 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.220+0000 7fe9d4400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc101d00 0x7fe9cc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d4400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc102620 0x7fe9cc198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d4400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc199490 con 0x7fe9cc101d00 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d4400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc199600 con 0x7fe9cc102620 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc101d00 0x7fe9cc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc101d00 0x7fe9cc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55590/0 (socket says 172.21.15.176:55590) 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1a00700 1 -- 172.21.15.176:0/1070552511 learned_addr learned my addr 172.21.15.176:0/1070552511 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1000700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc102620 0x7fe9cc198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1a00700 1 -- 172.21.15.176:0/1070552511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc102620 msgr2=0x7fe9cc198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1a00700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc102620 0x7fe9cc198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1a00700 1 -- 172.21.15.176:0/1070552511 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b80095f0 con 0x7fe9cc101d00 2024-03-13T14:57:34.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.221+0000 7fe9d1000700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc102620 0x7fe9cc198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:57:34.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.222+0000 7fe9d1a00700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc101d00 0x7fe9cc198950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c930 tx=0x7fe9c000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:34.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.222+0000 7fe9ca000700 1 -- 172.21.15.176:0/1070552511 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9c0007ad0 con 0x7fe9cc101d00 2024-03-13T14:57:34.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.222+0000 7fe9ca000700 1 -- 172.21.15.176:0/1070552511 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9c0007c30 con 0x7fe9cc101d00 2024-03-13T14:57:34.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.222+0000 7fe9ca000700 1 -- 172.21.15.176:0/1070552511 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9c0018610 con 0x7fe9cc101d00 2024-03-13T14:57:34.210 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.222+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc19d480 con 0x7fe9cc101d00 2024-03-13T14:57:34.210 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.222+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc19d9d0 con 0x7fe9cc101d00 2024-03-13T14:57:34.210 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.223+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc10a670 con 0x7fe9cc101d00 2024-03-13T14:57:34.210 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.224+0000 7fe9ca000700 1 -- 172.21.15.176:0/1070552511 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe9c000f450 con 0x7fe9cc101d00 2024-03-13T14:57:34.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.225+0000 7fe9ca000700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe9b406dfa0 0x7fe9b4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:34.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.225+0000 7fe9ca000700 1 -- 172.21.15.176:0/1070552511 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fe9c008dff0 con 0x7fe9cc101d00 2024-03-13T14:57:34.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.226+0000 7fe9d1000700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe9b406dfa0 0x7fe9b4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:34.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.226+0000 7fe9d1000700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe9b406dfa0 0x7fe9b4070460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006010 tx=0x7fe9b800b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:34.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.230+0000 7fe9ca000700 1 -- 172.21.15.176:0/1070552511 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fe9c005af00 con 0x7fe9cc101d00 2024-03-13T14:57:34.482 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:34 smithi176 ceph-mon[112742]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/225 objects degraded (9.333%), 9 pgs degraded) 2024-03-13T14:57:34.482 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:34 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T14:57:34.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.495+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9cc02cd50 con 0x7fe9b406dfa0 2024-03-13T14:57:34.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.497+0000 7fe9ca000700 1 -- 172.21.15.176:0/1070552511 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fe9cc02cd50 con 0x7fe9b406dfa0 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe9b406dfa0 msgr2=0x7fe9b4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe9b406dfa0 0x7fe9b4070460 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006010 tx=0x7fe9b800b540 comp rx=0 tx=0).stop 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc101d00 msgr2=0x7fe9cc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc101d00 0x7fe9cc198950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000c930 tx=0x7fe9c000ccf0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 shutdown_connections 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe9b406dfa0 0x7fe9b4070460 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe9cc101d00 0x7fe9cc198950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 --2- 172.21.15.176:0/1070552511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc102620 0x7fe9cc198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.487 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 >> 172.21.15.176:0/1070552511 conn(0x7fe9cc0fb570 msgr2=0x7fe9cc100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:34.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 shutdown_connections 2024-03-13T14:57:34.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.501+0000 7fe9d4400700 1 -- 172.21.15.176:0/1070552511 wait complete. 2024-03-13T14:57:34.504 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:57:34.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.671+0000 7f39be400700 1 -- 172.21.15.176:0/3481240899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 msgr2=0x7f39b8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.671+0000 7f39be400700 1 --2- 172.21.15.176:0/3481240899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b8101180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009910 tx=0x7f39a8009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:34.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.672+0000 7f39be400700 1 -- 172.21.15.176:0/3481240899 shutdown_connections 2024-03-13T14:57:34.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.672+0000 7f39be400700 1 --2- 172.21.15.176:0/3481240899 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b8101750 0x7f39b81057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.672+0000 7f39be400700 1 --2- 172.21.15.176:0/3481240899 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b8101180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.658 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.672+0000 7f39be400700 1 -- 172.21.15.176:0/3481240899 >> 172.21.15.176:0/3481240899 conn(0x7f39b80fc630 msgr2=0x7f39b80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:34.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.672+0000 7f39be400700 1 -- 172.21.15.176:0/3481240899 shutdown_connections 2024-03-13T14:57:34.659 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.672+0000 7f39be400700 1 -- 172.21.15.176:0/3481240899 wait complete. 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.673+0000 7f39be400700 1 Processor -- start 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.673+0000 7f39be400700 1 -- start start 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.673+0000 7f39be400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.673+0000 7f39be400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b8101750 0x7f39b819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.673+0000 7f39be400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b819f2c0 con 0x7f39b8100da0 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.673+0000 7f39be400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b8198800 con 0x7f39b8101750 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55610/0 (socket says 172.21.15.176:55610) 2024-03-13T14:57:34.660 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b7400700 1 -- 172.21.15.176:0/3422638382 learned_addr learned my addr 172.21.15.176:0/3422638382 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:34.661 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b6a00700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b8101750 0x7f39b819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:34.661 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b7400700 1 -- 172.21.15.176:0/3422638382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b8101750 msgr2=0x7f39b819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.661 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b7400700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b8101750 0x7f39b819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.661 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b7400700 1 -- 172.21.15.176:0/3422638382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80095f0 con 0x7f39b8100da0 2024-03-13T14:57:34.662 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.674+0000 7f39b7400700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b819e780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39a8006010 tx=0x7f39a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:34.662 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.675+0000 7f39afe00700 1 -- 172.21.15.176:0/3422638382 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a801d070 con 0x7f39b8100da0 2024-03-13T14:57:34.662 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.675+0000 7f39afe00700 1 -- 172.21.15.176:0/3422638382 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39a8004fb0 con 0x7f39b8100da0 2024-03-13T14:57:34.662 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.675+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b8198a80 con 0x7f39b8100da0 2024-03-13T14:57:34.662 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.675+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b8198f70 con 0x7f39b8100da0 2024-03-13T14:57:34.662 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.675+0000 7f39afe00700 1 -- 172.21.15.176:0/3422638382 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a800f690 con 0x7f39b8100da0 2024-03-13T14:57:34.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.676+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b8104f20 con 0x7f39b8100da0 2024-03-13T14:57:34.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.677+0000 7f39afe00700 1 -- 172.21.15.176:0/3422638382 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f39a80228a0 con 0x7f39b8100da0 2024-03-13T14:57:34.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.678+0000 7f39afe00700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f399c06dfa0 0x7f399c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:34.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.678+0000 7f39afe00700 1 -- 172.21.15.176:0/3422638382 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f39a808f760 con 0x7f39b8100da0 2024-03-13T14:57:34.664 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.679+0000 7f39b6a00700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f399c06dfa0 0x7f399c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:34.665 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.679+0000 7f39b6a00700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f399c06dfa0 0x7f399c070460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39b80fcdb0 tx=0x7f39a4005d50 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:34.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.684+0000 7f39afe00700 1 -- 172.21.15.176:0/3422638382 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f39a805c5b0 con 0x7f39b8100da0 2024-03-13T14:57:34.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:34 smithi196 ceph-mon[86193]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/225 objects degraded (9.333%), 9 pgs degraded) 2024-03-13T14:57:34.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:34 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T14:57:34.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.942+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39b8199250 con 0x7f399c06dfa0 2024-03-13T14:57:34.930 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.944+0000 7f39afe00700 1 -- 172.21.15.176:0/3422638382 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f39b8199250 con 0x7f399c06dfa0 2024-03-13T14:57:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f399c06dfa0 msgr2=0x7f399c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f399c06dfa0 0x7f399c070460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39b80fcdb0 tx=0x7f39a4005d50 comp rx=0 tx=0).stop 2024-03-13T14:57:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 msgr2=0x7f39b819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:34.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b819e780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f39a8006010 tx=0x7f39a800be20 comp rx=0 tx=0).stop 2024-03-13T14:57:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 shutdown_connections 2024-03-13T14:57:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f399c06dfa0 0x7f399c070460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39b8100da0 0x7f39b819e780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 --2- 172.21.15.176:0/3422638382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39b8101750 0x7f39b819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 >> 172.21.15.176:0/3422638382 conn(0x7f39b80fc630 msgr2=0x7f39b80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 shutdown_connections 2024-03-13T14:57:34.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:34.947+0000 7f39be400700 1 -- 172.21.15.176:0/3422638382 wait complete. 2024-03-13T14:57:35.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 -- 172.21.15.176:0/709514299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b10075be0 msgr2=0x7f4b101114f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 --2- 172.21.15.176:0/709514299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b10075be0 0x7f4b101114f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009910 tx=0x7f4afc009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:35.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 -- 172.21.15.176:0/709514299 shutdown_connections 2024-03-13T14:57:35.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 --2- 172.21.15.176:0/709514299 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b10075be0 0x7f4b101114f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 --2- 172.21.15.176:0/709514299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b100752c0 0x7f4b100756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 -- 172.21.15.176:0/709514299 >> 172.21.15.176:0/709514299 conn(0x7f4b100fe880 msgr2=0x7f4b10100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:35.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 -- 172.21.15.176:0/709514299 shutdown_connections 2024-03-13T14:57:35.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.100+0000 7f4b17800700 1 -- 172.21.15.176:0/709514299 wait complete. 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.101+0000 7f4b17800700 1 Processor -- start 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.101+0000 7f4b17800700 1 -- start start 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b17800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b100752c0 0x7f4b101a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b17800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b10075be0 0x7f4b101a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b17800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b101a3690 con 0x7f4b100752c0 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b17800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1019cb40 con 0x7f4b10075be0 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b14e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b100752c0 0x7f4b101a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b14e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b100752c0 0x7f4b101a2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55628/0 (socket says 172.21.15.176:55628) 2024-03-13T14:57:35.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b14e00700 1 -- 172.21.15.176:0/4170037276 learned_addr learned my addr 172.21.15.176:0/4170037276 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:35.089 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b0fe00700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b10075be0 0x7f4b101a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:35.089 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.102+0000 7f4b14e00700 1 -- 172.21.15.176:0/4170037276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b10075be0 msgr2=0x7f4b101a3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.089 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b14e00700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b10075be0 0x7f4b101a3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.089 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b14e00700 1 -- 172.21.15.176:0/4170037276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc0095f0 con 0x7f4b100752c0 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b0fe00700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b10075be0 0x7f4b101a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b14e00700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b100752c0 0x7f4b101a2ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400ada0 tx=0x7f4b0400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b0d600700 1 -- 172.21.15.176:0/4170037276 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b0400f070 con 0x7f4b100752c0 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1019ce20 con 0x7f4b100752c0 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b0d600700 1 -- 172.21.15.176:0/4170037276 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b04007bf0 con 0x7f4b100752c0 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b0d600700 1 -- 172.21.15.176:0/4170037276 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b04005690 con 0x7f4b100752c0 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.103+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1019d370 con 0x7f4b100752c0 2024-03-13T14:57:35.090 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.104+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1010ec60 con 0x7f4b100752c0 2024-03-13T14:57:35.091 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.106+0000 7f4b0d600700 1 -- 172.21.15.176:0/4170037276 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4b04005870 con 0x7f4b100752c0 2024-03-13T14:57:35.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.107+0000 7f4b0d600700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4af806dfa0 0x7f4af8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.107+0000 7f4b0d600700 1 -- 172.21.15.176:0/4170037276 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f4b0408d740 con 0x7f4b100752c0 2024-03-13T14:57:35.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.107+0000 7f4b0fe00700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4af806dfa0 0x7f4af8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:35.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.108+0000 7f4b0fe00700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4af806dfa0 0x7f4af8070460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00b850 tx=0x7f4afc00b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:35.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.111+0000 7f4b0d600700 1 -- 172.21.15.176:0/4170037276 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4b0405a6b0 con 0x7f4b100752c0 2024-03-13T14:57:35.329 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.343+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b1019d6a0 con 0x7f4af806dfa0 2024-03-13T14:57:35.342 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.356+0000 7f4b0d600700 1 -- 172.21.15.176:0/4170037276 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4b1019d6a0 con 0x7f4af806dfa0 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (5m) 6s ago 19m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (19m) 6s ago 19m 8812k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (17m) 115s ago 17m 8238k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (2m) 6s ago 19m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (2m) 115s ago 17m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (4m) 6s ago 18m 73.6M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (11m) 6s ago 11m 87.3M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (11m) 6s ago 11m 17.9M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (11m) 115s ago 11m 141M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (11m) 115s ago 11m 16.9M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (8m) 6s ago 20m 504M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (7m) 115s ago 17m 449M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:57:35.343 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (3m) 6s ago 21m 47.3M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (3m) 115s ago 17m 36.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (6m) 6s ago 19m 7520k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (6m) 115s ago 17m 7910k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (90s) 6s ago 16m 122M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (44s) 6s ago 15m 97.2M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (14s) 6s ago 15m 11.1M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (14m) 115s ago 14m 306M 2241M 18.2.1 5be31c24972a d5726d8c4f1b 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (14m) 115s ago 14m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (13m) 115s ago 13m 230M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:57:35.344 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (6m) 6s ago 18m 69.1M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:57:35.345 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4af806dfa0 msgr2=0x7f4af8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4af806dfa0 0x7f4af8070460 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4afc00b850 tx=0x7f4afc00b540 comp rx=0 tx=0).stop 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b100752c0 msgr2=0x7f4b101a2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b100752c0 0x7f4b101a2ac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400ada0 tx=0x7f4b0400c330 comp rx=0 tx=0).stop 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 shutdown_connections 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4af806dfa0 0x7f4af8070460 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b100752c0 0x7f4b101a2ac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 --2- 172.21.15.176:0/4170037276 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b10075be0 0x7f4b101a3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.359+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 >> 172.21.15.176:0/4170037276 conn(0x7f4b100fe880 msgr2=0x7f4b10100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.360+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 shutdown_connections 2024-03-13T14:57:35.346 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.360+0000 7f4b17800700 1 -- 172.21.15.176:0/4170037276 wait complete. 2024-03-13T14:57:35.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 -- 172.21.15.176:0/3888678118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc281017a0 msgr2=0x7fcc281057f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 --2- 172.21.15.176:0/3888678118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc281017a0 0x7fcc281057f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcc14009910 tx=0x7fcc14009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:35.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 -- 172.21.15.176:0/3888678118 shutdown_connections 2024-03-13T14:57:35.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 --2- 172.21.15.176:0/3888678118 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc281017a0 0x7fcc281057f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 --2- 172.21.15.176:0/3888678118 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc28100df0 0x7fcc281011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 -- 172.21.15.176:0/3888678118 >> 172.21.15.176:0/3888678118 conn(0x7fcc280fc6a0 msgr2=0x7fcc280feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:35.508 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 -- 172.21.15.176:0/3888678118 shutdown_connections 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.517+0000 7fcc2e200700 1 -- 172.21.15.176:0/3888678118 wait complete. 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.518+0000 7fcc2e200700 1 Processor -- start 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc2e200700 1 -- start start 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc2e200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc28100df0 0x7fcc28073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc2e200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc281017a0 0x7fcc280735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc2e200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28073b10 con 0x7fcc28100df0 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc2e200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28073c50 con 0x7fcc281017a0 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc27400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc28100df0 0x7fcc28073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc27400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc28100df0 0x7fcc28073090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55650/0 (socket says 172.21.15.176:55650) 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.519+0000 7fcc27400700 1 -- 172.21.15.176:0/99812722 learned_addr learned my addr 172.21.15.176:0/99812722 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc26a00700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc281017a0 0x7fcc280735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:35.509 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc27400700 1 -- 172.21.15.176:0/99812722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc281017a0 msgr2=0x7fcc280735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc27400700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc281017a0 0x7fcc280735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc27400700 1 -- 172.21.15.176:0/99812722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc140095f0 con 0x7fcc28100df0 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc27400700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc28100df0 0x7fcc28073090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d8d0 tx=0x7fcc1800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc1fe00700 1 -- 172.21.15.176:0/99812722 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc18009940 con 0x7fcc28100df0 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc1fe00700 1 -- 172.21.15.176:0/99812722 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc18010460 con 0x7fcc28100df0 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.520+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc281a2970 con 0x7fcc28100df0 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.521+0000 7fcc1fe00700 1 -- 172.21.15.176:0/99812722 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc1800f5f0 con 0x7fcc28100df0 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.521+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc281a2d80 con 0x7fcc28100df0 2024-03-13T14:57:35.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.522+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc281091f0 con 0x7fcc28100df0 2024-03-13T14:57:35.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.523+0000 7fcc1fe00700 1 -- 172.21.15.176:0/99812722 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fcc180105d0 con 0x7fcc28100df0 2024-03-13T14:57:35.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.524+0000 7fcc1fe00700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fcc0c06dfa0 0x7fcc0c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.524+0000 7fcc1fe00700 1 -- 172.21.15.176:0/99812722 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fcc1808deb0 con 0x7fcc28100df0 2024-03-13T14:57:35.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.524+0000 7fcc26a00700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fcc0c06dfa0 0x7fcc0c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:35.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.525+0000 7fcc26a00700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fcc0c06dfa0 0x7fcc0c070460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc14000c00 tx=0x7fcc14005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:35.514 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.528+0000 7fcc1fe00700 1 -- 172.21.15.176:0/99812722 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fcc18057300 con 0x7fcc28100df0 2024-03-13T14:57:35.603 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:35 smithi176 ceph-mon[112742]: pgmap v141: 65 pgs: 10 peering, 55 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:35.603 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:35 smithi176 ceph-mon[112742]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:35.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.845+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcc2802ccd0 con 0x7fcc28100df0 2024-03-13T14:57:35.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.845+0000 7fcc1fe00700 1 -- 172.21.15.176:0/99812722 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fcc18016330 con 0x7fcc28100df0 2024-03-13T14:57:35.831 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 3, 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 3 2024-03-13T14:57:35.832 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 7, 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 7 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:57:35.833 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fcc0c06dfa0 msgr2=0x7fcc0c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fcc0c06dfa0 0x7fcc0c070460 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc14000c00 tx=0x7fcc14005fd0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc28100df0 msgr2=0x7fcc28073090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc28100df0 0x7fcc28073090 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800d8d0 tx=0x7fcc1800dc90 comp rx=0 tx=0).stop 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 shutdown_connections 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fcc0c06dfa0 0x7fcc0c070460 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fcc28100df0 0x7fcc28073090 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 --2- 172.21.15.176:0/99812722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcc281017a0 0x7fcc280735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.848+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 >> 172.21.15.176:0/99812722 conn(0x7fcc280fc6a0 msgr2=0x7fcc280fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.849+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 shutdown_connections 2024-03-13T14:57:35.835 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:35.849+0000 7fcc2e200700 1 -- 172.21.15.176:0/99812722 wait complete. 2024-03-13T14:57:35.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:35 smithi196 ceph-mon[86193]: pgmap v141: 65 pgs: 10 peering, 55 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:35.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:35 smithi196 ceph-mon[86193]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:35.996 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.006+0000 7fa3c3e00700 1 -- 172.21.15.176:0/441620643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4101b60 msgr2=0x7fa3c4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:35.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.006+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/441620643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4101b60 0x7fa3c4101f40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009910 tx=0x7fa3b4009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:35.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.006+0000 7fa3c3e00700 1 -- 172.21.15.176:0/441620643 shutdown_connections 2024-03-13T14:57:35.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.006+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/441620643 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4102480 0x7fa3c410a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.006+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/441620643 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4101b60 0x7fa3c4101f40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:35.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.006+0000 7fa3c3e00700 1 -- 172.21.15.176:0/441620643 >> 172.21.15.176:0/441620643 conn(0x7fa3c40fb390 msgr2=0x7fa3c40fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:35.997 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.006+0000 7fa3c3e00700 1 -- 172.21.15.176:0/441620643 shutdown_connections 2024-03-13T14:57:35.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.007+0000 7fa3c3e00700 1 -- 172.21.15.176:0/441620643 wait complete. 2024-03-13T14:57:35.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.007+0000 7fa3c3e00700 1 Processor -- start 2024-03-13T14:57:35.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c3e00700 1 -- start start 2024-03-13T14:57:35.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c3e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101b60 0x7fa3c41966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.998 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4102480 0x7fa3c4196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:35.999 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c3e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c41972c0 con 0x7fa3c4102480 2024-03-13T14:57:35.999 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c3e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4199fa0 con 0x7fa3c4101b60 2024-03-13T14:57:35.999 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c2000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4102480 0x7fa3c4196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:35.999 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c2000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4102480 0x7fa3c4196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55680/0 (socket says 172.21.15.176:55680) 2024-03-13T14:57:35.999 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c2000700 1 -- 172.21.15.176:0/3668676921 learned_addr learned my addr 172.21.15.176:0/3668676921 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:35.999 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.008+0000 7fa3c2a00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101b60 0x7fa3c41966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3c2000700 1 -- 172.21.15.176:0/3668676921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101b60 msgr2=0x7fa3c41966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3c2000700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101b60 0x7fa3c41966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3c2000700 1 -- 172.21.15.176:0/3668676921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b40095f0 con 0x7fa3c4102480 2024-03-13T14:57:36.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3c2000700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4102480 0x7fa3c4196c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000b3d0 tx=0x7fa3b000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:36.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3bb400700 1 -- 172.21.15.176:0/3668676921 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b0011800 con 0x7fa3c4102480 2024-03-13T14:57:36.000 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3bb400700 1 -- 172.21.15.176:0/3668676921 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3b0011e20 con 0x7fa3c4102480 2024-03-13T14:57:36.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3bb400700 1 -- 172.21.15.176:0/3668676921 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b0010ea0 con 0x7fa3c4102480 2024-03-13T14:57:36.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.009+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c419a280 con 0x7fa3c4102480 2024-03-13T14:57:36.001 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.010+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c419a780 con 0x7fa3c4102480 2024-03-13T14:57:36.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.011+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c4108070 con 0x7fa3c4102480 2024-03-13T14:57:36.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.012+0000 7fa3bb400700 1 -- 172.21.15.176:0/3668676921 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa3b0010380 con 0x7fa3c4102480 2024-03-13T14:57:36.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.012+0000 7fa3bb400700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa3a806dfa0 0x7fa3a8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:36.003 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.013+0000 7fa3bb400700 1 -- 172.21.15.176:0/3668676921 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fa3b008d8e0 con 0x7fa3c4102480 2024-03-13T14:57:36.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.013+0000 7fa3c2a00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa3a806dfa0 0x7fa3a8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.013+0000 7fa3c2a00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa3a806dfa0 0x7fa3a8070460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400b5c0 tx=0x7fa3b401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:36.004 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.017+0000 7fa3bb400700 1 -- 172.21.15.176:0/3668676921 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa3b0056ca0 con 0x7fa3c4102480 2024-03-13T14:57:36.261 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.275+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa3c404f2e0 con 0x7fa3c4102480 2024-03-13T14:57:36.262 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.276+0000 7fa3bb400700 1 -- 172.21.15.176:0/3668676921 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7fa3b008be60 con 0x7fa3c4102480 2024-03-13T14:57:36.262 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:57:36.262 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:57:36.262 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:57:36.263 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:57:36.264 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:36.265 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:57:36.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa3a806dfa0 msgr2=0x7fa3a8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa3a806dfa0 0x7fa3a8070460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400b5c0 tx=0x7fa3b401a040 comp rx=0 tx=0).stop 2024-03-13T14:57:36.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4102480 msgr2=0x7fa3c4196c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.266 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4102480 0x7fa3c4196c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000b3d0 tx=0x7fa3b000b790 comp rx=0 tx=0).stop 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 shutdown_connections 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa3a806dfa0 0x7fa3a8070460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3c4101b60 0x7fa3c41966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 --2- 172.21.15.176:0/3668676921 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa3c4102480 0x7fa3c4196c30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.279+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 >> 172.21.15.176:0/3668676921 conn(0x7fa3c40fb390 msgr2=0x7fa3c41051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.280+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 shutdown_connections 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.280+0000 7fa3c3e00700 1 -- 172.21.15.176:0/3668676921 wait complete. 2024-03-13T14:57:36.267 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:57:36.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 -- 172.21.15.176:0/1786729747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 msgr2=0x7f575810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 --2- 172.21.15.176:0/1786729747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575810cf00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5744009910 tx=0x7f5744009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:36.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 -- 172.21.15.176:0/1786729747 shutdown_connections 2024-03-13T14:57:36.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 --2- 172.21.15.176:0/1786729747 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575810cf00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 --2- 172.21.15.176:0/1786729747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57580ff890 0x7f57580ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 -- 172.21.15.176:0/1786729747 >> 172.21.15.176:0/1786729747 conn(0x7f57580fb570 msgr2=0x7f57580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:36.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 -- 172.21.15.176:0/1786729747 shutdown_connections 2024-03-13T14:57:36.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.431+0000 7f575f600700 1 -- 172.21.15.176:0/1786729747 wait complete. 2024-03-13T14:57:36.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.433+0000 7f575f600700 1 Processor -- start 2024-03-13T14:57:36.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.433+0000 7f575f600700 1 -- start start 2024-03-13T14:57:36.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f575f600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57580ff890 0x7f575819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:36.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f575f600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:36.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f575f600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575819f2d0 con 0x7f5758100240 2024-03-13T14:57:36.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f575f600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5758198780 con 0x7f57580ff890 2024-03-13T14:57:36.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f5757e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f5757e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575819ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55694/0 (socket says 172.21.15.176:55694) 2024-03-13T14:57:36.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f5757e00700 1 -- 172.21.15.176:0/1387649307 learned_addr learned my addr 172.21.15.176:0/1387649307 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:36.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.434+0000 7f575cc00700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57580ff890 0x7f575819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f5757e00700 1 -- 172.21.15.176:0/1387649307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57580ff890 msgr2=0x7f575819e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f5757e00700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57580ff890 0x7f575819e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f5757e00700 1 -- 172.21.15.176:0/1387649307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57440095f0 con 0x7f5758100240 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f5757e00700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575819ec40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5744005b40 tx=0x7f57440056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f5755600700 1 -- 172.21.15.176:0/1387649307 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f574401d070 con 0x7f5758100240 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f5755600700 1 -- 172.21.15.176:0/1387649307 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f574400f460 con 0x7f5758100240 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5758198a00 con 0x7f5758100240 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f5755600700 1 -- 172.21.15.176:0/1387649307 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5744021640 con 0x7f5758100240 2024-03-13T14:57:36.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.435+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5758198f50 con 0x7f5758100240 2024-03-13T14:57:36.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.437+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575810a670 con 0x7f5758100240 2024-03-13T14:57:36.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.437+0000 7f5755600700 1 -- 172.21.15.176:0/1387649307 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f574400f5d0 con 0x7f5758100240 2024-03-13T14:57:36.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.439+0000 7f5755600700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f574006dfa0 0x7f5740070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:36.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.439+0000 7f5755600700 1 -- 172.21.15.176:0/1387649307 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6675+0+0 (secure 0 0 0) 0x7f574408f210 con 0x7f5758100240 2024-03-13T14:57:36.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.439+0000 7f575cc00700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f574006dfa0 0x7f5740070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.425 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.440+0000 7f575cc00700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f574006dfa0 0x7f5740070460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f574c005fd0 tx=0x7f574c00a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:36.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.443+0000 7f5755600700 1 -- 172.21.15.176:0/1387649307 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f574405c060 con 0x7f5758100240 2024-03-13T14:57:36.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:36.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:36.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:36.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:36.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:36.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/99812722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:36.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:36 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3668676921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:57:36.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.687+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f575802cce0 con 0x7f574006dfa0 2024-03-13T14:57:36.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.689+0000 7f5755600700 1 -- 172.21.15.176:0/1387649307 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f575802cce0 con 0x7f574006dfa0 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "9/23 daemons upgraded", 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading osd daemons", 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:57:36.676 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:57:36.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f574006dfa0 msgr2=0x7f5740070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f574006dfa0 0x7f5740070460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f574c005fd0 tx=0x7f574c00a560 comp rx=0 tx=0).stop 2024-03-13T14:57:36.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 msgr2=0x7f575819ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.678 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575819ec40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5744005b40 tx=0x7f57440056a0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 shutdown_connections 2024-03-13T14:57:36.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f574006dfa0 0x7f5740070460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57580ff890 0x7f575819e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 --2- 172.21.15.176:0/1387649307 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5758100240 0x7f575819ec40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.692+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 >> 172.21.15.176:0/1387649307 conn(0x7f57580fb570 msgr2=0x7f57580fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:36.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.693+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 shutdown_connections 2024-03-13T14:57:36.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.693+0000 7f575f600700 1 -- 172.21.15.176:0/1387649307 wait complete. 2024-03-13T14:57:36.836 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.850+0000 7fe059c00700 1 -- 172.21.15.176:0/1688160541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 msgr2=0x7fe05410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.850+0000 7fe059c00700 1 --2- 172.21.15.176:0/1688160541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe05410d660 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe044009910 tx=0x7fe044009c20 comp rx=0 tx=0).stop 2024-03-13T14:57:36.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.850+0000 7fe059c00700 1 -- 172.21.15.176:0/1688160541 shutdown_connections 2024-03-13T14:57:36.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.850+0000 7fe059c00700 1 --2- 172.21.15.176:0/1688160541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0541071d0 0x7fe054107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.850+0000 7fe059c00700 1 --2- 172.21.15.176:0/1688160541 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe05410d660 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.850+0000 7fe059c00700 1 -- 172.21.15.176:0/1688160541 >> 172.21.15.176:0/1688160541 conn(0x7fe0540769e0 msgr2=0x7fe054078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:36.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.850+0000 7fe059c00700 1 -- 172.21.15.176:0/1688160541 shutdown_connections 2024-03-13T14:57:36.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.851+0000 7fe059c00700 1 -- 172.21.15.176:0/1688160541 wait complete. 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.851+0000 7fe059c00700 1 Processor -- start 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe059c00700 1 -- start start 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe059c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0541071d0 0x7fe0541a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe059c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe0541a2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe059c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0541a3640 con 0x7fe05410d280 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe059c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05419cb00 con 0x7fe0541071d0 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe052000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe0541a2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe052000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe0541a2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55714/0 (socket says 172.21.15.176:55714) 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.852+0000 7fe052000700 1 -- 172.21.15.176:0/4269274709 learned_addr learned my addr 172.21.15.176:0/4269274709 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe052a00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0541071d0 0x7fe0541a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe052000700 1 -- 172.21.15.176:0/4269274709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0541071d0 msgr2=0x7fe0541a2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe052000700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0541071d0 0x7fe0541a2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe052000700 1 -- 172.21.15.176:0/4269274709 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0440095f0 con 0x7fe05410d280 2024-03-13T14:57:36.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe052a00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0541071d0 0x7fe0541a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T14:57:36.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe052000700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe0541a2f60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe04000d8d0 tx=0x7fe04000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:36.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe04b400700 1 -- 172.21.15.176:0/4269274709 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe040009940 con 0x7fe05410d280 2024-03-13T14:57:36.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe04b400700 1 -- 172.21.15.176:0/4269274709 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe040010460 con 0x7fe05410d280 2024-03-13T14:57:36.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05419cde0 con 0x7fe05410d280 2024-03-13T14:57:36.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.853+0000 7fe04b400700 1 -- 172.21.15.176:0/4269274709 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04000f5f0 con 0x7fe05410d280 2024-03-13T14:57:36.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.854+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05419d3b0 con 0x7fe05410d280 2024-03-13T14:57:36.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.855+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05404ea90 con 0x7fe05410d280 2024-03-13T14:57:36.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.856+0000 7fe04b400700 1 -- 172.21.15.176:0/4269274709 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe0400105d0 con 0x7fe05410d280 2024-03-13T14:57:36.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.857+0000 7fe04b400700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe03806dfa0 0x7fe038070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:57:36.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.857+0000 7fe04b400700 1 -- 172.21.15.176:0/4269274709 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6675+0+0 (secure 0 0 0) 0x7fe04008ddb0 con 0x7fe05410d280 2024-03-13T14:57:36.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.857+0000 7fe052a00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe03806dfa0 0x7fe038070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:57:36.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.858+0000 7fe052a00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe03806dfa0 0x7fe038070460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe044005b40 tx=0x7fe04401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:57:36.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:36.862+0000 7fe04b400700 1 -- 172.21.15.176:0/4269274709 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fe04005ac80 con 0x7fe05410d280 2024-03-13T14:57:36.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:36.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:36.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/99812722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:36.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:36 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3668676921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:57:37.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.172+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe05419dff0 con 0x7fe05410d280 2024-03-13T14:57:37.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.172+0000 7fe04b400700 1 -- 172.21.15.176:0/4269274709 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe04005a810 con 0x7fe05410d280 2024-03-13T14:57:37.159 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:57:37.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe03806dfa0 msgr2=0x7fe038070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe03806dfa0 0x7fe038070460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe044005b40 tx=0x7fe04401a040 comp rx=0 tx=0).stop 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 msgr2=0x7fe0541a2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe0541a2f60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe04000d8d0 tx=0x7fe04000dc90 comp rx=0 tx=0).stop 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 shutdown_connections 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe03806dfa0 0x7fe038070460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0541071d0 0x7fe0541a2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 --2- 172.21.15.176:0/4269274709 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe05410d280 0x7fe0541a2f60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 >> 172.21.15.176:0/4269274709 conn(0x7fe0540769e0 msgr2=0x7fe054078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.175+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 shutdown_connections 2024-03-13T14:57:37.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:57:37.176+0000 7fe059c00700 1 -- 172.21.15.176:0/4269274709 wait complete. 2024-03-13T14:57:37.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:37 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:37 smithi176 ceph-mon[112742]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-13T14:57:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:37 smithi176 ceph-mon[112742]: pgmap v142: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-13T14:57:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:37 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/4269274709' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:57:37.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:37 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:37.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:37 smithi196 ceph-mon[86193]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-03-13T14:57:37.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:37 smithi196 ceph-mon[86193]: pgmap v142: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-03-13T14:57:37.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:37 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/4269274709' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:57:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:38 smithi176 ceph-mon[112742]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:38.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:38 smithi196 ceph-mon[86193]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:57:39.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:39 smithi196 ceph-mon[86193]: pgmap v143: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:39.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:39 smithi176 ceph-mon[112742]: pgmap v143: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:41.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:41 smithi176 ceph-mon[112742]: pgmap v144: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-13T14:57:41.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:41 smithi196 ceph-mon[86193]: pgmap v144: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s 2024-03-13T14:57:43.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:43 smithi176 ceph-mon[112742]: pgmap v145: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 943 B/s rd, 1 op/s 2024-03-13T14:57:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:43 smithi196 ceph-mon[86193]: pgmap v145: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 943 B/s rd, 1 op/s 2024-03-13T14:57:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:45 smithi176 ceph-mon[112742]: pgmap v146: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:45.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:45.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:45.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:45.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:45 smithi196 ceph-mon[86193]: pgmap v146: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:45.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:45.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:57:45.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:47 smithi176 ceph-mon[112742]: pgmap v147: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:47.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:47 smithi196 ceph-mon[86193]: pgmap v147: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:49.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:49 smithi176 ceph-mon[112742]: pgmap v148: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:49 smithi196 ceph-mon[86193]: pgmap v148: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:51 smithi176 ceph-mon[112742]: pgmap v149: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:51.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:51.906 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:51 smithi196 ceph-mon[86193]: pgmap v149: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:51.907 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:52.870 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:52 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:52.870 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:52 smithi196 ceph-mon[86193]: Upgrade: osd.3 is safe to restart 2024-03-13T14:57:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:52 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-03-13T14:57:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:52 smithi176 ceph-mon[112742]: Upgrade: osd.3 is safe to restart 2024-03-13T14:57:53.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:53 smithi196 ceph-mon[86193]: pgmap v150: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:53.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:53.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-13T14:57:53.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:53 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:53.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:53 smithi176 ceph-mon[112742]: pgmap v150: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:53.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:57:53.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-13T14:57:53.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:53 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:57:54.816 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:54 smithi196 ceph-mon[86193]: Upgrade: Updating osd.3 2024-03-13T14:57:54.816 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:54 smithi196 ceph-mon[86193]: Deploying daemon osd.3 on smithi196 2024-03-13T14:57:54.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:54 smithi176 ceph-mon[112742]: Upgrade: Updating osd.3 2024-03-13T14:57:54.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:54 smithi176 ceph-mon[112742]: Deploying daemon osd.3 on smithi196 2024-03-13T14:57:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:55 smithi176 ceph-mon[112742]: pgmap v151: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:55 smithi196 ceph-mon[86193]: pgmap v151: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:57.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:57 smithi196 ceph-mon[86193]: pgmap v152: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:57.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:57 smithi176 ceph-mon[112742]: pgmap v152: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:57:58.627 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:57:58 smithi196 systemd[1]: Stopping Ceph osd.3 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:57:58.927 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:57:58 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[43397]: 2024-03-13T14:57:58.640+0000 7f16b1c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:57:58.927 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:57:58 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[43397]: 2024-03-13T14:57:58.640+0000 7f16b1c00700 -1 osd.3 81 *** Got signal Terminated *** 2024-03-13T14:57:58.927 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:57:58 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[43397]: 2024-03-13T14:57:58.640+0000 7f16b1c00700 -1 osd.3 81 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T14:57:59.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:59 smithi176 ceph-mon[112742]: pgmap v153: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:59.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:57:59 smithi176 ceph-mon[112742]: osd.3 marked itself down and dead 2024-03-13T14:57:59.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:59 smithi196 ceph-mon[86193]: pgmap v153: 65 pgs: 65 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:57:59.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:57:59 smithi196 ceph-mon[86193]: osd.3 marked itself down and dead 2024-03-13T14:58:00.394 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:00 smithi196 podman[95630]: 2024-03-13 14:58:00.111427911 +0000 UTC m=+1.627991711 container stop d5726d8c4f1b6f1235e1e78deee71df0ec88808864281fe8ebd0706c362c8010 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-13T14:58:00.394 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:00 smithi196 podman[95630]: 2024-03-13 14:58:00.140713522 +0000 UTC m=+1.657277322 container died d5726d8c4f1b6f1235e1e78deee71df0ec88808864281fe8ebd0706c362c8010 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=HEAD) 2024-03-13T14:58:00.677 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:00 smithi196 podman[95630]: 2024-03-13 14:58:00.5173781 +0000 UTC m=+2.033941974 container cleanup d5726d8c4f1b6f1235e1e78deee71df0ec88808864281fe8ebd0706c362c8010 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, ceph=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-13T14:58:00.677 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:00 smithi196 bash[95630]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3 2024-03-13T14:58:00.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:58:00.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: osdmap e82: 6 total, 5 up, 6 in 2024-03-13T14:58:00.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 3]}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.8"}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.e"}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.18"}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]: dispatch 2024-03-13T14:58:00.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1b"}]: dispatch 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: osdmap e82: 6 total, 5 up, 6 in 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]}]: dispatch 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]}]: dispatch 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 3]}]: dispatch 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.8"}]: dispatch 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.e"}]: dispatch 2024-03-13T14:58:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.18"}]: dispatch 2024-03-13T14:58:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]: dispatch 2024-03-13T14:58:00.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1b"}]: dispatch 2024-03-13T14:58:01.177 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:00 smithi196 podman[95650]: 2024-03-13 14:58:00.843974284 +0000 UTC m=+0.710165576 container remove d5726d8c4f1b6f1235e1e78deee71df0ec88808864281fe8ebd0706c362c8010 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_CLEAN=True) 2024-03-13T14:58:01.843 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: pgmap v155: 65 pgs: 15 stale+active+clean, 50 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-13T14:58:01.843 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 3]}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.8"}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.e"}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.18"}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1b"}]': finished 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: osdmap e83: 6 total, 5 up, 6 in 2024-03-13T14:58:01.844 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:01 smithi196 ceph-mon[86193]: osdmap e84: 6 total, 5 up, 6 in 2024-03-13T14:58:01.845 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:01 smithi196 podman[95737]: 2024-03-13 14:58:01.429831135 +0000 UTC m=+0.035082738 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:58:01.845 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:01 smithi196 podman[95737]: 2024-03-13 14:58:01.581395845 +0000 UTC m=+0.186647488 container create c0df90423d4c78f59b26f240629d2c274c9619eb7d7eecbf6d09a2befa42310c (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-13T14:58:01.845 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:01 smithi196 podman[95737]: 2024-03-13 14:58:01.857268891 +0000 UTC m=+0.462520570 container init c0df90423d4c78f59b26f240629d2c274c9619eb7d7eecbf6d09a2befa42310c (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, ceph=True) 2024-03-13T14:58:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: pgmap v155: 65 pgs: 15 stale+active+clean, 50 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s 2024-03-13T14:58:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.9", "id": [0, 3]}]': finished 2024-03-13T14:58:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.15", "id": [0, 2]}]': finished 2024-03-13T14:58:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1e", "id": [0, 3]}]': finished 2024-03-13T14:58:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.8"}]': finished 2024-03-13T14:58:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.e"}]': finished 2024-03-13T14:58:01.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.18"}]': finished 2024-03-13T14:58:01.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.16"}]': finished 2024-03-13T14:58:01.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "3.1b"}]': finished 2024-03-13T14:58:01.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: osdmap e83: 6 total, 5 up, 6 in 2024-03-13T14:58:01.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:01 smithi176 ceph-mon[112742]: osdmap e84: 6 total, 5 up, 6 in 2024-03-13T14:58:02.177 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:01 smithi196 podman[95737]: 2024-03-13 14:58:01.868520091 +0000 UTC m=+0.473771705 container start c0df90423d4c78f59b26f240629d2c274c9619eb7d7eecbf6d09a2befa42310c (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-13T14:58:02.177 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:01 smithi196 podman[95737]: 2024-03-13 14:58:01.972451899 +0000 UTC m=+0.577703581 container attach c0df90423d4c78f59b26f240629d2c274c9619eb7d7eecbf6d09a2befa42310c (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-13T14:58:02.634 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:02 smithi196 podman[95737]: 2024-03-13 14:58:02.296420755 +0000 UTC m=+0.901672397 container died c0df90423d4c78f59b26f240629d2c274c9619eb7d7eecbf6d09a2befa42310c (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T14:58:02.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:02 smithi196 ceph-mon[86193]: osdmap e85: 6 total, 5 up, 6 in 2024-03-13T14:58:02.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:02 smithi196 ceph-mon[86193]: pgmap v159: 65 pgs: 9 active+undersized, 3 remapped+peering, 6 active+undersized+degraded, 6 stale+active+clean, 41 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23/225 objects degraded (10.222%) 2024-03-13T14:58:03.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:02 smithi176 ceph-mon[112742]: osdmap e85: 6 total, 5 up, 6 in 2024-03-13T14:58:03.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:02 smithi176 ceph-mon[112742]: pgmap v159: 65 pgs: 9 active+undersized, 3 remapped+peering, 6 active+undersized+degraded, 6 stale+active+clean, 41 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23/225 objects degraded (10.222%) 2024-03-13T14:58:03.362 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:02 smithi196 podman[95763]: 2024-03-13 14:58:02.992843369 +0000 UTC m=+0.680647387 container remove c0df90423d4c78f59b26f240629d2c274c9619eb7d7eecbf6d09a2befa42310c (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-03-13T14:58:03.362 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:03 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.3.service: Deactivated successfully. 2024-03-13T14:58:03.362 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:03 smithi196 systemd[1]: Stopped Ceph osd.3 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:58:03.362 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:03 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.3.service: Consumed 2min 15.046s CPU time. 2024-03-13T14:58:03.676 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:03 smithi196 systemd[1]: Starting Ceph osd.3 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:58:04.075 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:03 smithi196 ceph-mon[86193]: Health check failed: Degraded data redundancy: 23/225 objects degraded (10.222%), 6 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:04.075 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:03 smithi196 ceph-mon[86193]: osdmap e86: 6 total, 5 up, 6 in 2024-03-13T14:58:04.076 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:03 smithi196 podman[95870]: 2024-03-13 14:58:03.642251198 +0000 UTC m=+0.045584699 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:58:04.076 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:03 smithi196 podman[95870]: 2024-03-13 14:58:03.760443566 +0000 UTC m=+0.163777106 container create 7502913e79916238a6066506400f860e8ea5ac84ad9106d8ebd59e80c6345776 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:58:04.076 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 podman[95870]: 2024-03-13 14:58:04.089456627 +0000 UTC m=+0.492790171 container init 7502913e79916238a6066506400f860e8ea5ac84ad9106d8ebd59e80c6345776 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-03-13T14:58:04.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:03 smithi176 ceph-mon[112742]: Health check failed: Degraded data redundancy: 23/225 objects degraded (10.222%), 6 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:04.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:03 smithi176 ceph-mon[112742]: osdmap e86: 6 total, 5 up, 6 in 2024-03-13T14:58:04.426 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 podman[95870]: 2024-03-13 14:58:04.101790453 +0000 UTC m=+0.505123960 container start 7502913e79916238a6066506400f860e8ea5ac84ad9106d8ebd59e80c6345776 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-13T14:58:04.427 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 podman[95870]: 2024-03-13 14:58:04.188222192 +0000 UTC m=+0.591555779 container attach 7502913e79916238a6066506400f860e8ea5ac84ad9106d8ebd59e80c6345776 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-03-13T14:58:05.122 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-mon[86193]: osdmap e87: 6 total, 5 up, 6 in 2024-03-13T14:58:05.123 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-mon[86193]: pgmap v162: 65 pgs: 4 active+clean+remapped, 18 active+undersized, 4 remapped+peering, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 6/225 objects misplaced (2.667%); 4.2 MiB/s, 6 objects/s recovering 2024-03-13T14:58:05.123 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate[95886]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-13T14:58:05.123 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate[95886]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-13T14:58:05.123 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 bash[95870]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-13T14:58:05.123 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 bash[95870]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-03-13T14:58:05.123 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 bash[95870]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-13T14:58:05.123 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 bash[95870]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 bash[95870]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 bash[95870]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 bash[95870]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate[95886]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate[95886]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate[95886]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate[95886]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate[95886]: --> ceph-volume raw activate successful for osd ID: 3 2024-03-13T14:58:05.124 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:05 smithi196 podman[95870]: 2024-03-13 14:58:05.007342563 +0000 UTC m=+1.410676102 container died 7502913e79916238a6066506400f860e8ea5ac84ad9106d8ebd59e80c6345776 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-03-13T14:58:05.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:04 smithi176 ceph-mon[112742]: osdmap e87: 6 total, 5 up, 6 in 2024-03-13T14:58:05.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:04 smithi176 ceph-mon[112742]: pgmap v162: 65 pgs: 4 active+clean+remapped, 18 active+undersized, 4 remapped+peering, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 6/225 objects misplaced (2.667%); 4.2 MiB/s, 6 objects/s recovering 2024-03-13T14:58:06.176 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:05 smithi196 podman[96006]: 2024-03-13 14:58:05.9170973 +0000 UTC m=+0.887054902 container remove 7502913e79916238a6066506400f860e8ea5ac84ad9106d8ebd59e80c6345776 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-13T14:58:06.566 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:06 smithi196 podman[96066]: 2024-03-13 14:58:06.180656382 +0000 UTC m=+0.035684927 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:58:06.566 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:06 smithi196 podman[96066]: 2024-03-13 14:58:06.284895454 +0000 UTC m=+0.139924017 container create a186117c65ba776ba9db2f1af9ed12450ee0bff9100fff3f67e26616c4330b62 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD) 2024-03-13T14:58:06.566 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:06 smithi196 podman[96066]: 2024-03-13 14:58:06.580213581 +0000 UTC m=+0.435242151 container init a186117c65ba776ba9db2f1af9ed12450ee0bff9100fff3f67e26616c4330b62 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-13T14:58:06.926 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:06 smithi196 podman[96066]: 2024-03-13 14:58:06.590962919 +0000 UTC m=+0.445991464 container start a186117c65ba776ba9db2f1af9ed12450ee0bff9100fff3f67e26616c4330b62 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-13T14:58:06.927 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:06 smithi196 ceph-osd[96085]: -- 172.21.15.196:0/1475538911 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560971493680 con 0x56097145d000 2024-03-13T14:58:06.927 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:06 smithi196 bash[96066]: a186117c65ba776ba9db2f1af9ed12450ee0bff9100fff3f67e26616c4330b62 2024-03-13T14:58:06.927 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:06 smithi196 systemd[1]: Started Ceph osd.3 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.344+0000 7fc62c400700 1 -- 172.21.15.176:0/1951516104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 msgr2=0x7fc6241011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.344+0000 7fc62c400700 1 --2- 172.21.15.176:0/1951516104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc6241011f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc618009910 tx=0x7fc618009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.345+0000 7fc62c400700 1 -- 172.21.15.176:0/1951516104 shutdown_connections 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.345+0000 7fc62c400700 1 --2- 172.21.15.176:0/1951516104 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6241017c0 0x7fc624105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.345+0000 7fc62c400700 1 --2- 172.21.15.176:0/1951516104 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc6241011f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.345+0000 7fc62c400700 1 -- 172.21.15.176:0/1951516104 >> 172.21.15.176:0/1951516104 conn(0x7fc6240fc6c0 msgr2=0x7fc6240feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.345+0000 7fc62c400700 1 -- 172.21.15.176:0/1951516104 shutdown_connections 2024-03-13T14:58:07.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.345+0000 7fc62c400700 1 -- 172.21.15.176:0/1951516104 wait complete. 2024-03-13T14:58:07.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.346+0000 7fc62c400700 1 Processor -- start 2024-03-13T14:58:07.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.346+0000 7fc62c400700 1 -- start start 2024-03-13T14:58:07.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc62c400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc62419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:07.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc62c400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6241017c0 0x7fc62419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:07.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc62c400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62419f390 con 0x7fc624100e10 2024-03-13T14:58:07.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc62c400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc624198840 con 0x7fc6241017c0 2024-03-13T14:58:07.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc62419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:07.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc62419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55808/0 (socket says 172.21.15.176:55808) 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629a00700 1 -- 172.21.15.176:0/2143369162 learned_addr learned my addr 172.21.15.176:0/2143369162 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629000700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6241017c0 0x7fc62419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629a00700 1 -- 172.21.15.176:0/2143369162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6241017c0 msgr2=0x7fc62419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629a00700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6241017c0 0x7fc62419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629a00700 1 -- 172.21.15.176:0/2143369162 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6180095f0 con 0x7fc624100e10 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.347+0000 7fc629000700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6241017c0 0x7fc62419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.348+0000 7fc629a00700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc62419e7c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b5c0 tx=0x7fc61800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.348+0000 7fc622000700 1 -- 172.21.15.176:0/2143369162 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61801d070 con 0x7fc624100e10 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.348+0000 7fc622000700 1 -- 172.21.15.176:0/2143369162 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc618005e00 con 0x7fc624100e10 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.348+0000 7fc622000700 1 -- 172.21.15.176:0/2143369162 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc618017400 con 0x7fc624100e10 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.348+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc624198ac0 con 0x7fc624100e10 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.348+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc624198fb0 con 0x7fc624100e10 2024-03-13T14:58:07.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.349+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc624104f90 con 0x7fc624100e10 2024-03-13T14:58:07.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.350+0000 7fc622000700 1 -- 172.21.15.176:0/2143369162 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc61800f460 con 0x7fc624100e10 2024-03-13T14:58:07.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.351+0000 7fc622000700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc60c06dfa0 0x7fc60c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:07.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.351+0000 7fc622000700 1 -- 172.21.15.176:0/2143369162 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6881+0+0 (secure 0 0 0) 0x7fc61808f410 con 0x7fc624100e10 2024-03-13T14:58:07.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.352+0000 7fc629000700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc60c06dfa0 0x7fc60c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:07.338 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.352+0000 7fc629000700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc60c06dfa0 0x7fc60c070460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc610005fd0 tx=0x7fc610005ee0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:07.342 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.356+0000 7fc622000700 1 -- 172.21.15.176:0/2143369162 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc61805c230 con 0x7fc624100e10 2024-03-13T14:58:07.422 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:07 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T14:58:07.158+0000 7f4e523067c0 -1 Falling back to public interface 2024-03-13T14:58:07.575 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.589+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6241991d0 con 0x7fc60c06dfa0 2024-03-13T14:58:07.576 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.591+0000 7fc622000700 1 -- 172.21.15.176:0/2143369162 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc6241991d0 con 0x7fc60c06dfa0 2024-03-13T14:58:07.579 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.593+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc60c06dfa0 msgr2=0x7fc60c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.593+0000 7fc62c400700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc60c06dfa0 0x7fc60c070460 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc610005fd0 tx=0x7fc610005ee0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.593+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 msgr2=0x7fc62419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.593+0000 7fc62c400700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc62419e7c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b5c0 tx=0x7fc61800be20 comp rx=0 tx=0).stop 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.594+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 shutdown_connections 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.594+0000 7fc62c400700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc60c06dfa0 0x7fc60c070460 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.594+0000 7fc62c400700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc624100e10 0x7fc62419e7c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.594+0000 7fc62c400700 1 --2- 172.21.15.176:0/2143369162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6241017c0 0x7fc62419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.594+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 >> 172.21.15.176:0/2143369162 conn(0x7fc6240fc6c0 msgr2=0x7fc6240fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.594+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 shutdown_connections 2024-03-13T14:58:07.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.594+0000 7fc62c400700 1 -- 172.21.15.176:0/2143369162 wait complete. 2024-03-13T14:58:07.594 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:58:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:07 smithi176 ceph-mon[112742]: pgmap v163: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 1 remapped+peering, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 11/225 objects misplaced (4.889%); 2.9 MiB/s, 4 objects/s recovering 2024-03-13T14:58:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:07.594 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:58:07.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:07 smithi196 ceph-mon[86193]: pgmap v163: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 1 remapped+peering, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 11/225 objects misplaced (4.889%); 2.9 MiB/s, 4 objects/s recovering 2024-03-13T14:58:07.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:07.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:07.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.750+0000 7f0bbb800700 1 -- 172.21.15.176:0/3630431175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 msgr2=0x7f0bb41033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.750+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3630431175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb41033c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009910 tx=0x7f0ba8009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.751+0000 7f0bbb800700 1 -- 172.21.15.176:0/3630431175 shutdown_connections 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.751+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3630431175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb4103990 0x7f0bb41079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.751+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3630431175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb41033c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.751+0000 7f0bbb800700 1 -- 172.21.15.176:0/3630431175 >> 172.21.15.176:0/3630431175 conn(0x7f0bb40fe870 msgr2=0x7f0bb4100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.751+0000 7f0bbb800700 1 -- 172.21.15.176:0/3630431175 shutdown_connections 2024-03-13T14:58:07.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.751+0000 7f0bbb800700 1 -- 172.21.15.176:0/3630431175 wait complete. 2024-03-13T14:58:07.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.752+0000 7f0bbb800700 1 Processor -- start 2024-03-13T14:58:07.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.752+0000 7f0bbb800700 1 -- start start 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bbb800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bbb800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb4103990 0x7f0bb419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bbb800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb419f360 con 0x7f0bb4102fe0 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bbb800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb4198810 con 0x7f0bb4103990 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bb8e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb4103990 0x7f0bb419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bb3e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb4103990 0x7f0bb419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:39386/0 (socket says 172.21.15.176:39386) 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bb8e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55826/0 (socket says 172.21.15.176:55826) 2024-03-13T14:58:07.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.753+0000 7f0bb3e00700 1 -- 172.21.15.176:0/3014059449 learned_addr learned my addr 172.21.15.176:0/3014059449 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:07.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bb8e00700 1 -- 172.21.15.176:0/3014059449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb4103990 msgr2=0x7f0bb419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bb8e00700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb4103990 0x7f0bb419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bb8e00700 1 -- 172.21.15.176:0/3014059449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80095f0 con 0x7f0bb4102fe0 2024-03-13T14:58:07.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bb8e00700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb419e790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009fd0 tx=0x7f0ba800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:07.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bb1600700 1 -- 172.21.15.176:0/3014059449 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba801d070 con 0x7f0bb4102fe0 2024-03-13T14:58:07.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bb1600700 1 -- 172.21.15.176:0/3014059449 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ba80050d0 con 0x7f0bb4102fe0 2024-03-13T14:58:07.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb4198a90 con 0x7f0bb4102fe0 2024-03-13T14:58:07.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bb1600700 1 -- 172.21.15.176:0/3014059449 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ba8003e90 con 0x7f0bb4102fe0 2024-03-13T14:58:07.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.754+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb4198f80 con 0x7f0bb4102fe0 2024-03-13T14:58:07.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.755+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb410b380 con 0x7f0bb4102fe0 2024-03-13T14:58:07.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.756+0000 7f0bb1600700 1 -- 172.21.15.176:0/3014059449 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0ba8003680 con 0x7f0bb4102fe0 2024-03-13T14:58:07.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.757+0000 7f0bb1600700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0b9c06dfa0 0x7f0b9c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:07.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.757+0000 7f0bb1600700 1 -- 172.21.15.176:0/3014059449 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6881+0+0 (secure 0 0 0) 0x7f0ba808f380 con 0x7f0bb4102fe0 2024-03-13T14:58:07.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.758+0000 7f0bb3e00700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0b9c06dfa0 0x7f0b9c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:07.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.758+0000 7f0bb3e00700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0b9c06dfa0 0x7f0b9c070460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40ff040 tx=0x7f0ba0009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:07.748 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:07.762+0000 7f0bb1600700 1 -- 172.21.15.176:0/3014059449 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f0ba805c100 con 0x7f0bb4102fe0 2024-03-13T14:58:07.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.002+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bb4199260 con 0x7f0b9c06dfa0 2024-03-13T14:58:07.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.005+0000 7f0bb1600700 1 -- 172.21.15.176:0/3014059449 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0bb4199260 con 0x7f0b9c06dfa0 2024-03-13T14:58:07.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0b9c06dfa0 msgr2=0x7f0b9c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0b9c06dfa0 0x7f0b9c070460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0bb40ff040 tx=0x7f0ba0009260 comp rx=0 tx=0).stop 2024-03-13T14:58:07.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 msgr2=0x7f0bb419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:07.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb419e790 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009fd0 tx=0x7f0ba800f690 comp rx=0 tx=0).stop 2024-03-13T14:58:07.994 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 shutdown_connections 2024-03-13T14:58:07.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0b9c06dfa0 0x7f0b9c070460 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0bb4102fe0 0x7f0bb419e790 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 --2- 172.21.15.176:0/3014059449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb4103990 0x7f0bb419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:07.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 >> 172.21.15.176:0/3014059449 conn(0x7f0bb40fe870 msgr2=0x7f0bb4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:07.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 shutdown_connections 2024-03-13T14:58:07.995 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.008+0000 7f0bbb800700 1 -- 172.21.15.176:0/3014059449 wait complete. 2024-03-13T14:58:08.151 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 -- 172.21.15.176:0/4018706966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 msgr2=0x7fe1f4073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/4018706966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f4073540 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009910 tx=0x7fe1e4009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:08.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 -- 172.21.15.176:0/4018706966 shutdown_connections 2024-03-13T14:58:08.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/4018706966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1f4073a80 0x7fe1f410cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/4018706966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f4073540 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 -- 172.21.15.176:0/4018706966 >> 172.21.15.176:0/4018706966 conn(0x7fe1f40fc650 msgr2=0x7fe1f40fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:08.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 -- 172.21.15.176:0/4018706966 shutdown_connections 2024-03-13T14:58:08.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.165+0000 7fe1f8e00700 1 -- 172.21.15.176:0/4018706966 wait complete. 2024-03-13T14:58:08.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.166+0000 7fe1f8e00700 1 Processor -- start 2024-03-13T14:58:08.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.166+0000 7fe1f8e00700 1 -- start start 2024-03-13T14:58:08.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.167+0000 7fe1f8e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.167+0000 7fe1f8e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1f4073a80 0x7fe1f419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.167+0000 7fe1f8e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f419f410 con 0x7fe1f4073160 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.167+0000 7fe1f8e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f41988c0 con 0x7fe1f4073a80 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.167+0000 7fe1f1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.167+0000 7fe1f1e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55856/0 (socket says 172.21.15.176:55856) 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.167+0000 7fe1f1e00700 1 -- 172.21.15.176:0/3832379087 learned_addr learned my addr 172.21.15.176:0/3832379087 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1f1400700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1f4073a80 0x7fe1f419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1f1e00700 1 -- 172.21.15.176:0/3832379087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1f4073a80 msgr2=0x7fe1f419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1f1e00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1f4073a80 0x7fe1f419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.154 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1f1e00700 1 -- 172.21.15.176:0/3832379087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e40095f0 con 0x7fe1f4073160 2024-03-13T14:58:08.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1f1e00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f419e840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40092e0 tx=0x7fe1e400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:08.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1eaa00700 1 -- 172.21.15.176:0/3832379087 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e401d070 con 0x7fe1f4073160 2024-03-13T14:58:08.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1eaa00700 1 -- 172.21.15.176:0/3832379087 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1e400f460 con 0x7fe1f4073160 2024-03-13T14:58:08.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.168+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f4198b40 con 0x7fe1f4073160 2024-03-13T14:58:08.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.169+0000 7fe1eaa00700 1 -- 172.21.15.176:0/3832379087 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e4017630 con 0x7fe1f4073160 2024-03-13T14:58:08.155 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.169+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f4199030 con 0x7fe1f4073160 2024-03-13T14:58:08.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.170+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f410a5c0 con 0x7fe1f4073160 2024-03-13T14:58:08.156 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.171+0000 7fe1eaa00700 1 -- 172.21.15.176:0/3832379087 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe1e400f5d0 con 0x7fe1f4073160 2024-03-13T14:58:08.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.172+0000 7fe1eaa00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1d806dfa0 0x7fe1d8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:08.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.172+0000 7fe1eaa00700 1 -- 172.21.15.176:0/3832379087 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6881+0+0 (secure 0 0 0) 0x7fe1e408f4e0 con 0x7fe1f4073160 2024-03-13T14:58:08.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.172+0000 7fe1f1400700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1d806dfa0 0x7fe1d8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:08.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.173+0000 7fe1f1400700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1d806dfa0 0x7fe1d8070460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006fd0 tx=0x7fe1e0008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:08.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.177+0000 7fe1eaa00700 1 -- 172.21.15.176:0/3832379087 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fe1e405c300 con 0x7fe1f4073160 2024-03-13T14:58:08.392 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.406+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1f4199250 con 0x7fe1d806dfa0 2024-03-13T14:58:08.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.437+0000 7fe1eaa00700 1 -- 172.21.15.176:0/3832379087 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe1f4199250 con 0x7fe1d806dfa0 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (6m) 39s ago 19m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (19m) 39s ago 19m 8812k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (18m) 2m ago 18m 8238k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (2m) 39s ago 19m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (2m) 2m ago 18m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (5m) 39s ago 19m 73.6M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (12m) 39s ago 12m 87.3M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (12m) 39s ago 12m 17.9M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (12m) 2m ago 12m 141M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (12m) 2m ago 12m 16.9M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:58:08.424 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (8m) 39s ago 21m 504M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (7m) 2m ago 18m 449M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (4m) 39s ago 21m 47.3M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (3m) 2m ago 18m 36.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (7m) 39s ago 19m 7520k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (7m) 2m ago 18m 7910k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (2m) 39s ago 17m 122M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (77s) 39s ago 16m 97.2M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (47s) 39s ago 15m 11.1M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 starting - - - 2241M 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (14m) 2m ago 14m 253M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (14m) 2m ago 14m 230M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:58:08.425 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (6m) 39s ago 18m 69.1M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:58:08.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1d806dfa0 msgr2=0x7fe1d8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1d806dfa0 0x7fe1d8070460 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0006fd0 tx=0x7fe1e0008040 comp rx=0 tx=0).stop 2024-03-13T14:58:08.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 msgr2=0x7fe1f419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f419e840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40092e0 tx=0x7fe1e400be00 comp rx=0 tx=0).stop 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 shutdown_connections 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1d806dfa0 0x7fe1d8070460 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1f4073160 0x7fe1f419e840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 --2- 172.21.15.176:0/3832379087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1f4073a80 0x7fe1f419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.440+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 >> 172.21.15.176:0/3832379087 conn(0x7fe1f40fc650 msgr2=0x7fe1f4107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.441+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 shutdown_connections 2024-03-13T14:58:08.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.441+0000 7fe1f8e00700 1 -- 172.21.15.176:0/3832379087 wait complete. 2024-03-13T14:58:08.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 -- 172.21.15.176:0/1908733370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 msgr2=0x7f70641057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.577 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 --2- 172.21.15.176:0/1908733370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f70641057a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7050009910 tx=0x7f7050009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:08.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 -- 172.21.15.176:0/1908733370 shutdown_connections 2024-03-13T14:58:08.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 --2- 172.21.15.176:0/1908733370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f70641057a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 --2- 172.21.15.176:0/1908733370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064100da0 0x7f7064101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 -- 172.21.15.176:0/1908733370 >> 172.21.15.176:0/1908733370 conn(0x7f70640fc630 msgr2=0x7f70640fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:08.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 -- 172.21.15.176:0/1908733370 shutdown_connections 2024-03-13T14:58:08.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.591+0000 7f706c400700 1 -- 172.21.15.176:0/1908733370 wait complete. 2024-03-13T14:58:08.579 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.592+0000 7f706c400700 1 Processor -- start 2024-03-13T14:58:08.579 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.592+0000 7f706c400700 1 -- start start 2024-03-13T14:58:08.579 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f706c400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064100da0 0x7f706419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:08.579 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f706c400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f706419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:08.579 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f706c400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706419d0d0 con 0x7f7064101750 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f706c400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064196580 con 0x7f7064100da0 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f7069000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f706419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f7069000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f706419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55880/0 (socket says 172.21.15.176:55880) 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f7069000700 1 -- 172.21.15.176:0/3198955290 learned_addr learned my addr 172.21.15.176:0/3198955290 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.593+0000 7f7069a00700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064100da0 0x7f706419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f7069000700 1 -- 172.21.15.176:0/3198955290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064100da0 msgr2=0x7f706419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f7069000700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064100da0 0x7f706419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.580 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f7069000700 1 -- 172.21.15.176:0/3198955290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500095f0 con 0x7f7064101750 2024-03-13T14:58:08.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f7069000700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f706419ca40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7050005b40 tx=0x7f705000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:08.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f7062000700 1 -- 172.21.15.176:0/3198955290 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f705001d070 con 0x7f7064101750 2024-03-13T14:58:08.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f7062000700 1 -- 172.21.15.176:0/3198955290 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f705000bed0 con 0x7f7064101750 2024-03-13T14:58:08.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7064196800 con 0x7f7064101750 2024-03-13T14:58:08.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f7062000700 1 -- 172.21.15.176:0/3198955290 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7050003e70 con 0x7f7064101750 2024-03-13T14:58:08.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.594+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7064196cf0 con 0x7f7064101750 2024-03-13T14:58:08.581 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.596+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70641091a0 con 0x7f7064101750 2024-03-13T14:58:08.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.596+0000 7f7062000700 1 -- 172.21.15.176:0/3198955290 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f70500050d0 con 0x7f7064101750 2024-03-13T14:58:08.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.597+0000 7f7062000700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f704c06dfa0 0x7f704c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:08.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.598+0000 7f7062000700 1 -- 172.21.15.176:0/3198955290 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6881+0+0 (secure 0 0 0) 0x7f705008ed70 con 0x7f7064101750 2024-03-13T14:58:08.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.598+0000 7f7069a00700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f704c06dfa0 0x7f704c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:08.584 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.599+0000 7f7069a00700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f704c06dfa0 0x7f704c070460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f705800f540 tx=0x7f705800b040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:08.589 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.602+0000 7f7062000700 1 -- 172.21.15.176:0/3198955290 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f705005bc10 con 0x7f7064101750 2024-03-13T14:58:08.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.895+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f706404ea90 con 0x7f7064101750 2024-03-13T14:58:08.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.897+0000 7f7062000700 1 -- 172.21.15.176:0/3198955290 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f705005b7a0 con 0x7f7064101750 2024-03-13T14:58:08.883 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:58:08.883 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:58:08.883 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:58:08.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:08.883 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:58:08.883 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:58:08.883 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 3 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-03-13T14:58:08.884 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 7 2024-03-13T14:58:08.885 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:58:08.885 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:58:08.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f704c06dfa0 msgr2=0x7f704c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f704c06dfa0 0x7f704c070460 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f705800f540 tx=0x7f705800b040 comp rx=0 tx=0).stop 2024-03-13T14:58:08.886 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 msgr2=0x7f706419ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:08.886 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f706419ca40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7050005b40 tx=0x7f705000f690 comp rx=0 tx=0).stop 2024-03-13T14:58:08.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 shutdown_connections 2024-03-13T14:58:08.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f704c06dfa0 0x7f704c070460 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7064100da0 0x7f706419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 --2- 172.21.15.176:0/3198955290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7064101750 0x7f706419ca40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:08.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 >> 172.21.15.176:0/3198955290 conn(0x7f70640fc630 msgr2=0x7f70640fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:08.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 shutdown_connections 2024-03-13T14:58:08.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:08.900+0000 7f706c400700 1 -- 172.21.15.176:0/3198955290 wait complete. 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.052+0000 7f4b37c00700 1 -- 172.21.15.176:0/758132018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 msgr2=0x7f4b30107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.052+0000 7f4b37c00700 1 --2- 172.21.15.176:0/758132018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b30107a00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009910 tx=0x7f4b1c009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.053+0000 7f4b37c00700 1 -- 172.21.15.176:0/758132018 shutdown_connections 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.053+0000 7f4b37c00700 1 --2- 172.21.15.176:0/758132018 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b30107a00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.053+0000 7f4b37c00700 1 --2- 172.21.15.176:0/758132018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b30103000 0x7f4b301033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.053+0000 7f4b37c00700 1 -- 172.21.15.176:0/758132018 >> 172.21.15.176:0/758132018 conn(0x7f4b300fe8d0 msgr2=0x7f4b30100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.053+0000 7f4b37c00700 1 -- 172.21.15.176:0/758132018 shutdown_connections 2024-03-13T14:58:09.039 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.053+0000 7f4b37c00700 1 -- 172.21.15.176:0/758132018 wait complete. 2024-03-13T14:58:09.041 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.054+0000 7f4b37c00700 1 Processor -- start 2024-03-13T14:58:09.041 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.054+0000 7f4b37c00700 1 -- start start 2024-03-13T14:58:09.041 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.054+0000 7f4b37c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b30103000 0x7f4b3019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.054+0000 7f4b37c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b3019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.054+0000 7f4b37c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3019f390 con 0x7f4b301039b0 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.054+0000 7f4b37c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b30198840 con 0x7f4b30103000 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b2fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b3019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b2fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b3019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55896/0 (socket says 172.21.15.176:55896) 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b2fe00700 1 -- 172.21.15.176:0/2466999631 learned_addr learned my addr 172.21.15.176:0/2466999631 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b35200700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b30103000 0x7f4b3019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b2fe00700 1 -- 172.21.15.176:0/2466999631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b30103000 msgr2=0x7f4b3019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b2fe00700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b30103000 0x7f4b3019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b2fe00700 1 -- 172.21.15.176:0/2466999631 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0095f0 con 0x7f4b301039b0 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.055+0000 7f4b2fe00700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b3019ed00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00bd70 tx=0x7f4b1c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.056+0000 7f4b2d600700 1 -- 172.21.15.176:0/2466999631 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1c01d070 con 0x7f4b301039b0 2024-03-13T14:58:09.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.056+0000 7f4b2d600700 1 -- 172.21.15.176:0/2466999631 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b1c005dc0 con 0x7f4b301039b0 2024-03-13T14:58:09.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.056+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b30198ac0 con 0x7f4b301039b0 2024-03-13T14:58:09.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.056+0000 7f4b2d600700 1 -- 172.21.15.176:0/2466999631 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1c017400 con 0x7f4b301039b0 2024-03-13T14:58:09.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.056+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b30198fb0 con 0x7f4b301039b0 2024-03-13T14:58:09.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.057+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3010b400 con 0x7f4b301039b0 2024-03-13T14:58:09.043 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.058+0000 7f4b2d600700 1 -- 172.21.15.176:0/2466999631 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4b1c00f7f0 con 0x7f4b301039b0 2024-03-13T14:58:09.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.059+0000 7f4b2d600700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b1806dfa0 0x7f4b18070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.060+0000 7f4b2d600700 1 -- 172.21.15.176:0/2466999631 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6881+0+0 (secure 0 0 0) 0x7f4b1c08f460 con 0x7f4b301039b0 2024-03-13T14:58:09.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.060+0000 7f4b35200700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b1806dfa0 0x7f4b18070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.060+0000 7f4b35200700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b1806dfa0 0x7f4b18070460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005950 tx=0x7f4b240058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:09.050 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.064+0000 7f4b2d600700 1 -- 172.21.15.176:0/2466999631 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4b1c05c1e0 con 0x7f4b301039b0 2024-03-13T14:58:09.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.317+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4b3004ea90 con 0x7f4b301039b0 2024-03-13T14:58:09.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.318+0000 7f4b2d600700 1 -- 172.21.15.176:0/2466999631 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f4b1c05bd70 con 0x7f4b301039b0 2024-03-13T14:58:09.305 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:58:09.306 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:58:09.307 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:09.308 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:09.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b1806dfa0 msgr2=0x7f4b18070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b1806dfa0 0x7f4b18070460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4b24005950 tx=0x7f4b240058e0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 msgr2=0x7f4b3019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b3019ed00 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00bd70 tx=0x7f4b1c00be50 comp rx=0 tx=0).stop 2024-03-13T14:58:09.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 shutdown_connections 2024-03-13T14:58:09.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b1806dfa0 0x7f4b18070460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b30103000 0x7f4b3019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 --2- 172.21.15.176:0/2466999631 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b301039b0 0x7f4b3019ed00 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 >> 172.21.15.176:0/2466999631 conn(0x7f4b300fe8d0 msgr2=0x7f4b30100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:09.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 shutdown_connections 2024-03-13T14:58:09.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.322+0000 7f4b37c00700 1 -- 172.21.15.176:0/2466999631 wait complete. 2024-03-13T14:58:09.310 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:58:09.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 -- 172.21.15.176:0/422556932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 msgr2=0x7f96681057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 --2- 172.21.15.176:0/422556932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96681057a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9654009910 tx=0x7f9654009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:09.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 -- 172.21.15.176:0/422556932 shutdown_connections 2024-03-13T14:58:09.452 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 --2- 172.21.15.176:0/422556932 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96681057a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.453 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 --2- 172.21.15.176:0/422556932 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9668100da0 0x7f9668101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.453 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 -- 172.21.15.176:0/422556932 >> 172.21.15.176:0/422556932 conn(0x7f96680fc630 msgr2=0x7f96680fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:09.453 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 -- 172.21.15.176:0/422556932 shutdown_connections 2024-03-13T14:58:09.453 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.466+0000 7f9670400700 1 -- 172.21.15.176:0/422556932 wait complete. 2024-03-13T14:58:09.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.467+0000 7f9670400700 1 Processor -- start 2024-03-13T14:58:09.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.467+0000 7f9670400700 1 -- start start 2024-03-13T14:58:09.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f9670400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9668100da0 0x7f9668073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.454 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f9670400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96680735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f9670400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668073b10 con 0x7f9668101750 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f9670400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9668073c50 con 0x7f9668100da0 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f966d000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96680735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f966d000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96680735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55924/0 (socket says 172.21.15.176:55924) 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f966d000700 1 -- 172.21.15.176:0/1990149465 learned_addr learned my addr 172.21.15.176:0/1990149465 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.468+0000 7f966da00700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9668100da0 0x7f9668073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f966d000700 1 -- 172.21.15.176:0/1990149465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9668100da0 msgr2=0x7f9668073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f966d000700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9668100da0 0x7f9668073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f966d000700 1 -- 172.21.15.176:0/1990149465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96540095f0 con 0x7f9668101750 2024-03-13T14:58:09.455 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f966d000700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96680735d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f965400bda0 tx=0x7f965400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:09.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f9666000700 1 -- 172.21.15.176:0/1990149465 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965401d070 con 0x7f9668101750 2024-03-13T14:58:09.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f9666000700 1 -- 172.21.15.176:0/1990149465 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9654004fb0 con 0x7f9668101750 2024-03-13T14:58:09.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f9666000700 1 -- 172.21.15.176:0/1990149465 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965400f460 con 0x7f9668101750 2024-03-13T14:58:09.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9668073ed0 con 0x7f9668101750 2024-03-13T14:58:09.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.469+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96681a2be0 con 0x7f9668101750 2024-03-13T14:58:09.456 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.470+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96681091a0 con 0x7f9668101750 2024-03-13T14:58:09.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.471+0000 7f9666000700 1 -- 172.21.15.176:0/1990149465 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f9654022800 con 0x7f9668101750 2024-03-13T14:58:09.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.472+0000 7f9666000700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f965006dfa0 0x7f9650070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.472+0000 7f9666000700 1 -- 172.21.15.176:0/1990149465 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6881+0+0 (secure 0 0 0) 0x7f965408f3a0 con 0x7f9668101750 2024-03-13T14:58:09.458 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.472+0000 7f966da00700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f965006dfa0 0x7f9650070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.459 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.473+0000 7f966da00700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f965006dfa0 0x7f9650070460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f965c005ea0 tx=0x7f965c005e30 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:09.459 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:09 smithi176 ceph-mon[112742]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:09.459 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:09 smithi176 ceph-mon[112742]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:09.459 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:09 smithi176 ceph-mon[112742]: pgmap v164: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 1 remapped+peering, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 11/225 objects misplaced (4.889%); 2.4 MiB/s, 3 objects/s recovering 2024-03-13T14:58:09.459 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:09 smithi176 ceph-mon[112742]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:09.459 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:09 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3198955290' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:09.459 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:09 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2466999631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:58:09.462 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.477+0000 7f9666000700 1 -- 172.21.15.176:0/1990149465 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f9654058840 con 0x7f9668101750 2024-03-13T14:58:09.687 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.700+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f966802cce0 con 0x7f965006dfa0 2024-03-13T14:58:09.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.702+0000 7f9666000700 1 -- 172.21.15.176:0/1990149465 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f966802cce0 con 0x7f965006dfa0 2024-03-13T14:58:09.689 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:58:09.689 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "9/23 daemons upgraded", 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading osd daemons", 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:58:09.690 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:58:09.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f965006dfa0 msgr2=0x7f9650070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f965006dfa0 0x7f9650070460 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f965c005ea0 tx=0x7f965c005e30 comp rx=0 tx=0).stop 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 msgr2=0x7f96680735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96680735d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f965400bda0 tx=0x7f965400be80 comp rx=0 tx=0).stop 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 shutdown_connections 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f965006dfa0 0x7f9650070460 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9668100da0 0x7f9668073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 --2- 172.21.15.176:0/1990149465 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9668101750 0x7f96680735d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 >> 172.21.15.176:0/1990149465 conn(0x7f96680fc630 msgr2=0x7f96680fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.705+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 shutdown_connections 2024-03-13T14:58:09.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.706+0000 7f9670400700 1 -- 172.21.15.176:0/1990149465 wait complete. 2024-03-13T14:58:09.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 -- 172.21.15.176:0/2290116551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc81071d0 msgr2=0x7f4cc8107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/2290116551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc81071d0 0x7f4cc8107650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009910 tx=0x7f4cb4009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:09.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 -- 172.21.15.176:0/2290116551 shutdown_connections 2024-03-13T14:58:09.848 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/2290116551 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc81071d0 0x7f4cc8107650 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/2290116551 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc810d280 0x7f4cc810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 -- 172.21.15.176:0/2290116551 >> 172.21.15.176:0/2290116551 conn(0x7f4cc80769e0 msgr2=0x7f4cc8078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:09.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 -- 172.21.15.176:0/2290116551 shutdown_connections 2024-03-13T14:58:09.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.861+0000 7f4ccfa00700 1 -- 172.21.15.176:0/2290116551 wait complete. 2024-03-13T14:58:09.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.862+0000 7f4ccfa00700 1 Processor -- start 2024-03-13T14:58:09.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.862+0000 7f4ccfa00700 1 -- start start 2024-03-13T14:58:09.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4ccfa00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc81071d0 0x7f4cc819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4ccfa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc810d280 0x7f4cc819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4ccfa00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc819d880 con 0x7f4cc810d280 2024-03-13T14:58:09.850 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4ccfa00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc81a05b0 con 0x7f4cc81071d0 2024-03-13T14:58:09.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4cc7e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc810d280 0x7f4cc819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4cc7e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc810d280 0x7f4cc819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55932/0 (socket says 172.21.15.176:55932) 2024-03-13T14:58:09.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4cc7e00700 1 -- 172.21.15.176:0/1726702994 learned_addr learned my addr 172.21.15.176:0/1726702994 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:09.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4ccd000700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc81071d0 0x7f4cc819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4cc7e00700 1 -- 172.21.15.176:0/1726702994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc81071d0 msgr2=0x7f4cc819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:09.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.863+0000 7f4cc7e00700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc81071d0 0x7f4cc819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:09.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.864+0000 7f4cc7e00700 1 -- 172.21.15.176:0/1726702994 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb40095f0 con 0x7f4cc810d280 2024-03-13T14:58:09.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.864+0000 7f4cc7e00700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc810d280 0x7f4cc819d1f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b5c0 tx=0x7f4cb400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:09.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.864+0000 7f4cc5600700 1 -- 172.21.15.176:0/1726702994 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb401d070 con 0x7f4cc810d280 2024-03-13T14:58:09.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.864+0000 7f4cc5600700 1 -- 172.21.15.176:0/1726702994 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cb400f460 con 0x7f4cc810d280 2024-03-13T14:58:09.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.864+0000 7f4cc5600700 1 -- 172.21.15.176:0/1726702994 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb4017610 con 0x7f4cc810d280 2024-03-13T14:58:09.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.864+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc81a0830 con 0x7f4cc810d280 2024-03-13T14:58:09.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.864+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc81a0da0 con 0x7f4cc810d280 2024-03-13T14:58:09.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.865+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc810f6c0 con 0x7f4cc810d280 2024-03-13T14:58:09.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.866+0000 7f4cc5600700 1 -- 172.21.15.176:0/1726702994 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4cb400f5d0 con 0x7f4cc810d280 2024-03-13T14:58:09.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.867+0000 7f4cc5600700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4cb006dfa0 0x7f4cb0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:09.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.867+0000 7f4cc5600700 1 -- 172.21.15.176:0/1726702994 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6881+0+0 (secure 0 0 0) 0x7f4cb408f5a0 con 0x7f4cc810d280 2024-03-13T14:58:09.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.868+0000 7f4ccd000700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4cb006dfa0 0x7f4cb0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:09.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.868+0000 7f4ccd000700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4cb006dfa0 0x7f4cb0070460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc005fd0 tx=0x7f4cbc00a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:09.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:09.872+0000 7f4cc5600700 1 -- 172.21.15.176:0/1726702994 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4cb40587f0 con 0x7f4cc810d280 2024-03-13T14:58:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:09 smithi196 ceph-mon[86193]: from='client.34338 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:09 smithi196 ceph-mon[86193]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:09 smithi196 ceph-mon[86193]: pgmap v164: 65 pgs: 7 active+clean+remapped, 18 active+undersized, 1 remapped+peering, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 11/225 objects misplaced (4.889%); 2.4 MiB/s, 3 objects/s recovering 2024-03-13T14:58:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:09 smithi196 ceph-mon[86193]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:09 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3198955290' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:09 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2466999631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:58:10.171 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.184+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4cc804ea90 con 0x7f4cc810d280 2024-03-13T14:58:10.172 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.186+0000 7f4cc5600700 1 -- 172.21.15.176:0/1726702994 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1049 (secure 0 0 0) 0x7f4cb405beb0 con 0x7f4cc810d280 2024-03-13T14:58:10.172 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 40/225 objects degraded (17.778%), 14 pgs degraded 2024-03-13T14:58:10.173 INFO:teuthology.orchestra.run.smithi176.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-13T14:58:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: osd.3 (root=default,host=smithi196) is down 2024-03-13T14:58:10.173 INFO:teuthology.orchestra.run.smithi176.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 40/225 objects degraded (17.778%), 14 pgs degraded 2024-03-13T14:58:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-03-13T14:58:10.173 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-03-13T14:58:10.174 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-03-13T14:58:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-03-13T14:58:10.175 INFO:teuthology.orchestra.run.smithi176.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-03-13T14:58:10.175 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.188+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4cb006dfa0 msgr2=0x7f4cb0070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:10.176 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.188+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4cb006dfa0 0x7f4cb0070460 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc005fd0 tx=0x7f4cbc00a560 comp rx=0 tx=0).stop 2024-03-13T14:58:10.176 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.188+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc810d280 msgr2=0x7f4cc819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:10.176 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.188+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc810d280 0x7f4cc819d1f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400b5c0 tx=0x7f4cb400be00 comp rx=0 tx=0).stop 2024-03-13T14:58:10.176 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.189+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 shutdown_connections 2024-03-13T14:58:10.176 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.189+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4cb006dfa0 0x7f4cb0070460 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:10.176 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.189+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cc81071d0 0x7f4cc819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:10.176 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.189+0000 7f4ccfa00700 1 --2- 172.21.15.176:0/1726702994 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4cc810d280 0x7f4cc819d1f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:10.177 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.189+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 >> 172.21.15.176:0/1726702994 conn(0x7f4cc80769e0 msgr2=0x7f4cc8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:10.177 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.189+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 shutdown_connections 2024-03-13T14:58:10.177 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:10.189+0000 7f4ccfa00700 1 -- 172.21.15.176:0/1726702994 wait complete. 2024-03-13T14:58:10.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:10 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1726702994' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:58:10.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:10 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1726702994' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:58:11.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:11 smithi176 ceph-mon[112742]: from='client.34358 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:11.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:11 smithi176 ceph-mon[112742]: pgmap v165: 65 pgs: 8 active+clean+remapped, 18 active+undersized, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 17/225 objects misplaced (7.556%); 2.0 MiB/s, 2 objects/s recovering 2024-03-13T14:58:11.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:11 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 40/225 objects degraded (17.778%), 14 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:11.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:11.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:11 smithi196 ceph-mon[86193]: from='client.34358 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:11 smithi196 ceph-mon[86193]: pgmap v165: 65 pgs: 8 active+clean+remapped, 18 active+undersized, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 40/225 objects degraded (17.778%); 17/225 objects misplaced (7.556%); 2.0 MiB/s, 2 objects/s recovering 2024-03-13T14:58:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:11 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 40/225 objects degraded (17.778%), 14 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:11.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:11.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:13.176 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:12 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T14:58:12.912+0000 7f4e523067c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-03-13T14:58:13.676 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:13 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T14:58:13.513+0000 7f4e523067c0 -1 osd.3 81 log_to_monitors true 2024-03-13T14:58:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:13 smithi196 ceph-mon[86193]: pgmap v166: 65 pgs: 8 active+clean+remapped, 18 active+undersized, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 40/225 objects degraded (17.778%); 17/225 objects misplaced (7.556%); 1.7 MiB/s, 2 objects/s recovering 2024-03-13T14:58:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:13.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:13 smithi176 ceph-mon[112742]: pgmap v166: 65 pgs: 8 active+clean+remapped, 18 active+undersized, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s; 40/225 objects degraded (17.778%); 17/225 objects misplaced (7.556%); 1.7 MiB/s, 2 objects/s recovering 2024-03-13T14:58:13.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:13.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:14.528 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:14 smithi196 ceph-mon[86193]: from='osd.3 [v2:172.21.15.196:6800/1554416376,v1:172.21.15.196:6801/1554416376]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:58:14.528 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:14 smithi196 ceph-mon[86193]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:58:14.528 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 14:58:14 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T14:58:14.497+0000 7f4e41600700 -1 osd.3 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T14:58:14.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:14 smithi176 ceph-mon[112742]: from='osd.3 [v2:172.21.15.196:6800/1554416376,v1:172.21.15.196:6801/1554416376]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:58:14.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:14 smithi176 ceph-mon[112742]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-13T14:58:15.592 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:15 smithi196 ceph-mon[86193]: pgmap v167: 65 pgs: 8 active+clean+remapped, 18 active+undersized, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 971 B/s rd, 1 op/s; 40/225 objects degraded (17.778%); 17/225 objects misplaced (7.556%) 2024-03-13T14:58:15.593 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:15 smithi196 ceph-mon[86193]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-13T14:58:15.593 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:15 smithi196 ceph-mon[86193]: from='osd.3 [v2:172.21.15.196:6800/1554416376,v1:172.21.15.196:6801/1554416376]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:15.593 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:15 smithi196 ceph-mon[86193]: osdmap e88: 6 total, 5 up, 6 in 2024-03-13T14:58:15.593 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:15 smithi196 ceph-mon[86193]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:15.593 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:15.593 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:15.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:15 smithi176 ceph-mon[112742]: pgmap v167: 65 pgs: 8 active+clean+remapped, 18 active+undersized, 14 active+undersized+degraded, 25 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 971 B/s rd, 1 op/s; 40/225 objects degraded (17.778%); 17/225 objects misplaced (7.556%) 2024-03-13T14:58:15.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:15 smithi176 ceph-mon[112742]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-13T14:58:15.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:15 smithi176 ceph-mon[112742]: from='osd.3 [v2:172.21.15.196:6800/1554416376,v1:172.21.15.196:6801/1554416376]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:15.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:15 smithi176 ceph-mon[112742]: osdmap e88: 6 total, 5 up, 6 in 2024-03-13T14:58:15.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:15 smithi176 ceph-mon[112742]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:15.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:15.878 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:16.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:16 smithi176 ceph-mon[112742]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:58:16.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:16 smithi176 ceph-mon[112742]: osd.3 [v2:172.21.15.196:6800/1554416376,v1:172.21.15.196:6801/1554416376] boot 2024-03-13T14:58:16.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:16 smithi176 ceph-mon[112742]: osdmap e89: 6 total, 6 up, 6 in 2024-03-13T14:58:16.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:58:16.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:16 smithi176 ceph-mon[112742]: osdmap e90: 6 total, 6 up, 6 in 2024-03-13T14:58:16.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:16 smithi196 ceph-mon[86193]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:58:16.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:16 smithi196 ceph-mon[86193]: osd.3 [v2:172.21.15.196:6800/1554416376,v1:172.21.15.196:6801/1554416376] boot 2024-03-13T14:58:16.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:16 smithi196 ceph-mon[86193]: osdmap e89: 6 total, 6 up, 6 in 2024-03-13T14:58:16.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-13T14:58:16.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:16 smithi196 ceph-mon[86193]: osdmap e90: 6 total, 6 up, 6 in 2024-03-13T14:58:17.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:17 smithi176 ceph-mon[112742]: pgmap v171: 65 pgs: 3 remapped+peering, 5 active+clean+remapped, 13 active+undersized, 13 active+undersized+degraded, 31 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 36/225 objects degraded (16.000%); 12/225 objects misplaced (5.333%) 2024-03-13T14:58:17.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:17 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 36/225 objects degraded (16.000%), 13 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:17.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:17 smithi176 ceph-mon[112742]: osdmap e91: 6 total, 6 up, 6 in 2024-03-13T14:58:17.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:17 smithi196 ceph-mon[86193]: pgmap v171: 65 pgs: 3 remapped+peering, 5 active+clean+remapped, 13 active+undersized, 13 active+undersized+degraded, 31 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s; 36/225 objects degraded (16.000%); 12/225 objects misplaced (5.333%) 2024-03-13T14:58:17.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:17 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 36/225 objects degraded (16.000%), 13 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:17.931 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:17 smithi196 ceph-mon[86193]: osdmap e91: 6 total, 6 up, 6 in 2024-03-13T14:58:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:18 smithi176 ceph-mon[112742]: osdmap e92: 6 total, 6 up, 6 in 2024-03-13T14:58:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:18 smithi176 ceph-mon[112742]: pgmap v174: 65 pgs: 3 remapped+peering, 4 active+clean+remapped, 1 active+recovering+undersized+remapped, 11 active+undersized, 12 active+undersized+degraded, 34 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 33/225 objects degraded (14.667%); 12/225 objects misplaced (5.333%); 0 B/s, 13 objects/s recovering 2024-03-13T14:58:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:58:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:58:19.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:18 smithi196 ceph-mon[86193]: osdmap e92: 6 total, 6 up, 6 in 2024-03-13T14:58:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:18 smithi196 ceph-mon[86193]: pgmap v174: 65 pgs: 3 remapped+peering, 4 active+clean+remapped, 1 active+recovering+undersized+remapped, 11 active+undersized, 12 active+undersized+degraded, 34 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 33/225 objects degraded (14.667%); 12/225 objects misplaced (5.333%); 0 B/s, 13 objects/s recovering 2024-03-13T14:58:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:58:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:58:19.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:20.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: osdmap e93: 6 total, 6 up, 6 in 2024-03-13T14:58:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:58:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:20.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:19 smithi176 ceph-mon[112742]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: osdmap e93: 6 total, 6 up, 6 in 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:20.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:19 smithi196 ceph-mon[86193]: Upgrade: unsafe to stop osd(s) at this time (13 PGs are or would become offline) 2024-03-13T14:58:21.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:20 smithi176 ceph-mon[112742]: osdmap e94: 6 total, 6 up, 6 in 2024-03-13T14:58:21.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:20 smithi176 ceph-mon[112742]: pgmap v177: 65 pgs: 3 peering, 2 remapped+peering, 1 active+recovering+undersized+remapped, 59 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 5.0 MiB/s, 26 objects/s recovering 2024-03-13T14:58:21.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:20 smithi176 ceph-mon[112742]: osdmap e95: 6 total, 6 up, 6 in 2024-03-13T14:58:21.176 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:20 smithi196 ceph-mon[86193]: osdmap e94: 6 total, 6 up, 6 in 2024-03-13T14:58:21.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:20 smithi196 ceph-mon[86193]: pgmap v177: 65 pgs: 3 peering, 2 remapped+peering, 1 active+recovering+undersized+remapped, 59 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 5.0 MiB/s, 26 objects/s recovering 2024-03-13T14:58:21.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:20 smithi196 ceph-mon[86193]: osdmap e95: 6 total, 6 up, 6 in 2024-03-13T14:58:22.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:21 smithi176 ceph-mon[112742]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/225 objects degraded (14.667%), 12 pgs degraded) 2024-03-13T14:58:22.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:21 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T14:58:22.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:21 smithi176 ceph-mon[112742]: osdmap e96: 6 total, 6 up, 6 in 2024-03-13T14:58:22.176 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:21 smithi196 ceph-mon[86193]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/225 objects degraded (14.667%), 12 pgs degraded) 2024-03-13T14:58:22.176 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:21 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T14:58:22.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:21 smithi196 ceph-mon[86193]: osdmap e96: 6 total, 6 up, 6 in 2024-03-13T14:58:23.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:22 smithi176 ceph-mon[112742]: pgmap v180: 65 pgs: 3 peering, 1 active+recovering+undersized+remapped, 61 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 9.0 MiB/s, 188 objects/s recovering 2024-03-13T14:58:23.176 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:22 smithi196 ceph-mon[86193]: pgmap v180: 65 pgs: 3 peering, 1 active+recovering+undersized+remapped, 61 active+clean; 204 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 9.0 MiB/s, 188 objects/s recovering 2024-03-13T14:58:25.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:25 smithi176 ceph-mon[112742]: pgmap v181: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 2.9 MiB/s, 245 objects/s recovering 2024-03-13T14:58:25.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:25 smithi196 ceph-mon[86193]: pgmap v181: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 2.9 MiB/s, 245 objects/s recovering 2024-03-13T14:58:27.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:27 smithi176 ceph-mon[112742]: pgmap v182: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 784 B/s rd, 1 op/s; 2.5 MiB/s, 208 objects/s recovering 2024-03-13T14:58:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:27 smithi196 ceph-mon[86193]: pgmap v182: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 784 B/s rd, 1 op/s; 2.5 MiB/s, 208 objects/s recovering 2024-03-13T14:58:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:29 smithi176 ceph-mon[112742]: pgmap v183: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 2.0 MiB/s, 169 objects/s recovering 2024-03-13T14:58:29.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:29 smithi196 ceph-mon[86193]: pgmap v183: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 2.0 MiB/s, 169 objects/s recovering 2024-03-13T14:58:31.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:31 smithi196 ceph-mon[86193]: pgmap v184: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 1.7 MiB/s, 142 objects/s recovering 2024-03-13T14:58:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:31.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:31 smithi176 ceph-mon[112742]: pgmap v184: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s rd, 2 op/s; 1.7 MiB/s, 142 objects/s recovering 2024-03-13T14:58:31.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:31.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:31.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:33.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:33 smithi196 ceph-mon[86193]: pgmap v185: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 970 B/s rd, 1 op/s; 0 B/s, 62 objects/s recovering 2024-03-13T14:58:33.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:33 smithi176 ceph-mon[112742]: pgmap v185: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 970 B/s rd, 1 op/s; 0 B/s, 62 objects/s recovering 2024-03-13T14:58:34.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:34 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:34.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:34 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:35 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:35 smithi196 ceph-mon[86193]: Upgrade: osd.4 is safe to restart 2024-03-13T14:58:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:35 smithi196 ceph-mon[86193]: pgmap v186: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 54 objects/s recovering 2024-03-13T14:58:35.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:35 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-03-13T14:58:35.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:35 smithi176 ceph-mon[112742]: Upgrade: osd.4 is safe to restart 2024-03-13T14:58:35.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:35 smithi176 ceph-mon[112742]: pgmap v186: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 0 B/s, 54 objects/s recovering 2024-03-13T14:58:37.614 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:37 smithi196 ceph-mon[86193]: pgmap v187: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:58:37.614 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:37 smithi196 ceph-mon[86193]: Upgrade: Updating osd.4 2024-03-13T14:58:37.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:37 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:37.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:37 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-13T14:58:37.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:37 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:58:37.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:37 smithi196 ceph-mon[86193]: Deploying daemon osd.4 on smithi196 2024-03-13T14:58:37.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:37 smithi176 ceph-mon[112742]: pgmap v187: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:58:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:37 smithi176 ceph-mon[112742]: Upgrade: Updating osd.4 2024-03-13T14:58:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:37 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:37 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-13T14:58:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:37 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:58:37.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:37 smithi176 ceph-mon[112742]: Deploying daemon osd.4 on smithi196 2024-03-13T14:58:39.775 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:39 smithi196 ceph-mon[86193]: pgmap v188: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:58:39.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:39 smithi176 ceph-mon[112742]: pgmap v188: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:58:40.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 -- 172.21.15.176:0/4163743909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c100970 msgr2=0x7f6e6c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:40.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 --2- 172.21.15.176:0/4163743909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c100970 0x7f6e6c100df0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c009910 tx=0x7f6e5c009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:40.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 -- 172.21.15.176:0/4163743909 shutdown_connections 2024-03-13T14:58:40.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 --2- 172.21.15.176:0/4163743909 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c100970 0x7f6e6c100df0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 --2- 172.21.15.176:0/4163743909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e6c106970 0x7f6e6c106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 -- 172.21.15.176:0/4163743909 >> 172.21.15.176:0/4163743909 conn(0x7f6e6c0fc440 msgr2=0x7f6e6c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:40.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 -- 172.21.15.176:0/4163743909 shutdown_connections 2024-03-13T14:58:40.350 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.363+0000 7f6e73200700 1 -- 172.21.15.176:0/4163743909 wait complete. 2024-03-13T14:58:40.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.364+0000 7f6e73200700 1 Processor -- start 2024-03-13T14:58:40.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.365+0000 7f6e73200700 1 -- start start 2024-03-13T14:58:40.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.365+0000 7f6e73200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e6c100970 0x7f6e6c19e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:40.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.365+0000 7f6e73200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c106970 0x7f6e6c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:40.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.365+0000 7f6e73200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c19f210 con 0x7f6e6c106970 2024-03-13T14:58:40.351 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.365+0000 7f6e73200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c1986c0 con 0x7f6e6c100970 2024-03-13T14:58:40.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.365+0000 7f6e71400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c106970 0x7f6e6c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:40.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.366+0000 7f6e71400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c106970 0x7f6e6c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44748/0 (socket says 172.21.15.176:44748) 2024-03-13T14:58:40.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.366+0000 7f6e71400700 1 -- 172.21.15.176:0/2343709543 learned_addr learned my addr 172.21.15.176:0/2343709543 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:40.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.366+0000 7f6e71e00700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e6c100970 0x7f6e6c19e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:40.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.366+0000 7f6e71400700 1 -- 172.21.15.176:0/2343709543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e6c100970 msgr2=0x7f6e6c19e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:40.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.366+0000 7f6e71400700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e6c100970 0x7f6e6c19e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.352 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.366+0000 7f6e71400700 1 -- 172.21.15.176:0/2343709543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0095f0 con 0x7f6e6c106970 2024-03-13T14:58:40.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.366+0000 7f6e71400700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c106970 0x7f6e6c19eb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00b5c0 tx=0x7f6e5c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:40.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.367+0000 7f6e66a00700 1 -- 172.21.15.176:0/2343709543 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5c01d070 con 0x7f6e6c106970 2024-03-13T14:58:40.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.367+0000 7f6e66a00700 1 -- 172.21.15.176:0/2343709543 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e5c00f460 con 0x7f6e6c106970 2024-03-13T14:58:40.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.367+0000 7f6e66a00700 1 -- 172.21.15.176:0/2343709543 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5c017630 con 0x7f6e6c106970 2024-03-13T14:58:40.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.367+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c198940 con 0x7f6e6c106970 2024-03-13T14:58:40.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.367+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c198e30 con 0x7f6e6c106970 2024-03-13T14:58:40.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.368+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c04f2e0 con 0x7f6e6c106970 2024-03-13T14:58:40.355 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.369+0000 7f6e66a00700 1 -- 172.21.15.176:0/2343709543 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6e5c00f5d0 con 0x7f6e6c106970 2024-03-13T14:58:40.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.370+0000 7f6e66a00700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e5406dfa0 0x7f6e54070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:40.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.370+0000 7f6e66a00700 1 -- 172.21.15.176:0/2343709543 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f6e5c08f270 con 0x7f6e6c106970 2024-03-13T14:58:40.356 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.371+0000 7f6e71e00700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e5406dfa0 0x7f6e54070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:40.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.371+0000 7f6e71e00700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e5406dfa0 0x7f6e54070460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009a10 tx=0x7f6e60009190 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:40.366 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.380+0000 7f6e66a00700 1 -- 172.21.15.176:0/2343709543 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f6e5c05c220 con 0x7f6e6c106970 2024-03-13T14:58:40.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.621+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e6c199320 con 0x7f6e5406dfa0 2024-03-13T14:58:40.609 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.623+0000 7f6e66a00700 1 -- 172.21.15.176:0/2343709543 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f6e6c199320 con 0x7f6e5406dfa0 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e5406dfa0 msgr2=0x7f6e54070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e5406dfa0 0x7f6e54070460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009a10 tx=0x7f6e60009190 comp rx=0 tx=0).stop 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c106970 msgr2=0x7f6e6c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c106970 0x7f6e6c19eb80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00b5c0 tx=0x7f6e5c00bd50 comp rx=0 tx=0).stop 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 shutdown_connections 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e5406dfa0 0x7f6e54070460 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e6c100970 0x7f6e6c19e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 --2- 172.21.15.176:0/2343709543 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e6c106970 0x7f6e6c19eb80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 >> 172.21.15.176:0/2343709543 conn(0x7f6e6c0fc440 msgr2=0x7f6e6c0fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 shutdown_connections 2024-03-13T14:58:40.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.627+0000 7f6e73200700 1 -- 172.21.15.176:0/2343709543 wait complete. 2024-03-13T14:58:40.629 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:58:40.771 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 -- 172.21.15.176:0/1074488444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 msgr2=0x7f7d481033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:40.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 --2- 172.21.15.176:0/1074488444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d481033e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009910 tx=0x7f7d38009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:40.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 -- 172.21.15.176:0/1074488444 shutdown_connections 2024-03-13T14:58:40.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 --2- 172.21.15.176:0/1074488444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d481039b0 0x7f7d48107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 --2- 172.21.15.176:0/1074488444 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d481033e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 -- 172.21.15.176:0/1074488444 >> 172.21.15.176:0/1074488444 conn(0x7f7d480fe8d0 msgr2=0x7f7d48100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:40.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 -- 172.21.15.176:0/1074488444 shutdown_connections 2024-03-13T14:58:40.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.785+0000 7f7d4e000700 1 -- 172.21.15.176:0/1074488444 wait complete. 2024-03-13T14:58:40.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.786+0000 7f7d4e000700 1 Processor -- start 2024-03-13T14:58:40.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.786+0000 7f7d4e000700 1 -- start start 2024-03-13T14:58:40.773 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.787+0000 7f7d4e000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d4819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.787+0000 7f7d4e000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d481039b0 0x7f7d4819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.787+0000 7f7d4e000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4819f3c0 con 0x7f7d48103000 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.787+0000 7f7d4e000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d48198870 con 0x7f7d481039b0 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.787+0000 7f7d47400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d4819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.787+0000 7f7d47400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d4819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44766/0 (socket says 172.21.15.176:44766) 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.787+0000 7f7d47400700 1 -- 172.21.15.176:0/3413167750 learned_addr learned my addr 172.21.15.176:0/3413167750 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d46a00700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d481039b0 0x7f7d4819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d47400700 1 -- 172.21.15.176:0/3413167750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d481039b0 msgr2=0x7f7d4819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d47400700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d481039b0 0x7f7d4819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d47400700 1 -- 172.21.15.176:0/3413167750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d380095f0 con 0x7f7d48103000 2024-03-13T14:58:40.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d47400700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d4819e7f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006010 tx=0x7f7d3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:40.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d3fe00700 1 -- 172.21.15.176:0/3413167750 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d3801d070 con 0x7f7d48103000 2024-03-13T14:58:40.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d3fe00700 1 -- 172.21.15.176:0/3413167750 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d3800f460 con 0x7f7d48103000 2024-03-13T14:58:40.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d48198af0 con 0x7f7d48103000 2024-03-13T14:58:40.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d3fe00700 1 -- 172.21.15.176:0/3413167750 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d38017630 con 0x7f7d48103000 2024-03-13T14:58:40.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.788+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d48198fe0 con 0x7f7d48103000 2024-03-13T14:58:40.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.789+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4810b3a0 con 0x7f7d48103000 2024-03-13T14:58:40.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.790+0000 7f7d3fe00700 1 -- 172.21.15.176:0/3413167750 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7d3800f5d0 con 0x7f7d48103000 2024-03-13T14:58:40.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.791+0000 7f7d3fe00700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7d2c06dfa0 0x7f7d2c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:40.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.791+0000 7f7d3fe00700 1 -- 172.21.15.176:0/3413167750 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f7d3808f270 con 0x7f7d48103000 2024-03-13T14:58:40.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.792+0000 7f7d46a00700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7d2c06dfa0 0x7f7d2c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:40.778 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.792+0000 7f7d46a00700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7d2c06dfa0 0x7f7d2c070460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d34005950 tx=0x7f7d340058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:40.782 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:40.796+0000 7f7d3fe00700 1 -- 172.21.15.176:0/3413167750 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f7d3805c180 con 0x7f7d48103000 2024-03-13T14:58:41.022 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.036+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d481992c0 con 0x7f7d2c06dfa0 2024-03-13T14:58:41.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.038+0000 7f7d3fe00700 1 -- 172.21.15.176:0/3413167750 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7d481992c0 con 0x7f7d2c06dfa0 2024-03-13T14:58:41.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7d2c06dfa0 msgr2=0x7f7d2c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7d2c06dfa0 0x7f7d2c070460 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7d34005950 tx=0x7f7d340058e0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 msgr2=0x7f7d4819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d4819e7f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7d38006010 tx=0x7f7d3800be20 comp rx=0 tx=0).stop 2024-03-13T14:58:41.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 shutdown_connections 2024-03-13T14:58:41.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7d2c06dfa0 0x7f7d2c070460 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.027 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7d48103000 0x7f7d4819e7f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 --2- 172.21.15.176:0/3413167750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7d481039b0 0x7f7d4819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.041+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 >> 172.21.15.176:0/3413167750 conn(0x7f7d480fe8d0 msgr2=0x7f7d480ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:41.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.042+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 shutdown_connections 2024-03-13T14:58:41.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.042+0000 7f7d4e000700 1 -- 172.21.15.176:0/3413167750 wait complete. 2024-03-13T14:58:41.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 -- 172.21.15.176:0/1578051475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 msgr2=0x7fb0241033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 --2- 172.21.15.176:0/1578051475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb0241033e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb014009910 tx=0x7fb014009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:41.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 -- 172.21.15.176:0/1578051475 shutdown_connections 2024-03-13T14:58:41.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 --2- 172.21.15.176:0/1578051475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0241039b0 0x7fb024107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 --2- 172.21.15.176:0/1578051475 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb0241033e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 -- 172.21.15.176:0/1578051475 >> 172.21.15.176:0/1578051475 conn(0x7fb0240fe8d0 msgr2=0x7fb024100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:41.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 -- 172.21.15.176:0/1578051475 shutdown_connections 2024-03-13T14:58:41.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.200+0000 7fb029200700 1 -- 172.21.15.176:0/1578051475 wait complete. 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.201+0000 7fb029200700 1 Processor -- start 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb029200700 1 -- start start 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb029200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb02419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb029200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0241039b0 0x7fb02419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb029200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02419f4d0 con 0x7fb024103000 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb029200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb024198980 con 0x7fb0241039b0 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb021e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb02419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb021e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb02419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44790/0 (socket says 172.21.15.176:44790) 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.202+0000 7fb021e00700 1 -- 172.21.15.176:0/1399448959 learned_addr learned my addr 172.21.15.176:0/1399448959 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:41.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.203+0000 7fb021400700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0241039b0 0x7fb02419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:41.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.203+0000 7fb021e00700 1 -- 172.21.15.176:0/1399448959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0241039b0 msgr2=0x7fb02419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.203+0000 7fb021e00700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0241039b0 0x7fb02419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.203+0000 7fb021e00700 1 -- 172.21.15.176:0/1399448959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0140095f0 con 0x7fb024103000 2024-03-13T14:58:41.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.203+0000 7fb021e00700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb02419e900 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b5c0 tx=0x7fb01400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:41.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.203+0000 7fb01aa00700 1 -- 172.21.15.176:0/1399448959 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01401d070 con 0x7fb024103000 2024-03-13T14:58:41.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.204+0000 7fb01aa00700 1 -- 172.21.15.176:0/1399448959 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb01400f460 con 0x7fb024103000 2024-03-13T14:58:41.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.204+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb024198c00 con 0x7fb024103000 2024-03-13T14:58:41.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.204+0000 7fb01aa00700 1 -- 172.21.15.176:0/1399448959 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb014017610 con 0x7fb024103000 2024-03-13T14:58:41.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.204+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0241990f0 con 0x7fb024103000 2024-03-13T14:58:41.191 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.205+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb02410b3a0 con 0x7fb024103000 2024-03-13T14:58:41.192 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.206+0000 7fb01aa00700 1 -- 172.21.15.176:0/1399448959 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb01400f5d0 con 0x7fb024103000 2024-03-13T14:58:41.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.207+0000 7fb01aa00700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb00806dfa0 0x7fb008070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:41.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.207+0000 7fb01aa00700 1 -- 172.21.15.176:0/1399448959 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fb01408f3f0 con 0x7fb024103000 2024-03-13T14:58:41.193 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.207+0000 7fb021400700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb00806dfa0 0x7fb008070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:41.194 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.208+0000 7fb021400700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb00806dfa0 0x7fb008070460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb010006fd0 tx=0x7fb010008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:41.198 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.212+0000 7fb01aa00700 1 -- 172.21.15.176:0/1399448959 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fb01405c320 con 0x7fb024103000 2024-03-13T14:58:41.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.440+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb02402cd30 con 0x7fb00806dfa0 2024-03-13T14:58:41.457 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.472+0000 7fb01aa00700 1 -- 172.21.15.176:0/1399448959 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb02402cd30 con 0x7fb00806dfa0 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (6m) 73s ago 20m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (20m) 73s ago 20m 8812k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (18m) 30s ago 18m 8442k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (3m) 73s ago 20m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (3m) 30s ago 18m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (6m) 73s ago 19m 73.6M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:58:41.458 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (12m) 73s ago 12m 87.3M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (12m) 73s ago 12m 17.9M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (12m) 30s ago 12m 140M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (12m) 30s ago 12m 17.9M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (9m) 73s ago 21m 504M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (8m) 30s ago 18m 450M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (4m) 73s ago 22m 47.3M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (4m) 30s ago 18m 39.3M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (7m) 73s ago 20m 7520k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (7m) 30s ago 18m 7894k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (2m) 73s ago 17m 122M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (110s) 73s ago 17m 97.2M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (80s) 73s ago 16m 11.1M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (35s) 30s ago 15m 11.2M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T14:58:41.459 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (15m) 30s ago 15m 281M 2241M 18.2.1 5be31c24972a 8ee4b4d23410 2024-03-13T14:58:41.460 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (14m) 30s ago 14m 295M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:58:41.460 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (7m) 73s ago 19m 69.1M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:58:41.462 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb00806dfa0 msgr2=0x7fb008070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.462 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb00806dfa0 0x7fb008070460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb010006fd0 tx=0x7fb010008040 comp rx=0 tx=0).stop 2024-03-13T14:58:41.462 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 msgr2=0x7fb02419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.462 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb02419e900 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b5c0 tx=0x7fb01400be00 comp rx=0 tx=0).stop 2024-03-13T14:58:41.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 shutdown_connections 2024-03-13T14:58:41.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb00806dfa0 0x7fb008070460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb024103000 0x7fb02419e900 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 --2- 172.21.15.176:0/1399448959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0241039b0 0x7fb02419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 >> 172.21.15.176:0/1399448959 conn(0x7fb0240fe8d0 msgr2=0x7fb0240fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:41.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 shutdown_connections 2024-03-13T14:58:41.463 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.476+0000 7fb029200700 1 -- 172.21.15.176:0/1399448959 wait complete. 2024-03-13T14:58:41.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 -- 172.21.15.176:0/3384751207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094101770 msgr2=0x7fc0941057c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 --2- 172.21.15.176:0/3384751207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094101770 0x7fc0941057c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc080009910 tx=0x7fc080009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:41.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 -- 172.21.15.176:0/3384751207 shutdown_connections 2024-03-13T14:58:41.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 --2- 172.21.15.176:0/3384751207 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094101770 0x7fc0941057c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 --2- 172.21.15.176:0/3384751207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc094100dc0 0x7fc0941011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 -- 172.21.15.176:0/3384751207 >> 172.21.15.176:0/3384751207 conn(0x7fc0940fc650 msgr2=0x7fc0940fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:41.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 -- 172.21.15.176:0/3384751207 shutdown_connections 2024-03-13T14:58:41.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.624+0000 7fc09c400700 1 -- 172.21.15.176:0/3384751207 wait complete. 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.625+0000 7fc09c400700 1 Processor -- start 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.625+0000 7fc09c400700 1 -- start start 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc09c400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094100dc0 0x7fc09419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc09c400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc094101770 0x7fc09419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc09c400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09419f410 con 0x7fc094100dc0 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc09c400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0941988c0 con 0x7fc094101770 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc099a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094100dc0 0x7fc09419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc099a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094100dc0 0x7fc09419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:44822/0 (socket says 172.21.15.176:44822) 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc099a00700 1 -- 172.21.15.176:0/3729724039 learned_addr learned my addr 172.21.15.176:0/3729724039 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc099000700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc094101770 0x7fc09419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:41.613 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc099a00700 1 -- 172.21.15.176:0/3729724039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc094101770 msgr2=0x7fc09419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc099a00700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc094101770 0x7fc09419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.626+0000 7fc099a00700 1 -- 172.21.15.176:0/3729724039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0800095f0 con 0x7fc094100dc0 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.627+0000 7fc099a00700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094100dc0 0x7fc09419e840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc08800b4b0 tx=0x7fc08800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.627+0000 7fc092000700 1 -- 172.21.15.176:0/3729724039 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc088011800 con 0x7fc094100dc0 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.627+0000 7fc092000700 1 -- 172.21.15.176:0/3729724039 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc088011e20 con 0x7fc094100dc0 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.627+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc094198ba0 con 0x7fc094100dc0 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.627+0000 7fc092000700 1 -- 172.21.15.176:0/3729724039 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc08800f2e0 con 0x7fc094100dc0 2024-03-13T14:58:41.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.627+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941990f0 con 0x7fc094100dc0 2024-03-13T14:58:41.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.628+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc094104f40 con 0x7fc094100dc0 2024-03-13T14:58:41.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.629+0000 7fc092000700 1 -- 172.21.15.176:0/3729724039 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc088010380 con 0x7fc094100dc0 2024-03-13T14:58:41.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.630+0000 7fc092000700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc07c06dfa0 0x7fc07c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:41.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.630+0000 7fc092000700 1 -- 172.21.15.176:0/3729724039 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc08808d990 con 0x7fc094100dc0 2024-03-13T14:58:41.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.630+0000 7fc099000700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc07c06dfa0 0x7fc07c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:41.616 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.631+0000 7fc099000700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc07c06dfa0 0x7fc07c070460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0940fcf10 tx=0x7fc08001a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:41.620 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.635+0000 7fc092000700 1 -- 172.21.15.176:0/3729724039 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc08805a950 con 0x7fc094100dc0 2024-03-13T14:58:41.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:41 smithi176 ceph-mon[112742]: pgmap v189: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:58:41.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:41 smithi196 ceph-mon[86193]: pgmap v189: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:58:41.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:41 smithi196 systemd[1]: Stopping Ceph osd.4 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:58:41.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.941+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0941993c0 con 0x7fc094100dc0 2024-03-13T14:58:41.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.942+0000 7fc092000700 1 -- 172.21.15.176:0/3729724039 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fc08805a4e0 con 0x7fc094100dc0 2024-03-13T14:58:41.928 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:58:41.928 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:58:41.928 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:58:41.928 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 6, 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 8 2024-03-13T14:58:41.929 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:58:41.930 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:58:41.930 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc07c06dfa0 msgr2=0x7fc07c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc07c06dfa0 0x7fc07c070460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0940fcf10 tx=0x7fc08001a040 comp rx=0 tx=0).stop 2024-03-13T14:58:41.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094100dc0 msgr2=0x7fc09419e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:41.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094100dc0 0x7fc09419e840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc08800b4b0 tx=0x7fc08800b870 comp rx=0 tx=0).stop 2024-03-13T14:58:41.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 shutdown_connections 2024-03-13T14:58:41.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc07c06dfa0 0x7fc07c070460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc094100dc0 0x7fc09419e840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 --2- 172.21.15.176:0/3729724039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc094101770 0x7fc09419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:41.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 >> 172.21.15.176:0/3729724039 conn(0x7fc0940fc650 msgr2=0x7fc0940fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:41.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.945+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 shutdown_connections 2024-03-13T14:58:41.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:41.946+0000 7fc09c400700 1 -- 172.21.15.176:0/3729724039 wait complete. 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.099+0000 7f5adb600700 1 -- 172.21.15.176:0/3451067472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 msgr2=0x7f5ad410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.099+0000 7f5adb600700 1 --2- 172.21.15.176:0/3451067472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad410d660 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009910 tx=0x7f5ac8009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.100+0000 7f5adb600700 1 -- 172.21.15.176:0/3451067472 shutdown_connections 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.100+0000 7f5adb600700 1 --2- 172.21.15.176:0/3451067472 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ad41071d0 0x7f5ad4107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.100+0000 7f5adb600700 1 --2- 172.21.15.176:0/3451067472 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad410d660 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.100+0000 7f5adb600700 1 -- 172.21.15.176:0/3451067472 >> 172.21.15.176:0/3451067472 conn(0x7f5ad40769e0 msgr2=0x7f5ad4078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.100+0000 7f5adb600700 1 -- 172.21.15.176:0/3451067472 shutdown_connections 2024-03-13T14:58:42.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.100+0000 7f5adb600700 1 -- 172.21.15.176:0/3451067472 wait complete. 2024-03-13T14:58:42.091 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.101+0000 7f5adb600700 1 Processor -- start 2024-03-13T14:58:42.091 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.101+0000 7f5adb600700 1 -- start start 2024-03-13T14:58:42.091 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5adb600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ad41071d0 0x7f5ad419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5adb600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5adb600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad419d8a0 con 0x7f5ad410d280 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5adb600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad41a05d0 con 0x7f5ad41071d0 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5ad3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5ad3e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47582/0 (socket says 172.21.15.176:47582) 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5ad3e00700 1 -- 172.21.15.176:0/3116144796 learned_addr learned my addr 172.21.15.176:0/3116144796 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.102+0000 7f5ad8c00700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ad41071d0 0x7f5ad419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5ad3e00700 1 -- 172.21.15.176:0/3116144796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ad41071d0 msgr2=0x7f5ad419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5ad3e00700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ad41071d0 0x7f5ad419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5ad3e00700 1 -- 172.21.15.176:0/3116144796 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac80095f0 con 0x7f5ad410d280 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5ad3e00700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad419d210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c8a0 tx=0x7f5ac000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5ad1600700 1 -- 172.21.15.176:0/3116144796 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ac000cea0 con 0x7f5ad410d280 2024-03-13T14:58:42.092 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad41a08b0 con 0x7f5ad410d280 2024-03-13T14:58:42.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5ad1600700 1 -- 172.21.15.176:0/3116144796 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ac0004830 con 0x7f5ad410d280 2024-03-13T14:58:42.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.103+0000 7f5ad1600700 1 -- 172.21.15.176:0/3116144796 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ac00055b0 con 0x7f5ad410d280 2024-03-13T14:58:42.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.104+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad41a0e80 con 0x7f5ad410d280 2024-03-13T14:58:42.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.105+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad404ea90 con 0x7f5ad410d280 2024-03-13T14:58:42.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.106+0000 7f5ad1600700 1 -- 172.21.15.176:0/3116144796 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5ac00049a0 con 0x7f5ad410d280 2024-03-13T14:58:42.093 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.106+0000 7f5ad1600700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5abc06dfa0 0x7f5abc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.094 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.107+0000 7f5ad1600700 1 -- 172.21.15.176:0/3116144796 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f5ac008db10 con 0x7f5ad410d280 2024-03-13T14:58:42.094 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.107+0000 7f5ad8c00700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5abc06dfa0 0x7f5abc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.094 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.107+0000 7f5ad8c00700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5abc06dfa0 0x7f5abc070460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006010 tx=0x7f5ac801a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:42.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.111+0000 7f5ad1600700 1 -- 172.21.15.176:0/3116144796 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f5ac0056e70 con 0x7f5ad410d280 2024-03-13T14:58:42.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.363+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5ad41a1200 con 0x7f5ad410d280 2024-03-13T14:58:42.349 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.364+0000 7f5ad1600700 1 -- 172.21.15.176:0/3116144796 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f5ac001d770 con 0x7f5ad410d280 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:58:42.350 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:58:42.351 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:58:42.352 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:58:42.353 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:42.353 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:58:42.353 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.367+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5abc06dfa0 msgr2=0x7f5abc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.367+0000 7f5adb600700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5abc06dfa0 0x7f5abc070460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8006010 tx=0x7f5ac801a040 comp rx=0 tx=0).stop 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.367+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 msgr2=0x7f5ad419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.367+0000 7f5adb600700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad419d210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c8a0 tx=0x7f5ac000cc60 comp rx=0 tx=0).stop 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.368+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 shutdown_connections 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.368+0000 7f5adb600700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5abc06dfa0 0x7f5abc070460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.368+0000 7f5adb600700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ad41071d0 0x7f5ad419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.368+0000 7f5adb600700 1 --2- 172.21.15.176:0/3116144796 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5ad410d280 0x7f5ad419d210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.368+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 >> 172.21.15.176:0/3116144796 conn(0x7f5ad40769e0 msgr2=0x7f5ad4078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.368+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 shutdown_connections 2024-03-13T14:58:42.354 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.368+0000 7f5adb600700 1 -- 172.21.15.176:0/3116144796 wait complete. 2024-03-13T14:58:42.355 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:58:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[50002]: 2024-03-13T14:58:42.078+0000 7f61d7600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:58:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[50002]: 2024-03-13T14:58:42.078+0000 7f61d7600700 -1 osd.4 96 *** Got signal Terminated *** 2024-03-13T14:58:42.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[50002]: 2024-03-13T14:58:42.078+0000 7f61d7600700 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T14:58:42.483 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:42 smithi176 ceph-mon[112742]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:42.483 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:42 smithi176 ceph-mon[112742]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:42.483 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:42 smithi176 ceph-mon[112742]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:42.483 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:42 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3729724039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:42.484 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:42 smithi176 ceph-mon[112742]: osd.4 marked itself down and dead 2024-03-13T14:58:42.484 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:42 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3116144796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:58:42.513 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 -- 172.21.15.176:0/872842661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774101750 msgr2=0x7fc7741057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.513 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 --2- 172.21.15.176:0/872842661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774101750 0x7fc7741057a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009910 tx=0x7fc75c009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:42.513 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 -- 172.21.15.176:0/872842661 shutdown_connections 2024-03-13T14:58:42.513 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 --2- 172.21.15.176:0/872842661 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774101750 0x7fc7741057a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.513 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 --2- 172.21.15.176:0/872842661 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc774100da0 0x7fc774101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.514 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 -- 172.21.15.176:0/872842661 >> 172.21.15.176:0/872842661 conn(0x7fc7740fc630 msgr2=0x7fc7740fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:42.514 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 -- 172.21.15.176:0/872842661 shutdown_connections 2024-03-13T14:58:42.514 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.527+0000 7fc77a000700 1 -- 172.21.15.176:0/872842661 wait complete. 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.528+0000 7fc77a000700 1 Processor -- start 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.528+0000 7fc77a000700 1 -- start start 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc77a000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774100da0 0x7fc77419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc77a000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc774101750 0x7fc77419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc77a000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77419f350 con 0x7fc774100da0 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc77a000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc774198800 con 0x7fc774101750 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc773400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774100da0 0x7fc77419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc773400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774100da0 0x7fc77419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47598/0 (socket says 172.21.15.176:47598) 2024-03-13T14:58:42.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc773400700 1 -- 172.21.15.176:0/1243961087 learned_addr learned my addr 172.21.15.176:0/1243961087 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:42.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc76be00700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc774101750 0x7fc77419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc773400700 1 -- 172.21.15.176:0/1243961087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc774101750 msgr2=0x7fc77419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc773400700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc774101750 0x7fc77419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.529+0000 7fc773400700 1 -- 172.21.15.176:0/1243961087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0095f0 con 0x7fc774100da0 2024-03-13T14:58:42.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.530+0000 7fc773400700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774100da0 0x7fc77419e780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc76400d8d0 tx=0x7fc76400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:42.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.530+0000 7fc770c00700 1 -- 172.21.15.176:0/1243961087 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc764009940 con 0x7fc774100da0 2024-03-13T14:58:42.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.530+0000 7fc770c00700 1 -- 172.21.15.176:0/1243961087 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc764010460 con 0x7fc774100da0 2024-03-13T14:58:42.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.530+0000 7fc770c00700 1 -- 172.21.15.176:0/1243961087 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc76400f5f0 con 0x7fc774100da0 2024-03-13T14:58:42.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.530+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc774198ae0 con 0x7fc774100da0 2024-03-13T14:58:42.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.530+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc774199030 con 0x7fc774100da0 2024-03-13T14:58:42.517 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.531+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc774104f20 con 0x7fc774100da0 2024-03-13T14:58:42.518 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.532+0000 7fc770c00700 1 -- 172.21.15.176:0/1243961087 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc7640105d0 con 0x7fc774100da0 2024-03-13T14:58:42.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.533+0000 7fc770c00700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc75406dfa0 0x7fc754070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.519 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.533+0000 7fc770c00700 1 -- 172.21.15.176:0/1243961087 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc76408ddf0 con 0x7fc774100da0 2024-03-13T14:58:42.520 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.534+0000 7fc76be00700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc75406dfa0 0x7fc754070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.520 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.534+0000 7fc76be00700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc75406dfa0 0x7fc754070460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc7740fcd70 tx=0x7fc75c01a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:42.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.538+0000 7fc770c00700 1 -- 172.21.15.176:0/1243961087 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc76405ad20 con 0x7fc774100da0 2024-03-13T14:58:42.764 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.778+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc774199310 con 0x7fc75406dfa0 2024-03-13T14:58:42.766 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.780+0000 7fc770c00700 1 -- 172.21.15.176:0/1243961087 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fc774199310 con 0x7fc75406dfa0 2024-03-13T14:58:42.766 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "10/23 daemons upgraded", 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading osd daemons", 2024-03-13T14:58:42.767 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:58:42.768 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:58:42.769 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.783+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc75406dfa0 msgr2=0x7fc754070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.769 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.783+0000 7fc77a000700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc75406dfa0 0x7fc754070460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc7740fcd70 tx=0x7fc75c01a040 comp rx=0 tx=0).stop 2024-03-13T14:58:42.769 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.783+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774100da0 msgr2=0x7fc77419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.769 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.783+0000 7fc77a000700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774100da0 0x7fc77419e780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc76400d8d0 tx=0x7fc76400dc90 comp rx=0 tx=0).stop 2024-03-13T14:58:42.770 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.784+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 shutdown_connections 2024-03-13T14:58:42.770 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.784+0000 7fc77a000700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc75406dfa0 0x7fc754070460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.770 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.784+0000 7fc77a000700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc774100da0 0x7fc77419e780 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.770 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.784+0000 7fc77a000700 1 --2- 172.21.15.176:0/1243961087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc774101750 0x7fc77419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.770 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.784+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 >> 172.21.15.176:0/1243961087 conn(0x7fc7740fc630 msgr2=0x7fc7740fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:42.770 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.784+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 shutdown_connections 2024-03-13T14:58:42.770 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.784+0000 7fc77a000700 1 -- 172.21.15.176:0/1243961087 wait complete. 2024-03-13T14:58:42.921 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 -- 172.21.15.176:0/1056087798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0101750 msgr2=0x7fadc01057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.921 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 --2- 172.21.15.176:0/1056087798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0101750 0x7fadc01057a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fada8009910 tx=0x7fada8009c20 comp rx=0 tx=0).stop 2024-03-13T14:58:42.921 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 -- 172.21.15.176:0/1056087798 shutdown_connections 2024-03-13T14:58:42.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 --2- 172.21.15.176:0/1056087798 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0101750 0x7fadc01057a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 --2- 172.21.15.176:0/1056087798 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0100da0 0x7fadc0101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 -- 172.21.15.176:0/1056087798 >> 172.21.15.176:0/1056087798 conn(0x7fadc00fc630 msgr2=0x7fadc00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:42.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 -- 172.21.15.176:0/1056087798 shutdown_connections 2024-03-13T14:58:42.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.935+0000 7fadc5600700 1 -- 172.21.15.176:0/1056087798 wait complete. 2024-03-13T14:58:42.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.937+0000 7fadc5600700 1 Processor -- start 2024-03-13T14:58:42.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.937+0000 7fadc5600700 1 -- start start 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.937+0000 7fadc5600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0100da0 0x7fadc019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.937+0000 7fadc5600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0101750 0x7fadc019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.937+0000 7fadc5600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc019d0d0 con 0x7fadc0100da0 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.937+0000 7fadc5600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0196580 con 0x7fadc0101750 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadbea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0100da0 0x7fadc019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadbea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0100da0 0x7fadc019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:47626/0 (socket says 172.21.15.176:47626) 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadbea00700 1 -- 172.21.15.176:0/2743385966 learned_addr learned my addr 172.21.15.176:0/2743385966 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadb6000700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0101750 0x7fadc019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadbea00700 1 -- 172.21.15.176:0/2743385966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0101750 msgr2=0x7fadc019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:42.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadbea00700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0101750 0x7fadc019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:42.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadbea00700 1 -- 172.21.15.176:0/2743385966 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80095f0 con 0x7fadc0100da0 2024-03-13T14:58:42.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.938+0000 7fadb6000700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0101750 0x7fadc019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:58:42.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.939+0000 7fadbea00700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0100da0 0x7fadc019c500 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c8a0 tx=0x7fadb000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:42.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.939+0000 7fadb7e00700 1 -- 172.21.15.176:0/2743385966 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadb000cea0 con 0x7fadc0100da0 2024-03-13T14:58:42.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.939+0000 7fadb7e00700 1 -- 172.21.15.176:0/2743385966 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadb0004830 con 0x7fadc0100da0 2024-03-13T14:58:42.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.939+0000 7fadb7e00700 1 -- 172.21.15.176:0/2743385966 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadb00055b0 con 0x7fadc0100da0 2024-03-13T14:58:42.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.939+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc0196860 con 0x7fadc0100da0 2024-03-13T14:58:42.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.939+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc0196d60 con 0x7fadc0100da0 2024-03-13T14:58:42.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.940+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc01091a0 con 0x7fadc0100da0 2024-03-13T14:58:42.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-mon[86193]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:42.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-mon[86193]: from='client.34372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:42.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-mon[86193]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:42.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3729724039' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:58:42.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-mon[86193]: osd.4 marked itself down and dead 2024-03-13T14:58:42.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:42 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3116144796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:58:42.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.941+0000 7fadb7e00700 1 -- 172.21.15.176:0/2743385966 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fadb00049a0 con 0x7fadc0100da0 2024-03-13T14:58:42.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.942+0000 7fadb7e00700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fada406dfa0 0x7fada4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:58:42.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.942+0000 7fadb7e00700 1 -- 172.21.15.176:0/2743385966 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fadb008db20 con 0x7fadc0100da0 2024-03-13T14:58:42.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.942+0000 7fadb6000700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fada406dfa0 0x7fada4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:58:42.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.943+0000 7fadb6000700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fada406dfa0 0x7fada4070460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fada8006010 tx=0x7fada800b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:58:42.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:42.946+0000 7fadb7e00700 1 -- 172.21.15.176:0/2743385966 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fadb005a9b0 con 0x7fadc0100da0 2024-03-13T14:58:43.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.234+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fadc004ea90 con 0x7fadc0100da0 2024-03-13T14:58:43.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.235+0000 7fadb7e00700 1 -- 172.21.15.176:0/2743385966 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7fadb0004c50 con 0x7fadc0100da0 2024-03-13T14:58:43.221 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_WARN 1 osds down 2024-03-13T14:58:43.221 INFO:teuthology.orchestra.run.smithi176.stdout:[WRN] OSD_DOWN: 1 osds down 2024-03-13T14:58:43.221 INFO:teuthology.orchestra.run.smithi176.stdout: osd.4 (root=default,host=smithi196) is down 2024-03-13T14:58:43.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fada406dfa0 msgr2=0x7fada4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:43.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fada406dfa0 0x7fada4070460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fada8006010 tx=0x7fada800b540 comp rx=0 tx=0).stop 2024-03-13T14:58:43.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0100da0 msgr2=0x7fadc019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:58:43.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0100da0 0x7fadc019c500 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fadb000c8a0 tx=0x7fadb000cc60 comp rx=0 tx=0).stop 2024-03-13T14:58:43.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 shutdown_connections 2024-03-13T14:58:43.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fada406dfa0 0x7fada4070460 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:43.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fadc0100da0 0x7fadc019c500 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:43.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 --2- 172.21.15.176:0/2743385966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fadc0101750 0x7fadc019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:58:43.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 >> 172.21.15.176:0/2743385966 conn(0x7fadc00fc630 msgr2=0x7fadc00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:58:43.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 shutdown_connections 2024-03-13T14:58:43.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:58:43.238+0000 7fadc5600700 1 -- 172.21.15.176:0/2743385966 wait complete. 2024-03-13T14:58:43.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:43 smithi196 podman[100957]: 2024-03-13 14:58:43.156540115 +0000 UTC m=+1.342834115 container died 8ee4b4d23410c55d9da234150be1f4a9601765482ed40dd989f1da6917029b46 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=HEAD, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_BRANCH=HEAD) 2024-03-13T14:58:43.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:43 smithi196 podman[100957]: 2024-03-13 14:58:43.613734946 +0000 UTC m=+1.800028942 container cleanup 8ee4b4d23410c55d9da234150be1f4a9601765482ed40dd989f1da6917029b46 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=HEAD) 2024-03-13T14:58:43.875 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:43 smithi196 bash[100957]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4 2024-03-13T14:58:43.876 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:43 smithi196 ceph-mon[86193]: pgmap v190: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:58:43.876 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:43 smithi196 ceph-mon[86193]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:58:43.876 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:43 smithi196 ceph-mon[86193]: osdmap e97: 6 total, 5 up, 6 in 2024-03-13T14:58:43.876 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:43 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2743385966' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:58:43.876 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:43 smithi196 ceph-mon[86193]: osdmap e98: 6 total, 5 up, 6 in 2024-03-13T14:58:43.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:43 smithi176 ceph-mon[112742]: pgmap v190: 65 pgs: 65 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T14:58:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:43 smithi176 ceph-mon[112742]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:58:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:43 smithi176 ceph-mon[112742]: osdmap e97: 6 total, 5 up, 6 in 2024-03-13T14:58:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:43 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2743385966' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:58:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:43 smithi176 ceph-mon[112742]: osdmap e98: 6 total, 5 up, 6 in 2024-03-13T14:58:44.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:43 smithi196 podman[100976]: 2024-03-13 14:58:43.889948531 +0000 UTC m=+0.713963154 container remove 8ee4b4d23410c55d9da234150be1f4a9601765482ed40dd989f1da6917029b46 (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:58:44.867 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:44 smithi196 ceph-mon[86193]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:44.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:44 smithi196 podman[101061]: 2024-03-13 14:58:44.682350701 +0000 UTC m=+0.126504029 container create 7a93adb0c1508e78615a61f1005d7b68e7fe644edd969fb17a2b3b770ffde367 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T14:58:44.867 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:44 smithi196 podman[101061]: 2024-03-13 14:58:44.591093073 +0000 UTC m=+0.035246366 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:58:44.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:44 smithi176 ceph-mon[112742]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:58:45.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:44 smithi196 podman[101061]: 2024-03-13 14:58:44.986273015 +0000 UTC m=+0.430426356 container init 7a93adb0c1508e78615a61f1005d7b68e7fe644edd969fb17a2b3b770ffde367 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, ceph=True) 2024-03-13T14:58:45.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:44 smithi196 podman[101061]: 2024-03-13 14:58:44.997251421 +0000 UTC m=+0.441404729 container start 7a93adb0c1508e78615a61f1005d7b68e7fe644edd969fb17a2b3b770ffde367 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222) 2024-03-13T14:58:45.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:45 smithi196 podman[101061]: 2024-03-13 14:58:45.093104605 +0000 UTC m=+0.537257976 container attach 7a93adb0c1508e78615a61f1005d7b68e7fe644edd969fb17a2b3b770ffde367 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-13T14:58:45.677 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:45 smithi196 podman[101061]: 2024-03-13 14:58:45.435276633 +0000 UTC m=+0.879429956 container died 7a93adb0c1508e78615a61f1005d7b68e7fe644edd969fb17a2b3b770ffde367 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-03-13T14:58:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:45 smithi196 ceph-mon[86193]: pgmap v193: 65 pgs: 2 active+undersized, 1 active+undersized+degraded, 6 activating+undersized, 4 stale+active+clean, 52 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 6/225 objects degraded (2.667%) 2024-03-13T14:58:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:45 smithi196 ceph-mon[86193]: Health check failed: Degraded data redundancy: 6/225 objects degraded (2.667%), 1 pg degraded (PG_DEGRADED) 2024-03-13T14:58:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:45.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:45 smithi176 ceph-mon[112742]: pgmap v193: 65 pgs: 2 active+undersized, 1 active+undersized+degraded, 6 activating+undersized, 4 stale+active+clean, 52 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s; 6/225 objects degraded (2.667%) 2024-03-13T14:58:45.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:45 smithi176 ceph-mon[112742]: Health check failed: Degraded data redundancy: 6/225 objects degraded (2.667%), 1 pg degraded (PG_DEGRADED) 2024-03-13T14:58:45.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:45.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:58:46.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:46 smithi196 podman[101087]: 2024-03-13 14:58:46.171264126 +0000 UTC m=+0.716296772 container remove 7a93adb0c1508e78615a61f1005d7b68e7fe644edd969fb17a2b3b770ffde367 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-13T14:58:46.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:46 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.4.service: Deactivated successfully. 2024-03-13T14:58:46.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:46 smithi196 systemd[1]: Stopped Ceph osd.4 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:58:46.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:46 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.4.service: Consumed 1min 49.241s CPU time. 2024-03-13T14:58:46.921 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:46 smithi196 systemd[1]: Starting Ceph osd.4 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:58:47.176 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:46 smithi196 podman[101193]: 2024-03-13 14:58:46.842268763 +0000 UTC m=+0.036733787 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:58:47.177 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:46 smithi196 podman[101193]: 2024-03-13 14:58:46.955474146 +0000 UTC m=+0.149939200 container create 9139889ec7cba6380edda6a7175c4815246dd659b80beba3020dd35939f374b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222) 2024-03-13T14:58:47.578 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:47 smithi196 ceph-mon[86193]: pgmap v194: 65 pgs: 10 active+undersized, 8 active+undersized+degraded, 6 activating+undersized, 2 stale+active+clean, 39 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 31/225 objects degraded (13.778%) 2024-03-13T14:58:47.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:47 smithi196 podman[101193]: 2024-03-13 14:58:47.30118818 +0000 UTC m=+0.495653241 container init 9139889ec7cba6380edda6a7175c4815246dd659b80beba3020dd35939f374b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-03-13T14:58:47.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:47 smithi196 podman[101193]: 2024-03-13 14:58:47.314002872 +0000 UTC m=+0.508467916 container start 9139889ec7cba6380edda6a7175c4815246dd659b80beba3020dd35939f374b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-13T14:58:47.578 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:47 smithi196 podman[101193]: 2024-03-13 14:58:47.591548738 +0000 UTC m=+0.786013828 container attach 9139889ec7cba6380edda6a7175c4815246dd659b80beba3020dd35939f374b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T14:58:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:47 smithi176 ceph-mon[112742]: pgmap v194: 65 pgs: 10 active+undersized, 8 active+undersized+degraded, 6 activating+undersized, 2 stale+active+clean, 39 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 31/225 objects degraded (13.778%) 2024-03-13T14:58:48.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate[101207]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-13T14:58:48.427 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate[101207]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 bash[101193]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 bash[101193]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 bash[101193]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 bash[101193]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 bash[101193]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 bash[101193]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 bash[101193]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate[101207]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate[101207]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate[101207]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate[101207]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate[101207]: --> ceph-volume raw activate successful for osd ID: 4 2024-03-13T14:58:48.428 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 podman[101193]: 2024-03-13 14:58:48.228932526 +0000 UTC m=+1.423397590 container died 9139889ec7cba6380edda6a7175c4815246dd659b80beba3020dd35939f374b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-03-13T14:58:49.281 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:48 smithi196 podman[101330]: 2024-03-13 14:58:48.953383532 +0000 UTC m=+0.706040262 container remove 9139889ec7cba6380edda6a7175c4815246dd659b80beba3020dd35939f374b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-activate, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-13T14:58:49.593 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:49 smithi196 ceph-mon[86193]: pgmap v195: 65 pgs: 14 active+undersized, 12 active+undersized+degraded, 6 activating+undersized, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:49.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:49 smithi196 podman[101391]: 2024-03-13 14:58:49.203565526 +0000 UTC m=+0.034668132 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:58:49.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:49 smithi196 podman[101391]: 2024-03-13 14:58:49.312447616 +0000 UTC m=+0.143550256 container create 7f510f7a757cc86d57b2a153291fccf2aec1f2dbaa2c6d17ad8b73a8fa115285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T14:58:49.594 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:49 smithi196 podman[101391]: 2024-03-13 14:58:49.608707134 +0000 UTC m=+0.439809775 container init 7f510f7a757cc86d57b2a153291fccf2aec1f2dbaa2c6d17ad8b73a8fa115285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T14:58:49.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:49 smithi176 ceph-mon[112742]: pgmap v195: 65 pgs: 14 active+undersized, 12 active+undersized+degraded, 6 activating+undersized, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:49.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:49 smithi196 podman[101391]: 2024-03-13 14:58:49.619560344 +0000 UTC m=+0.450662964 container start 7f510f7a757cc86d57b2a153291fccf2aec1f2dbaa2c6d17ad8b73a8fa115285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, ceph=True) 2024-03-13T14:58:49.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:49 smithi196 ceph-osd[101410]: -- 172.21.15.196:0/1675241385 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55756b149680 con 0x55756b112c00 2024-03-13T14:58:49.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:49 smithi196 bash[101391]: 7f510f7a757cc86d57b2a153291fccf2aec1f2dbaa2c6d17ad8b73a8fa115285 2024-03-13T14:58:49.927 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:49 smithi196 systemd[1]: Started Ceph osd.4 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:58:50.676 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T14:58:50.193+0000 7f3f6d46b7c0 -1 Falling back to public interface 2024-03-13T14:58:51.176 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:58:51.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-mon[86193]: pgmap v196: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:51.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:50 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 39/225 objects degraded (17.333%), 12 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:51.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:50 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:50 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:50 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:58:51.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:50 smithi176 ceph-mon[112742]: pgmap v196: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:51.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:50 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:50 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:51.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:50 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 39/225 objects degraded (17.333%), 12 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:53.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:53 smithi196 ceph-mon[86193]: pgmap v197: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 619 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:53.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:53 smithi176 ceph-mon[112742]: pgmap v197: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 619 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:55 smithi176 ceph-mon[112742]: pgmap v198: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 562 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:55.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:55 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:55.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:55 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:55.926 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:55 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T14:58:55.628+0000 7f3f6d46b7c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-03-13T14:58:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:55 smithi196 ceph-mon[86193]: pgmap v198: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 562 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:55.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:55 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:55.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:55 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:56.426 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:56 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T14:58:56.159+0000 7f3f6d46b7c0 -1 osd.4 96 log_to_monitors true 2024-03-13T14:58:56.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:56 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 39/225 objects degraded (17.333%), 15 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:56.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:56 smithi176 ceph-mon[112742]: from='osd.4 [v2:172.21.15.196:6808/1908907228,v1:172.21.15.196:6809/1908907228]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:58:56.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:56 smithi176 ceph-mon[112742]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:58:56.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:56 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 39/225 objects degraded (17.333%), 15 pgs degraded (PG_DEGRADED) 2024-03-13T14:58:56.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:56 smithi196 ceph-mon[86193]: from='osd.4 [v2:172.21.15.196:6808/1908907228,v1:172.21.15.196:6809/1908907228]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:58:56.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:56 smithi196 ceph-mon[86193]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-13T14:58:57.818 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T14:58:57.501+0000 7f3f57800700 -1 osd.4 96 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T14:58:57.818 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-mon[86193]: pgmap v199: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:57.818 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-mon[86193]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-13T14:58:57.818 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-mon[86193]: osdmap e99: 6 total, 5 up, 6 in 2024-03-13T14:58:57.818 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-mon[86193]: from='osd.4 [v2:172.21.15.196:6808/1908907228,v1:172.21.15.196:6809/1908907228]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:57.819 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-mon[86193]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:57.819 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:57.819 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:57 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:57.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:57 smithi176 ceph-mon[112742]: pgmap v199: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:57.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:57 smithi176 ceph-mon[112742]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-13T14:58:57.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:57 smithi176 ceph-mon[112742]: osdmap e99: 6 total, 5 up, 6 in 2024-03-13T14:58:57.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:57 smithi176 ceph-mon[112742]: from='osd.4 [v2:172.21.15.196:6808/1908907228,v1:172.21.15.196:6809/1908907228]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:57.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:57 smithi176 ceph-mon[112742]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:58:57.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:57 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:57.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:57 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:58:58.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:58 smithi196 ceph-mon[86193]: from='osd.4 ' entity='osd.4' 2024-03-13T14:58:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:58 smithi176 ceph-mon[112742]: from='osd.4 ' entity='osd.4' 2024-03-13T14:58:59.605 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:59 smithi196 ceph-mon[86193]: pgmap v201: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 942 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:59.605 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:59 smithi196 ceph-mon[86193]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:58:59.605 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:59 smithi196 ceph-mon[86193]: osd.4 [v2:172.21.15.196:6808/1908907228,v1:172.21.15.196:6809/1908907228] boot 2024-03-13T14:58:59.606 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:59 smithi196 ceph-mon[86193]: osdmap e100: 6 total, 6 up, 6 in 2024-03-13T14:58:59.606 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:58:59 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:58:59.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:59 smithi176 ceph-mon[112742]: pgmap v201: 65 pgs: 17 active+undersized, 15 active+undersized+degraded, 33 active+clean; 204 MiB data, 942 MiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 39/225 objects degraded (17.333%) 2024-03-13T14:58:59.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:59 smithi176 ceph-mon[112742]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:58:59.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:59 smithi176 ceph-mon[112742]: osd.4 [v2:172.21.15.196:6808/1908907228,v1:172.21.15.196:6809/1908907228] boot 2024-03-13T14:58:59.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:59 smithi176 ceph-mon[112742]: osdmap e100: 6 total, 6 up, 6 in 2024-03-13T14:58:59.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:58:59 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-13T14:59:00.807 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:00 smithi176 ceph-mon[112742]: osdmap e101: 6 total, 6 up, 6 in 2024-03-13T14:59:00.807 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:00.807 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:00.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:00 smithi196 ceph-mon[86193]: osdmap e101: 6 total, 6 up, 6 in 2024-03-13T14:59:00.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:00.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:01.828 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:01 smithi196 ceph-mon[86193]: pgmap v204: 65 pgs: 11 peering, 11 active+undersized, 10 active+undersized+degraded, 33 active+clean; 204 MiB data, 942 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 20/225 objects degraded (8.889%) 2024-03-13T14:59:01.829 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:01 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 20/225 objects degraded (8.889%), 10 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:01.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:01 smithi176 ceph-mon[112742]: pgmap v204: 65 pgs: 11 peering, 11 active+undersized, 10 active+undersized+degraded, 33 active+clean; 204 MiB data, 942 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 20/225 objects degraded (8.889%) 2024-03-13T14:59:01.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:01 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 20/225 objects degraded (8.889%), 10 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: pgmap v205: 65 pgs: 11 peering, 5 active+undersized, 7 active+undersized+degraded, 42 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 0 op/s; 12/225 objects degraded (5.333%) 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:03.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: pgmap v205: 65 pgs: 11 peering, 5 active+undersized, 7 active+undersized+degraded, 42 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 341 B/s rd, 0 op/s; 12/225 objects degraded (5.333%) 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:03.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:04.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:04 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:04.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:04 smithi196 ceph-mon[86193]: Upgrade: unsafe to stop osd(s) at this time (4 PGs are or would become offline) 2024-03-13T14:59:04.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:04 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:04.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:04 smithi176 ceph-mon[112742]: Upgrade: unsafe to stop osd(s) at this time (4 PGs are or would become offline) 2024-03-13T14:59:05.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:05 smithi196 ceph-mon[86193]: pgmap v206: 65 pgs: 10 peering, 55 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 258 B/s rd, 0 op/s 2024-03-13T14:59:05.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:05 smithi196 ceph-mon[86193]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/225 objects degraded (5.333%), 7 pgs degraded) 2024-03-13T14:59:05.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:05 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T14:59:05.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:05 smithi176 ceph-mon[112742]: pgmap v206: 65 pgs: 10 peering, 55 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 258 B/s rd, 0 op/s 2024-03-13T14:59:05.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:05 smithi176 ceph-mon[112742]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/225 objects degraded (5.333%), 7 pgs degraded) 2024-03-13T14:59:05.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:05 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T14:59:07.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:07 smithi196 ceph-mon[86193]: pgmap v207: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-03-13T14:59:07.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:07 smithi176 ceph-mon[112742]: pgmap v207: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 895 B/s rd, 1 op/s 2024-03-13T14:59:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:09 smithi176 ceph-mon[112742]: pgmap v208: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 724 B/s rd, 1 op/s 2024-03-13T14:59:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:09 smithi196 ceph-mon[86193]: pgmap v208: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 724 B/s rd, 1 op/s 2024-03-13T14:59:11.755 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:11 smithi196 ceph-mon[86193]: pgmap v209: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 940 B/s rd, 1 op/s 2024-03-13T14:59:11.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:11 smithi176 ceph-mon[112742]: pgmap v209: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 940 B/s rd, 1 op/s 2024-03-13T14:59:13.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 -- 172.21.15.176:0/1978508144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073390 msgr2=0x7fbe30073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:13.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 --2- 172.21.15.176:0/1978508144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073390 0x7fbe30073770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009940 tx=0x7fbe24009c50 comp rx=0 tx=0).stop 2024-03-13T14:59:13.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 -- 172.21.15.176:0/1978508144 shutdown_connections 2024-03-13T14:59:13.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 --2- 172.21.15.176:0/1978508144 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30073cb0 0x7fbe3010cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 --2- 172.21.15.176:0/1978508144 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073390 0x7fbe30073770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 -- 172.21.15.176:0/1978508144 >> 172.21.15.176:0/1978508144 conn(0x7fbe300fc4d0 msgr2=0x7fbe300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:13.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 -- 172.21.15.176:0/1978508144 shutdown_connections 2024-03-13T14:59:13.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.408+0000 7fbe36200700 1 -- 172.21.15.176:0/1978508144 wait complete. 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.409+0000 7fbe36200700 1 Processor -- start 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.409+0000 7fbe36200700 1 -- start start 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe36200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30073390 0x7fbe3019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe36200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073cb0 0x7fbe3019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe36200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3019f150 con 0x7fbe30073cb0 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe36200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30198600 con 0x7fbe30073390 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe2fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073cb0 0x7fbe3019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe2fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073cb0 0x7fbe3019eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60636/0 (socket says 172.21.15.176:60636) 2024-03-13T14:59:13.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe2fe00700 1 -- 172.21.15.176:0/121470220 learned_addr learned my addr 172.21.15.176:0/121470220 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.410+0000 7fbe34e00700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30073390 0x7fbe3019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe2fe00700 1 -- 172.21.15.176:0/121470220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30073390 msgr2=0x7fbe3019e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe2fe00700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30073390 0x7fbe3019e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe2fe00700 1 -- 172.21.15.176:0/121470220 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe240095f0 con 0x7fbe30073cb0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe34e00700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30073390 0x7fbe3019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe2fe00700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073cb0 0x7fbe3019eac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00c8f0 tx=0x7fbe1c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe2d600700 1 -- 172.21.15.176:0/121470220 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c007a50 con 0x7fbe30073cb0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe2d600700 1 -- 172.21.15.176:0/121470220 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe1c007bb0 con 0x7fbe30073cb0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe301988e0 con 0x7fbe30073cb0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe2d600700 1 -- 172.21.15.176:0/121470220 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c018630 con 0x7fbe30073cb0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.411+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe30198e30 con 0x7fbe30073cb0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.412+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3010a3a0 con 0x7fbe30073cb0 2024-03-13T14:59:13.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.414+0000 7fbe2d600700 1 -- 172.21.15.176:0/121470220 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fbe1c007d20 con 0x7fbe30073cb0 2024-03-13T14:59:13.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.415+0000 7fbe2d600700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbe1806dfa0 0x7fbe18070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:13.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.415+0000 7fbe2d600700 1 -- 172.21.15.176:0/121470220 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fbe1c08de20 con 0x7fbe30073cb0 2024-03-13T14:59:13.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.415+0000 7fbe34e00700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbe1806dfa0 0x7fbe18070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:13.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.416+0000 7fbe34e00700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbe1806dfa0 0x7fbe18070460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000c00 tx=0x7fbe24005fb0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:13.405 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.420+0000 7fbe2d600700 1 -- 172.21.15.176:0/121470220 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fbe1c05acb0 con 0x7fbe30073cb0 2024-03-13T14:59:13.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.663+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe30199150 con 0x7fbe1806dfa0 2024-03-13T14:59:13.649 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:13 smithi176 ceph-mon[112742]: pgmap v210: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:13.651 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.665+0000 7fbe2d600700 1 -- 172.21.15.176:0/121470220 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbe30199150 con 0x7fbe1806dfa0 2024-03-13T14:59:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbe1806dfa0 msgr2=0x7fbe18070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:13.654 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbe1806dfa0 0x7fbe18070460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000c00 tx=0x7fbe24005fb0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073cb0 msgr2=0x7fbe3019eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073cb0 0x7fbe3019eac0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00c8f0 tx=0x7fbe1c00ccb0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 shutdown_connections 2024-03-13T14:59:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbe1806dfa0 0x7fbe18070460 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe30073390 0x7fbe3019e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 --2- 172.21.15.176:0/121470220 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbe30073cb0 0x7fbe3019eac0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.655 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 >> 172.21.15.176:0/121470220 conn(0x7fbe300fc4d0 msgr2=0x7fbe301074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:13.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 shutdown_connections 2024-03-13T14:59:13.656 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.669+0000 7fbe36200700 1 -- 172.21.15.176:0/121470220 wait complete. 2024-03-13T14:59:13.670 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.827+0000 7f7cee200700 1 -- 172.21.15.176:0/362908808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 msgr2=0x7f7ce8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.827+0000 7f7cee200700 1 --2- 172.21.15.176:0/362908808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce8107a00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009910 tx=0x7f7cd4009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.828+0000 7f7cee200700 1 -- 172.21.15.176:0/362908808 shutdown_connections 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.828+0000 7f7cee200700 1 --2- 172.21.15.176:0/362908808 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce8107a00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.828+0000 7f7cee200700 1 --2- 172.21.15.176:0/362908808 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ce8103000 0x7f7ce81033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.828+0000 7f7cee200700 1 -- 172.21.15.176:0/362908808 >> 172.21.15.176:0/362908808 conn(0x7f7ce80fe8d0 msgr2=0x7f7ce8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.828+0000 7f7cee200700 1 -- 172.21.15.176:0/362908808 shutdown_connections 2024-03-13T14:59:13.814 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.828+0000 7f7cee200700 1 -- 172.21.15.176:0/362908808 wait complete. 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.829+0000 7f7cee200700 1 Processor -- start 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.829+0000 7f7cee200700 1 -- start start 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.829+0000 7f7cee200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ce8103000 0x7f7ce819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.829+0000 7f7cee200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.829+0000 7f7cee200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce819f390 con 0x7f7ce81039b0 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.829+0000 7f7cee200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce8198840 con 0x7f7ce8103000 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce6a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60654/0 (socket says 172.21.15.176:60654) 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce6a00700 1 -- 172.21.15.176:0/563979726 learned_addr learned my addr 172.21.15.176:0/563979726 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce7400700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ce8103000 0x7f7ce819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce6a00700 1 -- 172.21.15.176:0/563979726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ce8103000 msgr2=0x7f7ce819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce6a00700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ce8103000 0x7f7ce819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:13.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce6a00700 1 -- 172.21.15.176:0/563979726 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd40095f0 con 0x7f7ce81039b0 2024-03-13T14:59:13.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.830+0000 7f7ce6a00700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce819ed00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400bda0 tx=0x7f7cd400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:13.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.831+0000 7f7cdfe00700 1 -- 172.21.15.176:0/563979726 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd401d070 con 0x7f7ce81039b0 2024-03-13T14:59:13.817 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.831+0000 7f7cdfe00700 1 -- 172.21.15.176:0/563979726 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd400f460 con 0x7f7ce81039b0 2024-03-13T14:59:13.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.831+0000 7f7cdfe00700 1 -- 172.21.15.176:0/563979726 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd4017610 con 0x7f7ce81039b0 2024-03-13T14:59:13.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.831+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce8198ac0 con 0x7f7ce81039b0 2024-03-13T14:59:13.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.831+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce8198f60 con 0x7f7ce81039b0 2024-03-13T14:59:13.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.832+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce810b400 con 0x7f7ce81039b0 2024-03-13T14:59:13.818 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.833+0000 7f7cdfe00700 1 -- 172.21.15.176:0/563979726 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f7cd400fa40 con 0x7f7ce81039b0 2024-03-13T14:59:13.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.834+0000 7f7cdfe00700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ccc06dfa0 0x7f7ccc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:13.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.834+0000 7f7cdfe00700 1 -- 172.21.15.176:0/563979726 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f7cd408f2f0 con 0x7f7ce81039b0 2024-03-13T14:59:13.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.834+0000 7f7ce7400700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ccc06dfa0 0x7f7ccc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:13.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.835+0000 7f7ce7400700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ccc06dfa0 0x7f7ccc070460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8005950 tx=0x7f7cd80058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:13.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:13.839+0000 7f7cdfe00700 1 -- 172.21.15.176:0/563979726 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f7cd405c180 con 0x7f7ce81039b0 2024-03-13T14:59:13.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:13 smithi196 ceph-mon[86193]: pgmap v210: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:14.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.078+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ce802cd80 con 0x7f7ccc06dfa0 2024-03-13T14:59:14.066 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.080+0000 7f7cdfe00700 1 -- 172.21.15.176:0/563979726 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f7ce802cd80 con 0x7f7ccc06dfa0 2024-03-13T14:59:14.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ccc06dfa0 msgr2=0x7f7ccc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ccc06dfa0 0x7f7ccc070460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8005950 tx=0x7f7cd80058e0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 msgr2=0x7f7ce819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce819ed00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400bda0 tx=0x7f7cd400be80 comp rx=0 tx=0).stop 2024-03-13T14:59:14.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 shutdown_connections 2024-03-13T14:59:14.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f7ccc06dfa0 0x7f7ccc070460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ce8103000 0x7f7ce819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 --2- 172.21.15.176:0/563979726 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f7ce81039b0 0x7f7ce819ed00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 >> 172.21.15.176:0/563979726 conn(0x7f7ce80fe8d0 msgr2=0x7f7ce8100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:14.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.083+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 shutdown_connections 2024-03-13T14:59:14.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.084+0000 7f7cee200700 1 -- 172.21.15.176:0/563979726 wait complete. 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.225+0000 7f39e4c00700 1 -- 172.21.15.176:0/970094453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 msgr2=0x7f39e010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.225+0000 7f39e4c00700 1 --2- 172.21.15.176:0/970094453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e010cf00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009910 tx=0x7f39c8009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.226+0000 7f39e4c00700 1 -- 172.21.15.176:0/970094453 shutdown_connections 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.226+0000 7f39e4c00700 1 --2- 172.21.15.176:0/970094453 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e010cf00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.226+0000 7f39e4c00700 1 --2- 172.21.15.176:0/970094453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e00ff890 0x7f39e00ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.226+0000 7f39e4c00700 1 -- 172.21.15.176:0/970094453 >> 172.21.15.176:0/970094453 conn(0x7f39e00fb570 msgr2=0x7f39e00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.226+0000 7f39e4c00700 1 -- 172.21.15.176:0/970094453 shutdown_connections 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.226+0000 7f39e4c00700 1 -- 172.21.15.176:0/970094453 wait complete. 2024-03-13T14:59:14.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39e4c00700 1 Processor -- start 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39e4c00700 1 -- start start 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39e4c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e00ff890 0x7f39e019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39e4c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39e4c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e019f2d0 con 0x7f39e0100240 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39e4c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e0198780 con 0x7f39e00ff890 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39d7e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39d7e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e019ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60674/0 (socket says 172.21.15.176:60674) 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.227+0000 7f39d7e00700 1 -- 172.21.15.176:0/2481154770 learned_addr learned my addr 172.21.15.176:0/2481154770 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39dde00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e00ff890 0x7f39e019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39d7e00700 1 -- 172.21.15.176:0/2481154770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e00ff890 msgr2=0x7f39e019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39d7e00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e00ff890 0x7f39e019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39d7e00700 1 -- 172.21.15.176:0/2481154770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c80095f0 con 0x7f39e0100240 2024-03-13T14:59:14.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39d7e00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e019ec40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005b40 tx=0x7f39c80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:14.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39d6a00700 1 -- 172.21.15.176:0/2481154770 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39c801d070 con 0x7f39e0100240 2024-03-13T14:59:14.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39d6a00700 1 -- 172.21.15.176:0/2481154770 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39c800f460 con 0x7f39e0100240 2024-03-13T14:59:14.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e0198a00 con 0x7f39e0100240 2024-03-13T14:59:14.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.228+0000 7f39d6a00700 1 -- 172.21.15.176:0/2481154770 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39c8021640 con 0x7f39e0100240 2024-03-13T14:59:14.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.229+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e0198f50 con 0x7f39e0100240 2024-03-13T14:59:14.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.230+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e010a670 con 0x7f39e0100240 2024-03-13T14:59:14.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.231+0000 7f39d6a00700 1 -- 172.21.15.176:0/2481154770 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f39c800f5d0 con 0x7f39e0100240 2024-03-13T14:59:14.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.232+0000 7f39d6a00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f39c406dfa0 0x7f39c4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:14.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.232+0000 7f39d6a00700 1 -- 172.21.15.176:0/2481154770 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f39c808f130 con 0x7f39e0100240 2024-03-13T14:59:14.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.232+0000 7f39dde00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f39c406dfa0 0x7f39c4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:14.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.233+0000 7f39dde00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f39c406dfa0 0x7f39c4070460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39d00060b0 tx=0x7f39d0007b80 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:14.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.237+0000 7f39d6a00700 1 -- 172.21.15.176:0/2481154770 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f39c805c040 con 0x7f39e0100240 2024-03-13T14:59:14.453 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.467+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f39e0199280 con 0x7f39c406dfa0 2024-03-13T14:59:14.469 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.483+0000 7f39d6a00700 1 -- 172.21.15.176:0/2481154770 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f39e0199280 con 0x7f39c406dfa0 2024-03-13T14:59:14.469 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:59:14.469 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (7m) 106s ago 20m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:59:14.469 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (21m) 106s ago 21m 8812k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:59:14.469 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (19m) 19s ago 19m 8652k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:59:14.469 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (3m) 106s ago 21m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:59:14.469 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (3m) 19s ago 19m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (6m) 106s ago 20m 73.6M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (13m) 106s ago 13m 87.3M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (13m) 106s ago 13m 17.9M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (13m) 19s ago 13m 140M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (13m) 19s ago 13m 18.2M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (10m) 106s ago 22m 504M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (8m) 19s ago 19m 451M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (5m) 106s ago 22m 47.3M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (4m) 19s ago 19m 39.3M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (8m) 106s ago 20m 7520k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (8m) 19s ago 19m 7646k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (3m) 106s ago 18m 122M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T14:59:14.470 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (2m) 106s ago 17m 97.2M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T14:59:14.471 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (113s) 106s ago 17m 11.1M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T14:59:14.471 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (68s) 19s ago 16m 142M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T14:59:14.471 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (25s) 19s ago 15m 11.2M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T14:59:14.471 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (15m) 19s ago 15m 322M 2241M 18.2.1 5be31c24972a 2805450c8460 2024-03-13T14:59:14.471 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (7m) 106s ago 19m 69.1M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:59:14.472 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f39c406dfa0 msgr2=0x7f39c4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.472 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f39c406dfa0 0x7f39c4070460 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39d00060b0 tx=0x7f39d0007b80 comp rx=0 tx=0).stop 2024-03-13T14:59:14.472 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 msgr2=0x7f39e019ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.472 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e019ec40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f39c8005b40 tx=0x7f39c80056a0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.472 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 shutdown_connections 2024-03-13T14:59:14.472 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f39c406dfa0 0x7f39c4070460 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.472 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39e00ff890 0x7f39e019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.473 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 --2- 172.21.15.176:0/2481154770 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f39e0100240 0x7f39e019ec40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.473 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 >> 172.21.15.176:0/2481154770 conn(0x7f39e00fb570 msgr2=0x7f39e00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:14.473 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 shutdown_connections 2024-03-13T14:59:14.473 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.486+0000 7f39e4c00700 1 -- 172.21.15.176:0/2481154770 wait complete. 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 -- 172.21.15.176:0/3429009476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c181071d0 msgr2=0x7f2c18107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3429009476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c181071d0 0x7f2c18107650 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c04009910 tx=0x7f2c04009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 -- 172.21.15.176:0/3429009476 shutdown_connections 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3429009476 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c181071d0 0x7f2c18107650 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3429009476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c1810d280 0x7f2c1810d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 -- 172.21.15.176:0/3429009476 >> 172.21.15.176:0/3429009476 conn(0x7f2c180769e0 msgr2=0x7f2c18078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 -- 172.21.15.176:0/3429009476 shutdown_connections 2024-03-13T14:59:14.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.639+0000 7f2c1e600700 1 -- 172.21.15.176:0/3429009476 wait complete. 2024-03-13T14:59:14.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.640+0000 7f2c1e600700 1 Processor -- start 2024-03-13T14:59:14.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.640+0000 7f2c1e600700 1 -- start start 2024-03-13T14:59:14.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c1e600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c181071d0 0x7f2c1819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:14.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c1e600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c1810d280 0x7f2c1819d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c1e600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1819d880 con 0x7f2c1810d280 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c1e600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181a05b0 con 0x7f2c181071d0 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c16a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c1810d280 0x7f2c1819d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c16a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c1810d280 0x7f2c1819d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60698/0 (socket says 172.21.15.176:60698) 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c16a00700 1 -- 172.21.15.176:0/3984737930 learned_addr learned my addr 172.21.15.176:0/3984737930 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.641+0000 7f2c17400700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c181071d0 0x7f2c1819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c16a00700 1 -- 172.21.15.176:0/3984737930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c181071d0 msgr2=0x7f2c1819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c16a00700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c181071d0 0x7f2c1819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c16a00700 1 -- 172.21.15.176:0/3984737930 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c040095f0 con 0x7f2c1810d280 2024-03-13T14:59:14.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c16a00700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c1810d280 0x7f2c1819d1f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006010 tx=0x7f2c0400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:14.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c0fe00700 1 -- 172.21.15.176:0/3984737930 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c0401d070 con 0x7f2c1810d280 2024-03-13T14:59:14.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c0fe00700 1 -- 172.21.15.176:0/3984737930 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c0400f460 con 0x7f2c1810d280 2024-03-13T14:59:14.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c0fe00700 1 -- 172.21.15.176:0/3984737930 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c04017630 con 0x7f2c1810d280 2024-03-13T14:59:14.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.642+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c181a0830 con 0x7f2c1810d280 2024-03-13T14:59:14.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.643+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c181a0da0 con 0x7f2c1810d280 2024-03-13T14:59:14.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.644+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1810f6c0 con 0x7f2c1810d280 2024-03-13T14:59:14.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.645+0000 7f2c0fe00700 1 -- 172.21.15.176:0/3984737930 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2c0400f5d0 con 0x7f2c1810d280 2024-03-13T14:59:14.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.645+0000 7f2c0fe00700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2bfc06dfa0 0x7f2bfc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:14.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.646+0000 7f2c0fe00700 1 -- 172.21.15.176:0/3984737930 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f2c0408f1d0 con 0x7f2c1810d280 2024-03-13T14:59:14.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.646+0000 7f2c17400700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2bfc06dfa0 0x7f2bfc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.646+0000 7f2c17400700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2bfc06dfa0 0x7f2bfc070460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009d10 tx=0x7f2c08009040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.650+0000 7f2c0fe00700 1 -- 172.21.15.176:0/3984737930 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f2c04058650 con 0x7f2c1810d280 2024-03-13T14:59:14.937 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.951+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2c1804ea90 con 0x7f2c1810d280 2024-03-13T14:59:14.937 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.952+0000 7f2c0fe00700 1 -- 172.21.15.176:0/3984737930 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f2c0405bd10 con 0x7f2c1810d280 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:14.938 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 5 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 5, 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 9 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:59:14.939 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:59:14.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.954+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2bfc06dfa0 msgr2=0x7f2bfc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.954+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2bfc06dfa0 0x7f2bfc070460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009d10 tx=0x7f2c08009040 comp rx=0 tx=0).stop 2024-03-13T14:59:14.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.954+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c1810d280 msgr2=0x7f2c1819d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:14.940 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.954+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c1810d280 0x7f2c1819d1f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c04006010 tx=0x7f2c0400be20 comp rx=0 tx=0).stop 2024-03-13T14:59:14.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.955+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 shutdown_connections 2024-03-13T14:59:14.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.955+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f2bfc06dfa0 0x7f2bfc070460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.955+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2c181071d0 0x7f2c1819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.955+0000 7f2c1e600700 1 --2- 172.21.15.176:0/3984737930 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f2c1810d280 0x7f2c1819d1f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:14.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.955+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 >> 172.21.15.176:0/3984737930 conn(0x7f2c180769e0 msgr2=0x7f2c18078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:14.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.955+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 shutdown_connections 2024-03-13T14:59:14.941 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:14.955+0000 7f2c1e600700 1 -- 172.21.15.176:0/3984737930 wait complete. 2024-03-13T14:59:15.096 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.110+0000 7f0ea1800700 1 -- 172.21.15.176:0/4182011189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c10d280 msgr2=0x7f0e9c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.110+0000 7f0ea1800700 1 --2- 172.21.15.176:0/4182011189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c10d280 0x7f0e9c10d660 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009910 tx=0x7f0e8c009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:15.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.110+0000 7f0ea1800700 1 -- 172.21.15.176:0/4182011189 shutdown_connections 2024-03-13T14:59:15.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.110+0000 7f0ea1800700 1 --2- 172.21.15.176:0/4182011189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e9c1071d0 0x7f0e9c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.110+0000 7f0ea1800700 1 --2- 172.21.15.176:0/4182011189 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c10d280 0x7f0e9c10d660 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.111+0000 7f0ea1800700 1 -- 172.21.15.176:0/4182011189 >> 172.21.15.176:0/4182011189 conn(0x7f0e9c0769e0 msgr2=0x7f0e9c078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:15.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.111+0000 7f0ea1800700 1 -- 172.21.15.176:0/4182011189 shutdown_connections 2024-03-13T14:59:15.097 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.111+0000 7f0ea1800700 1 -- 172.21.15.176:0/4182011189 wait complete. 2024-03-13T14:59:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0ea1800700 1 Processor -- start 2024-03-13T14:59:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0ea1800700 1 -- start start 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0ea1800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c1071d0 0x7f0e9c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0ea1800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e9c10d280 0x7f0e9c19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0ea1800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c19d8a0 con 0x7f0e9c1071d0 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0ea1800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c1a05d0 con 0x7f0e9c10d280 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0e9aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c1071d0 0x7f0e9c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.112+0000 7f0e9aa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c1071d0 0x7f0e9c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60708/0 (socket says 172.21.15.176:60708) 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.113+0000 7f0e9aa00700 1 -- 172.21.15.176:0/1761613337 learned_addr learned my addr 172.21.15.176:0/1761613337 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.113+0000 7f0e9a000700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e9c10d280 0x7f0e9c19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.113+0000 7f0e9aa00700 1 -- 172.21.15.176:0/1761613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e9c10d280 msgr2=0x7f0e9c19d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.113+0000 7f0e9aa00700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e9c10d280 0x7f0e9c19d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.113+0000 7f0e9aa00700 1 -- 172.21.15.176:0/1761613337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0095f0 con 0x7f0e9c1071d0 2024-03-13T14:59:15.100 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.113+0000 7f0e9aa00700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c1071d0 0x7f0e9c19ccd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0092e0 tx=0x7f0e8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.114+0000 7f0e93400700 1 -- 172.21.15.176:0/1761613337 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e8c01d070 con 0x7f0e9c1071d0 2024-03-13T14:59:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.114+0000 7f0e93400700 1 -- 172.21.15.176:0/1761613337 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e8c005e00 con 0x7f0e9c1071d0 2024-03-13T14:59:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.114+0000 7f0e93400700 1 -- 172.21.15.176:0/1761613337 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e8c017400 con 0x7f0e9c1071d0 2024-03-13T14:59:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.114+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9c1a0850 con 0x7f0e9c1071d0 2024-03-13T14:59:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.114+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9c1a0d70 con 0x7f0e9c1071d0 2024-03-13T14:59:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.115+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9c04ea90 con 0x7f0e9c1071d0 2024-03-13T14:59:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.116+0000 7f0e93400700 1 -- 172.21.15.176:0/1761613337 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0e8c00f460 con 0x7f0e9c1071d0 2024-03-13T14:59:15.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.117+0000 7f0e93400700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e8006dfa0 0x7f0e80070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.117+0000 7f0e93400700 1 -- 172.21.15.176:0/1761613337 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f0e8c08f3a0 con 0x7f0e9c1071d0 2024-03-13T14:59:15.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.117+0000 7f0e9a000700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e8006dfa0 0x7f0e80070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.118+0000 7f0e9a000700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e8006dfa0 0x7f0e80070460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005950 tx=0x7f0e880058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:15.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.122+0000 7f0e93400700 1 -- 172.21.15.176:0/1761613337 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f0e8c058700 con 0x7f0e9c1071d0 2024-03-13T14:59:15.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.386+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0e9c19dff0 con 0x7f0e9c1071d0 2024-03-13T14:59:15.373 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.387+0000 7f0e93400700 1 -- 172.21.15.176:0/1761613337 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 18 v18) v1 ==== 76+0+1877 (secure 0 0 0) 0x7f0e8c05bdc0 con 0x7f0e9c1071d0 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:e18 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 18 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:59:15.374 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:46:02.172324+0000 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 47 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:59:15.375 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:14484} state up:standby-replay seq 3 join_fscid=1 addr [v2:172.21.15.176:6828/4135138463,v1:172.21.15.176:6829/4135138463] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:59:15.376 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:15.377 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:15.377 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:59:15.377 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:15.377 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:59:15.377 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.390+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e8006dfa0 msgr2=0x7f0e80070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.390+0000 7f0ea1800700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e8006dfa0 0x7f0e80070460 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0e88005950 tx=0x7f0e880058e0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.390+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c1071d0 msgr2=0x7f0e9c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.390+0000 7f0ea1800700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c1071d0 0x7f0e9c19ccd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0092e0 tx=0x7f0e8c00be20 comp rx=0 tx=0).stop 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.391+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 shutdown_connections 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.391+0000 7f0ea1800700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e8006dfa0 0x7f0e80070460 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.391+0000 7f0ea1800700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e9c1071d0 0x7f0e9c19ccd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.391+0000 7f0ea1800700 1 --2- 172.21.15.176:0/1761613337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e9c10d280 0x7f0e9c19d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.391+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 >> 172.21.15.176:0/1761613337 conn(0x7f0e9c0769e0 msgr2=0x7f0e9c078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.391+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 shutdown_connections 2024-03-13T14:59:15.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.391+0000 7f0ea1800700 1 -- 172.21.15.176:0/1761613337 wait complete. 2024-03-13T14:59:15.379 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 18 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 -- 172.21.15.176:0/809534429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 msgr2=0x7f599810d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 --2- 172.21.15.176:0/809534429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599810d660 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f598c009910 tx=0x7f598c009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 -- 172.21.15.176:0/809534429 shutdown_connections 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 --2- 172.21.15.176:0/809534429 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59981071d0 0x7f5998107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 --2- 172.21.15.176:0/809534429 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599810d660 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 -- 172.21.15.176:0/809534429 >> 172.21.15.176:0/809534429 conn(0x7f59980769e0 msgr2=0x7f5998078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 -- 172.21.15.176:0/809534429 shutdown_connections 2024-03-13T14:59:15.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.542+0000 7f599f800700 1 -- 172.21.15.176:0/809534429 wait complete. 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.543+0000 7f599f800700 1 Processor -- start 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.543+0000 7f599f800700 1 -- start start 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f599f800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59981071d0 0x7f599819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f599f800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f599f800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599819d8a0 con 0x7f599810d280 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f599f800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59981a05d0 con 0x7f59981071d0 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f5997e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f5997e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60730/0 (socket says 172.21.15.176:60730) 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f5997e00700 1 -- 172.21.15.176:0/493165459 learned_addr learned my addr 172.21.15.176:0/493165459 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f599ce00700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59981071d0 0x7f599819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f5997e00700 1 -- 172.21.15.176:0/493165459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59981071d0 msgr2=0x7f599819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f5997e00700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59981071d0 0x7f599819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.544+0000 7f5997e00700 1 -- 172.21.15.176:0/493165459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0095f0 con 0x7f599810d280 2024-03-13T14:59:15.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.545+0000 7f5997e00700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599819d210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f598400d8d0 tx=0x7f598400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:15.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.545+0000 7f5995600700 1 -- 172.21.15.176:0/493165459 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5984009940 con 0x7f599810d280 2024-03-13T14:59:15.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.545+0000 7f5995600700 1 -- 172.21.15.176:0/493165459 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5984010460 con 0x7f599810d280 2024-03-13T14:59:15.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.545+0000 7f5995600700 1 -- 172.21.15.176:0/493165459 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598400f5f0 con 0x7f599810d280 2024-03-13T14:59:15.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.545+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59981a08b0 con 0x7f599810d280 2024-03-13T14:59:15.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.545+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59981a0e80 con 0x7f599810d280 2024-03-13T14:59:15.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.546+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599804ea90 con 0x7f599810d280 2024-03-13T14:59:15.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.547+0000 7f5995600700 1 -- 172.21.15.176:0/493165459 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f59840105d0 con 0x7f599810d280 2024-03-13T14:59:15.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.548+0000 7f5995600700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f598006dfa0 0x7f5980070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.549+0000 7f5995600700 1 -- 172.21.15.176:0/493165459 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f598408de90 con 0x7f599810d280 2024-03-13T14:59:15.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.549+0000 7f599ce00700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f598006dfa0 0x7f5980070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.550+0000 7f599ce00700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f598006dfa0 0x7f5980070460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f598c006010 tx=0x7f598c01a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:15.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.554+0000 7f5995600700 1 -- 172.21.15.176:0/493165459 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f59840571f0 con 0x7f599810d280 2024-03-13T14:59:15.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:15.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:15.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: pgmap v211: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:15.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:15.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3984737930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:15.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1761613337' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:59:15.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:15.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: pgmap v211: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3984737930' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1761613337' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:15.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:15.771 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.786+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59981a11d0 con 0x7f598006dfa0 2024-03-13T14:59:15.774 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.788+0000 7f5995600700 1 -- 172.21.15.176:0/493165459 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f59981a11d0 con 0x7f598006dfa0 2024-03-13T14:59:15.774 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:59:15.774 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:59:15.774 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:59:15.774 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:59:15.774 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "11/23 daemons upgraded", 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading osd daemons", 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:59:15.775 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f598006dfa0 msgr2=0x7f5980070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f598006dfa0 0x7f5980070460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f598c006010 tx=0x7f598c01a040 comp rx=0 tx=0).stop 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 msgr2=0x7f599819d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599819d210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f598400d8d0 tx=0x7f598400dc90 comp rx=0 tx=0).stop 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 shutdown_connections 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f598006dfa0 0x7f5980070460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59981071d0 0x7f599819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 --2- 172.21.15.176:0/493165459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f599810d280 0x7f599819d210 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.791+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 >> 172.21.15.176:0/493165459 conn(0x7f59980769e0 msgr2=0x7f5998078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:15.778 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.792+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 shutdown_connections 2024-03-13T14:59:15.778 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.792+0000 7f599f800700 1 -- 172.21.15.176:0/493165459 wait complete. 2024-03-13T14:59:15.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.941+0000 7f9b04a00700 1 -- 172.21.15.176:0/4107270787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 msgr2=0x7f9b00107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.941+0000 7f9b04a00700 1 --2- 172.21.15.176:0/4107270787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b00107650 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009910 tx=0x7f9aec009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:15.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.941+0000 7f9b04a00700 1 -- 172.21.15.176:0/4107270787 shutdown_connections 2024-03-13T14:59:15.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.941+0000 7f9b04a00700 1 --2- 172.21.15.176:0/4107270787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b00107650 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.941+0000 7f9b04a00700 1 --2- 172.21.15.176:0/4107270787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b0010d280 0x7f9b0010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.941+0000 7f9b04a00700 1 -- 172.21.15.176:0/4107270787 >> 172.21.15.176:0/4107270787 conn(0x7f9b000769e0 msgr2=0x7f9b00078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:15.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.942+0000 7f9b04a00700 1 -- 172.21.15.176:0/4107270787 shutdown_connections 2024-03-13T14:59:15.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.942+0000 7f9b04a00700 1 -- 172.21.15.176:0/4107270787 wait complete. 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.943+0000 7f9b04a00700 1 Processor -- start 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.943+0000 7f9b04a00700 1 -- start start 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.943+0000 7f9b04a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b0019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.943+0000 7f9b04a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b0010d280 0x7f9b0019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.943+0000 7f9b04a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0019da30 con 0x7f9b001071d0 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.943+0000 7f9b04a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b001a17c0 con 0x7f9b0010d280 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.943+0000 7f9afde00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b0019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9afde00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b0019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:60750/0 (socket says 172.21.15.176:60750) 2024-03-13T14:59:15.930 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9afde00700 1 -- 172.21.15.176:0/2858217852 learned_addr learned my addr 172.21.15.176:0/2858217852 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:15.930 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9afd400700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b0010d280 0x7f9b0019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9afde00700 1 -- 172.21.15.176:0/2858217852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b0010d280 msgr2=0x7f9b0019d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9afde00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b0010d280 0x7f9b0019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9afde00700 1 -- 172.21.15.176:0/2858217852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec0095f0 con 0x7f9b001071d0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9afde00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b0019ce10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b4b0 tx=0x7f9af000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.944+0000 7f9af6a00700 1 -- 172.21.15.176:0/2858217852 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af0011800 con 0x7f9b001071d0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.945+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b001a1aa0 con 0x7f9b001071d0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.945+0000 7f9af6a00700 1 -- 172.21.15.176:0/2858217852 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9af0011e20 con 0x7f9b001071d0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.945+0000 7f9af6a00700 1 -- 172.21.15.176:0/2858217852 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af000f2e0 con 0x7f9b001071d0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.945+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b001a2070 con 0x7f9b001071d0 2024-03-13T14:59:15.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.946+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b0010f6f0 con 0x7f9b001071d0 2024-03-13T14:59:15.933 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.947+0000 7f9af6a00700 1 -- 172.21.15.176:0/2858217852 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f9af0010380 con 0x7f9b001071d0 2024-03-13T14:59:15.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.948+0000 7f9af6a00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f9ae406dfa0 0x7f9ae4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:15.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.948+0000 7f9af6a00700 1 -- 172.21.15.176:0/2858217852 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f9af008d960 con 0x7f9b001071d0 2024-03-13T14:59:15.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.948+0000 7f9afd400700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f9ae406dfa0 0x7f9ae4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:15.934 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.949+0000 7f9afd400700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f9ae406dfa0 0x7f9ae4070460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9aec006010 tx=0x7f9aec00b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:15.938 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:15.953+0000 7f9af6a00700 1 -- 172.21.15.176:0/2858217852 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f9af0056d40 con 0x7f9b001071d0 2024-03-13T14:59:16.242 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.257+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9b0004ea90 con 0x7f9b001071d0 2024-03-13T14:59:16.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.258+0000 7f9af6a00700 1 -- 172.21.15.176:0/2858217852 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9af005a400 con 0x7f9b001071d0 2024-03-13T14:59:16.243 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:59:16.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f9ae406dfa0 msgr2=0x7f9ae4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f9ae406dfa0 0x7f9ae4070460 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9aec006010 tx=0x7f9aec00b540 comp rx=0 tx=0).stop 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 msgr2=0x7f9b0019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b0019ce10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b4b0 tx=0x7f9af000b870 comp rx=0 tx=0).stop 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 shutdown_connections 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f9ae406dfa0 0x7f9ae4070460 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9b001071d0 0x7f9b0019ce10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 --2- 172.21.15.176:0/2858217852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b0010d280 0x7f9b0019d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:16.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 >> 172.21.15.176:0/2858217852 conn(0x7f9b000769e0 msgr2=0x7f9b000784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:16.248 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 shutdown_connections 2024-03-13T14:59:16.248 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:16.261+0000 7f9b04a00700 1 -- 172.21.15.176:0/2858217852 wait complete. 2024-03-13T14:59:16.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:16.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:16 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2858217852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:59:16.921 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:16.922 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:16 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2858217852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:59:17.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:17 smithi176 ceph-mon[112742]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:17.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:17 smithi176 ceph-mon[112742]: pgmap v212: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:59:17.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:17 smithi196 ceph-mon[86193]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:17.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:17 smithi196 ceph-mon[86193]: pgmap v212: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:59:18.854 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:18.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:19 smithi176 ceph-mon[112742]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:19 smithi176 ceph-mon[112742]: Upgrade: osd.5 is safe to restart 2024-03-13T14:59:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:19 smithi176 ceph-mon[112742]: pgmap v213: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:19.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:19 smithi196 ceph-mon[86193]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-03-13T14:59:19.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:19 smithi196 ceph-mon[86193]: Upgrade: osd.5 is safe to restart 2024-03-13T14:59:19.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:19 smithi196 ceph-mon[86193]: pgmap v213: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:21.292 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:21 smithi196 ceph-mon[86193]: Upgrade: Updating osd.5 2024-03-13T14:59:21.292 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:21.292 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-13T14:59:21.292 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:21 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:21.292 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:21 smithi196 ceph-mon[86193]: Deploying daemon osd.5 on smithi196 2024-03-13T14:59:21.292 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:21 smithi196 ceph-mon[86193]: pgmap v214: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:59:21.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:21 smithi176 ceph-mon[112742]: Upgrade: Updating osd.5 2024-03-13T14:59:21.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:21.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-13T14:59:21.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:21 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:21.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:21 smithi176 ceph-mon[112742]: Deploying daemon osd.5 on smithi196 2024-03-13T14:59:21.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:21 smithi176 ceph-mon[112742]: pgmap v214: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:59:23.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:23 smithi196 ceph-mon[86193]: pgmap v215: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:23.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:23 smithi176 ceph-mon[112742]: pgmap v215: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:25.426 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:25 smithi196 systemd[1]: Stopping Ceph osd.5 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:59:25.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:25 smithi176 ceph-mon[112742]: pgmap v216: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:25.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:25 smithi196 ceph-mon[86193]: pgmap v216: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T14:59:25.927 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:25 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[56604]: 2024-03-13T14:59:25.538+0000 7fb347000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T14:59:25.927 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:25 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[56604]: 2024-03-13T14:59:25.538+0000 7fb347000700 -1 osd.5 101 *** Got signal Terminated *** 2024-03-13T14:59:25.927 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:25 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[56604]: 2024-03-13T14:59:25.538+0000 7fb347000700 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T14:59:26.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:26 smithi176 ceph-mon[112742]: osd.5 marked itself down and dead 2024-03-13T14:59:26.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:26 smithi196 ceph-mon[86193]: osd.5 marked itself down and dead 2024-03-13T14:59:27.426 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:27 smithi196 podman[106296]: 2024-03-13 14:59:27.167204504 +0000 UTC m=+1.777533227 container died 2805450c8460baa3447d9f8a7335dc26a93ccad4a280fce29de26b2d34f282dc (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.1, RELEASE=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-13T14:59:27.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:27 smithi176 ceph-mon[112742]: pgmap v217: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:59:27.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:27 smithi176 ceph-mon[112742]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:59:27.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:27 smithi176 ceph-mon[112742]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-13T14:59:27.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:27 smithi176 ceph-mon[112742]: osdmap e102: 6 total, 5 up, 6 in 2024-03-13T14:59:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:27 smithi196 ceph-mon[86193]: pgmap v217: 65 pgs: 65 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T14:59:27.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:27 smithi196 ceph-mon[86193]: Health check failed: 1 osds down (OSD_DOWN) 2024-03-13T14:59:27.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:27 smithi196 ceph-mon[86193]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-03-13T14:59:27.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:27 smithi196 ceph-mon[86193]: osdmap e102: 6 total, 5 up, 6 in 2024-03-13T14:59:27.927 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:27 smithi196 podman[106296]: 2024-03-13 14:59:27.663878918 +0000 UTC m=+2.274207634 container cleanup 2805450c8460baa3447d9f8a7335dc26a93ccad4a280fce29de26b2d34f282dc (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=-18.2.1, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.vendor=CentOS) 2024-03-13T14:59:27.927 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:27 smithi196 bash[106296]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5 2024-03-13T14:59:28.426 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:27 smithi196 podman[106316]: 2024-03-13 14:59:27.964992552 +0000 UTC m=+0.792238781 container remove 2805450c8460baa3447d9f8a7335dc26a93ccad4a280fce29de26b2d34f282dc (image=quay.io/ceph/ceph@sha256:9f35728f6070a596500c0804814a12ab6b98e05067316dc64876fb4b28d04af3, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20240222, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.1, ceph=True, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-13T14:59:28.768 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:28 smithi196 ceph-mon[86193]: osdmap e103: 6 total, 5 up, 6 in 2024-03-13T14:59:28.768 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:28 smithi196 podman[106404]: 2024-03-13 14:59:28.508800863 +0000 UTC m=+0.035043661 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:59:28.768 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:28 smithi196 podman[106404]: 2024-03-13 14:59:28.632754837 +0000 UTC m=+0.158997672 container create 25a6d6bd3aa4608ceca75d97616d48c2569d22b5a462d6c651bd5542f43473e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:59:28.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:28 smithi176 ceph-mon[112742]: osdmap e103: 6 total, 5 up, 6 in 2024-03-13T14:59:29.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:28 smithi196 podman[106404]: 2024-03-13 14:59:28.886629577 +0000 UTC m=+0.412872419 container init 25a6d6bd3aa4608ceca75d97616d48c2569d22b5a462d6c651bd5542f43473e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-13T14:59:29.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:28 smithi196 podman[106404]: 2024-03-13 14:59:28.896989018 +0000 UTC m=+0.423231840 container start 25a6d6bd3aa4608ceca75d97616d48c2569d22b5a462d6c651bd5542f43473e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-13T14:59:29.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:28 smithi196 podman[106404]: 2024-03-13 14:59:28.998702565 +0000 UTC m=+0.524945443 container attach 25a6d6bd3aa4608ceca75d97616d48c2569d22b5a462d6c651bd5542f43473e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-13T14:59:29.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:29 smithi196 ceph-mon[86193]: pgmap v220: 65 pgs: 11 peering, 7 stale+active+clean, 47 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-13T14:59:29.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:29 smithi196 ceph-mon[86193]: Health check failed: Reduced data availability: 1 pg inactive, 6 pgs peering (PG_AVAILABILITY) 2024-03-13T14:59:29.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:29 smithi196 podman[106404]: 2024-03-13 14:59:29.325835616 +0000 UTC m=+0.852078438 container died 25a6d6bd3aa4608ceca75d97616d48c2569d22b5a462d6c651bd5542f43473e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-13T14:59:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:29 smithi176 ceph-mon[112742]: pgmap v220: 65 pgs: 11 peering, 7 stale+active+clean, 47 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 639 B/s rd, 1 op/s 2024-03-13T14:59:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:29 smithi176 ceph-mon[112742]: Health check failed: Reduced data availability: 1 pg inactive, 6 pgs peering (PG_AVAILABILITY) 2024-03-13T14:59:30.427 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:30 smithi196 podman[106430]: 2024-03-13 14:59:30.09671785 +0000 UTC m=+0.753363395 container remove 25a6d6bd3aa4608ceca75d97616d48c2569d22b5a462d6c651bd5542f43473e5 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T14:59:30.427 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:30 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.5.service: Deactivated successfully. 2024-03-13T14:59:30.427 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:30 smithi196 systemd[1]: Stopped Ceph osd.5 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:59:30.427 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:30 smithi196 systemd[1]: ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.5.service: Consumed 1min 59.821s CPU time. 2024-03-13T14:59:30.887 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:30 smithi196 systemd[1]: Starting Ceph osd.5 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T14:59:31.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:30 smithi196 podman[106530]: 2024-03-13 14:59:30.806037234 +0000 UTC m=+0.033845473 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:59:31.177 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:30 smithi196 podman[106530]: 2024-03-13 14:59:30.914222102 +0000 UTC m=+0.142030380 container create cd20e2250f1865304d8df80174cad44667a05382c15846f95614bb8806965018 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T14:59:31.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:31.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:31.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:31 smithi196 ceph-mon[86193]: pgmap v221: 65 pgs: 9 active+undersized, 9 active+undersized+degraded, 11 peering, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 26/225 objects degraded (11.556%) 2024-03-13T14:59:31.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:31 smithi196 podman[106530]: 2024-03-13 14:59:31.268393527 +0000 UTC m=+0.496201846 container init cd20e2250f1865304d8df80174cad44667a05382c15846f95614bb8806965018 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-13T14:59:31.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:31 smithi196 podman[106530]: 2024-03-13 14:59:31.280795868 +0000 UTC m=+0.508604129 container start cd20e2250f1865304d8df80174cad44667a05382c15846f95614bb8806965018 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-13T14:59:31.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:31 smithi196 podman[106530]: 2024-03-13 14:59:31.392003912 +0000 UTC m=+0.619812193 container attach cd20e2250f1865304d8df80174cad44667a05382c15846f95614bb8806965018 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-13T14:59:31.798 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:31.798 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:31.798 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:31 smithi176 ceph-mon[112742]: pgmap v221: 65 pgs: 9 active+undersized, 9 active+undersized+degraded, 11 peering, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 26/225 objects degraded (11.556%) 2024-03-13T14:59:32.398 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate[106545]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate[106545]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 bash[106530]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 bash[106530]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 bash[106530]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 bash[106530]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 bash[106530]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 bash[106530]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 bash[106530]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate[106545]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate[106545]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate[106545]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-03-13T14:59:32.399 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate[106545]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-13T14:59:32.400 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate[106545]: --> ceph-volume raw activate successful for osd ID: 5 2024-03-13T14:59:32.400 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 podman[106530]: 2024-03-13 14:59:32.177672105 +0000 UTC m=+1.405480387 container died cd20e2250f1865304d8df80174cad44667a05382c15846f95614bb8806965018 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-13T14:59:32.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:32 smithi176 ceph-mon[112742]: Health check failed: Degraded data redundancy: 26/225 objects degraded (11.556%), 9 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:32.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:32 smithi196 ceph-mon[86193]: Health check failed: Degraded data redundancy: 26/225 objects degraded (11.556%), 9 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:33.140 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:32 smithi196 podman[106671]: 2024-03-13 14:59:32.862280667 +0000 UTC m=+0.666458247 container remove cd20e2250f1865304d8df80174cad44667a05382c15846f95614bb8806965018 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-13T14:59:33.394 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:33 smithi196 podman[106736]: 2024-03-13 14:59:33.117026344 +0000 UTC m=+0.030402912 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T14:59:33.395 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:33 smithi196 podman[106736]: 2024-03-13 14:59:33.255146687 +0000 UTC m=+0.168523293 container create 43366d2dc8dacc75d72d1bfe86c5007ee5c7563c5f45c31476ea48f38d1ae452 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-13T14:59:33.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:33 smithi196 ceph-mon[86193]: pgmap v222: 65 pgs: 12 active+undersized, 12 active+undersized+degraded, 5 peering, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:33.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:33 smithi196 podman[106736]: 2024-03-13 14:59:33.541620414 +0000 UTC m=+0.454997064 container init 43366d2dc8dacc75d72d1bfe86c5007ee5c7563c5f45c31476ea48f38d1ae452 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222) 2024-03-13T14:59:33.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:33 smithi196 podman[106736]: 2024-03-13 14:59:33.554051985 +0000 UTC m=+0.467428624 container start 43366d2dc8dacc75d72d1bfe86c5007ee5c7563c5f45c31476ea48f38d1ae452 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-13T14:59:33.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:33 smithi196 ceph-osd[106756]: -- 172.21.15.196:0/1199290449 <== mon.1 v2:172.21.15.196:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560dc05f5680 con 0x560dc05bf000 2024-03-13T14:59:33.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:33 smithi196 bash[106736]: 43366d2dc8dacc75d72d1bfe86c5007ee5c7563c5f45c31476ea48f38d1ae452 2024-03-13T14:59:33.677 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:33 smithi196 systemd[1]: Started Ceph osd.5 for f60e35da-e146-11ee-95c9-87774f69a715. 2024-03-13T14:59:33.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:33 smithi176 ceph-mon[112742]: pgmap v222: 65 pgs: 12 active+undersized, 12 active+undersized+degraded, 5 peering, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:34.426 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:34 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T14:59:34.137+0000 7f58415ca7c0 -1 Falling back to public interface 2024-03-13T14:59:35.064 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:34 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:35.064 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:34 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:35.064 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:34 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:35.064 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:34 smithi196 ceph-mon[86193]: pgmap v223: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:35.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:34 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:35.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:34 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:35.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:34 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:35.127 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:34 smithi176 ceph-mon[112742]: pgmap v223: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 127 B/s rd, 0 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:36.102 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:35 smithi196 ceph-mon[86193]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-03-13T14:59:36.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:35 smithi176 ceph-mon[112742]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 3 pgs peering) 2024-03-13T14:59:37.033 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:36 smithi196 ceph-mon[86193]: pgmap v224: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 618 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:37.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:36 smithi176 ceph-mon[112742]: pgmap v224: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 618 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:39.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:39.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:39.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:39 smithi176 ceph-mon[112742]: pgmap v225: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 562 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:39 smithi196 ceph-mon[86193]: pgmap v225: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 946 MiB used, 535 GiB / 536 GiB avail; 562 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:39.676 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:39 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T14:59:39.268+0000 7f58415ca7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-03-13T14:59:40.176 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:39 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T14:59:39.787+0000 7f58415ca7c0 -1 osd.5 101 log_to_monitors true 2024-03-13T14:59:40.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:40 smithi176 ceph-mon[112742]: from='osd.5 [v2:172.21.15.196:6816/3411147075,v1:172.21.15.196:6817/3411147075]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:59:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:40 smithi176 ceph-mon[112742]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:59:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:40 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:40.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:40 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:40.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:40 smithi196 ceph-mon[86193]: from='osd.5 [v2:172.21.15.196:6816/3411147075,v1:172.21.15.196:6817/3411147075]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:59:40.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:40 smithi196 ceph-mon[86193]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-13T14:59:40.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:40 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:40.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:40 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:41.611 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 14:59:41 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T14:59:41.382+0000 7f582ba00700 -1 osd.5 101 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-13T14:59:41.611 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:41 smithi196 ceph-mon[86193]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-13T14:59:41.611 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:41 smithi196 ceph-mon[86193]: from='osd.5 [v2:172.21.15.196:6816/3411147075,v1:172.21.15.196:6817/3411147075]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:59:41.612 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:41 smithi196 ceph-mon[86193]: osdmap e104: 6 total, 5 up, 6 in 2024-03-13T14:59:41.612 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:41 smithi196 ceph-mon[86193]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:59:41.612 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:41 smithi196 ceph-mon[86193]: pgmap v227: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:41.612 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:41 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 34/225 objects degraded (15.111%), 12 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:41 smithi176 ceph-mon[112742]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-13T14:59:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:41 smithi176 ceph-mon[112742]: from='osd.5 [v2:172.21.15.196:6816/3411147075,v1:172.21.15.196:6817/3411147075]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:59:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:41 smithi176 ceph-mon[112742]: osdmap e104: 6 total, 5 up, 6 in 2024-03-13T14:59:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:41 smithi176 ceph-mon[112742]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-03-13T14:59:41.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:41 smithi176 ceph-mon[112742]: pgmap v227: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 1023 B/s rd, 1 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:41.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:41 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 34/225 objects degraded (15.111%), 12 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:42.390 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:42 smithi176 ceph-mon[112742]: from='osd.5 ' entity='osd.5' 2024-03-13T14:59:42.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:42 smithi196 ceph-mon[86193]: from='osd.5 ' entity='osd.5' 2024-03-13T14:59:43.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:43 smithi196 ceph-mon[86193]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:59:43.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:43 smithi196 ceph-mon[86193]: osd.5 [v2:172.21.15.196:6816/3411147075,v1:172.21.15.196:6817/3411147075] boot 2024-03-13T14:59:43.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:43 smithi196 ceph-mon[86193]: osdmap e105: 6 total, 6 up, 6 in 2024-03-13T14:59:43.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:43 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:59:43.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:43 smithi196 ceph-mon[86193]: pgmap v229: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:43.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:43 smithi196 ceph-mon[86193]: osdmap e106: 6 total, 6 up, 6 in 2024-03-13T14:59:43.758 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:43 smithi176 ceph-mon[112742]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-03-13T14:59:43.758 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:43 smithi176 ceph-mon[112742]: osd.5 [v2:172.21.15.196:6816/3411147075,v1:172.21.15.196:6817/3411147075] boot 2024-03-13T14:59:43.758 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:43 smithi176 ceph-mon[112742]: osdmap e105: 6 total, 6 up, 6 in 2024-03-13T14:59:43.758 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:43 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-13T14:59:43.758 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:43 smithi176 ceph-mon[112742]: pgmap v229: 65 pgs: 17 active+undersized, 12 active+undersized+degraded, 36 active+clean; 204 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s; 34/225 objects degraded (15.111%) 2024-03-13T14:59:43.758 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:43 smithi176 ceph-mon[112742]: osdmap e106: 6 total, 6 up, 6 in 2024-03-13T14:59:45.784 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:45 smithi196 ceph-mon[86193]: pgmap v231: 65 pgs: 2 peering, 12 active+undersized, 11 active+undersized+degraded, 40 active+clean; 204 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 32/225 objects degraded (14.222%) 2024-03-13T14:59:45.784 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:45.785 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:45 smithi176 ceph-mon[112742]: pgmap v231: 65 pgs: 2 peering, 12 active+undersized, 11 active+undersized+degraded, 40 active+clean; 204 MiB data, 858 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s; 32/225 objects degraded (14.222%) 2024-03-13T14:59:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T14:59:46.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 -- 172.21.15.176:0/1605055151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628100970 msgr2=0x7f6628100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:46.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 --2- 172.21.15.176:0/1605055151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628100970 0x7f6628100df0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6610009910 tx=0x7f6610009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:46.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 -- 172.21.15.176:0/1605055151 shutdown_connections 2024-03-13T14:59:46.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 --2- 172.21.15.176:0/1605055151 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628100970 0x7f6628100df0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 --2- 172.21.15.176:0/1605055151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6628106970 0x7f6628106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 -- 172.21.15.176:0/1605055151 >> 172.21.15.176:0/1605055151 conn(0x7f66280fc440 msgr2=0x7f66280fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:46.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 -- 172.21.15.176:0/1605055151 shutdown_connections 2024-03-13T14:59:46.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.447+0000 7f6627e00700 1 -- 172.21.15.176:0/1605055151 wait complete. 2024-03-13T14:59:46.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.448+0000 7f6627e00700 1 Processor -- start 2024-03-13T14:59:46.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.448+0000 7f6627e00700 1 -- start start 2024-03-13T14:59:46.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f6627e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6628100970 0x7f6628073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:46.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f6627e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628106970 0x7f66280735d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:46.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f6627e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628073b10 con 0x7f6628106970 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f6627e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6628073c50 con 0x7f6628100970 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f661fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628106970 0x7f66280735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f661fe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628106970 0x7f66280735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:49998/0 (socket says 172.21.15.176:49998) 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f661fe00700 1 -- 172.21.15.176:0/3934195435 learned_addr learned my addr 172.21.15.176:0/3934195435 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.449+0000 7f6626a00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6628100970 0x7f6628073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f661fe00700 1 -- 172.21.15.176:0/3934195435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6628100970 msgr2=0x7f6628073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f661fe00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6628100970 0x7f6628073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f661fe00700 1 -- 172.21.15.176:0/3934195435 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100095f0 con 0x7f6628106970 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f6626a00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6628100970 0x7f6628073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f661fe00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628106970 0x7f66280735d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6610006010 tx=0x7f661000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f661f400700 1 -- 172.21.15.176:0/3934195435 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661001d070 con 0x7f6628106970 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f661f400700 1 -- 172.21.15.176:0/3934195435 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6610004fb0 con 0x7f6628106970 2024-03-13T14:59:46.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f661f400700 1 -- 172.21.15.176:0/3934195435 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f661000f690 con 0x7f6628106970 2024-03-13T14:59:46.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.450+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6628073ed0 con 0x7f6628106970 2024-03-13T14:59:46.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.451+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281a2b70 con 0x7f6628106970 2024-03-13T14:59:46.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.452+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6628108f90 con 0x7f6628106970 2024-03-13T14:59:46.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.453+0000 7f661f400700 1 -- 172.21.15.176:0/3934195435 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f66100228a0 con 0x7f6628106970 2024-03-13T14:59:46.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.454+0000 7f661f400700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f660c06dfa0 0x7f660c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:46.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.454+0000 7f661f400700 1 -- 172.21.15.176:0/3934195435 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f661008f620 con 0x7f6628106970 2024-03-13T14:59:46.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.454+0000 7f6626a00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f660c06dfa0 0x7f660c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:46.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.455+0000 7f6626a00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f660c06dfa0 0x7f660c070460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6618007900 tx=0x7f6618008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:46.444 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.459+0000 7f661f400700 1 -- 172.21.15.176:0/3934195435 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f6610058a00 con 0x7f6628106970 2024-03-13T14:59:46.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.704+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f662802cd50 con 0x7f660c06dfa0 2024-03-13T14:59:46.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: Health check update: Degraded data redundancy: 32/225 objects degraded (14.222%), 11 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:46.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:46.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:59:46.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-13T14:59:46.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-13T14:59:46.692 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-13T14:59:46.692 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-13T14:59:46.692 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-13T14:59:46.692 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-13T14:59:46.692 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-13T14:59:46.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.706+0000 7f661f400700 1 -- 172.21.15.176:0/3934195435 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f662802cd50 con 0x7f660c06dfa0 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f660c06dfa0 msgr2=0x7f660c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f660c06dfa0 0x7f660c070460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6618007900 tx=0x7f6618008040 comp rx=0 tx=0).stop 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628106970 msgr2=0x7f66280735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628106970 0x7f66280735d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6610006010 tx=0x7f661000be20 comp rx=0 tx=0).stop 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 shutdown_connections 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f660c06dfa0 0x7f660c070460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6628100970 0x7f6628073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 --2- 172.21.15.176:0/3934195435 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6628106970 0x7f66280735d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 >> 172.21.15.176:0/3934195435 conn(0x7f66280fc440 msgr2=0x7f66280fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:46.696 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 shutdown_connections 2024-03-13T14:59:46.696 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.709+0000 7f6627e00700 1 -- 172.21.15.176:0/3934195435 wait complete. 2024-03-13T14:59:46.711 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T14:59:46.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 -- 172.21.15.176:0/248915813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073130 msgr2=0x7f33b0073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:46.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 --2- 172.21.15.176:0/248915813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073130 0x7f33b0073510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009910 tx=0x7f33a4009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:46.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 -- 172.21.15.176:0/248915813 shutdown_connections 2024-03-13T14:59:46.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 --2- 172.21.15.176:0/248915813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b0073a50 0x7f33b0111440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 --2- 172.21.15.176:0/248915813 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073130 0x7f33b0073510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 -- 172.21.15.176:0/248915813 >> 172.21.15.176:0/248915813 conn(0x7f33b00fc440 msgr2=0x7f33b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:46.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 -- 172.21.15.176:0/248915813 shutdown_connections 2024-03-13T14:59:46.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.870+0000 7f33b6400700 1 -- 172.21.15.176:0/248915813 wait complete. 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.871+0000 7f33b6400700 1 Processor -- start 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.871+0000 7f33b6400700 1 -- start start 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.872+0000 7f33b6400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b0073130 0x7f33b01a28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.872+0000 7f33b6400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073a50 0x7f33b01a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.872+0000 7f33b6400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b01a3470 con 0x7f33b0073a50 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.872+0000 7f33b6400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b019c920 con 0x7f33b0073130 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.872+0000 7f33afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073a50 0x7f33b01a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.872+0000 7f33afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073a50 0x7f33b01a2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50018/0 (socket says 172.21.15.176:50018) 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.872+0000 7f33afe00700 1 -- 172.21.15.176:0/2798766119 learned_addr learned my addr 172.21.15.176:0/2798766119 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:46.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.873+0000 7f33b5000700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b0073130 0x7f33b01a28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:46.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.873+0000 7f33afe00700 1 -- 172.21.15.176:0/2798766119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b0073130 msgr2=0x7f33b01a28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:46.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.873+0000 7f33afe00700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b0073130 0x7f33b01a28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:46.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.873+0000 7f33afe00700 1 -- 172.21.15.176:0/2798766119 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40095f0 con 0x7f33b0073a50 2024-03-13T14:59:46.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.873+0000 7f33afe00700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073a50 0x7f33b01a2de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c8a0 tx=0x7f33a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:46.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.873+0000 7f33ad600700 1 -- 172.21.15.176:0/2798766119 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33a000cea0 con 0x7f33b0073a50 2024-03-13T14:59:46.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.874+0000 7f33ad600700 1 -- 172.21.15.176:0/2798766119 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33a0004830 con 0x7f33b0073a50 2024-03-13T14:59:46.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.874+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b019cc00 con 0x7f33b0073a50 2024-03-13T14:59:46.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.874+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b019d150 con 0x7f33b0073a50 2024-03-13T14:59:46.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.874+0000 7f33ad600700 1 -- 172.21.15.176:0/2798766119 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33a00055b0 con 0x7f33b0073a50 2024-03-13T14:59:46.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.875+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b010ebc0 con 0x7f33b0073a50 2024-03-13T14:59:46.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.876+0000 7f33ad600700 1 -- 172.21.15.176:0/2798766119 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f33a00049a0 con 0x7f33b0073a50 2024-03-13T14:59:46.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.877+0000 7f33ad600700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f339806dfa0 0x7f3398070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:46.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.877+0000 7f33ad600700 1 -- 172.21.15.176:0/2798766119 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f33a008db20 con 0x7f33b0073a50 2024-03-13T14:59:46.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.877+0000 7f33b5000700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f339806dfa0 0x7f3398070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:46.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.878+0000 7f33b5000700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f339806dfa0 0x7f3398070460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006010 tx=0x7f33a401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:46.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:46.882+0000 7f33ad600700 1 -- 172.21.15.176:0/2798766119 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f33a005a9b0 con 0x7f33b0073a50 2024-03-13T14:59:46.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: Health check update: Degraded data redundancy: 32/225 objects degraded (14.222%), 11 pgs degraded (PG_DEGRADED) 2024-03-13T14:59:46.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:46.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-03-13T14:59:46.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-03-13T14:59:46.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-03-13T14:59:46.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-03-13T14:59:46.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-03-13T14:59:46.929 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-03-13T14:59:47.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.123+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33b019d490 con 0x7f339806dfa0 2024-03-13T14:59:47.112 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.126+0000 7f33ad600700 1 -- 172.21.15.176:0/2798766119 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f33b019d490 con 0x7f339806dfa0 2024-03-13T14:59:47.115 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f339806dfa0 msgr2=0x7f3398070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.115 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f339806dfa0 0x7f3398070460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f33a4006010 tx=0x7f33a401a040 comp rx=0 tx=0).stop 2024-03-13T14:59:47.115 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073a50 msgr2=0x7f33b01a2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.115 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073a50 0x7f33b01a2de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c8a0 tx=0x7f33a000cc60 comp rx=0 tx=0).stop 2024-03-13T14:59:47.115 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 shutdown_connections 2024-03-13T14:59:47.115 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f339806dfa0 0x7f3398070460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.116 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33b0073130 0x7f33b01a28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.116 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 --2- 172.21.15.176:0/2798766119 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f33b0073a50 0x7f33b01a2de0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.116 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.129+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 >> 172.21.15.176:0/2798766119 conn(0x7f33b00fc440 msgr2=0x7f33b0102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:47.116 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.130+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 shutdown_connections 2024-03-13T14:59:47.116 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.130+0000 7f33b6400700 1 -- 172.21.15.176:0/2798766119 wait complete. 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 -- 172.21.15.176:0/923171101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 msgr2=0x7fbee010ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 --2- 172.21.15.176:0/923171101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee010ab20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009910 tx=0x7fbecc009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 -- 172.21.15.176:0/923171101 shutdown_connections 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 --2- 172.21.15.176:0/923171101 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee010ab20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 --2- 172.21.15.176:0/923171101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee0101d10 0x7fbee01020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 -- 172.21.15.176:0/923171101 >> 172.21.15.176:0/923171101 conn(0x7fbee00fb540 msgr2=0x7fbee00fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 -- 172.21.15.176:0/923171101 shutdown_connections 2024-03-13T14:59:47.272 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.286+0000 7fbee4a00700 1 -- 172.21.15.176:0/923171101 wait complete. 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.287+0000 7fbee4a00700 1 Processor -- start 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.287+0000 7fbee4a00700 1 -- start start 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbee4a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee0101d10 0x7fbee00ff970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbee4a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee00ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbee4a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee01003f0 con 0x7fbee0102630 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbee4a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee0100530 con 0x7fbee0101d10 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbedd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee00ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbedd400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee00ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50038/0 (socket says 172.21.15.176:50038) 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbedd400700 1 -- 172.21.15.176:0/3094911280 learned_addr learned my addr 172.21.15.176:0/3094911280 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:47.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbedde00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee0101d10 0x7fbee00ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:47.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.288+0000 7fbedd400700 1 -- 172.21.15.176:0/3094911280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee0101d10 msgr2=0x7fbee00ff970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbedd400700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee0101d10 0x7fbee00ff970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbedd400700 1 -- 172.21.15.176:0/3094911280 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0095f0 con 0x7fbee0102630 2024-03-13T14:59:47.275 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbedde00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee0101d10 0x7fbee00ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:59:47.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbedd400700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee00ffeb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbecc005f50 tx=0x7fbecc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:47.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbed6a00700 1 -- 172.21.15.176:0/3094911280 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbecc01d070 con 0x7fbee0102630 2024-03-13T14:59:47.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbed6a00700 1 -- 172.21.15.176:0/3094911280 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbecc00f460 con 0x7fbee0102630 2024-03-13T14:59:47.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbed6a00700 1 -- 172.21.15.176:0/3094911280 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbecc017630 con 0x7fbee0102630 2024-03-13T14:59:47.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee01007b0 con 0x7fbee0102630 2024-03-13T14:59:47.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.289+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee01a2c90 con 0x7fbee0102630 2024-03-13T14:59:47.276 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.291+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee0108220 con 0x7fbee0102630 2024-03-13T14:59:47.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.291+0000 7fbed6a00700 1 -- 172.21.15.176:0/3094911280 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fbecc00f5d0 con 0x7fbee0102630 2024-03-13T14:59:47.277 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.292+0000 7fbed6a00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbec406dfa0 0x7fbec4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:47.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.292+0000 7fbed6a00700 1 -- 172.21.15.176:0/3094911280 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fbecc08f290 con 0x7fbee0102630 2024-03-13T14:59:47.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.293+0000 7fbedde00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbec406dfa0 0x7fbec4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:47.278 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.293+0000 7fbedde00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbec406dfa0 0x7fbec4070460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbed0005fd0 tx=0x7fbed0005dc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:47.282 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.297+0000 7fbed6a00700 1 -- 172.21.15.176:0/3094911280 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fbecc058670 con 0x7fbee0102630 2024-03-13T14:59:47.546 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.560+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbee00611d0 con 0x7fbec406dfa0 2024-03-13T14:59:47.578 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.592+0000 7fbed6a00700 1 -- 172.21.15.176:0/3094911280 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fbee00611d0 con 0x7fbec406dfa0 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (7m) 2m ago 21m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (21m) 2m ago 21m 8812k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (20m) 9s ago 20m 8900k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (4m) 2m ago 21m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (4m) 9s ago 19m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (7m) 2m ago 20m 73.6M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (13m) 2m ago 13m 87.3M - 18.2.1 5be31c24972a e932595696da 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (14m) 2m ago 14m 17.9M - 18.2.1 5be31c24972a 46196c8def03 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (14m) 9s ago 14m 140M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T14:59:47.579 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (13m) 9s ago 13m 18.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (10m) 2m ago 23m 504M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (9m) 9s ago 19m 451M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (5m) 2m ago 23m 47.3M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (5m) 9s ago 19m 41.0M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (8m) 2m ago 21m 7520k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (8m) 9s ago 19m 7671k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (3m) 2m ago 18m 122M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (2m) 2m ago 18m 97.2M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (2m) 2m ago 17m 11.1M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (101s) 9s ago 16m 140M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (58s) 9s ago 16m 100M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (14s) 9s ago 15m 11.2M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 43366d2dc8da 2024-03-13T14:59:47.580 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (8m) 2m ago 20m 69.1M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbec406dfa0 msgr2=0x7fbec4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbec406dfa0 0x7fbec4070460 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbed0005fd0 tx=0x7fbed0005dc0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 msgr2=0x7fbee00ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee00ffeb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbecc005f50 tx=0x7fbecc00be20 comp rx=0 tx=0).stop 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 shutdown_connections 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbec406dfa0 0x7fbec4070460 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbee0101d10 0x7fbee00ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 --2- 172.21.15.176:0/3094911280 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbee0102630 0x7fbee00ffeb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.582 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 >> 172.21.15.176:0/3094911280 conn(0x7fbee00fb540 msgr2=0x7fbee0105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:47.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 shutdown_connections 2024-03-13T14:59:47.583 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.596+0000 7fbee4a00700 1 -- 172.21.15.176:0/3094911280 wait complete. 2024-03-13T14:59:47.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 -- 172.21.15.176:0/3011075675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 msgr2=0x7f9540105e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 --2- 172.21.15.176:0/3011075675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540105e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9530009910 tx=0x7f9530009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:47.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 -- 172.21.15.176:0/3011075675 shutdown_connections 2024-03-13T14:59:47.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 --2- 172.21.15.176:0/3011075675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95400ffbd0 0x7f9540100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 --2- 172.21.15.176:0/3011075675 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540105e20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 -- 172.21.15.176:0/3011075675 >> 172.21.15.176:0/3011075675 conn(0x7f95400755a0 msgr2=0x7f95400759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:47.735 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all osd 2024-03-13T14:59:47.735 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: pgmap v232: 65 pgs: 2 peering, 1 active+undersized, 2 active+undersized+degraded, 60 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 506 B/s rd, 1 op/s; 4/225 objects degraded (1.778%); 2.6 MiB/s, 0 objects/s recovering 2024-03-13T14:59:47.735 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: Upgrade: Setting require_osd_release to 19 squid 2024-03-13T14:59:47.735 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-13T14:59:47.735 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-13T14:59:47.735 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: osdmap e107: 6 total, 6 up, 6 in 2024-03-13T14:59:47.736 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:47.736 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-13T14:59:47.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 -- 172.21.15.176:0/3011075675 shutdown_connections 2024-03-13T14:59:47.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.748+0000 7f9544e00700 1 -- 172.21.15.176:0/3011075675 wait complete. 2024-03-13T14:59:47.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.749+0000 7f9544e00700 1 Processor -- start 2024-03-13T14:59:47.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.749+0000 7f9544e00700 1 -- start start 2024-03-13T14:59:47.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9544e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95400ffbd0 0x7f9540101610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:47.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9544e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540101b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:47.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9544e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9540102090 con 0x7f9540105a40 2024-03-13T14:59:47.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9544e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95401021d0 con 0x7f95400ffbd0 2024-03-13T14:59:47.740 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9537e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9537e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50066/0 (socket says 172.21.15.176:50066) 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9537e00700 1 -- 172.21.15.176:0/2256557598 learned_addr learned my addr 172.21.15.176:0/2256557598 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f9537e00700 1 -- 172.21.15.176:0/2256557598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95400ffbd0 msgr2=0x7f9540101610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.750+0000 7f953de00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95400ffbd0 0x7f9540101610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9537e00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95400ffbd0 0x7f9540101610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9537e00700 1 -- 172.21.15.176:0/2256557598 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95300095f0 con 0x7f9540105a40 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9537e00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540101b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f952800c8a0 tx=0x7f952800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9536a00700 1 -- 172.21.15.176:0/2256557598 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f952800cea0 con 0x7f9540105a40 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9536a00700 1 -- 172.21.15.176:0/2256557598 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9528004830 con 0x7f9540105a40 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95401a2940 con 0x7f9540105a40 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9536a00700 1 -- 172.21.15.176:0/2256557598 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95280055b0 con 0x7f9540105a40 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.751+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95401a2d50 con 0x7f9540105a40 2024-03-13T14:59:47.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.753+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9540109050 con 0x7f9540105a40 2024-03-13T14:59:47.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.755+0000 7f9536a00700 1 -- 172.21.15.176:0/2256557598 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f95280049a0 con 0x7f9540105a40 2024-03-13T14:59:47.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.755+0000 7f9536a00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f952406dfa0 0x7f9524070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:47.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.756+0000 7f9536a00700 1 -- 172.21.15.176:0/2256557598 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f952808da70 con 0x7f9540105a40 2024-03-13T14:59:47.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.756+0000 7f953de00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f952406dfa0 0x7f9524070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:47.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.757+0000 7f953de00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f952406dfa0 0x7f9524070460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9530000c00 tx=0x7f9530005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:47.746 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:47.760+0000 7f9536a00700 1 -- 172.21.15.176:0/2256557598 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f9528056e50 con 0x7f9540105a40 2024-03-13T14:59:47.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all osd 2024-03-13T14:59:47.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: pgmap v232: 65 pgs: 2 peering, 1 active+undersized, 2 active+undersized+degraded, 60 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 506 B/s rd, 1 op/s; 4/225 objects degraded (1.778%); 2.6 MiB/s, 0 objects/s recovering 2024-03-13T14:59:47.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: Upgrade: Setting require_osd_release to 19 squid 2024-03-13T14:59:47.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-03-13T14:59:47.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-03-13T14:59:47.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: osdmap e107: 6 total, 6 up, 6 in 2024-03-13T14:59:47.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:47.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-03-13T14:59:48.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.077+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f954004ea90 con 0x7f9540105a40 2024-03-13T14:59:48.064 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.079+0000 7f9536a00700 1 -- 172.21.15.176:0/2256557598 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+701 (secure 0 0 0) 0x7f9528004c50 con 0x7f9540105a40 2024-03-13T14:59:48.064 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T14:59:48.065 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 4, 2024-03-13T14:59:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 10 2024-03-13T14:59:48.066 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T14:59:48.066 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:59:48.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f952406dfa0 msgr2=0x7f9524070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f952406dfa0 0x7f9524070460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9530000c00 tx=0x7f9530005fd0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 msgr2=0x7f9540101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540101b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f952800c8a0 tx=0x7f952800cc60 comp rx=0 tx=0).stop 2024-03-13T14:59:48.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 shutdown_connections 2024-03-13T14:59:48.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f952406dfa0 0x7f9524070460 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95400ffbd0 0x7f9540101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 --2- 172.21.15.176:0/2256557598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9540105a40 0x7f9540101b50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 >> 172.21.15.176:0/2256557598 conn(0x7f95400755a0 msgr2=0x7f95400685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:48.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 shutdown_connections 2024-03-13T14:59:48.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.082+0000 7f9544e00700 1 -- 172.21.15.176:0/2256557598 wait complete. 2024-03-13T14:59:48.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.236+0000 7fb272600700 1 -- 172.21.15.176:0/2889180099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 msgr2=0x7fb26c1033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.236+0000 7fb272600700 1 --2- 172.21.15.176:0/2889180099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c1033a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009910 tx=0x7fb25c009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:48.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.237+0000 7fb272600700 1 -- 172.21.15.176:0/2889180099 shutdown_connections 2024-03-13T14:59:48.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.237+0000 7fb272600700 1 --2- 172.21.15.176:0/2889180099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb26c103970 0x7fb26c1079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.237+0000 7fb272600700 1 --2- 172.21.15.176:0/2889180099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c1033a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.237+0000 7fb272600700 1 -- 172.21.15.176:0/2889180099 >> 172.21.15.176:0/2889180099 conn(0x7fb26c0fe870 msgr2=0x7fb26c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:48.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.237+0000 7fb272600700 1 -- 172.21.15.176:0/2889180099 shutdown_connections 2024-03-13T14:59:48.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.237+0000 7fb272600700 1 -- 172.21.15.176:0/2889180099 wait complete. 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.238+0000 7fb272600700 1 Processor -- start 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.238+0000 7fb272600700 1 -- start start 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.238+0000 7fb272600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.238+0000 7fb272600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb26c103970 0x7fb26c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.238+0000 7fb272600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c19f230 con 0x7fb26c102fc0 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.238+0000 7fb272600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c198770 con 0x7fb26c103970 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50086/0 (socket says 172.21.15.176:50086) 2024-03-13T14:59:48.224 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26b400700 1 -- 172.21.15.176:0/108965614 learned_addr learned my addr 172.21.15.176:0/108965614 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:48.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26b400700 1 -- 172.21.15.176:0/108965614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb26c103970 msgr2=0x7fb26c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26aa00700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb26c103970 0x7fb26c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:48.225 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26b400700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb26c103970 0x7fb26c19ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26b400700 1 -- 172.21.15.176:0/108965614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c0095f0 con 0x7fb26c102fc0 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.239+0000 7fb26b400700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c19e6f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006010 tx=0x7fb25c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.240+0000 7fb263e00700 1 -- 172.21.15.176:0/108965614 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb25c01d070 con 0x7fb26c102fc0 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.240+0000 7fb263e00700 1 -- 172.21.15.176:0/108965614 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb25c004fb0 con 0x7fb26c102fc0 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.240+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c1989f0 con 0x7fb26c102fc0 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.240+0000 7fb263e00700 1 -- 172.21.15.176:0/108965614 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb25c022ea0 con 0x7fb26c102fc0 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.240+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c198ee0 con 0x7fb26c102fc0 2024-03-13T14:59:48.226 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.241+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c10b360 con 0x7fb26c102fc0 2024-03-13T14:59:48.227 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.242+0000 7fb263e00700 1 -- 172.21.15.176:0/108965614 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb25c022470 con 0x7fb26c102fc0 2024-03-13T14:59:48.228 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.243+0000 7fb263e00700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb25006dfa0 0x7fb250070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:48.228 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.243+0000 7fb263e00700 1 -- 172.21.15.176:0/108965614 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fb25c08f380 con 0x7fb26c102fc0 2024-03-13T14:59:48.229 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.243+0000 7fb26aa00700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb25006dfa0 0x7fb250070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:48.229 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.244+0000 7fb26aa00700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb25006dfa0 0x7fb250070460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb26c199be0 tx=0x7fb258005d50 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:48.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.248+0000 7fb263e00700 1 -- 172.21.15.176:0/108965614 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fb25c05c360 con 0x7fb26c102fc0 2024-03-13T14:59:48.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.525+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb26c04ea90 con 0x7fb26c102fc0 2024-03-13T14:59:48.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.526+0000 7fb263e00700 1 -- 172.21.15.176:0/108965614 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1671 (secure 0 0 0) 0x7fb25c022720 con 0x7fb26c102fc0 2024-03-13T14:59:48.511 INFO:teuthology.orchestra.run.smithi176.stdout:e19 2024-03-13T14:59:48.511 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T14:59:48.511 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:59:48.511 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 19 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:59:47.444191+0000 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T14:59:48.512 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 0 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:59:48.513 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:48.514 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T14:59:48.514 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T14:59:48.514 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/668575330,v1:172.21.15.176:6827/668575330] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:59:48.514 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T14:59:48.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.529+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb25006dfa0 msgr2=0x7fb250070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.529+0000 7fb272600700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb25006dfa0 0x7fb250070460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb26c199be0 tx=0x7fb258005d50 comp rx=0 tx=0).stop 2024-03-13T14:59:48.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.529+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 msgr2=0x7fb26c19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.529+0000 7fb272600700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c19e6f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb25c006010 tx=0x7fb25c00be20 comp rx=0 tx=0).stop 2024-03-13T14:59:48.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.530+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 shutdown_connections 2024-03-13T14:59:48.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.530+0000 7fb272600700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb25006dfa0 0x7fb250070460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.530+0000 7fb272600700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb26c102fc0 0x7fb26c19e6f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.530+0000 7fb272600700 1 --2- 172.21.15.176:0/108965614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb26c103970 0x7fb26c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.530+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 >> 172.21.15.176:0/108965614 conn(0x7fb26c0fe870 msgr2=0x7fb26c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:48.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.530+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 shutdown_connections 2024-03-13T14:59:48.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.530+0000 7fb272600700 1 -- 172.21.15.176:0/108965614 wait complete. 2024-03-13T14:59:48.516 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 19 2024-03-13T14:59:48.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.682+0000 7f962ea00700 1 -- 172.21.15.176:0/4114089597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628068670 msgr2=0x7f9628068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.682+0000 7f962ea00700 1 --2- 172.21.15.176:0/4114089597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628068670 0x7f9628068a50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9618009910 tx=0x7f9618009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:48.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.683+0000 7f962ea00700 1 -- 172.21.15.176:0/4114089597 shutdown_connections 2024-03-13T14:59:48.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.683+0000 7f962ea00700 1 --2- 172.21.15.176:0/4114089597 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9628069020 0x7f9628105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.683+0000 7f962ea00700 1 --2- 172.21.15.176:0/4114089597 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628068670 0x7f9628068a50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.683+0000 7f962ea00700 1 -- 172.21.15.176:0/4114089597 >> 172.21.15.176:0/4114089597 conn(0x7f96280755a0 msgr2=0x7f96280759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:48.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.683+0000 7f962ea00700 1 -- 172.21.15.176:0/4114089597 shutdown_connections 2024-03-13T14:59:48.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.683+0000 7f962ea00700 1 -- 172.21.15.176:0/4114089597 wait complete. 2024-03-13T14:59:48.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.684+0000 7f962ea00700 1 Processor -- start 2024-03-13T14:59:48.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.684+0000 7f962ea00700 1 -- start start 2024-03-13T14:59:48.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.684+0000 7f962ea00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9628068670 0x7f96280ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.684+0000 7f962ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628069020 0x7f96281001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.684+0000 7f962ea00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9628100720 con 0x7f9628069020 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.684+0000 7f962ea00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9628100860 con 0x7f9628068670 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628069020 0x7f96281001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628069020 0x7f96281001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50120/0 (socket says 172.21.15.176:50120) 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627400700 1 -- 172.21.15.176:0/314261851 learned_addr learned my addr 172.21.15.176:0/314261851 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627e00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9628068670 0x7f96280ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627400700 1 -- 172.21.15.176:0/314261851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9628068670 msgr2=0x7f96280ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627400700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9628068670 0x7f96280ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627400700 1 -- 172.21.15.176:0/314261851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96180095f0 con 0x7f9628069020 2024-03-13T14:59:48.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.685+0000 7f9627400700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628069020 0x7f96281001e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f961400e8f0 tx=0x7f961400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:48.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.686+0000 7f9624c00700 1 -- 172.21.15.176:0/314261851 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9614009790 con 0x7f9628069020 2024-03-13T14:59:48.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.686+0000 7f9624c00700 1 -- 172.21.15.176:0/314261851 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f961400c810 con 0x7f9628069020 2024-03-13T14:59:48.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.686+0000 7f9624c00700 1 -- 172.21.15.176:0/314261851 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9614005580 con 0x7f9628069020 2024-03-13T14:59:48.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.686+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96281a2910 con 0x7f9628069020 2024-03-13T14:59:48.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.686+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96281a2d20 con 0x7f9628069020 2024-03-13T14:59:48.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.687+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9628109130 con 0x7f9628069020 2024-03-13T14:59:48.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.688+0000 7f9624c00700 1 -- 172.21.15.176:0/314261851 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f961400c980 con 0x7f9628069020 2024-03-13T14:59:48.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.689+0000 7f9624c00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f960c06dfa0 0x7f960c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:48.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.689+0000 7f9624c00700 1 -- 172.21.15.176:0/314261851 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f9614014070 con 0x7f9628069020 2024-03-13T14:59:48.674 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.689+0000 7f9627e00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f960c06dfa0 0x7f960c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:48.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.690+0000 7f9627e00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f960c06dfa0 0x7f960c070460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9618009fd0 tx=0x7f9618005a90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:48.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.693+0000 7f9624c00700 1 -- 172.21.15.176:0/314261851 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f9614057430 con 0x7f9628069020 2024-03-13T14:59:48.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:48.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:48.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-13T14:59:48.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: osdmap e108: 6 total, 6 up, 6 in 2024-03-13T14:59:48.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2256557598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-13T14:59:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 2 up:standby 2024-03-13T14:59:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:59:48.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:48 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:48.918 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.932+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f962802cd30 con 0x7f960c06dfa0 2024-03-13T14:59:48.920 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.935+0000 7f9624c00700 1 -- 172.21.15.176:0/314261851 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f962802cd30 con 0x7f960c06dfa0 2024-03-13T14:59:48.920 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T14:59:48.920 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T14:59:48.920 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T14:59:48.920 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "osd", 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "12/23 daemons upgraded", 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mds daemons", 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T14:59:48.921 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T14:59:48.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f960c06dfa0 msgr2=0x7f960c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f960c06dfa0 0x7f960c070460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9618009fd0 tx=0x7f9618005a90 comp rx=0 tx=0).stop 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628069020 msgr2=0x7f96281001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628069020 0x7f96281001e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f961400e8f0 tx=0x7f961400ec00 comp rx=0 tx=0).stop 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 shutdown_connections 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f960c06dfa0 0x7f960c070460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9628068670 0x7f96280ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 --2- 172.21.15.176:0/314261851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9628069020 0x7f96281001e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 >> 172.21.15.176:0/314261851 conn(0x7f96280755a0 msgr2=0x7f96280fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 shutdown_connections 2024-03-13T14:59:48.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:48.938+0000 7f962ea00700 1 -- 172.21.15.176:0/314261851 wait complete. 2024-03-13T14:59:48.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: osdmap e108: 6 total, 6 up, 6 in 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2256557598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 2 up:standby 2024-03-13T14:59:48.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:48.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:48.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T14:59:48.928 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:48 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 -- 172.21.15.176:0/4176956536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 msgr2=0x7f5c1c10ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 --2- 172.21.15.176:0/4176956536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c10ce40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009910 tx=0x7f5c04009c20 comp rx=0 tx=0).stop 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 -- 172.21.15.176:0/4176956536 shutdown_connections 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 --2- 172.21.15.176:0/4176956536 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c10ce40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 --2- 172.21.15.176:0/4176956536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c1c0731c0 0x7f5c1c0735a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 -- 172.21.15.176:0/4176956536 >> 172.21.15.176:0/4176956536 conn(0x7f5c1c0fc630 msgr2=0x7f5c1c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 -- 172.21.15.176:0/4176956536 shutdown_connections 2024-03-13T14:59:49.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.085+0000 7f5c21800700 1 -- 172.21.15.176:0/4176956536 wait complete. 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.086+0000 7f5c21800700 1 Processor -- start 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.086+0000 7f5c21800700 1 -- start start 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c21800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c1c0731c0 0x7f5c1c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c21800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c21800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c19f3e0 con 0x7f5c1c073ae0 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c21800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c198890 con 0x7f5c1c0731c0 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c13e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c13e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:50146/0 (socket says 172.21.15.176:50146) 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c13e00700 1 -- 172.21.15.176:0/1862451078 learned_addr learned my addr 172.21.15.176:0/1862451078 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c13e00700 1 -- 172.21.15.176:0/1862451078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c1c0731c0 msgr2=0x7f5c1c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:49.073 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c1aa00700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c1c0731c0 0x7f5c1c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:49.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c13e00700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c1c0731c0 0x7f5c1c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:49.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c13e00700 1 -- 172.21.15.176:0/1862451078 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c040095f0 con 0x7f5c1c073ae0 2024-03-13T14:59:49.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.087+0000 7f5c1aa00700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c1c0731c0 0x7f5c1c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T14:59:49.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.088+0000 7f5c13e00700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c19ed50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006010 tx=0x7f5c0400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:49.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.088+0000 7f5c13400700 1 -- 172.21.15.176:0/1862451078 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0401d070 con 0x7f5c1c073ae0 2024-03-13T14:59:49.074 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.088+0000 7f5c13400700 1 -- 172.21.15.176:0/1862451078 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c04004fb0 con 0x7f5c1c073ae0 2024-03-13T14:59:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.088+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c198b10 con 0x7f5c1c073ae0 2024-03-13T14:59:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.088+0000 7f5c13400700 1 -- 172.21.15.176:0/1862451078 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c0400f480 con 0x7f5c1c073ae0 2024-03-13T14:59:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.088+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c199000 con 0x7f5c1c073ae0 2024-03-13T14:59:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.090+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c10a540 con 0x7f5c1c073ae0 2024-03-13T14:59:49.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.090+0000 7f5c13400700 1 -- 172.21.15.176:0/1862451078 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f5c04022890 con 0x7f5c1c073ae0 2024-03-13T14:59:49.077 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.091+0000 7f5c13400700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5c0006dfa0 0x7f5c00070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T14:59:49.077 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.091+0000 7f5c13400700 1 -- 172.21.15.176:0/1862451078 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f5c0408f510 con 0x7f5c1c073ae0 2024-03-13T14:59:49.077 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.091+0000 7f5c1aa00700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5c0006dfa0 0x7f5c00070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T14:59:49.077 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.092+0000 7f5c1aa00700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5c0006dfa0 0x7f5c00070460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c007900 tx=0x7f5c0c008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T14:59:49.081 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.096+0000 7f5c13400700 1 -- 172.21.15.176:0/1862451078 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f5c0405c570 con 0x7f5c1c073ae0 2024-03-13T14:59:49.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.425+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5c1c04ea90 con 0x7f5c1c073ae0 2024-03-13T14:59:49.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.425+0000 7f5c13400700 1 -- 172.21.15.176:0/1862451078 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f5c04027020 con 0x7f5c1c073ae0 2024-03-13T14:59:49.411 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5c0006dfa0 msgr2=0x7f5c00070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5c0006dfa0 0x7f5c00070460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c007900 tx=0x7f5c0c008040 comp rx=0 tx=0).stop 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 msgr2=0x7f5c1c19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c19ed50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006010 tx=0x7f5c0400be20 comp rx=0 tx=0).stop 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 shutdown_connections 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f5c0006dfa0 0x7f5c00070460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c1c0731c0 0x7f5c1c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 --2- 172.21.15.176:0/1862451078 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f5c1c073ae0 0x7f5c1c19ed50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 >> 172.21.15.176:0/1862451078 conn(0x7f5c1c0fc630 msgr2=0x7f5c1c107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 shutdown_connections 2024-03-13T14:59:49.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T14:59:49.428+0000 7f5c21800700 1 -- 172.21.15.176:0/1862451078 wait complete. 2024-03-13T14:59:49.689 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: pgmap v235: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 2.7 MiB/s, 0 objects/s recovering 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/225 objects degraded (1.778%), 2 pgs degraded) 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/108965614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.690 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:49 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1862451078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:59:49.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: pgmap v235: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 511 B/s rd, 1 op/s; 2.7 MiB/s, 0 objects/s recovering 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/225 objects degraded (1.778%), 2 pgs degraded) 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/108965614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T14:59:49.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:49 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1862451078' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T14:59:50.585 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:50 smithi176 ceph-mon[112742]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:50.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:50 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.176:6828/46228150,v1:172.21.15.176:6829/46228150] up:boot 2024-03-13T14:59:50.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:50 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T14:59:50.586 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:50 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:59:50.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:50 smithi196 ceph-mon[86193]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T14:59:50.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:50 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.176:6828/46228150,v1:172.21.15.176:6829/46228150] up:boot 2024-03-13T14:59:50.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:50 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T14:59:50.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:50 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T14:59:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:51 smithi176 ceph-mon[112742]: pgmap v236: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 873 B/s rd, 1 op/s; 2.3 MiB/s, 0 objects/s recovering 2024-03-13T14:59:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:59:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:51 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:51.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:51 smithi196 ceph-mon[86193]: pgmap v236: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 873 B/s rd, 1 op/s; 2.3 MiB/s, 0 objects/s recovering 2024-03-13T14:59:51.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:51.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.ldskqt", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T14:59:51.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:51 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T14:59:52.759 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:52 smithi176 ceph-mon[112742]: Upgrade: Updating mds.cephfs.smithi176.ldskqt 2024-03-13T14:59:52.760 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:52 smithi176 ceph-mon[112742]: Deploying daemon mds.cephfs.smithi176.ldskqt on smithi176 2024-03-13T14:59:52.870 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:52 smithi196 ceph-mon[86193]: Upgrade: Updating mds.cephfs.smithi176.ldskqt 2024-03-13T14:59:52.870 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:52 smithi196 ceph-mon[86193]: Deploying daemon mds.cephfs.smithi176.ldskqt on smithi176 2024-03-13T14:59:53.833 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:53 smithi176 ceph-mon[112742]: pgmap v237: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 2.0 MiB/s, 0 objects/s recovering 2024-03-13T14:59:53.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:53 smithi196 ceph-mon[86193]: pgmap v237: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s; 2.0 MiB/s, 0 objects/s recovering 2024-03-13T14:59:55.584 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:55 smithi176 ceph-mon[112742]: pgmap v238: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 383 B/s rd, 0 op/s 2024-03-13T14:59:55.584 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:55 smithi176 ceph-mon[112742]: osdmap e109: 6 total, 6 up, 6 in 2024-03-13T14:59:55.585 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:55 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 2 up:standby 2024-03-13T14:59:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:55 smithi196 ceph-mon[86193]: pgmap v238: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 383 B/s rd, 0 op/s 2024-03-13T14:59:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:55 smithi196 ceph-mon[86193]: osdmap e109: 6 total, 6 up, 6 in 2024-03-13T14:59:55.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:55 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 2 up:standby 2024-03-13T14:59:56.867 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:56 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:56.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:56 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:57.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:57 smithi176 ceph-mon[112742]: pgmap v240: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 343 B/s rd, 0 op/s 2024-03-13T14:59:57.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:57 smithi196 ceph-mon[86193]: pgmap v240: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 343 B/s rd, 0 op/s 2024-03-13T14:59:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:58 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:boot 2024-03-13T14:59:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:58 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T14:59:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:59:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:58.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:58 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:58.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:58 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:boot 2024-03-13T14:59:58.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:58 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T14:59:58.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T14:59:58.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:58.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T14:59:58.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:58 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T14:59:59.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 14:59:59 smithi176 ceph-mon[112742]: pgmap v241: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 0 op/s 2024-03-13T14:59:59.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 14:59:59 smithi196 ceph-mon[86193]: pgmap v241: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 0 op/s 2024-03-13T15:00:00.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:00 smithi176 ceph-mon[112742]: overall HEALTH_OK 2024-03-13T15:00:00.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:00.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:00.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:00 smithi196 ceph-mon[86193]: overall HEALTH_OK 2024-03-13T15:00:00.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:00.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:01.828 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:01 smithi196 ceph-mon[86193]: pgmap v242: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:01.848 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:01 smithi176 ceph-mon[112742]: pgmap v242: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:03.830 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:03 smithi176 ceph-mon[112742]: pgmap v243: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:03.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:03 smithi196 ceph-mon[86193]: pgmap v243: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:05.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:05 smithi176 ceph-mon[112742]: pgmap v244: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:05.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:05 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:05.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:05 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:05.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:05 smithi196 ceph-mon[86193]: pgmap v244: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:05.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:05 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:05.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:05 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:07.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:07 smithi176 ceph-mon[112742]: pgmap v245: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:07.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:07.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:07.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:07 smithi196 ceph-mon[86193]: pgmap v245: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:07.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:07.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:09.696 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:09 smithi176 ceph-mon[112742]: pgmap v246: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:09 smithi196 ceph-mon[86193]: pgmap v246: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:11.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:11 smithi176 ceph-mon[112742]: pgmap v247: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:11 smithi196 ceph-mon[86193]: pgmap v247: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:13.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:13.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:13.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: pgmap v248: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:13.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:13.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: pgmap v248: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:13.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:15.709 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:15 smithi176 ceph-mon[112742]: pgmap v249: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:15.709 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:15.709 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T15:00:15.709 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:15.709 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:15.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:15 smithi196 ceph-mon[86193]: pgmap v249: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:15.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:15.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi176.hjrgov", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T15:00:15.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:15.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:16.769 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:16 smithi176 ceph-mon[112742]: Upgrade: Updating mds.cephfs.smithi176.hjrgov 2024-03-13T15:00:16.769 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:16 smithi176 ceph-mon[112742]: Deploying daemon mds.cephfs.smithi176.hjrgov on smithi176 2024-03-13T15:00:16.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:16 smithi196 ceph-mon[86193]: Upgrade: Updating mds.cephfs.smithi176.hjrgov 2024-03-13T15:00:16.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:16 smithi196 ceph-mon[86193]: Deploying daemon mds.cephfs.smithi176.hjrgov on smithi176 2024-03-13T15:00:17.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:17 smithi176 ceph-mon[112742]: pgmap v250: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:17.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:17 smithi196 ceph-mon[86193]: pgmap v250: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:19.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 -- 172.21.15.176:0/3571729794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c0ffa40 msgr2=0x7fce7c0ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 --2- 172.21.15.176:0/3571729794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c0ffa40 0x7fce7c0ffec0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fce68009910 tx=0x7fce68009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 -- 172.21.15.176:0/3571729794 shutdown_connections 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 --2- 172.21.15.176:0/3571729794 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c0ffa40 0x7fce7c0ffec0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 --2- 172.21.15.176:0/3571729794 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c105a40 0x7fce7c105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 -- 172.21.15.176:0/3571729794 >> 172.21.15.176:0/3571729794 conn(0x7fce7c0fb570 msgr2=0x7fce7c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 -- 172.21.15.176:0/3571729794 shutdown_connections 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.608+0000 7fce84600700 1 -- 172.21.15.176:0/3571729794 wait complete. 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce84600700 1 Processor -- start 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce84600700 1 -- start start 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce84600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c0ffa40 0x7fce7c0fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce84600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c105a40 0x7fce7c0fc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce84600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0fca00 con 0x7fce7c105a40 2024-03-13T15:00:19.596 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce84600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0fcb70 con 0x7fce7c0ffa40 2024-03-13T15:00:19.597 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce81200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c105a40 0x7fce7c0fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:19.597 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce81200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c105a40 0x7fce7c0fc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40904/0 (socket says 172.21.15.176:40904) 2024-03-13T15:00:19.597 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.610+0000 7fce81200700 1 -- 172.21.15.176:0/301952746 learned_addr learned my addr 172.21.15.176:0/301952746 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce81c00700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c0ffa40 0x7fce7c0fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce81200700 1 -- 172.21.15.176:0/301952746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c0ffa40 msgr2=0x7fce7c0fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce81200700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c0ffa40 0x7fce7c0fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce81200700 1 -- 172.21.15.176:0/301952746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce680095f0 con 0x7fce7c105a40 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce81200700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c105a40 0x7fce7c0fc430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fce6800b5c0 tx=0x7fce6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce7a000700 1 -- 172.21.15.176:0/301952746 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6801d070 con 0x7fce7c105a40 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce7a000700 1 -- 172.21.15.176:0/301952746 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce68005e00 con 0x7fce7c105a40 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce7a000700 1 -- 172.21.15.176:0/301952746 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce68017400 con 0x7fce7c105a40 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.611+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c1a2920 con 0x7fce7c105a40 2024-03-13T15:00:19.598 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.612+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1a2cd0 con 0x7fce7c105a40 2024-03-13T15:00:19.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.613+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c108fe0 con 0x7fce7c105a40 2024-03-13T15:00:19.599 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.614+0000 7fce7a000700 1 -- 172.21.15.176:0/301952746 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fce6800f460 con 0x7fce7c105a40 2024-03-13T15:00:19.600 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.615+0000 7fce7a000700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fce6406dfa0 0x7fce64070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:19.600 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.615+0000 7fce7a000700 1 -- 172.21.15.176:0/301952746 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fce6808f200 con 0x7fce7c105a40 2024-03-13T15:00:19.600 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.615+0000 7fce81c00700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fce6406dfa0 0x7fce64070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:19.601 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.616+0000 7fce81c00700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fce6406dfa0 0x7fce64070460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce70005950 tx=0x7fce700058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:19.605 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.620+0000 7fce7a000700 1 -- 172.21.15.176:0/301952746 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fce68058680 con 0x7fce7c105a40 2024-03-13T15:00:19.849 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.863+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce7c0611d0 con 0x7fce6406dfa0 2024-03-13T15:00:19.849 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:19 smithi176 ceph-mon[112742]: pgmap v251: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:19.849 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:19 smithi176 ceph-mon[112742]: osdmap e110: 6 total, 6 up, 6 in 2024-03-13T15:00:19.849 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:19 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 2 up:standby 2024-03-13T15:00:19.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.867+0000 7fce7a000700 1 -- 172.21.15.176:0/301952746 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fce7c0611d0 con 0x7fce6406dfa0 2024-03-13T15:00:19.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fce6406dfa0 msgr2=0x7fce64070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:19.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fce6406dfa0 0x7fce64070460 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce70005950 tx=0x7fce700058e0 comp rx=0 tx=0).stop 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c105a40 msgr2=0x7fce7c0fc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c105a40 0x7fce7c0fc430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fce6800b5c0 tx=0x7fce6800be20 comp rx=0 tx=0).stop 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 shutdown_connections 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fce6406dfa0 0x7fce64070460 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce7c0ffa40 0x7fce7c0fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 --2- 172.21.15.176:0/301952746 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fce7c105a40 0x7fce7c0fc430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 >> 172.21.15.176:0/301952746 conn(0x7fce7c0fb570 msgr2=0x7fce7c068f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 shutdown_connections 2024-03-13T15:00:19.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:19.870+0000 7fce84600700 1 -- 172.21.15.176:0/301952746 wait complete. 2024-03-13T15:00:19.872 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T15:00:19.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:19 smithi196 ceph-mon[86193]: pgmap v251: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:19.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:19 smithi196 ceph-mon[86193]: osdmap e110: 6 total, 6 up, 6 in 2024-03-13T15:00:19.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:19 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 2 up:standby 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 -- 172.21.15.176:0/3211015562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 msgr2=0x7fb2cc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/3211015562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc107650 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400b0a0 tx=0x7fb2b400b3b0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 -- 172.21.15.176:0/3211015562 shutdown_connections 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/3211015562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc107650 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/3211015562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc10d280 0x7fb2cc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 -- 172.21.15.176:0/3211015562 >> 172.21.15.176:0/3211015562 conn(0x7fb2cc0769e0 msgr2=0x7fb2cc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 -- 172.21.15.176:0/3211015562 shutdown_connections 2024-03-13T15:00:20.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.028+0000 7fb2d1a00700 1 -- 172.21.15.176:0/3211015562 wait complete. 2024-03-13T15:00:20.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.029+0000 7fb2d1a00700 1 Processor -- start 2024-03-13T15:00:20.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.029+0000 7fb2d1a00700 1 -- start start 2024-03-13T15:00:20.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2d1a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2d1a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc10d280 0x7fb2cc19d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2d1a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc19d880 con 0x7fb2cc1071d0 2024-03-13T15:00:20.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2d1a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc1a05b0 con 0x7fb2cc10d280 2024-03-13T15:00:20.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2caa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2caa00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40924/0 (socket says 172.21.15.176:40924) 2024-03-13T15:00:20.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2caa00700 1 -- 172.21.15.176:0/4218078498 learned_addr learned my addr 172.21.15.176:0/4218078498 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:20.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.030+0000 7fb2caa00700 1 -- 172.21.15.176:0/4218078498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc10d280 msgr2=0x7fb2cc19d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2c2000700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc10d280 0x7fb2cc19d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2caa00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc10d280 0x7fb2cc19d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2caa00700 1 -- 172.21.15.176:0/4218078498 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4009d00 con 0x7fb2cc1071d0 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2c2000700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc10d280 0x7fb2cc19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2caa00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc19ccb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00d900 tx=0x7fb2bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2c3e00700 1 -- 172.21.15.176:0/4218078498 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2bc0041f0 con 0x7fb2cc1071d0 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2c3e00700 1 -- 172.21.15.176:0/4218078498 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2bc004cf0 con 0x7fb2cc1071d0 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2c3e00700 1 -- 172.21.15.176:0/4218078498 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2bc00b750 con 0x7fb2cc1071d0 2024-03-13T15:00:20.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc1a0890 con 0x7fb2cc1071d0 2024-03-13T15:00:20.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.031+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc1a0de0 con 0x7fb2cc1071d0 2024-03-13T15:00:20.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.032+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc10f6c0 con 0x7fb2cc1071d0 2024-03-13T15:00:20.018 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.033+0000 7fb2c3e00700 1 -- 172.21.15.176:0/4218078498 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb2bc004350 con 0x7fb2cc1071d0 2024-03-13T15:00:20.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.034+0000 7fb2c3e00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb2b006dfa0 0x7fb2b0070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.034+0000 7fb2c3e00700 1 -- 172.21.15.176:0/4218078498 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fb2bc08dfe0 con 0x7fb2cc1071d0 2024-03-13T15:00:20.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.035+0000 7fb2c2000700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb2b006dfa0 0x7fb2b0070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.020 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.035+0000 7fb2c2000700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb2b006dfa0 0x7fb2b0070460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4015040 tx=0x7fb2b40097a0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:20.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.039+0000 7fb2c3e00700 1 -- 172.21.15.176:0/4218078498 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fb2bc0573a0 con 0x7fb2cc1071d0 2024-03-13T15:00:20.268 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.282+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2cc1a1070 con 0x7fb2b006dfa0 2024-03-13T15:00:20.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.285+0000 7fb2c3e00700 1 -- 172.21.15.176:0/4218078498 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb2cc1a1070 con 0x7fb2b006dfa0 2024-03-13T15:00:20.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.287+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb2b006dfa0 msgr2=0x7fb2b0070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.287+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb2b006dfa0 0x7fb2b0070460 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4015040 tx=0x7fb2b40097a0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.273 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 msgr2=0x7fb2cc19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc19ccb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00d900 tx=0x7fb2bc00dcc0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 shutdown_connections 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb2b006dfa0 0x7fb2b0070460 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb2cc1071d0 0x7fb2cc19ccb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 --2- 172.21.15.176:0/4218078498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc10d280 0x7fb2cc19d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 >> 172.21.15.176:0/4218078498 conn(0x7fb2cc0769e0 msgr2=0x7fb2cc078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 shutdown_connections 2024-03-13T15:00:20.274 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.288+0000 7fb2d1a00700 1 -- 172.21.15.176:0/4218078498 wait complete. 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 -- 172.21.15.176:0/2439258888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 msgr2=0x7fc2281033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 --2- 172.21.15.176:0/2439258888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281033c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009910 tx=0x7fc21c009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 -- 172.21.15.176:0/2439258888 shutdown_connections 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 --2- 172.21.15.176:0/2439258888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc228103990 0x7fc2281079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 --2- 172.21.15.176:0/2439258888 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281033c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 -- 172.21.15.176:0/2439258888 >> 172.21.15.176:0/2439258888 conn(0x7fc2280fe8d0 msgr2=0x7fc228100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 -- 172.21.15.176:0/2439258888 shutdown_connections 2024-03-13T15:00:20.434 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.448+0000 7fc230200700 1 -- 172.21.15.176:0/2439258888 wait complete. 2024-03-13T15:00:20.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.449+0000 7fc230200700 1 Processor -- start 2024-03-13T15:00:20.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.449+0000 7fc230200700 1 -- start start 2024-03-13T15:00:20.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc230200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.435 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc230200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc228103990 0x7fc228198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc230200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281994e0 con 0x7fc228102fe0 2024-03-13T15:00:20.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc230200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228199650 con 0x7fc228103990 2024-03-13T15:00:20.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc22d800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc22d800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40946/0 (socket says 172.21.15.176:40946) 2024-03-13T15:00:20.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc22d800700 1 -- 172.21.15.176:0/2365091022 learned_addr learned my addr 172.21.15.176:0/2365091022 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.450+0000 7fc22ce00700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc228103990 0x7fc228198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc22d800700 1 -- 172.21.15.176:0/2365091022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc228103990 msgr2=0x7fc228198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc22d800700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc228103990 0x7fc228198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc22d800700 1 -- 172.21.15.176:0/2365091022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0095f0 con 0x7fc228102fe0 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc22ce00700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc228103990 0x7fc228198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc22d800700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281989a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b5c0 tx=0x7fc21c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc226000700 1 -- 172.21.15.176:0/2365091022 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc21c01d070 con 0x7fc228102fe0 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc226000700 1 -- 172.21.15.176:0/2365091022 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc21c00f460 con 0x7fc228102fe0 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc226000700 1 -- 172.21.15.176:0/2365091022 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc21c017610 con 0x7fc228102fe0 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22819d470 con 0x7fc228102fe0 2024-03-13T15:00:20.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.451+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22819d960 con 0x7fc228102fe0 2024-03-13T15:00:20.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.453+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22810b380 con 0x7fc228102fe0 2024-03-13T15:00:20.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.454+0000 7fc226000700 1 -- 172.21.15.176:0/2365091022 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc21c00f5d0 con 0x7fc228102fe0 2024-03-13T15:00:20.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.455+0000 7fc226000700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc21006dfa0 0x7fc210070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.455+0000 7fc226000700 1 -- 172.21.15.176:0/2365091022 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fc21c08f330 con 0x7fc228102fe0 2024-03-13T15:00:20.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.455+0000 7fc22ce00700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc21006dfa0 0x7fc210070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.456+0000 7fc22ce00700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc21006dfa0 0x7fc210070460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc214005fd0 tx=0x7fc214005e20 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:20.445 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.460+0000 7fc226000700 1 -- 172.21.15.176:0/2365091022 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc21c05c2e0 con 0x7fc228102fe0 2024-03-13T15:00:20.568 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T15:00:20.685 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.700+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc22819dc90 con 0x7fc21006dfa0 2024-03-13T15:00:20.701 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.715+0000 7fc226000700 1 -- 172.21.15.176:0/2365091022 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc22819dc90 con 0x7fc21006dfa0 2024-03-13T15:00:20.701 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T15:00:20.701 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (8m) 15s ago 21m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T15:00:20.701 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (22m) 15s ago 22m 9235k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T15:00:20.701 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (20m) 42s ago 20m 8900k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (4m) 15s ago 22m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (4m) 42s ago 20m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (7m) 15s ago 21m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (14m) 15s ago 14m 11.3M - 18.2.1 5be31c24972a e932595696da 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (23s) 15s ago 14m 11.4M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5b6d2dc0d458 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (14m) 42s ago 14m 140M - 18.2.1 5be31c24972a a7815c840c9b 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (14m) 42s ago 14m 18.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (11m) 15s ago 23m 509M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (10m) 42s ago 20m 451M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (6m) 15s ago 23m 54.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (5m) 42s ago 20m 41.0M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T15:00:20.702 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (9m) 15s ago 21m 8019k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (9m) 42s ago 20m 7671k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (4m) 15s ago 19m 157M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (3m) 15s ago 18m 105M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (2m) 15s ago 18m 126M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (2m) 42s ago 17m 140M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (91s) 42s ago 16m 100M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (47s) 42s ago 16m 11.2M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 43366d2dc8da 2024-03-13T15:00:20.703 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (8m) 15s ago 21m 68.4M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T15:00:20.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc21006dfa0 msgr2=0x7fc210070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc21006dfa0 0x7fc210070460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc214005fd0 tx=0x7fc214005e20 comp rx=0 tx=0).stop 2024-03-13T15:00:20.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 msgr2=0x7fc2281989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281989a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b5c0 tx=0x7fc21c00be00 comp rx=0 tx=0).stop 2024-03-13T15:00:20.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 shutdown_connections 2024-03-13T15:00:20.704 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc21006dfa0 0x7fc210070460 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc228102fe0 0x7fc2281989a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.718+0000 7fc230200700 1 --2- 172.21.15.176:0/2365091022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc228103990 0x7fc228198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.719+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 >> 172.21.15.176:0/2365091022 conn(0x7fc2280fe8d0 msgr2=0x7fc2280fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:20.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.719+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 shutdown_connections 2024-03-13T15:00:20.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.719+0000 7fc230200700 1 -- 172.21.15.176:0/2365091022 wait complete. 2024-03-13T15:00:20.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 -- 172.21.15.176:0/3353131779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 msgr2=0x7ff754105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 --2- 172.21.15.176:0/3353131779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff754105790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff740009910 tx=0x7ff740009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:20.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 -- 172.21.15.176:0/3353131779 shutdown_connections 2024-03-13T15:00:20.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 --2- 172.21.15.176:0/3353131779 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff754105790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 --2- 172.21.15.176:0/3353131779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff754068670 0x7ff754068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 -- 172.21.15.176:0/3353131779 >> 172.21.15.176:0/3353131779 conn(0x7ff7540755a0 msgr2=0x7ff7540759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:20.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 -- 172.21.15.176:0/3353131779 shutdown_connections 2024-03-13T15:00:20.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.870+0000 7ff75be00700 1 -- 172.21.15.176:0/3353131779 wait complete. 2024-03-13T15:00:20.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.871+0000 7ff75be00700 1 Processor -- start 2024-03-13T15:00:20.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.872+0000 7ff75be00700 1 -- start start 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.872+0000 7ff75be00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff754068670 0x7ff75419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.872+0000 7ff75be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff75419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.872+0000 7ff75be00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75419f3b0 con 0x7ff754069020 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.872+0000 7ff75be00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754198860 con 0x7ff754068670 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.873+0000 7ff758a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff75419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.873+0000 7ff758a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff75419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40950/0 (socket says 172.21.15.176:40950) 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.873+0000 7ff758a00700 1 -- 172.21.15.176:0/2553051769 learned_addr learned my addr 172.21.15.176:0/2553051769 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:20.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.873+0000 7ff759400700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff754068670 0x7ff75419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.873+0000 7ff758a00700 1 -- 172.21.15.176:0/2553051769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff754068670 msgr2=0x7ff75419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:20.859 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.873+0000 7ff758a00700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff754068670 0x7ff75419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.873+0000 7ff758a00700 1 -- 172.21.15.176:0/2553051769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7400095f0 con 0x7ff754069020 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.874+0000 7ff758a00700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff75419ed20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff74000bda0 tx=0x7ff74000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.874+0000 7ff752000700 1 -- 172.21.15.176:0/2553051769 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74001d070 con 0x7ff754069020 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.874+0000 7ff752000700 1 -- 172.21.15.176:0/2553051769 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff74000f460 con 0x7ff754069020 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.874+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff754198ae0 con 0x7ff754069020 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.874+0000 7ff752000700 1 -- 172.21.15.176:0/2553051769 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff740017630 con 0x7ff754069020 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.874+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff754198fd0 con 0x7ff754069020 2024-03-13T15:00:20.860 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.875+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75404ea90 con 0x7ff754069020 2024-03-13T15:00:20.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.876+0000 7ff752000700 1 -- 172.21.15.176:0/2553051769 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff74000fab0 con 0x7ff754069020 2024-03-13T15:00:20.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.877+0000 7ff752000700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff73c06dfa0 0x7ff73c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:20.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.877+0000 7ff752000700 1 -- 172.21.15.176:0/2553051769 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6617+0+0 (secure 0 0 0) 0x7ff74008f1d0 con 0x7ff754069020 2024-03-13T15:00:20.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.877+0000 7ff759400700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff73c06dfa0 0x7ff73c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:20.863 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.878+0000 7ff759400700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff73c06dfa0 0x7ff73c070460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff748005950 tx=0x7ff74800a300 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:20.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:20.882+0000 7ff752000700 1 -- 172.21.15.176:0/2553051769 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7ff74005c180 con 0x7ff754069020 2024-03-13T15:00:20.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T15:00:21.169 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.184+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff754199d70 con 0x7ff754069020 2024-03-13T15:00:21.170 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.185+0000 7ff752000700 1 -- 172.21.15.176:0/2553051769 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7ff74005bd10 con 0x7ff754069020 2024-03-13T15:00:21.170 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 1 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:21.171 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T15:00:21.172 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 2, 2024-03-13T15:00:21.172 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 11 2024-03-13T15:00:21.172 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T15:00:21.172 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:00:21.173 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.187+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff73c06dfa0 msgr2=0x7ff73c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.173 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.187+0000 7ff75be00700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff73c06dfa0 0x7ff73c070460 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff748005950 tx=0x7ff74800a300 comp rx=0 tx=0).stop 2024-03-13T15:00:21.173 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 msgr2=0x7ff75419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.173 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff75419ed20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff74000bda0 tx=0x7ff74000bdd0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.173 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 shutdown_connections 2024-03-13T15:00:21.174 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff73c06dfa0 0x7ff73c070460 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.174 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff754068670 0x7ff75419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.174 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 --2- 172.21.15.176:0/2553051769 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff754069020 0x7ff75419ed20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.174 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 >> 172.21.15.176:0/2553051769 conn(0x7ff7540755a0 msgr2=0x7ff754102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:21.174 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 shutdown_connections 2024-03-13T15:00:21.174 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.188+0000 7ff75be00700 1 -- 172.21.15.176:0/2553051769 wait complete. 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 -- 172.21.15.176:0/958795304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 msgr2=0x7f19b010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 --2- 172.21.15.176:0/958795304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b010d660 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009910 tx=0x7f19a4009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 -- 172.21.15.176:0/958795304 shutdown_connections 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 --2- 172.21.15.176:0/958795304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19b01071d0 0x7f19b0107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 --2- 172.21.15.176:0/958795304 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b010d660 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 -- 172.21.15.176:0/958795304 >> 172.21.15.176:0/958795304 conn(0x7f19b00769e0 msgr2=0x7f19b0078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 -- 172.21.15.176:0/958795304 shutdown_connections 2024-03-13T15:00:21.331 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.345+0000 7f19b7600700 1 -- 172.21.15.176:0/958795304 wait complete. 2024-03-13T15:00:21.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.346+0000 7f19b7600700 1 Processor -- start 2024-03-13T15:00:21.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19b7600700 1 -- start start 2024-03-13T15:00:21.332 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19b7600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19b01071d0 0x7f19b019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19b7600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19b7600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b019da30 con 0x7f19b010d280 2024-03-13T15:00:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19b7600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01a1770 con 0x7f19b01071d0 2024-03-13T15:00:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19afe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:40972/0 (socket says 172.21.15.176:40972) 2024-03-13T15:00:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19afe00700 1 -- 172.21.15.176:0/1809039183 learned_addr learned my addr 172.21.15.176:0/1809039183 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:21.333 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.347+0000 7f19b4c00700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19b01071d0 0x7f19b019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19afe00700 1 -- 172.21.15.176:0/1809039183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19b01071d0 msgr2=0x7f19b019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19afe00700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19b01071d0 0x7f19b019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19afe00700 1 -- 172.21.15.176:0/1809039183 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a40095f0 con 0x7f19b010d280 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19afe00700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b019d350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f199c00b4b0 tx=0x7f199c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19ad600700 1 -- 172.21.15.176:0/1809039183 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199c011800 con 0x7f19b010d280 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19ad600700 1 -- 172.21.15.176:0/1809039183 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f199c011e20 con 0x7f19b010d280 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19ad600700 1 -- 172.21.15.176:0/1809039183 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f199c00f2e0 con 0x7f19b010d280 2024-03-13T15:00:21.334 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.348+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b01a1a50 con 0x7f19b010d280 2024-03-13T15:00:21.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.349+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b01a2020 con 0x7f19b010d280 2024-03-13T15:00:21.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.350+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b010f6f0 con 0x7f19b010d280 2024-03-13T15:00:21.335 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.350+0000 7f19ad600700 1 -- 172.21.15.176:0/1809039183 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f199c010380 con 0x7f19b010d280 2024-03-13T15:00:21.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.351+0000 7f19ad600700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f199806dfa0 0x7f1998070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:21.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.351+0000 7f19ad600700 1 -- 172.21.15.176:0/1809039183 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f199c08d8f0 con 0x7f19b010d280 2024-03-13T15:00:21.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.352+0000 7f19b4c00700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f199806dfa0 0x7f1998070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:21.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.352+0000 7f19b4c00700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f199806dfa0 0x7f1998070460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19a4006010 tx=0x7f19a401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:21.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.356+0000 7f19ad600700 1 -- 172.21.15.176:0/1809039183 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f199c056e20 con 0x7f19b010d280 2024-03-13T15:00:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.625+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f19b004ea90 con 0x7f19b010d280 2024-03-13T15:00:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.625+0000 7f19ad600700 1 -- 172.21.15.176:0/1809039183 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1671 (secure 0 0 0) 0x7f199c05a4e0 con 0x7f19b010d280 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:e23 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 19 2024-03-13T15:00:21.611 INFO:teuthology.orchestra.run.smithi176.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T14:59:47.444191+0000 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 0 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24249} 2024-03-13T15:00:21.612 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{0:24249} state up:active seq 7 join_fscid=1 addr [v2:172.21.15.196:6824/152396394,v1:172.21.15.196:6825/152396394] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:24255} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T15:00:21.613 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:00:21.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f199806dfa0 msgr2=0x7f1998070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f199806dfa0 0x7f1998070460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f19a4006010 tx=0x7f19a401a040 comp rx=0 tx=0).stop 2024-03-13T15:00:21.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 msgr2=0x7f19b019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b019d350 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f199c00b4b0 tx=0x7f199c00b870 comp rx=0 tx=0).stop 2024-03-13T15:00:21.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 shutdown_connections 2024-03-13T15:00:21.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f199806dfa0 0x7f1998070460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19b01071d0 0x7f19b019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 --2- 172.21.15.176:0/1809039183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f19b010d280 0x7f19b019d350 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 >> 172.21.15.176:0/1809039183 conn(0x7f19b00769e0 msgr2=0x7f19b00785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:21.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 shutdown_connections 2024-03-13T15:00:21.615 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.629+0000 7f19b7600700 1 -- 172.21.15.176:0/1809039183 wait complete. 2024-03-13T15:00:21.616 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 23 2024-03-13T15:00:21.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.790+0000 7f6b0d400700 1 -- 172.21.15.176:0/3530143209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b080731c0 msgr2=0x7f6b080735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.776 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.790+0000 7f6b0d400700 1 --2- 172.21.15.176:0/3530143209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b080731c0 0x7f6b080735a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009910 tx=0x7f6af8009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:21.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:21 smithi176 ceph-mon[112742]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:21.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:21 smithi176 ceph-mon[112742]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:21.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:21 smithi176 ceph-mon[112742]: pgmap v253: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:21.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:21 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2553051769' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:21.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.790+0000 7f6b0d400700 1 -- 172.21.15.176:0/3530143209 shutdown_connections 2024-03-13T15:00:21.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.790+0000 7f6b0d400700 1 --2- 172.21.15.176:0/3530143209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b08073ae0 0x7f6b0810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.790+0000 7f6b0d400700 1 --2- 172.21.15.176:0/3530143209 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b080731c0 0x7f6b080735a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.790+0000 7f6b0d400700 1 -- 172.21.15.176:0/3530143209 >> 172.21.15.176:0/3530143209 conn(0x7f6b080fc630 msgr2=0x7f6b080fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:21.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.791+0000 7f6b0d400700 1 -- 172.21.15.176:0/3530143209 shutdown_connections 2024-03-13T15:00:21.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.791+0000 7f6b0d400700 1 -- 172.21.15.176:0/3530143209 wait complete. 2024-03-13T15:00:21.778 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.792+0000 7f6b0d400700 1 Processor -- start 2024-03-13T15:00:21.778 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.792+0000 7f6b0d400700 1 -- start start 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b0d400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b080731c0 0x7f6b0819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b0d400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b08073ae0 0x7f6b0819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b0d400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0819f430 con 0x7f6b08073ae0 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b0d400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b081988e0 con 0x7f6b080731c0 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b05e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b08073ae0 0x7f6b0819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b05e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b08073ae0 0x7f6b0819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:41000/0 (socket says 172.21.15.176:41000) 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b05e00700 1 -- 172.21.15.176:0/1211690927 learned_addr learned my addr 172.21.15.176:0/1211690927 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b06800700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b080731c0 0x7f6b0819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b05e00700 1 -- 172.21.15.176:0/1211690927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b080731c0 msgr2=0x7f6b0819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b05e00700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b080731c0 0x7f6b0819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:21.779 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.793+0000 7f6b05e00700 1 -- 172.21.15.176:0/1211690927 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80095f0 con 0x7f6b08073ae0 2024-03-13T15:00:21.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.794+0000 7f6b05e00700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b08073ae0 0x7f6b0819eda0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b4b0 tx=0x7f6af400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:21.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.794+0000 7f6aff400700 1 -- 172.21.15.176:0/1211690927 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af4011800 con 0x7f6b08073ae0 2024-03-13T15:00:21.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.794+0000 7f6aff400700 1 -- 172.21.15.176:0/1211690927 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6af4011e20 con 0x7f6b08073ae0 2024-03-13T15:00:21.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.794+0000 7f6aff400700 1 -- 172.21.15.176:0/1211690927 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af400f2e0 con 0x7f6b08073ae0 2024-03-13T15:00:21.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.795+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b08198bc0 con 0x7f6b08073ae0 2024-03-13T15:00:21.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.795+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b08199090 con 0x7f6b08073ae0 2024-03-13T15:00:21.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.796+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0810a540 con 0x7f6b08073ae0 2024-03-13T15:00:21.782 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.797+0000 7f6aff400700 1 -- 172.21.15.176:0/1211690927 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6af4010380 con 0x7f6b08073ae0 2024-03-13T15:00:21.783 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.798+0000 7f6aff400700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6aec06dfa0 0x7f6aec070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:21.783 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.798+0000 7f6aff400700 1 -- 172.21.15.176:0/1211690927 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6617+0+0 (secure 0 0 0) 0x7f6af408d6c0 con 0x7f6b08073ae0 2024-03-13T15:00:21.783 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.798+0000 7f6b06800700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6aec06dfa0 0x7f6aec070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:21.784 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.799+0000 7f6b06800700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6aec06dfa0 0x7f6aec070460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b5c0 tx=0x7f6af801a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:21.788 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:21.803+0000 7f6aff400700 1 -- 172.21.15.176:0/1211690927 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f6af405a5f0 con 0x7f6b08073ae0 2024-03-13T15:00:21.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:21 smithi196 ceph-mon[86193]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:21.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:21 smithi196 ceph-mon[86193]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:21.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:21 smithi196 ceph-mon[86193]: pgmap v253: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:21.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:21 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2553051769' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:22.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.055+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b081993d0 con 0x7f6aec06dfa0 2024-03-13T15:00:22.042 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.057+0000 7f6aff400700 1 -- 172.21.15.176:0/1211690927 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6b081993d0 con 0x7f6aec06dfa0 2024-03-13T15:00:22.042 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:00:22.042 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "osd", 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "13/23 daemons upgraded", 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mds daemons", 2024-03-13T15:00:22.043 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T15:00:22.044 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:00:22.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6aec06dfa0 msgr2=0x7f6aec070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:22.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6aec06dfa0 0x7f6aec070460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6af800b5c0 tx=0x7f6af801a040 comp rx=0 tx=0).stop 2024-03-13T15:00:22.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b08073ae0 msgr2=0x7f6b0819eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:22.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b08073ae0 0x7f6b0819eda0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6af400b4b0 tx=0x7f6af400b870 comp rx=0 tx=0).stop 2024-03-13T15:00:22.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 shutdown_connections 2024-03-13T15:00:22.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6aec06dfa0 0x7f6aec070460 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b080731c0 0x7f6b0819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 --2- 172.21.15.176:0/1211690927 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6b08073ae0 0x7f6b0819eda0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 >> 172.21.15.176:0/1211690927 conn(0x7f6b080fc630 msgr2=0x7f6b08107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:22.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 shutdown_connections 2024-03-13T15:00:22.046 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.060+0000 7f6b0d400700 1 -- 172.21.15.176:0/1211690927 wait complete. 2024-03-13T15:00:22.200 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.214+0000 7fbac7a00700 1 -- 172.21.15.176:0/474737820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac01071d0 msgr2=0x7fbac0107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:22.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.214+0000 7fbac7a00700 1 --2- 172.21.15.176:0/474737820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac01071d0 0x7fbac0107650 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009910 tx=0x7fbaac009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:22.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.214+0000 7fbac7a00700 1 -- 172.21.15.176:0/474737820 shutdown_connections 2024-03-13T15:00:22.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.214+0000 7fbac7a00700 1 --2- 172.21.15.176:0/474737820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac01071d0 0x7fbac0107650 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.214+0000 7fbac7a00700 1 --2- 172.21.15.176:0/474737820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbac010d280 0x7fbac010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.214+0000 7fbac7a00700 1 -- 172.21.15.176:0/474737820 >> 172.21.15.176:0/474737820 conn(0x7fbac00769e0 msgr2=0x7fbac0078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:22.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.215+0000 7fbac7a00700 1 -- 172.21.15.176:0/474737820 shutdown_connections 2024-03-13T15:00:22.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.215+0000 7fbac7a00700 1 -- 172.21.15.176:0/474737820 wait complete. 2024-03-13T15:00:22.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.216+0000 7fbac7a00700 1 Processor -- start 2024-03-13T15:00:22.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.216+0000 7fbac7a00700 1 -- start start 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.216+0000 7fbac7a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbac01071d0 0x7fbac019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.216+0000 7fbac7a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac010d280 0x7fbac019d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.216+0000 7fbac7a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac019d880 con 0x7fbac010d280 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.216+0000 7fbac7a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbac01a05b0 con 0x7fbac01071d0 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbabfe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac010d280 0x7fbac019d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbabfe00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac010d280 0x7fbac019d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:34866/0 (socket says 172.21.15.176:34866) 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbabfe00700 1 -- 172.21.15.176:0/1992987329 learned_addr learned my addr 172.21.15.176:0/1992987329 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbac5000700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbac01071d0 0x7fbac019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbabfe00700 1 -- 172.21.15.176:0/1992987329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbac01071d0 msgr2=0x7fbac019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbabfe00700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbac01071d0 0x7fbac019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbabfe00700 1 -- 172.21.15.176:0/1992987329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac0095f0 con 0x7fbac010d280 2024-03-13T15:00:22.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.217+0000 7fbabfe00700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac010d280 0x7fbac019d1f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00b5c0 tx=0x7fbaac00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.218+0000 7fbabd600700 1 -- 172.21.15.176:0/1992987329 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac01d070 con 0x7fbac010d280 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.218+0000 7fbabd600700 1 -- 172.21.15.176:0/1992987329 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbaac00f460 con 0x7fbac010d280 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.218+0000 7fbabd600700 1 -- 172.21.15.176:0/1992987329 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbaac017610 con 0x7fbac010d280 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.218+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbac01a0830 con 0x7fbac010d280 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.218+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbac01a0da0 con 0x7fbac010d280 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.219+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbac010f6c0 con 0x7fbac010d280 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.220+0000 7fbabd600700 1 -- 172.21.15.176:0/1992987329 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fbaac00f5d0 con 0x7fbac010d280 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.221+0000 7fbabd600700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbaa806dfa0 0x7fbaa8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:22.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.222+0000 7fbabd600700 1 -- 172.21.15.176:0/1992987329 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6617+0+0 (secure 0 0 0) 0x7fbaac08f3f0 con 0x7fbac010d280 2024-03-13T15:00:22.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.222+0000 7fbac5000700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbaa806dfa0 0x7fbaa8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:22.209 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.222+0000 7fbac5000700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbaa806dfa0 0x7fbaa8070460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbab4005fd0 tx=0x7fbab400a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:22.211 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.226+0000 7fbabd600700 1 -- 172.21.15.176:0/1992987329 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fbaac0587d0 con 0x7fbac010d280 2024-03-13T15:00:22.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.546+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbac01a10d0 con 0x7fbac010d280 2024-03-13T15:00:22.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.546+0000 7fbabd600700 1 -- 172.21.15.176:0/1992987329 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbaac05be90 con 0x7fbac010d280 2024-03-13T15:00:22.532 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T15:00:22.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbaa806dfa0 msgr2=0x7fbaa8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:22.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbaa806dfa0 0x7fbaa8070460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbab4005fd0 tx=0x7fbab400a560 comp rx=0 tx=0).stop 2024-03-13T15:00:22.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac010d280 msgr2=0x7fbac019d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:22.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac010d280 0x7fbac019d1f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbaac00b5c0 tx=0x7fbaac00be00 comp rx=0 tx=0).stop 2024-03-13T15:00:22.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 shutdown_connections 2024-03-13T15:00:22.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbaa806dfa0 0x7fbaa8070460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbac01071d0 0x7fbac019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 --2- 172.21.15.176:0/1992987329 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbac010d280 0x7fbac019d1f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:22.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.549+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 >> 172.21.15.176:0/1992987329 conn(0x7fbac00769e0 msgr2=0x7fbac0078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:22.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.550+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 shutdown_connections 2024-03-13T15:00:22.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:22.550+0000 7fbac7a00700 1 -- 172.21.15.176:0/1992987329 wait complete. 2024-03-13T15:00:22.561 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:22 smithi176 ceph-mon[112742]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:22.561 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:22 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1809039183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:00:22.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:22 smithi196 ceph-mon[86193]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:22.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:22 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1809039183' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:00:23.857 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:23 smithi196 ceph-mon[86193]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:23.858 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:23 smithi196 ceph-mon[86193]: pgmap v254: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:23.858 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:23 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1992987329' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:00:23.858 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:23.858 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:23.858 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:23.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:23 smithi176 ceph-mon[112742]: from='client.34486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:23.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:23 smithi176 ceph-mon[112742]: pgmap v254: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:23.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:23 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1992987329' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:00:23.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:23.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:23.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:24.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:24 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.176:6828/552037800,v1:172.21.15.176:6829/552037800] up:boot 2024-03-13T15:00:24.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:24 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T15:00:24.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:24 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T15:00:24.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:24 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:24.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:24 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:24.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:24 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.176:6828/552037800,v1:172.21.15.176:6829/552037800] up:boot 2024-03-13T15:00:24.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:24 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.tcitrk=up:active} 3 up:standby 2024-03-13T15:00:24.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:24 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T15:00:24.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:24 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:24.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:24 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:25.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:25 smithi176 ceph-mon[112742]: pgmap v255: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:25.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:25 smithi196 ceph-mon[86193]: pgmap v255: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:27.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:27 smithi176 ceph-mon[112742]: pgmap v256: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:27 smithi196 ceph-mon[86193]: pgmap v256: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:29 smithi176 ceph-mon[112742]: pgmap v257: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:29.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:29 smithi196 ceph-mon[86193]: pgmap v257: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:31.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:31.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:31.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:31.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:31 smithi176 ceph-mon[112742]: pgmap v258: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:31 smithi196 ceph-mon[86193]: pgmap v258: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:33.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:33 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:33.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:33 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:33.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:33 smithi176 ceph-mon[112742]: pgmap v259: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:33.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:33 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:33.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:33 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:33.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:33 smithi196 ceph-mon[86193]: pgmap v259: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:35.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:35 smithi176 ceph-mon[112742]: pgmap v260: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:35.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:35 smithi196 ceph-mon[86193]: pgmap v260: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:37.088 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:36 smithi176 ceph-mon[112742]: pgmap v261: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:37.176 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:36 smithi196 ceph-mon[86193]: pgmap v261: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:38.782 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:38.782 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:38.782 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:38.783 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:00:38.783 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:38.783 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:38.783 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:38.783 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:38.783 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:38.783 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:38 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:38.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:38.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:38 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:39 smithi196 ceph-mon[86193]: pgmap v262: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:39.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:39 smithi176 ceph-mon[112742]: pgmap v262: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:41.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:41 smithi196 ceph-mon[86193]: Upgrade: Updating mds.cephfs.smithi196.tcitrk 2024-03-13T15:00:41.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:41 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:41.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:41 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T15:00:41.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:41 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:41.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:41 smithi196 ceph-mon[86193]: Deploying daemon mds.cephfs.smithi196.tcitrk on smithi196 2024-03-13T15:00:41.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:41 smithi196 ceph-mon[86193]: pgmap v263: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:41 smithi176 ceph-mon[112742]: Upgrade: Updating mds.cephfs.smithi196.tcitrk 2024-03-13T15:00:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:41 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:41 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.tcitrk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T15:00:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:41 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:00:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:41 smithi176 ceph-mon[112742]: Deploying daemon mds.cephfs.smithi196.tcitrk on smithi196 2024-03-13T15:00:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:41 smithi176 ceph-mon[112742]: pgmap v263: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:43.482 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[112738]: 2024-03-13T15:00:43.199+0000 7f67a1400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T15:00:43.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: pgmap v264: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:43.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-13T15:00:43.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T15:00:43.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: osdmap e111: 6 total, 6 up, 6 in 2024-03-13T15:00:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: Standby daemon mds.cephfs.smithi196.wabhcu assigned to filesystem cephfs as rank 0 2024-03-13T15:00:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-13T15:00:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-13T15:00:43.877 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:43 smithi176 ceph-mon[112742]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:replay} 2 up:standby 2024-03-13T15:00:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: pgmap v264: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail 2024-03-13T15:00:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-13T15:00:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T15:00:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: osdmap e111: 6 total, 6 up, 6 in 2024-03-13T15:00:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: Standby daemon mds.cephfs.smithi196.wabhcu assigned to filesystem cephfs as rank 0 2024-03-13T15:00:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-13T15:00:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-13T15:00:43.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:43 smithi196 ceph-mon[86193]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:replay} 2 up:standby 2024-03-13T15:00:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:45 smithi176 ceph-mon[112742]: pgmap v266: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-03-13T15:00:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:45.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:45 smithi196 ceph-mon[86193]: pgmap v266: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 1.2 MiB/s rd, 0 op/s 2024-03-13T15:00:45.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:45.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:00:47.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:47 smithi196 ceph-mon[86193]: pgmap v267: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 2 op/s 2024-03-13T15:00:47.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:47 smithi176 ceph-mon[112742]: pgmap v267: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s rd, 2 op/s 2024-03-13T15:00:49.732 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:49 smithi196 ceph-mon[86193]: pgmap v268: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s rd, 3 op/s 2024-03-13T15:00:49.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:49 smithi176 ceph-mon[112742]: pgmap v268: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s rd, 3 op/s 2024-03-13T15:00:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:51 smithi176 ceph-mon[112742]: pgmap v269: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-13T15:00:51.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:51 smithi196 ceph-mon[86193]: pgmap v269: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-13T15:00:52.702 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.716+0000 7f6513800700 1 -- 172.21.15.176:0/721874874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c068950 msgr2=0x7f650c068dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:52.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.716+0000 7f6513800700 1 --2- 172.21.15.176:0/721874874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c068950 0x7f650c068dd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009910 tx=0x7f64fc009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:52.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.717+0000 7f6513800700 1 -- 172.21.15.176:0/721874874 shutdown_connections 2024-03-13T15:00:52.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.717+0000 7f6513800700 1 --2- 172.21.15.176:0/721874874 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c068950 0x7f650c068dd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:52.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.717+0000 7f6513800700 1 --2- 172.21.15.176:0/721874874 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c105b50 0x7f650c068410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:52.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.717+0000 7f6513800700 1 -- 172.21.15.176:0/721874874 >> 172.21.15.176:0/721874874 conn(0x7f650c0755a0 msgr2=0x7f650c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:52.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.717+0000 7f6513800700 1 -- 172.21.15.176:0/721874874 shutdown_connections 2024-03-13T15:00:52.703 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.717+0000 7f6513800700 1 -- 172.21.15.176:0/721874874 wait complete. 2024-03-13T15:00:52.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.718+0000 7f6513800700 1 Processor -- start 2024-03-13T15:00:52.705 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.718+0000 7f6513800700 1 -- start start 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.718+0000 7f6513800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c068950 0x7f650c1a2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.718+0000 7f6513800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c105b50 0x7f650c1a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.718+0000 7f6513800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c1a36a0 con 0x7f650c105b50 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.718+0000 7f6513800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c19cb50 con 0x7f650c068950 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6503e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c105b50 0x7f650c1a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6503e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c105b50 0x7f650c1a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46716/0 (socket says 172.21.15.176:46716) 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6503e00700 1 -- 172.21.15.176:0/358602820 learned_addr learned my addr 172.21.15.176:0/358602820 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6510e00700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c068950 0x7f650c1a2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:52.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6503e00700 1 -- 172.21.15.176:0/358602820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c068950 msgr2=0x7f650c1a2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6503e00700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c068950 0x7f650c1a2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6503e00700 1 -- 172.21.15.176:0/358602820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0095f0 con 0x7f650c105b50 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6510e00700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c068950 0x7f650c1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.719+0000 7f6503e00700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c105b50 0x7f650c1a3010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64fc006010 tx=0x7f64fc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.720+0000 7f650a000700 1 -- 172.21.15.176:0/358602820 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64fc01d070 con 0x7f650c105b50 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.720+0000 7f650a000700 1 -- 172.21.15.176:0/358602820 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64fc004fb0 con 0x7f650c105b50 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.720+0000 7f650a000700 1 -- 172.21.15.176:0/358602820 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64fc022ea0 con 0x7f650c105b50 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.720+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c19cdd0 con 0x7f650c105b50 2024-03-13T15:00:52.707 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.720+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c19d2c0 con 0x7f650c105b50 2024-03-13T15:00:52.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.721+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c04ea90 con 0x7f650c105b50 2024-03-13T15:00:52.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.722+0000 7f650a000700 1 -- 172.21.15.176:0/358602820 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f64fc022470 con 0x7f650c105b50 2024-03-13T15:00:52.708 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.723+0000 7f650a000700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f64ec06dfa0 0x7f64ec070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:52.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.723+0000 7f650a000700 1 -- 172.21.15.176:0/358602820 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f64fc08f290 con 0x7f650c105b50 2024-03-13T15:00:52.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.723+0000 7f6510e00700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f64ec06dfa0 0x7f64ec070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:52.709 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.724+0000 7f6510e00700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f64ec06dfa0 0x7f64ec070460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64f4005fd0 tx=0x7f64f4005dc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:52.713 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.728+0000 7f650a000700 1 -- 172.21.15.176:0/358602820 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f64fc05c2a0 con 0x7f650c105b50 2024-03-13T15:00:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:52 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.196:6824/2671769958,v1:172.21.15.196:6825/2671769958] up:boot 2024-03-13T15:00:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:52 smithi176 ceph-mon[112742]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:replay} 3 up:standby 2024-03-13T15:00:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T15:00:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:52.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:52.910 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:52 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.196:6824/2671769958,v1:172.21.15.196:6825/2671769958] up:boot 2024-03-13T15:00:52.910 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:52 smithi196 ceph-mon[86193]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:replay} 3 up:standby 2024-03-13T15:00:52.910 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T15:00:52.910 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:52.910 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:52.910 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:00:52.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.971+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f650c19d700 con 0x7f64ec06dfa0 2024-03-13T15:00:52.958 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.973+0000 7f650a000700 1 -- 172.21.15.176:0/358602820 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f650c19d700 con 0x7f64ec06dfa0 2024-03-13T15:00:52.961 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f64ec06dfa0 msgr2=0x7f64ec070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f64ec06dfa0 0x7f64ec070460 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64f4005fd0 tx=0x7f64f4005dc0 comp rx=0 tx=0).stop 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c105b50 msgr2=0x7f650c1a3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c105b50 0x7f650c1a3010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f64fc006010 tx=0x7f64fc00be20 comp rx=0 tx=0).stop 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 shutdown_connections 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f64ec06dfa0 0x7f64ec070460 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c068950 0x7f650c1a2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 --2- 172.21.15.176:0/358602820 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f650c105b50 0x7f650c1a3010 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 >> 172.21.15.176:0/358602820 conn(0x7f650c0755a0 msgr2=0x7f650c101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 shutdown_connections 2024-03-13T15:00:52.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:52.976+0000 7f6513800700 1 -- 172.21.15.176:0/358602820 wait complete. 2024-03-13T15:00:52.977 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T15:00:53.119 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 -- 172.21.15.176:0/577498183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 msgr2=0x7f98d4073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 --2- 172.21.15.176:0/577498183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d4073510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009910 tx=0x7f98c4009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:53.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 -- 172.21.15.176:0/577498183 shutdown_connections 2024-03-13T15:00:53.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 --2- 172.21.15.176:0/577498183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d4073a50 0x7f98d4111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 --2- 172.21.15.176:0/577498183 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d4073510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 -- 172.21.15.176:0/577498183 >> 172.21.15.176:0/577498183 conn(0x7f98d40fc630 msgr2=0x7f98d40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:53.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 -- 172.21.15.176:0/577498183 shutdown_connections 2024-03-13T15:00:53.120 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.134+0000 7f98d9800700 1 -- 172.21.15.176:0/577498183 wait complete. 2024-03-13T15:00:53.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.135+0000 7f98d9800700 1 Processor -- start 2024-03-13T15:00:53.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.135+0000 7f98d9800700 1 -- start start 2024-03-13T15:00:53.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d9800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d9800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d4073a50 0x7f98d419d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d9800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d419d870 con 0x7f98d4073130 2024-03-13T15:00:53.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d9800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d41a0530 con 0x7f98d4073a50 2024-03-13T15:00:53.121 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d2a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d2a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46744/0 (socket says 172.21.15.176:46744) 2024-03-13T15:00:53.122 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d2a00700 1 -- 172.21.15.176:0/796402748 learned_addr learned my addr 172.21.15.176:0/796402748 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98cbe00700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d4073a50 0x7f98d419d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d2a00700 1 -- 172.21.15.176:0/796402748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d4073a50 msgr2=0x7f98d419d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d2a00700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d4073a50 0x7f98d419d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98d2a00700 1 -- 172.21.15.176:0/796402748 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c40095f0 con 0x7f98d4073130 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.136+0000 7f98cbe00700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d4073a50 0x7f98d419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.137+0000 7f98d2a00700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d419cca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f98c4006010 tx=0x7f98c400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.137+0000 7f98cb400700 1 -- 172.21.15.176:0/796402748 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98c401d070 con 0x7f98d4073130 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.137+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d41a0810 con 0x7f98d4073130 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.137+0000 7f98cb400700 1 -- 172.21.15.176:0/796402748 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98c400f460 con 0x7f98d4073130 2024-03-13T15:00:53.123 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.137+0000 7f98cb400700 1 -- 172.21.15.176:0/796402748 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98c4017630 con 0x7f98d4073130 2024-03-13T15:00:53.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.137+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d41a0d60 con 0x7f98d4073130 2024-03-13T15:00:53.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.138+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d410ed90 con 0x7f98d4073130 2024-03-13T15:00:53.124 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.140+0000 7f98cb400700 1 -- 172.21.15.176:0/796402748 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f98c400f5d0 con 0x7f98d4073130 2024-03-13T15:00:53.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.141+0000 7f98cb400700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f98b806dfa0 0x7f98b8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.141+0000 7f98cb400700 1 -- 172.21.15.176:0/796402748 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f98c408f2d0 con 0x7f98d4073130 2024-03-13T15:00:53.126 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.141+0000 7f98cbe00700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f98b806dfa0 0x7f98b8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.127 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.142+0000 7f98cbe00700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f98b806dfa0 0x7f98b8070460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005fd0 tx=0x7f98bc005ee0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:53.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.146+0000 7f98cb400700 1 -- 172.21.15.176:0/796402748 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f98c40585e0 con 0x7f98d4073130 2024-03-13T15:00:53.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.378+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f98d41a0ff0 con 0x7f98b806dfa0 2024-03-13T15:00:53.365 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.380+0000 7f98cb400700 1 -- 172.21.15.176:0/796402748 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f98d41a0ff0 con 0x7f98b806dfa0 2024-03-13T15:00:53.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f98b806dfa0 msgr2=0x7f98b8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f98b806dfa0 0x7f98b8070460 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005fd0 tx=0x7f98bc005ee0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 msgr2=0x7f98d419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d419cca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f98c4006010 tx=0x7f98c400be20 comp rx=0 tx=0).stop 2024-03-13T15:00:53.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 shutdown_connections 2024-03-13T15:00:53.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f98b806dfa0 0x7f98b8070460 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f98d4073130 0x7f98d419cca0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 --2- 172.21.15.176:0/796402748 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98d4073a50 0x7f98d419d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 >> 172.21.15.176:0/796402748 conn(0x7f98d40fc630 msgr2=0x7f98d4103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:53.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 shutdown_connections 2024-03-13T15:00:53.369 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.383+0000 7f98d9800700 1 -- 172.21.15.176:0/796402748 wait complete. 2024-03-13T15:00:53.521 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 -- 172.21.15.176:0/4086292368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 msgr2=0x7fdc9c10bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 --2- 172.21.15.176:0/4086292368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c10bc10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009910 tx=0x7fdc88009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:53.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 -- 172.21.15.176:0/4086292368 shutdown_connections 2024-03-13T15:00:53.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 --2- 172.21.15.176:0/4086292368 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c10bc10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 --2- 172.21.15.176:0/4086292368 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc9c107220 0x7fdc9c107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 -- 172.21.15.176:0/4086292368 >> 172.21.15.176:0/4086292368 conn(0x7fdc9c0767e0 msgr2=0x7fdc9c078c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:53.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 -- 172.21.15.176:0/4086292368 shutdown_connections 2024-03-13T15:00:53.522 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.536+0000 7fdc9b200700 1 -- 172.21.15.176:0/4086292368 wait complete. 2024-03-13T15:00:53.523 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.537+0000 7fdc9b200700 1 Processor -- start 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.537+0000 7fdc9b200700 1 -- start start 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc9b200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc9c107220 0x7fdc9c1a29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc9b200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c1a2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc9b200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c1a35c0 con 0x7fdc9c107bd0 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc9b200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c19ca70 con 0x7fdc9c107220 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc99400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c1a2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc99400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c1a2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46764/0 (socket says 172.21.15.176:46764) 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc99400700 1 -- 172.21.15.176:0/2182934370 learned_addr learned my addr 172.21.15.176:0/2182934370 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc99e00700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc9c107220 0x7fdc9c1a29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc99400700 1 -- 172.21.15.176:0/2182934370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc9c107220 msgr2=0x7fdc9c1a29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc99400700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc9c107220 0x7fdc9c1a29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.538+0000 7fdc99400700 1 -- 172.21.15.176:0/2182934370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880095f0 con 0x7fdc9c107bd0 2024-03-13T15:00:53.524 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.539+0000 7fdc99400700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c1a2f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005b40 tx=0x7fdc8800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:53.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.539+0000 7fdc92a00700 1 -- 172.21.15.176:0/2182934370 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc8801d070 con 0x7fdc9c107bd0 2024-03-13T15:00:53.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.539+0000 7fdc92a00700 1 -- 172.21.15.176:0/2182934370 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc8800bed0 con 0x7fdc9c107bd0 2024-03-13T15:00:53.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.539+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c19ccf0 con 0x7fdc9c107bd0 2024-03-13T15:00:53.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.539+0000 7fdc92a00700 1 -- 172.21.15.176:0/2182934370 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc88003e70 con 0x7fdc9c107bd0 2024-03-13T15:00:53.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.539+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c19d1e0 con 0x7fdc9c107bd0 2024-03-13T15:00:53.525 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.540+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9c10f610 con 0x7fdc9c107bd0 2024-03-13T15:00:53.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.541+0000 7fdc92a00700 1 -- 172.21.15.176:0/2182934370 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fdc880050d0 con 0x7fdc9c107bd0 2024-03-13T15:00:53.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.542+0000 7fdc92a00700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdc8006dfa0 0x7fdc80070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.542+0000 7fdc92a00700 1 -- 172.21.15.176:0/2182934370 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6703+0+0 (secure 0 0 0) 0x7fdc8808f120 con 0x7fdc9c107bd0 2024-03-13T15:00:53.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.542+0000 7fdc99e00700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdc8006dfa0 0x7fdc80070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.528 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.543+0000 7fdc99e00700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdc8006dfa0 0x7fdc80070460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009c60 tx=0x7fdc8c009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:53.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.547+0000 7fdc92a00700 1 -- 172.21.15.176:0/2182934370 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fdc8805bfe0 con 0x7fdc9c107bd0 2024-03-13T15:00:53.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:53 smithi176 ceph-mon[112742]: pgmap v270: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-13T15:00:53.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:53 smithi176 ceph-mon[112742]: reconnect by client.24287 192.168.0.1:0/2547707940 after 0.00299993 2024-03-13T15:00:53.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:53 smithi176 ceph-mon[112742]: reconnect by client.14538 192.168.0.1:0/3863864493 after 0.00299993 2024-03-13T15:00:53.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:53 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:reconnect 2024-03-13T15:00:53.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:53 smithi176 ceph-mon[112742]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:reconnect} 3 up:standby 2024-03-13T15:00:53.752 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.767+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc9c02cd30 con 0x7fdc8006dfa0 2024-03-13T15:00:53.764 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.779+0000 7fdc92a00700 1 -- 172.21.15.176:0/2182934370 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdc9c02cd30 con 0x7fdc8006dfa0 2024-03-13T15:00:53.764 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T15:00:53.764 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (8m) 23s ago 22m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (22m) 23s ago 22m 9261k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (21m) 75s ago 21m 8900k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (5m) 23s ago 22m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (5m) 75s ago 21m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (8m) 23s ago 21m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (31s) 23s ago 15m 11.1M - 19.0.0-1916-g9a75c81f 9552dbf7f115 edc682c345ac 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (56s) 23s ago 15m 11.9M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5b6d2dc0d458 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 starting - - - - 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (14m) 75s ago 14m 18.5M - 18.2.1 5be31c24972a ef3d2d38e0f8 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (11m) 23s ago 24m 509M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (10m) 75s ago 20m 451M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (7m) 23s ago 24m 55.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (6m) 75s ago 20m 41.0M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (10m) 23s ago 22m 8078k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T15:00:53.765 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (9m) 75s ago 20m 7671k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T15:00:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (4m) 23s ago 19m 159M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T15:00:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (4m) 23s ago 19m 107M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T15:00:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (3m) 23s ago 18m 127M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T15:00:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (2m) 75s ago 18m 140M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T15:00:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (2m) 75s ago 17m 100M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T15:00:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (80s) 75s ago 16m 11.2M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 43366d2dc8da 2024-03-13T15:00:53.766 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (9m) 23s ago 21m 68.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T15:00:53.767 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdc8006dfa0 msgr2=0x7fdc80070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.767 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdc8006dfa0 0x7fdc80070460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009c60 tx=0x7fdc8c009260 comp rx=0 tx=0).stop 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 msgr2=0x7fdc9c1a2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c1a2f30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005b40 tx=0x7fdc8800f690 comp rx=0 tx=0).stop 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 shutdown_connections 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdc8006dfa0 0x7fdc80070460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc9c107220 0x7fdc9c1a29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 --2- 172.21.15.176:0/2182934370 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdc9c107bd0 0x7fdc9c1a2f30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.782+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 >> 172.21.15.176:0/2182934370 conn(0x7fdc9c0767e0 msgr2=0x7fdc9c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.783+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 shutdown_connections 2024-03-13T15:00:53.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.783+0000 7fdc9b200700 1 -- 172.21.15.176:0/2182934370 wait complete. 2024-03-13T15:00:53.821 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:53 smithi196 ceph-mon[86193]: pgmap v270: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-13T15:00:53.821 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:53 smithi196 ceph-mon[86193]: reconnect by client.24287 192.168.0.1:0/2547707940 after 0.00299993 2024-03-13T15:00:53.821 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:53 smithi196 ceph-mon[86193]: reconnect by client.14538 192.168.0.1:0/3863864493 after 0.00299993 2024-03-13T15:00:53.821 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:53 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:reconnect 2024-03-13T15:00:53.822 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:53 smithi196 ceph-mon[86193]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:reconnect} 3 up:standby 2024-03-13T15:00:53.921 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 -- 172.21.15.176:0/2501224037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 msgr2=0x7fc05c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 --2- 172.21.15.176:0/2501224037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c1031f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009910 tx=0x7fc04c009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:53.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 -- 172.21.15.176:0/2501224037 shutdown_connections 2024-03-13T15:00:53.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 --2- 172.21.15.176:0/2501224037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc05c1037c0 0x7fc05c107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 --2- 172.21.15.176:0/2501224037 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c1031f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 -- 172.21.15.176:0/2501224037 >> 172.21.15.176:0/2501224037 conn(0x7fc05c0fe6c0 msgr2=0x7fc05c100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:53.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 -- 172.21.15.176:0/2501224037 shutdown_connections 2024-03-13T15:00:53.922 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.936+0000 7fc060c00700 1 -- 172.21.15.176:0/2501224037 wait complete. 2024-03-13T15:00:53.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.937+0000 7fc060c00700 1 Processor -- start 2024-03-13T15:00:53.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.937+0000 7fc060c00700 1 -- start start 2024-03-13T15:00:53.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.937+0000 7fc060c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c19e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.937+0000 7fc060c00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc05c1037c0 0x7fc05c19eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.937+0000 7fc060c00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c19f170 con 0x7fc05c102e10 2024-03-13T15:00:53.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.937+0000 7fc060c00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c198620 con 0x7fc05c1037c0 2024-03-13T15:00:53.924 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c19e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c19e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46790/0 (socket says 172.21.15.176:46790) 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05b400700 1 -- 172.21.15.176:0/1725059636 learned_addr learned my addr 172.21.15.176:0/1725059636 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05aa00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc05c1037c0 0x7fc05c19eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05b400700 1 -- 172.21.15.176:0/1725059636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc05c1037c0 msgr2=0x7fc05c19eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05b400700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc05c1037c0 0x7fc05c19eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05b400700 1 -- 172.21.15.176:0/1725059636 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0095f0 con 0x7fc05c102e10 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.938+0000 7fc05b400700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c19e5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b5c0 tx=0x7fc04c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.939+0000 7fc053e00700 1 -- 172.21.15.176:0/1725059636 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc04c01d070 con 0x7fc05c102e10 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.939+0000 7fc053e00700 1 -- 172.21.15.176:0/1725059636 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc04c005e00 con 0x7fc05c102e10 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.939+0000 7fc053e00700 1 -- 172.21.15.176:0/1725059636 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc04c017400 con 0x7fc05c102e10 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.939+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c1988a0 con 0x7fc05c102e10 2024-03-13T15:00:53.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.939+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c198d90 con 0x7fc05c102e10 2024-03-13T15:00:53.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.940+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c10b1b0 con 0x7fc05c102e10 2024-03-13T15:00:53.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.941+0000 7fc053e00700 1 -- 172.21.15.176:0/1725059636 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc04c00f460 con 0x7fc05c102e10 2024-03-13T15:00:53.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.942+0000 7fc053e00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc04006dfa0 0x7fc040070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:53.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.942+0000 7fc053e00700 1 -- 172.21.15.176:0/1725059636 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6703+0+0 (secure 0 0 0) 0x7fc04c08f3f0 con 0x7fc05c102e10 2024-03-13T15:00:53.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.942+0000 7fc05aa00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc04006dfa0 0x7fc040070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:53.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.943+0000 7fc05aa00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc04006dfa0 0x7fc040070460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc048005950 tx=0x7fc0480058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:53.931 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:53.946+0000 7fc053e00700 1 -- 172.21.15.176:0/1725059636 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc04c05c230 con 0x7fc05c102e10 2024-03-13T15:00:54.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.279+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc05c04f2e0 con 0x7fc05c102e10 2024-03-13T15:00:54.265 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.280+0000 7fc053e00700 1 -- 172.21.15.176:0/1725059636 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fc04c05bdc0 con 0x7fc05c102e10 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T15:00:54.266 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 3 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 18.2.1 (7fe91d5d5842e04be3b4f514d6dd990c54b29c76) reef (stable)": 1, 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 13 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T15:00:54.267 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:00:54.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc04006dfa0 msgr2=0x7fc040070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.269 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc04006dfa0 0x7fc040070460 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc048005950 tx=0x7fc0480058e0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 msgr2=0x7fc05c19e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c19e5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00b5c0 tx=0x7fc04c00be20 comp rx=0 tx=0).stop 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 shutdown_connections 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc04006dfa0 0x7fc040070460 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc05c102e10 0x7fc05c19e5a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 --2- 172.21.15.176:0/1725059636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc05c1037c0 0x7fc05c19eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.284+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 >> 172.21.15.176:0/1725059636 conn(0x7fc05c0fe6c0 msgr2=0x7fc05c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.285+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 shutdown_connections 2024-03-13T15:00:54.270 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.285+0000 7fc060c00700 1 -- 172.21.15.176:0/1725059636 wait complete. 2024-03-13T15:00:54.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 -- 172.21.15.176:0/1187007198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 msgr2=0x7f12841033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 --2- 172.21.15.176:0/1187007198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f12841033e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f127400b0a0 tx=0x7f127400b3b0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 -- 172.21.15.176:0/1187007198 shutdown_connections 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 --2- 172.21.15.176:0/1187007198 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12841039b0 0x7f1284107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 --2- 172.21.15.176:0/1187007198 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f12841033e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 -- 172.21.15.176:0/1187007198 >> 172.21.15.176:0/1187007198 conn(0x7f12840fe8d0 msgr2=0x7f1284100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 -- 172.21.15.176:0/1187007198 shutdown_connections 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.441+0000 7f128a000700 1 -- 172.21.15.176:0/1187007198 wait complete. 2024-03-13T15:00:54.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.442+0000 7f128a000700 1 Processor -- start 2024-03-13T15:00:54.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.442+0000 7f128a000700 1 -- start start 2024-03-13T15:00:54.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f128a000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f128419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:54.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f128a000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12841039b0 0x7f128419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:54.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f128a000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128419f400 con 0x7f1284103000 2024-03-13T15:00:54.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f128a000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12841988b0 con 0x7f12841039b0 2024-03-13T15:00:54.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f1283400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f128419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:54.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f1283400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f128419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46810/0 (socket says 172.21.15.176:46810) 2024-03-13T15:00:54.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f1283400700 1 -- 172.21.15.176:0/2058435593 learned_addr learned my addr 172.21.15.176:0/2058435593 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:54.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.443+0000 7f127aa00700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12841039b0 0x7f128419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f1283400700 1 -- 172.21.15.176:0/2058435593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12841039b0 msgr2=0x7f128419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f1283400700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12841039b0 0x7f128419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f1283400700 1 -- 172.21.15.176:0/2058435593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274009d00 con 0x7f1284103000 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f127aa00700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12841039b0 0x7f128419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f1283400700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f128419e830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f127400b070 tx=0x7f127400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f1280c00700 1 -- 172.21.15.176:0/2058435593 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f127400e040 con 0x7f1284103000 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f1280c00700 1 -- 172.21.15.176:0/2058435593 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1274003de0 con 0x7f1284103000 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f1280c00700 1 -- 172.21.15.176:0/2058435593 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f127401d8e0 con 0x7f1284103000 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1284198b30 con 0x7f1284103000 2024-03-13T15:00:54.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.444+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1284199020 con 0x7f1284103000 2024-03-13T15:00:54.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.446+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128410b3a0 con 0x7f1284103000 2024-03-13T15:00:54.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.447+0000 7f1280c00700 1 -- 172.21.15.176:0/2058435593 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f1274004040 con 0x7f1284103000 2024-03-13T15:00:54.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.447+0000 7f1280c00700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f126806dfa0 0x7f1268070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:54.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.448+0000 7f1280c00700 1 -- 172.21.15.176:0/2058435593 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f127408f590 con 0x7f1284103000 2024-03-13T15:00:54.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.448+0000 7f127aa00700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f126806dfa0 0x7f1268070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:54.433 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.448+0000 7f127aa00700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f126806dfa0 0x7f1268070460 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f126c005950 tx=0x7f126c0058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:54.437 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.452+0000 7f1280c00700 1 -- 172.21.15.176:0/2058435593 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f127405c4f0 con 0x7f1284103000 2024-03-13T15:00:54.715 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.729+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f128410ad80 con 0x7f1284103000 2024-03-13T15:00:54.716 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.731+0000 7f1280c00700 1 -- 172.21.15.176:0/2058435593 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1853 (secure 0 0 0) 0x7f127405c080 con 0x7f1284103000 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:e30 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 30 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T15:00:53.816937+0000 2024-03-13T15:00:54.717 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 111 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=24255} 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T15:00:54.718 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{0:24255} state up:active seq 228 join_fscid=1 addr [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] compat {c=[1],r=[1],i=[7ff]}] 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:00:54.719 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{-1:34494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6828/552037800,v1:172.21.15.176:6829/552037800] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:00:54.720 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{-1:44257} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.196:6824/2671769958,v1:172.21.15.196:6825/2671769958] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:00:54.720 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f126806dfa0 msgr2=0x7f1268070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.720 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f126806dfa0 0x7f1268070460 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f126c005950 tx=0x7f126c0058e0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.720 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 msgr2=0x7f128419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.720 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f128419e830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f127400b070 tx=0x7f127400cdd0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 shutdown_connections 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f126806dfa0 0x7f1268070460 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1284103000 0x7f128419e830 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 --2- 172.21.15.176:0/2058435593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12841039b0 0x7f128419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 >> 172.21.15.176:0/2058435593 conn(0x7f12840fe8d0 msgr2=0x7f12840ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 shutdown_connections 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.734+0000 7f128a000700 1 -- 172.21.15.176:0/2058435593 wait complete. 2024-03-13T15:00:54.721 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 30 2024-03-13T15:00:54.798 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:54 smithi196 ceph-mon[86193]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:54.799 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:54 smithi196 ceph-mon[86193]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:54.799 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:54 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:rejoin 2024-03-13T15:00:54.799 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:54 smithi196 ceph-mon[86193]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:rejoin} 3 up:standby 2024-03-13T15:00:54.799 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:54 smithi196 ceph-mon[86193]: daemon mds.cephfs.smithi196.wabhcu is now active in filesystem cephfs as rank 0 2024-03-13T15:00:54.799 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:54 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1725059636' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:54.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:54 smithi176 ceph-mon[112742]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:54.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:54 smithi176 ceph-mon[112742]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:54.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:54 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:rejoin 2024-03-13T15:00:54.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:54 smithi176 ceph-mon[112742]: fsmap cephfs:1/1 {0=cephfs.smithi196.wabhcu=up:rejoin} 3 up:standby 2024-03-13T15:00:54.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:54 smithi176 ceph-mon[112742]: daemon mds.cephfs.smithi196.wabhcu is now active in filesystem cephfs as rank 0 2024-03-13T15:00:54.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:54 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1725059636' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:00:54.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 -- 172.21.15.176:0/247905728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 msgr2=0x7f4ae81057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 --2- 172.21.15.176:0/247905728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 0x7f4ae81057a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009910 tx=0x7f4ad4009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:54.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 -- 172.21.15.176:0/247905728 shutdown_connections 2024-03-13T15:00:54.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 --2- 172.21.15.176:0/247905728 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 0x7f4ae81057a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 --2- 172.21.15.176:0/247905728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ae8100da0 0x7f4ae8101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 -- 172.21.15.176:0/247905728 >> 172.21.15.176:0/247905728 conn(0x7f4ae80fc650 msgr2=0x7f4ae80fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:54.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 -- 172.21.15.176:0/247905728 shutdown_connections 2024-03-13T15:00:54.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.893+0000 7f4aece00700 1 -- 172.21.15.176:0/247905728 wait complete. 2024-03-13T15:00:54.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.894+0000 7f4aece00700 1 Processor -- start 2024-03-13T15:00:54.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.895+0000 7f4aece00700 1 -- start start 2024-03-13T15:00:54.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.895+0000 7f4aece00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ae8100da0 0x7f4ae819c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:54.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.895+0000 7f4aece00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 0x7f4ae819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:54.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.895+0000 7f4aece00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae819d0e0 con 0x7f4ae8101750 2024-03-13T15:00:54.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.895+0000 7f4aece00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae8196590 con 0x7f4ae8100da0 2024-03-13T15:00:54.883 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.895+0000 7f4ae5400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 0x7f4ae819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:54.883 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.895+0000 7f4ae5e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ae8100da0 0x7f4ae819c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:54.883 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.896+0000 7f4ae5e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ae8100da0 0x7f4ae819c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:47046/0 (socket says 172.21.15.176:47046) 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.896+0000 7f4ae5e00700 1 -- 172.21.15.176:0/4082280000 learned_addr learned my addr 172.21.15.176:0/4082280000 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.896+0000 7f4ae5400700 1 -- 172.21.15.176:0/4082280000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ae8100da0 msgr2=0x7f4ae819c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.896+0000 7f4ae5400700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ae8100da0 0x7f4ae819c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.896+0000 7f4ae5400700 1 -- 172.21.15.176:0/4082280000 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad40095f0 con 0x7f4ae8101750 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.896+0000 7f4ae5400700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 0x7f4ae819ca50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.897+0000 7f4adea00700 1 -- 172.21.15.176:0/4082280000 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad401d070 con 0x7f4ae8101750 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.897+0000 7f4adea00700 1 -- 172.21.15.176:0/4082280000 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ad400bed0 con 0x7f4ae8101750 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.897+0000 7f4adea00700 1 -- 172.21.15.176:0/4082280000 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ad4003e10 con 0x7f4ae8101750 2024-03-13T15:00:54.884 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.897+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae8196810 con 0x7f4ae8101750 2024-03-13T15:00:54.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.897+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae8196c80 con 0x7f4ae8101750 2024-03-13T15:00:54.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.898+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae81091a0 con 0x7f4ae8101750 2024-03-13T15:00:54.885 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.900+0000 7f4adea00700 1 -- 172.21.15.176:0/4082280000 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4ad40050d0 con 0x7f4ae8101750 2024-03-13T15:00:54.886 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.901+0000 7f4adea00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4acc06dfa0 0x7f4acc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:54.886 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.901+0000 7f4adea00700 1 -- 172.21.15.176:0/4082280000 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f4ad408f340 con 0x7f4ae8101750 2024-03-13T15:00:54.886 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.901+0000 7f4ae5e00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4acc06dfa0 0x7f4acc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:54.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.902+0000 7f4ae5e00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4acc06dfa0 0x7f4acc070460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009d20 tx=0x7f4ad8009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:54.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:54.904+0000 7f4adea00700 1 -- 172.21.15.176:0/4082280000 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4ad405c280 con 0x7f4ae8101750 2024-03-13T15:00:55.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.146+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ae8196f60 con 0x7f4acc06dfa0 2024-03-13T15:00:55.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.149+0000 7f4adea00700 1 -- 172.21.15.176:0/4082280000 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4ae8196f60 con 0x7f4acc06dfa0 2024-03-13T15:00:55.134 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "crash", 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "osd", 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr" 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "14/23 daemons upgraded", 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mds daemons", 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T15:00:55.135 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:00:55.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4acc06dfa0 msgr2=0x7f4acc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:55.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4acc06dfa0 0x7f4acc070460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009d20 tx=0x7f4ad8009260 comp rx=0 tx=0).stop 2024-03-13T15:00:55.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 msgr2=0x7f4ae819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:55.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 0x7f4ae819ca50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006010 tx=0x7f4ad400f690 comp rx=0 tx=0).stop 2024-03-13T15:00:55.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 shutdown_connections 2024-03-13T15:00:55.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4acc06dfa0 0x7f4acc070460 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ae8100da0 0x7f4ae819c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 --2- 172.21.15.176:0/4082280000 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4ae8101750 0x7f4ae819ca50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 >> 172.21.15.176:0/4082280000 conn(0x7f4ae80fc650 msgr2=0x7f4ae80fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:55.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.152+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 shutdown_connections 2024-03-13T15:00:55.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.153+0000 7f4aece00700 1 -- 172.21.15.176:0/4082280000 wait complete. 2024-03-13T15:00:55.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.332+0000 7f070a200700 1 -- 172.21.15.176:0/1530468614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f070410d280 msgr2=0x7f070410d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:55.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.332+0000 7f070a200700 1 --2- 172.21.15.176:0/1530468614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f070410d280 0x7f070410d660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009910 tx=0x7f06f4009c20 comp rx=0 tx=0).stop 2024-03-13T15:00:55.318 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.332+0000 7f070a200700 1 -- 172.21.15.176:0/1530468614 shutdown_connections 2024-03-13T15:00:55.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.332+0000 7f070a200700 1 --2- 172.21.15.176:0/1530468614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07041071d0 0x7f0704107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.332+0000 7f070a200700 1 --2- 172.21.15.176:0/1530468614 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f070410d280 0x7f070410d660 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.332+0000 7f070a200700 1 -- 172.21.15.176:0/1530468614 >> 172.21.15.176:0/1530468614 conn(0x7f07040769e0 msgr2=0x7f0704078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:55.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.333+0000 7f070a200700 1 -- 172.21.15.176:0/1530468614 shutdown_connections 2024-03-13T15:00:55.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.333+0000 7f070a200700 1 -- 172.21.15.176:0/1530468614 wait complete. 2024-03-13T15:00:55.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f070a200700 1 Processor -- start 2024-03-13T15:00:55.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f070a200700 1 -- start start 2024-03-13T15:00:55.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f070a200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07041071d0 0x7f070419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:55.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f070a200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070410d280 0x7f070419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:55.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f070a200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070419d8a0 con 0x7f07041071d0 2024-03-13T15:00:55.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f070a200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07041a05d0 con 0x7f070410d280 2024-03-13T15:00:55.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f0703400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07041071d0 0x7f070419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f0703400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07041071d0 0x7f070419ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:46862/0 (socket says 172.21.15.176:46862) 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.334+0000 7f0703400700 1 -- 172.21.15.176:0/1363228459 learned_addr learned my addr 172.21.15.176:0/1363228459 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f0702a00700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070410d280 0x7f070419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f0703400700 1 -- 172.21.15.176:0/1363228459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070410d280 msgr2=0x7f070419d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f0703400700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070410d280 0x7f070419d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f0703400700 1 -- 172.21.15.176:0/1363228459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f40095f0 con 0x7f07041071d0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f0703400700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07041071d0 0x7f070419ccd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06f4006010 tx=0x7f06f400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f06fbe00700 1 -- 172.21.15.176:0/1363228459 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f401d070 con 0x7f07041071d0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f06fbe00700 1 -- 172.21.15.176:0/1363228459 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06f4004fb0 con 0x7f07041071d0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07041a0850 con 0x7f07041071d0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.335+0000 7f06fbe00700 1 -- 172.21.15.176:0/1363228459 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06f400f690 con 0x7f07041071d0 2024-03-13T15:00:55.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.336+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07041a0dc0 con 0x7f07041071d0 2024-03-13T15:00:55.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.337+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070404ea90 con 0x7f07041071d0 2024-03-13T15:00:55.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.338+0000 7f06fbe00700 1 -- 172.21.15.176:0/1363228459 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f06f40228a0 con 0x7f07041071d0 2024-03-13T15:00:55.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.339+0000 7f06fbe00700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f06e806dfa0 0x7f06e8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:00:55.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.339+0000 7f06fbe00700 1 -- 172.21.15.176:0/1363228459 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6703+0+0 (secure 0 0 0) 0x7f06f408f5f0 con 0x7f07041071d0 2024-03-13T15:00:55.324 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.339+0000 7f0702a00700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f06e806dfa0 0x7f06e8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:00:55.325 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.340+0000 7f0702a00700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f06e806dfa0 0x7f06e8070460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005fd0 tx=0x7f06f000a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:00:55.328 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.343+0000 7f06fbe00700 1 -- 172.21.15.176:0/1363228459 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f06f4058a20 con 0x7f07041071d0 2024-03-13T15:00:55.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.658+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f070419dff0 con 0x7f07041071d0 2024-03-13T15:00:55.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.659+0000 7f06fbe00700 1 -- 172.21.15.176:0/1363228459 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f06f4027020 con 0x7f07041071d0 2024-03-13T15:00:55.644 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T15:00:55.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f06e806dfa0 msgr2=0x7f06e8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:55.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f06e806dfa0 0x7f06e8070460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f06f0005fd0 tx=0x7f06f000a560 comp rx=0 tx=0).stop 2024-03-13T15:00:55.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07041071d0 msgr2=0x7f070419ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:00:55.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07041071d0 0x7f070419ccd0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06f4006010 tx=0x7f06f400be20 comp rx=0 tx=0).stop 2024-03-13T15:00:55.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 shutdown_connections 2024-03-13T15:00:55.648 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f06e806dfa0 0x7f06e8070460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f07041071d0 0x7f070419ccd0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 --2- 172.21.15.176:0/1363228459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f070410d280 0x7f070419d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:00:55.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.662+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 >> 172.21.15.176:0/1363228459 conn(0x7f07040769e0 msgr2=0x7f0704078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:00:55.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.663+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 shutdown_connections 2024-03-13T15:00:55.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:00:55.663+0000 7f070a200700 1 -- 172.21.15.176:0/1363228459 wait complete. 2024-03-13T15:00:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:55 smithi176 ceph-mon[112742]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:55 smithi176 ceph-mon[112742]: pgmap v271: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-13T15:00:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:55 smithi176 ceph-mon[112742]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-13T15:00:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:55 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T15:00:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:55 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:active 2024-03-13T15:00:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:55 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi196.wabhcu=up:active} 3 up:standby 2024-03-13T15:00:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:55 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2058435593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:00:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:55 smithi196 ceph-mon[86193]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:55 smithi196 ceph-mon[86193]: pgmap v271: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-03-13T15:00:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:55 smithi196 ceph-mon[86193]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-13T15:00:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:55 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T15:00:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:55 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.196:6826/1275358571,v1:172.21.15.196:6827/1275358571] up:active 2024-03-13T15:00:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:55 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi196.wabhcu=up:active} 3 up:standby 2024-03-13T15:00:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:55 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2058435593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:00:56.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:56 smithi196 ceph-mon[86193]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:56.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:56 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1363228459' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:00:56.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:56 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:56.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:56 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:56.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:56 smithi176 ceph-mon[112742]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:00:56.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:56 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1363228459' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:00:56.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:56 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:56.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:56 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:57.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:57 smithi176 ceph-mon[112742]: pgmap v272: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-13T15:00:57.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:57 smithi196 ceph-mon[86193]: pgmap v272: 65 pgs: 65 active+clean; 204 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-13T15:00:59.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:59 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:59.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:59 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:59.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:00:59 smithi176 ceph-mon[112742]: pgmap v273: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-13T15:00:59.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:59 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:59.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:59 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:00:59.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:00:59 smithi196 ceph-mon[86193]: pgmap v273: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 4 op/s 2024-03-13T15:01:01.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:01.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:01.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:01 smithi196 ceph-mon[86193]: pgmap v274: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-03-13T15:01:01.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:01.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:01.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:01 smithi176 ceph-mon[112742]: pgmap v274: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-03-13T15:01:03.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:03 smithi196 ceph-mon[86193]: pgmap v275: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:03.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:03.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:03.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:03.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:01:03.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:03 smithi176 ceph-mon[112742]: pgmap v275: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:01:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:04.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:04.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:04.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:04.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:04.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:04 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:04.887 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:04.887 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:04.887 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:04.887 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:04.887 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:04 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:05.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:05 smithi176 ceph-mon[112742]: pgmap v276: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:05.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:05 smithi196 ceph-mon[86193]: pgmap v276: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:07.313 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:07 smithi196 ceph-mon[86193]: Upgrade: Updating mds.cephfs.smithi196.wabhcu 2024-03-13T15:01:07.314 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:07.314 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T15:01:07.314 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:07.314 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:07 smithi196 ceph-mon[86193]: Deploying daemon mds.cephfs.smithi196.wabhcu on smithi196 2024-03-13T15:01:07.314 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:07 smithi196 ceph-mon[86193]: pgmap v277: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:07.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:07 smithi176 ceph-mon[112742]: Upgrade: Updating mds.cephfs.smithi196.wabhcu 2024-03-13T15:01:07.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:07.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi196.wabhcu", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-13T15:01:07.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:07.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:07 smithi176 ceph-mon[112742]: Deploying daemon mds.cephfs.smithi196.wabhcu on smithi196 2024-03-13T15:01:07.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:07 smithi176 ceph-mon[112742]: pgmap v277: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 74 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:09.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:08 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[112738]: 2024-03-13T15:01:08.960+0000 7f67a1400700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: pgmap v278: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: osdmap e112: 6 total, 6 up, 6 in 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: Standby daemon mds.cephfs.smithi176.ldskqt assigned to filesystem cephfs as rank 0 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-13T15:01:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:09 smithi176 ceph-mon[112742]: fsmap cephfs:1/1 {0=cephfs.smithi176.ldskqt=up:replay} 2 up:standby 2024-03-13T15:01:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: pgmap v278: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-03-13T15:01:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-13T15:01:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-13T15:01:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: osdmap e112: 6 total, 6 up, 6 in 2024-03-13T15:01:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: Standby daemon mds.cephfs.smithi176.ldskqt assigned to filesystem cephfs as rank 0 2024-03-13T15:01:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-13T15:01:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-03-13T15:01:09.927 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:09 smithi196 ceph-mon[86193]: fsmap cephfs:1/1 {0=cephfs.smithi176.ldskqt=up:replay} 2 up:standby 2024-03-13T15:01:11.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:11 smithi176 ceph-mon[112742]: pgmap v280: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-03-13T15:01:11.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T15:01:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:11 smithi196 ceph-mon[86193]: pgmap v280: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 1 op/s 2024-03-13T15:01:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T15:01:13.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:13 smithi176 ceph-mon[112742]: pgmap v281: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-03-13T15:01:13.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:13 smithi196 ceph-mon[86193]: pgmap v281: 65 pgs: 65 active+clean; 204 MiB data, 866 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s rd, 1 op/s 2024-03-13T15:01:15.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:15 smithi196 ceph-mon[86193]: pgmap v282: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-03-13T15:01:15.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:15.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:15.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:15 smithi176 ceph-mon[112742]: pgmap v282: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 10 MiB/s rd, 3 op/s 2024-03-13T15:01:15.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:15.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:17.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:17 smithi176 ceph-mon[112742]: pgmap v283: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-13T15:01:17.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:17 smithi196 ceph-mon[86193]: pgmap v283: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-13T15:01:19.563 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:19 smithi196 ceph-mon[86193]: pgmap v284: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-13T15:01:19.564 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:19 smithi196 ceph-mon[86193]: reconnect by client.24287 192.168.0.1:0/2547707940 after 0.00299997 2024-03-13T15:01:19.564 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:19 smithi196 ceph-mon[86193]: reconnect by client.14538 192.168.0.1:0/3863864493 after 0.00399997 2024-03-13T15:01:19.564 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:19 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:reconnect 2024-03-13T15:01:19.564 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:19 smithi196 ceph-mon[86193]: fsmap cephfs:1/1 {0=cephfs.smithi176.ldskqt=up:reconnect} 2 up:standby 2024-03-13T15:01:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:19 smithi176 ceph-mon[112742]: pgmap v284: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-13T15:01:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:19 smithi176 ceph-mon[112742]: reconnect by client.24287 192.168.0.1:0/2547707940 after 0.00299997 2024-03-13T15:01:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:19 smithi176 ceph-mon[112742]: reconnect by client.14538 192.168.0.1:0/3863864493 after 0.00399997 2024-03-13T15:01:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:19 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:reconnect 2024-03-13T15:01:19.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:19 smithi176 ceph-mon[112742]: fsmap cephfs:1/1 {0=cephfs.smithi176.ldskqt=up:reconnect} 2 up:standby 2024-03-13T15:01:20.675 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:20 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:rejoin 2024-03-13T15:01:20.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:20 smithi196 ceph-mon[86193]: fsmap cephfs:1/1 {0=cephfs.smithi176.ldskqt=up:rejoin} 2 up:standby 2024-03-13T15:01:20.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:20 smithi196 ceph-mon[86193]: daemon mds.cephfs.smithi176.ldskqt is now active in filesystem cephfs as rank 0 2024-03-13T15:01:20.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:20 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:rejoin 2024-03-13T15:01:20.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:20 smithi176 ceph-mon[112742]: fsmap cephfs:1/1 {0=cephfs.smithi176.ldskqt=up:rejoin} 2 up:standby 2024-03-13T15:01:20.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:20 smithi176 ceph-mon[112742]: daemon mds.cephfs.smithi176.ldskqt is now active in filesystem cephfs as rank 0 2024-03-13T15:01:21.832 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:21 smithi196 ceph-mon[86193]: pgmap v285: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-13T15:01:21.832 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:21 smithi196 ceph-mon[86193]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-13T15:01:21.832 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:21 smithi196 ceph-mon[86193]: Cluster is now healthy 2024-03-13T15:01:21.832 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:21 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:active 2024-03-13T15:01:21.832 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:21 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2 up:standby 2024-03-13T15:01:21.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:21 smithi176 ceph-mon[112742]: pgmap v285: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-03-13T15:01:21.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:21 smithi176 ceph-mon[112742]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-13T15:01:21.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:21 smithi176 ceph-mon[112742]: Cluster is now healthy 2024-03-13T15:01:21.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:21 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] up:active 2024-03-13T15:01:21.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:21 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 2 up:standby 2024-03-13T15:01:23.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:23.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:23.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:23.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:23 smithi196 ceph-mon[86193]: pgmap v286: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-03-13T15:01:23.615 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:23 smithi196 ceph-mon[86193]: mds.? [v2:172.21.15.196:6826/2135805809,v1:172.21.15.196:6827/2135805809] up:boot 2024-03-13T15:01:23.616 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:23 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 3 up:standby 2024-03-13T15:01:23.616 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:23 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T15:01:23.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:23.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:23.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:23.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:23 smithi176 ceph-mon[112742]: pgmap v286: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-03-13T15:01:23.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:23 smithi176 ceph-mon[112742]: mds.? [v2:172.21.15.196:6826/2135805809,v1:172.21.15.196:6827/2135805809] up:boot 2024-03-13T15:01:23.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:23 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 3 up:standby 2024-03-13T15:01:23.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:23 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T15:01:25.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:25 smithi196 ceph-mon[86193]: pgmap v287: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 0 B/s wr, 6 op/s 2024-03-13T15:01:25.820 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:25 smithi176 ceph-mon[112742]: pgmap v287: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 0 B/s wr, 6 op/s 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 -- 172.21.15.176:0/1741505840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c1017c0 msgr2=0x7f394c105810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 --2- 172.21.15.176:0/1741505840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c1017c0 0x7f394c105810 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3938009910 tx=0x7f3938009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 -- 172.21.15.176:0/1741505840 shutdown_connections 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 --2- 172.21.15.176:0/1741505840 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c1017c0 0x7f394c105810 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 --2- 172.21.15.176:0/1741505840 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f394c100e10 0x7f394c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 -- 172.21.15.176:0/1741505840 >> 172.21.15.176:0/1741505840 conn(0x7f394c0fc6c0 msgr2=0x7f394c0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 -- 172.21.15.176:0/1741505840 shutdown_connections 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.834+0000 7f3953600700 1 -- 172.21.15.176:0/1741505840 wait complete. 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.835+0000 7f3953600700 1 Processor -- start 2024-03-13T15:01:25.821 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.836+0000 7f3953600700 1 -- start start 2024-03-13T15:01:25.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.836+0000 7f3953600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c100e10 0x7f394c071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:25.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.836+0000 7f3953600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f394c1017c0 0x7f394c072310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:25.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.836+0000 7f3953600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c074f50 con 0x7f394c100e10 2024-03-13T15:01:25.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.836+0000 7f3953600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c072850 con 0x7f394c1017c0 2024-03-13T15:01:25.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f3950c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c100e10 0x7f394c071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:25.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f3950c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c100e10 0x7f394c071dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39662/0 (socket says 172.21.15.176:39662) 2024-03-13T15:01:25.823 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f3950c00700 1 -- 172.21.15.176:0/1630493023 learned_addr learned my addr 172.21.15.176:0/1630493023 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f394be00700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f394c1017c0 0x7f394c072310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f3950c00700 1 -- 172.21.15.176:0/1630493023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f394c1017c0 msgr2=0x7f394c072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f3950c00700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f394c1017c0 0x7f394c072310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f3950c00700 1 -- 172.21.15.176:0/1630493023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39380095f0 con 0x7f394c100e10 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.837+0000 7f3950c00700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c100e10 0x7f394c071dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f394000e920 tx=0x7f394000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.838+0000 7f3949600700 1 -- 172.21.15.176:0/1630493023 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f394000c9d0 con 0x7f394c100e10 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.838+0000 7f3949600700 1 -- 172.21.15.176:0/1630493023 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3940010430 con 0x7f394c100e10 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.838+0000 7f3949600700 1 -- 172.21.15.176:0/1630493023 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3940018630 con 0x7f394c100e10 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.838+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c072b30 con 0x7f394c100e10 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.838+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c1ab650 con 0x7f394c100e10 2024-03-13T15:01:25.824 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.839+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c109210 con 0x7f394c100e10 2024-03-13T15:01:25.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.840+0000 7f3949600700 1 -- 172.21.15.176:0/1630493023 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f39400105a0 con 0x7f394c100e10 2024-03-13T15:01:25.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.841+0000 7f3949600700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f393406dfa0 0x7f3934070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:25.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.841+0000 7f3949600700 1 -- 172.21.15.176:0/1630493023 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f3940014070 con 0x7f394c100e10 2024-03-13T15:01:25.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.841+0000 7f394be00700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f393406dfa0 0x7f3934070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:25.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.842+0000 7f394be00700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f393406dfa0 0x7f3934070460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3938005b40 tx=0x7f3938005a90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:25.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:25.846+0000 7f3949600700 1 -- 172.21.15.176:0/1630493023 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f39400531e0 con 0x7f394c100e10 2024-03-13T15:01:26.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.084+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f394c02ccc0 con 0x7f393406dfa0 2024-03-13T15:01:26.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.086+0000 7f3949600700 1 -- 172.21.15.176:0/1630493023 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f394c02ccc0 con 0x7f393406dfa0 2024-03-13T15:01:26.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f393406dfa0 msgr2=0x7f3934070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f393406dfa0 0x7f3934070460 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3938005b40 tx=0x7f3938005a90 comp rx=0 tx=0).stop 2024-03-13T15:01:26.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c100e10 msgr2=0x7f394c071dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c100e10 0x7f394c071dd0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f394000e920 tx=0x7f394000ec30 comp rx=0 tx=0).stop 2024-03-13T15:01:26.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 shutdown_connections 2024-03-13T15:01:26.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f393406dfa0 0x7f3934070460 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f394c100e10 0x7f394c071dd0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 --2- 172.21.15.176:0/1630493023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f394c1017c0 0x7f394c072310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.089+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 >> 172.21.15.176:0/1630493023 conn(0x7f394c0fc6c0 msgr2=0x7f394c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:26.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.090+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 shutdown_connections 2024-03-13T15:01:26.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.090+0000 7f3953600700 1 -- 172.21.15.176:0/1630493023 wait complete. 2024-03-13T15:01:26.092 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T15:01:26.230 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.244+0000 7f0531200700 1 -- 172.21.15.176:0/2202450606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c10d280 msgr2=0x7f052c10d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.230 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.244+0000 7f0531200700 1 --2- 172.21.15.176:0/2202450606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c10d280 0x7f052c10d660 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f051c009910 tx=0x7f051c009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:26.230 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.245+0000 7f0531200700 1 -- 172.21.15.176:0/2202450606 shutdown_connections 2024-03-13T15:01:26.231 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.245+0000 7f0531200700 1 --2- 172.21.15.176:0/2202450606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f052c1071d0 0x7f052c107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.231 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.245+0000 7f0531200700 1 --2- 172.21.15.176:0/2202450606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c10d280 0x7f052c10d660 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.231 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.245+0000 7f0531200700 1 -- 172.21.15.176:0/2202450606 >> 172.21.15.176:0/2202450606 conn(0x7f052c076990 msgr2=0x7f052c078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:26.231 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.245+0000 7f0531200700 1 -- 172.21.15.176:0/2202450606 shutdown_connections 2024-03-13T15:01:26.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.245+0000 7f0531200700 1 -- 172.21.15.176:0/2202450606 wait complete. 2024-03-13T15:01:26.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.246+0000 7f0531200700 1 Processor -- start 2024-03-13T15:01:26.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.246+0000 7f0531200700 1 -- start start 2024-03-13T15:01:26.232 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.246+0000 7f0531200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c1071d0 0x7f052c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:26.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.246+0000 7f0531200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f052c10d280 0x7f052c19d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:26.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.246+0000 7f0531200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c19d890 con 0x7f052c1071d0 2024-03-13T15:01:26.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.246+0000 7f0531200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052c1a05c0 con 0x7f052c10d280 2024-03-13T15:01:26.234 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.246+0000 7f0529e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c1071d0 0x7f052c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0529e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c1071d0 0x7f052c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39680/0 (socket says 172.21.15.176:39680) 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0529e00700 1 -- 172.21.15.176:0/2693085240 learned_addr learned my addr 172.21.15.176:0/2693085240 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0529400700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f052c10d280 0x7f052c19d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0529e00700 1 -- 172.21.15.176:0/2693085240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f052c10d280 msgr2=0x7f052c19d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0529e00700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f052c10d280 0x7f052c19d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0529e00700 1 -- 172.21.15.176:0/2693085240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c0095f0 con 0x7f052c1071d0 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0529e00700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c1071d0 0x7f052c19ccc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f051c0092e0 tx=0x7f051c00f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0522a00700 1 -- 172.21.15.176:0/2693085240 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f051c01d070 con 0x7f052c1071d0 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0522a00700 1 -- 172.21.15.176:0/2693085240 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f051c0052a0 con 0x7f052c1071d0 2024-03-13T15:01:26.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0522a00700 1 -- 172.21.15.176:0/2693085240 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f051c017630 con 0x7f052c1071d0 2024-03-13T15:01:26.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.247+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052c1a0840 con 0x7f052c1071d0 2024-03-13T15:01:26.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.248+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052c1a0db0 con 0x7f052c1071d0 2024-03-13T15:01:26.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.249+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052c04ea90 con 0x7f052c1071d0 2024-03-13T15:01:26.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.250+0000 7f0522a00700 1 -- 172.21.15.176:0/2693085240 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f051c0178f0 con 0x7f052c1071d0 2024-03-13T15:01:26.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.251+0000 7f0522a00700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f051006dfa0 0x7f0510070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:26.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.251+0000 7f0522a00700 1 -- 172.21.15.176:0/2693085240 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f051c08f2f0 con 0x7f052c1071d0 2024-03-13T15:01:26.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.251+0000 7f0529400700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f051006dfa0 0x7f0510070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:26.237 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.252+0000 7f0529400700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f051006dfa0 0x7f0510070460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f052c19e2a0 tx=0x7f0518009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:26.241 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.256+0000 7f0522a00700 1 -- 172.21.15.176:0/2693085240 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f051c0585a0 con 0x7f052c1071d0 2024-03-13T15:01:26.481 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.496+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f052c1a1140 con 0x7f051006dfa0 2024-03-13T15:01:26.482 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.498+0000 7f0522a00700 1 -- 172.21.15.176:0/2693085240 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f052c1a1140 con 0x7f051006dfa0 2024-03-13T15:01:26.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f051006dfa0 msgr2=0x7f0510070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f051006dfa0 0x7f0510070460 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f052c19e2a0 tx=0x7f0518009260 comp rx=0 tx=0).stop 2024-03-13T15:01:26.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c1071d0 msgr2=0x7f052c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c1071d0 0x7f052c19ccc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f051c0092e0 tx=0x7f051c00f460 comp rx=0 tx=0).stop 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 shutdown_connections 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f051006dfa0 0x7f0510070460 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f052c1071d0 0x7f052c19ccc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 --2- 172.21.15.176:0/2693085240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f052c10d280 0x7f052c19d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.500+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 >> 172.21.15.176:0/2693085240 conn(0x7f052c076990 msgr2=0x7f052c078310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.501+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 shutdown_connections 2024-03-13T15:01:26.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.501+0000 7f0531200700 1 -- 172.21.15.176:0/2693085240 wait complete. 2024-03-13T15:01:26.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 -- 172.21.15.176:0/2260375059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde4010d280 msgr2=0x7fde4010d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 --2- 172.21.15.176:0/2260375059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde4010d280 0x7fde4010d660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde30009910 tx=0x7fde30009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:26.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 -- 172.21.15.176:0/2260375059 shutdown_connections 2024-03-13T15:01:26.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 --2- 172.21.15.176:0/2260375059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde401071d0 0x7fde40107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 --2- 172.21.15.176:0/2260375059 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde4010d280 0x7fde4010d660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 -- 172.21.15.176:0/2260375059 >> 172.21.15.176:0/2260375059 conn(0x7fde400769e0 msgr2=0x7fde40078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:26.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 -- 172.21.15.176:0/2260375059 shutdown_connections 2024-03-13T15:01:26.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.657+0000 7fde45600700 1 -- 172.21.15.176:0/2260375059 wait complete. 2024-03-13T15:01:26.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.658+0000 7fde45600700 1 Processor -- start 2024-03-13T15:01:26.643 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.658+0000 7fde45600700 1 -- start start 2024-03-13T15:01:26.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde45600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde401071d0 0x7fde4019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:26.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde45600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde4010d280 0x7fde4019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:26.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde45600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4019d8a0 con 0x7fde401071d0 2024-03-13T15:01:26.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde45600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde401a05d0 con 0x7fde4010d280 2024-03-13T15:01:26.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde3ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde401071d0 0x7fde4019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:26.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde3ea00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde401071d0 0x7fde4019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39690/0 (socket says 172.21.15.176:39690) 2024-03-13T15:01:26.644 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde3ea00700 1 -- 172.21.15.176:0/515519956 learned_addr learned my addr 172.21.15.176:0/515519956 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.659+0000 7fde3e000700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde4010d280 0x7fde4019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde3ea00700 1 -- 172.21.15.176:0/515519956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde4010d280 msgr2=0x7fde4019d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde3ea00700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde4010d280 0x7fde4019d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde3ea00700 1 -- 172.21.15.176:0/515519956 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde300095f0 con 0x7fde401071d0 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde3ea00700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde401071d0 0x7fde4019ccd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde3000b5c0 tx=0x7fde3000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde37400700 1 -- 172.21.15.176:0/515519956 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde3001d070 con 0x7fde401071d0 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde37400700 1 -- 172.21.15.176:0/515519956 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde3000f460 con 0x7fde401071d0 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde37400700 1 -- 172.21.15.176:0/515519956 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde30017610 con 0x7fde401071d0 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.660+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde401a0850 con 0x7fde401071d0 2024-03-13T15:01:26.646 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.661+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde401a0dc0 con 0x7fde401071d0 2024-03-13T15:01:26.647 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.662+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4004ea90 con 0x7fde401071d0 2024-03-13T15:01:26.649 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.664+0000 7fde37400700 1 -- 172.21.15.176:0/515519956 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fde3000f5d0 con 0x7fde401071d0 2024-03-13T15:01:26.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.665+0000 7fde37400700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fde2406dfa0 0x7fde24070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:26.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.665+0000 7fde37400700 1 -- 172.21.15.176:0/515519956 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fde3008f5f0 con 0x7fde401071d0 2024-03-13T15:01:26.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.665+0000 7fde3e000700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fde2406dfa0 0x7fde24070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:26.650 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.666+0000 7fde3e000700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fde2406dfa0 0x7fde24070460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fde2c005fd0 tx=0x7fde2c00a560 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:26.653 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.668+0000 7fde37400700 1 -- 172.21.15.176:0/515519956 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fde300589a0 con 0x7fde401071d0 2024-03-13T15:01:26.897 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.912+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde401a1220 con 0x7fde2406dfa0 2024-03-13T15:01:26.910 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.925+0000 7fde37400700 1 -- 172.21.15.176:0/515519956 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fde401a1220 con 0x7fde2406dfa0 2024-03-13T15:01:26.910 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T15:01:26.910 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (9m) 56s ago 22m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T15:01:26.910 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (23m) 56s ago 23m 9261k - 18.2.1 5be31c24972a 7d20640e307c 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (21m) 0s ago 21m 9227k - 18.2.1 5be31c24972a c31d1d1ffeb8 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (5m) 56s ago 23m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (5m) 0s ago 21m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (8m) 56s ago 22m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (64s) 56s ago 15m 11.1M - 19.0.0-1916-g9a75c81f 9552dbf7f115 edc682c345ac 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (89s) 56s ago 15m 11.9M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5b6d2dc0d458 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (35s) 0s ago 15m 11.9M - 19.0.0-1916-g9a75c81f 9552dbf7f115 a9081f4c2ead 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (5s) 0s ago 15m 10.8M - 19.0.0-1916-g9a75c81f 9552dbf7f115 2e3acf7133a0 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (12m) 56s ago 24m 509M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (11m) 0s ago 21m 452M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (7m) 56s ago 24m 55.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (7m) 0s ago 21m 45.2M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (10m) 56s ago 23m 8078k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (10m) 0s ago 21m 7796k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T15:01:26.911 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (5m) 56s ago 20m 159M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T15:01:26.912 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (4m) 56s ago 19m 107M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T15:01:26.912 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (4m) 56s ago 19m 127M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T15:01:26.912 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (3m) 0s ago 18m 136M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T15:01:26.912 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (2m) 0s ago 18m 104M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T15:01:26.912 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (113s) 0s ago 17m 102M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 43366d2dc8da 2024-03-13T15:01:26.912 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (9m) 56s ago 22m 68.5M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T15:01:26.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.927+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fde2406dfa0 msgr2=0x7fde24070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.927+0000 7fde45600700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fde2406dfa0 0x7fde24070460 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fde2c005fd0 tx=0x7fde2c00a560 comp rx=0 tx=0).stop 2024-03-13T15:01:26.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.927+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde401071d0 msgr2=0x7fde4019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:26.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde401071d0 0x7fde4019ccd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fde3000b5c0 tx=0x7fde3000be00 comp rx=0 tx=0).stop 2024-03-13T15:01:26.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 shutdown_connections 2024-03-13T15:01:26.913 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fde2406dfa0 0x7fde24070460 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.914 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fde401071d0 0x7fde4019ccd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.914 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 --2- 172.21.15.176:0/515519956 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde4010d280 0x7fde4019d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:26.914 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 >> 172.21.15.176:0/515519956 conn(0x7fde400769e0 msgr2=0x7fde40078320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:26.914 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 shutdown_connections 2024-03-13T15:01:26.914 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:26.928+0000 7fde45600700 1 -- 172.21.15.176:0/515519956 wait complete. 2024-03-13T15:01:27.066 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 -- 172.21.15.176:0/1873306993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704069020 msgr2=0x7ff704105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1873306993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704069020 0x7ff704105790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009910 tx=0x7ff6f0009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:27.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 -- 172.21.15.176:0/1873306993 shutdown_connections 2024-03-13T15:01:27.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1873306993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704069020 0x7ff704105790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1873306993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704068670 0x7ff704068a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 -- 172.21.15.176:0/1873306993 >> 172.21.15.176:0/1873306993 conn(0x7ff7040755a0 msgr2=0x7ff7040759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:27.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 -- 172.21.15.176:0/1873306993 shutdown_connections 2024-03-13T15:01:27.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.081+0000 7ff70ae00700 1 -- 172.21.15.176:0/1873306993 wait complete. 2024-03-13T15:01:27.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.082+0000 7ff70ae00700 1 Processor -- start 2024-03-13T15:01:27.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.082+0000 7ff70ae00700 1 -- start start 2024-03-13T15:01:27.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.083+0000 7ff70ae00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704068670 0x7ff704196700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.083+0000 7ff70ae00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704069020 0x7ff704196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.083+0000 7ff70ae00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7041972d0 con 0x7ff704068670 2024-03-13T15:01:27.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.083+0000 7ff70ae00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70419a000 con 0x7ff704069020 2024-03-13T15:01:27.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.083+0000 7ff703e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704068670 0x7ff704196700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:27.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.083+0000 7ff703e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704068670 0x7ff704196700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39698/0 (socket says 172.21.15.176:39698) 2024-03-13T15:01:27.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.083+0000 7ff703e00700 1 -- 172.21.15.176:0/1560654968 learned_addr learned my addr 172.21.15.176:0/1560654968 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff703e00700 1 -- 172.21.15.176:0/1560654968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704069020 msgr2=0x7ff704196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff703e00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704069020 0x7ff704196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff703e00700 1 -- 172.21.15.176:0/1560654968 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f00095f0 con 0x7ff704068670 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff703e00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704068670 0x7ff704196700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400d8d0 tx=0x7ff6f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff700c00700 1 -- 172.21.15.176:0/1560654968 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6f4009940 con 0x7ff704068670 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff700c00700 1 -- 172.21.15.176:0/1560654968 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6f4010460 con 0x7ff704068670 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70419a2e0 con 0x7ff704068670 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff700c00700 1 -- 172.21.15.176:0/1560654968 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6f400f5f0 con 0x7ff704068670 2024-03-13T15:01:27.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.084+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70419a830 con 0x7ff704068670 2024-03-13T15:01:27.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.085+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7041090a0 con 0x7ff704068670 2024-03-13T15:01:27.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.086+0000 7ff700c00700 1 -- 172.21.15.176:0/1560654968 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff6f40105d0 con 0x7ff704068670 2024-03-13T15:01:27.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.087+0000 7ff700c00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff6e806dfa0 0x7ff6e8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.087+0000 7ff700c00700 1 -- 172.21.15.176:0/1560654968 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7ff6f408df40 con 0x7ff704068670 2024-03-13T15:01:27.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.088+0000 7ff703400700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff6e806dfa0 0x7ff6e8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:27.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.088+0000 7ff703400700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff6e806dfa0 0x7ff6e8070460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006010 tx=0x7ff6f000b540 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:27.077 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.092+0000 7ff700c00700 1 -- 172.21.15.176:0/1560654968 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7ff6f40571f0 con 0x7ff704068670 2024-03-13T15:01:27.372 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.387+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff70404ea90 con 0x7ff704068670 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.388+0000 7ff700c00700 1 -- 172.21.15.176:0/1560654968 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff6f405a8b0 con 0x7ff704068670 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:01:27.373 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 14 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T15:01:27.374 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:01:27.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff6e806dfa0 msgr2=0x7ff6e8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff6e806dfa0 0x7ff6e8070460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0006010 tx=0x7ff6f000b540 comp rx=0 tx=0).stop 2024-03-13T15:01:27.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704068670 msgr2=0x7ff704196700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704068670 0x7ff704196700 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400d8d0 tx=0x7ff6f400dc90 comp rx=0 tx=0).stop 2024-03-13T15:01:27.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 shutdown_connections 2024-03-13T15:01:27.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff6e806dfa0 0x7ff6e8070460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff704068670 0x7ff704196700 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 --2- 172.21.15.176:0/1560654968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff704069020 0x7ff704196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 >> 172.21.15.176:0/1560654968 conn(0x7ff7040755a0 msgr2=0x7ff704105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:27.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 shutdown_connections 2024-03-13T15:01:27.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.391+0000 7ff70ae00700 1 -- 172.21.15.176:0/1560654968 wait complete. 2024-03-13T15:01:27.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 -- 172.21.15.176:0/1801425250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 msgr2=0x7fad44107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 --2- 172.21.15.176:0/1801425250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad44107650 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fad30009910 tx=0x7fad30009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:27.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 -- 172.21.15.176:0/1801425250 shutdown_connections 2024-03-13T15:01:27.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 --2- 172.21.15.176:0/1801425250 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad44107650 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 --2- 172.21.15.176:0/1801425250 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4410d280 0x7fad4410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 -- 172.21.15.176:0/1801425250 >> 172.21.15.176:0/1801425250 conn(0x7fad440769e0 msgr2=0x7fad44078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:27.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 -- 172.21.15.176:0/1801425250 shutdown_connections 2024-03-13T15:01:27.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.544+0000 7fad4ae00700 1 -- 172.21.15.176:0/1801425250 wait complete. 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.545+0000 7fad4ae00700 1 Processor -- start 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.545+0000 7fad4ae00700 1 -- start start 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad4ae00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad4419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad4ae00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4410d280 0x7fad4419d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad4ae00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4419d9b0 con 0x7fad441071d0 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad4ae00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad441a1740 con 0x7fad4410d280 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad43e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad4419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:27.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad43e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad4419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39712/0 (socket says 172.21.15.176:39712) 2024-03-13T15:01:27.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad43e00700 1 -- 172.21.15.176:0/316283606 learned_addr learned my addr 172.21.15.176:0/316283606 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:27.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad43e00700 1 -- 172.21.15.176:0/316283606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4410d280 msgr2=0x7fad4419d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad43e00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4410d280 0x7fad4419d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.546+0000 7fad43e00700 1 -- 172.21.15.176:0/316283606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad300095f0 con 0x7fad441071d0 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.547+0000 7fad43e00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad4419cd90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad3400d8d0 tx=0x7fad3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.547+0000 7fad40c00700 1 -- 172.21.15.176:0/316283606 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad34009940 con 0x7fad441071d0 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.547+0000 7fad40c00700 1 -- 172.21.15.176:0/316283606 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad34010460 con 0x7fad441071d0 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.547+0000 7fad40c00700 1 -- 172.21.15.176:0/316283606 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad3400f5f0 con 0x7fad441071d0 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.547+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad441a1a20 con 0x7fad441071d0 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.547+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad441a1f70 con 0x7fad441071d0 2024-03-13T15:01:27.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.548+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4404ea90 con 0x7fad441071d0 2024-03-13T15:01:27.534 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.549+0000 7fad40c00700 1 -- 172.21.15.176:0/316283606 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fad340105d0 con 0x7fad441071d0 2024-03-13T15:01:27.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.550+0000 7fad40c00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fad2806dfa0 0x7fad28070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.550+0000 7fad40c00700 1 -- 172.21.15.176:0/316283606 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fad3408dd80 con 0x7fad441071d0 2024-03-13T15:01:27.535 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.550+0000 7fad43400700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fad2806dfa0 0x7fad28070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:27.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.551+0000 7fad43400700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fad2806dfa0 0x7fad28070460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad3000b5c0 tx=0x7fad30005960 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:27.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.555+0000 7fad40c00700 1 -- 172.21.15.176:0/316283606 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fad34057150 con 0x7fad441071d0 2024-03-13T15:01:27.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:27 smithi176 ceph-mon[112742]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:27.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:27 smithi176 ceph-mon[112742]: pgmap v288: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 8.4 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-03-13T15:01:27.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:27 smithi176 ceph-mon[112742]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:27.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:27 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:27.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:27 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:27.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:27 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1560654968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:27.825 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.840+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad441a2490 con 0x7fad441071d0 2024-03-13T15:01:27.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.841+0000 7fad40c00700 1 -- 172.21.15.176:0/316283606 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fad34016770 con 0x7fad441071d0 2024-03-13T15:01:27.826 INFO:teuthology.orchestra.run.smithi176.stdout:e36 2024-03-13T15:01:27.826 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T15:01:27.826 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T15:01:27.826 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 35 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T15:01:19.671648+0000 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T15:01:27.827 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 112 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=34462} 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T15:01:27.828 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{0:34462} state up:active seq 24 join_fscid=1 addr [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:01:27.829 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:01:27.829 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:01:27.829 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T15:01:27.829 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:01:27.829 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{-1:34494} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.176:6828/552037800,v1:172.21.15.176:6829/552037800] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:01:27.829 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{-1:44257} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.196:6824/2671769958,v1:172.21.15.196:6825/2671769958] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:01:27.829 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:44281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.196:6826/2135805809,v1:172.21.15.196:6827/2135805809] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:01:27.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.843+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fad2806dfa0 msgr2=0x7fad28070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.843+0000 7fad4ae00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fad2806dfa0 0x7fad28070460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad3000b5c0 tx=0x7fad30005960 comp rx=0 tx=0).stop 2024-03-13T15:01:27.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.843+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 msgr2=0x7fad4419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.843+0000 7fad4ae00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad4419cd90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fad3400d8d0 tx=0x7fad3400dc90 comp rx=0 tx=0).stop 2024-03-13T15:01:27.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.844+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 shutdown_connections 2024-03-13T15:01:27.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.844+0000 7fad4ae00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fad2806dfa0 0x7fad28070460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.844+0000 7fad4ae00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fad441071d0 0x7fad4419cd90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.844+0000 7fad4ae00700 1 --2- 172.21.15.176:0/316283606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad4410d280 0x7fad4419d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.844+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 >> 172.21.15.176:0/316283606 conn(0x7fad440769e0 msgr2=0x7fad44078480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:27.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.844+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 shutdown_connections 2024-03-13T15:01:27.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.844+0000 7fad4ae00700 1 -- 172.21.15.176:0/316283606 wait complete. 2024-03-13T15:01:27.831 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 36 2024-03-13T15:01:27.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:27 smithi196 ceph-mon[86193]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:27 smithi196 ceph-mon[86193]: pgmap v288: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 8.4 MiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-03-13T15:01:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:27 smithi196 ceph-mon[86193]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:27 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:27 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:27.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:27 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1560654968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.999+0000 7f279ae00700 1 -- 172.21.15.176:0/1159728752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 msgr2=0x7f2794107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:27.999+0000 7f279ae00700 1 --2- 172.21.15.176:0/1159728752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f2794107650 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2780009910 tx=0x7f2780009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.000+0000 7f279ae00700 1 -- 172.21.15.176:0/1159728752 shutdown_connections 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.000+0000 7f279ae00700 1 --2- 172.21.15.176:0/1159728752 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f2794107650 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.000+0000 7f279ae00700 1 --2- 172.21.15.176:0/1159728752 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f279410d280 0x7f279410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.000+0000 7f279ae00700 1 -- 172.21.15.176:0/1159728752 >> 172.21.15.176:0/1159728752 conn(0x7f27940769e0 msgr2=0x7f2794078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.000+0000 7f279ae00700 1 -- 172.21.15.176:0/1159728752 shutdown_connections 2024-03-13T15:01:27.985 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.000+0000 7f279ae00700 1 -- 172.21.15.176:0/1159728752 wait complete. 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f279ae00700 1 Processor -- start 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f279ae00700 1 -- start start 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f279ae00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f279419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f279ae00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f279410d280 0x7f279419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f279ae00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279419d880 con 0x7f27941071d0 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f279ae00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27941a05b0 con 0x7f279410d280 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f2793e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f279419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.001+0000 7f2793e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f279419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39730/0 (socket says 172.21.15.176:39730) 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.002+0000 7f2793e00700 1 -- 172.21.15.176:0/770934094 learned_addr learned my addr 172.21.15.176:0/770934094 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.002+0000 7f2793400700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f279410d280 0x7f279419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.002+0000 7f2793e00700 1 -- 172.21.15.176:0/770934094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f279410d280 msgr2=0x7f279419d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:27.987 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.002+0000 7f2793e00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f279410d280 0x7f279419d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:27.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.002+0000 7f2793e00700 1 -- 172.21.15.176:0/770934094 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27800095f0 con 0x7f27941071d0 2024-03-13T15:01:27.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.002+0000 7f2793e00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f279419ccb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f278400e920 tx=0x7f278400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:27.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.003+0000 7f2790c00700 1 -- 172.21.15.176:0/770934094 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f278400c9d0 con 0x7f27941071d0 2024-03-13T15:01:27.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.003+0000 7f2790c00700 1 -- 172.21.15.176:0/770934094 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2784010430 con 0x7f27941071d0 2024-03-13T15:01:27.988 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.003+0000 7f2790c00700 1 -- 172.21.15.176:0/770934094 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2784018630 con 0x7f27941071d0 2024-03-13T15:01:27.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.003+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27941a0890 con 0x7f27941071d0 2024-03-13T15:01:27.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.003+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27941a0e30 con 0x7f27941071d0 2024-03-13T15:01:27.989 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.004+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279410f6c0 con 0x7f27941071d0 2024-03-13T15:01:27.990 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.005+0000 7f2790c00700 1 -- 172.21.15.176:0/770934094 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f27840105a0 con 0x7f27941071d0 2024-03-13T15:01:27.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.006+0000 7f2790c00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f277806dfa0 0x7f2778070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:27.991 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.006+0000 7f2790c00700 1 -- 172.21.15.176:0/770934094 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f2784014070 con 0x7f27941071d0 2024-03-13T15:01:27.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.007+0000 7f2793400700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f277806dfa0 0x7f2778070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:27.992 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.007+0000 7f2793400700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f277806dfa0 0x7f2778070460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f279419e330 tx=0x7f2780005a90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:27.996 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.011+0000 7f2790c00700 1 -- 172.21.15.176:0/770934094 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f27840531e0 con 0x7f27941071d0 2024-03-13T15:01:28.231 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.246+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27941a10a0 con 0x7f277806dfa0 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.248+0000 7f2790c00700 1 -- 172.21.15.176:0/770934094 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f27941a10a0 con 0x7f277806dfa0 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stdout: "mds", 2024-03-13T15:01:28.233 INFO:teuthology.orchestra.run.smithi176.stdout: "osd", 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr", 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout: "crash" 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "16/23 daemons upgraded", 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading mds daemons", 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T15:01:28.234 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:01:28.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f277806dfa0 msgr2=0x7f2778070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:28.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f277806dfa0 0x7f2778070460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f279419e330 tx=0x7f2780005a90 comp rx=0 tx=0).stop 2024-03-13T15:01:28.235 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 msgr2=0x7f279419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f279419ccb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f278400e920 tx=0x7f278400ec30 comp rx=0 tx=0).stop 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 shutdown_connections 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f277806dfa0 0x7f2778070460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27941071d0 0x7f279419ccb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 --2- 172.21.15.176:0/770934094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f279410d280 0x7f279419d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.250+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 >> 172.21.15.176:0/770934094 conn(0x7f27940769e0 msgr2=0x7f2794078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.251+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 shutdown_connections 2024-03-13T15:01:28.236 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.251+0000 7f279ae00700 1 -- 172.21.15.176:0/770934094 wait complete. 2024-03-13T15:01:28.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2458266471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08103960 msgr2=0x7ffb081079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:28.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2458266471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08103960 0x7ffb081079b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009910 tx=0x7ffaf4009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:28.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2458266471 shutdown_connections 2024-03-13T15:01:28.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2458266471 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08103960 0x7ffb081079b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.387 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2458266471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb08102fb0 0x7ffb08103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2458266471 >> 172.21.15.176:0/2458266471 conn(0x7ffb080fe840 msgr2=0x7ffb08100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:28.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2458266471 shutdown_connections 2024-03-13T15:01:28.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.402+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2458266471 wait complete. 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.403+0000 7ffb0dc00700 1 Processor -- start 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.403+0000 7ffb0dc00700 1 -- start start 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.403+0000 7ffb0dc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08102fb0 0x7ffb0819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.403+0000 7ffb0dc00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb08103960 0x7ffb0819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.403+0000 7ffb0dc00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0819f360 con 0x7ffb08102fb0 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.403+0000 7ffb0dc00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb08198810 con 0x7ffb08103960 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.404+0000 7ffb06a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08102fb0 0x7ffb0819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:28.389 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.404+0000 7ffb06a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08102fb0 0x7ffb0819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39744/0 (socket says 172.21.15.176:39744) 2024-03-13T15:01:28.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.404+0000 7ffb06a00700 1 -- 172.21.15.176:0/2965737150 learned_addr learned my addr 172.21.15.176:0/2965737150 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:28.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.404+0000 7ffb06a00700 1 -- 172.21.15.176:0/2965737150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb08103960 msgr2=0x7ffb0819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:28.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.404+0000 7ffb06a00700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb08103960 0x7ffb0819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.404+0000 7ffb06a00700 1 -- 172.21.15.176:0/2965737150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40095f0 con 0x7ffb08102fb0 2024-03-13T15:01:28.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.404+0000 7ffb06a00700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08102fb0 0x7ffb0819e790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c8a0 tx=0x7ffaf800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:28.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.405+0000 7ffaff400700 1 -- 172.21.15.176:0/2965737150 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf800cea0 con 0x7ffb08102fb0 2024-03-13T15:01:28.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.405+0000 7ffaff400700 1 -- 172.21.15.176:0/2965737150 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaf8004830 con 0x7ffb08102fb0 2024-03-13T15:01:28.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.405+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb08198af0 con 0x7ffb08102fb0 2024-03-13T15:01:28.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.405+0000 7ffaff400700 1 -- 172.21.15.176:0/2965737150 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaf80055b0 con 0x7ffb08102fb0 2024-03-13T15:01:28.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.405+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb08199040 con 0x7ffb08102fb0 2024-03-13T15:01:28.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.406+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0810b3b0 con 0x7ffb08102fb0 2024-03-13T15:01:28.391 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.407+0000 7ffaff400700 1 -- 172.21.15.176:0/2965737150 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ffaf80049a0 con 0x7ffb08102fb0 2024-03-13T15:01:28.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.408+0000 7ffaff400700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ffaec06dfa0 0x7ffaec070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:28.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.408+0000 7ffaff400700 1 -- 172.21.15.176:0/2965737150 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7ffaf808dbd0 con 0x7ffb08102fb0 2024-03-13T15:01:28.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.408+0000 7ffb06000700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ffaec06dfa0 0x7ffaec070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:28.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.409+0000 7ffb06000700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ffaec06dfa0 0x7ffaec070460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b5c0 tx=0x7ffaf4005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:28.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.413+0000 7ffaff400700 1 -- 172.21.15.176:0/2965737150 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7ffaf805a9b0 con 0x7ffb08102fb0 2024-03-13T15:01:28.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.721+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffb08199310 con 0x7ffb08102fb0 2024-03-13T15:01:28.706 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.721+0000 7ffaff400700 1 -- 172.21.15.176:0/2965737150 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ffaf8004c50 con 0x7ffb08102fb0 2024-03-13T15:01:28.707 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T15:01:28.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ffaec06dfa0 msgr2=0x7ffaec070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:28.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ffaec06dfa0 0x7ffaec070460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b5c0 tx=0x7ffaf4005fd0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08102fb0 msgr2=0x7ffb0819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:28.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08102fb0 0x7ffb0819e790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800c8a0 tx=0x7ffaf800cc60 comp rx=0 tx=0).stop 2024-03-13T15:01:28.710 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 shutdown_connections 2024-03-13T15:01:28.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ffaec06dfa0 0x7ffaec070460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ffb08102fb0 0x7ffb0819e790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 --2- 172.21.15.176:0/2965737150 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb08103960 0x7ffb0819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:28.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 >> 172.21.15.176:0/2965737150 conn(0x7ffb080fe840 msgr2=0x7ffb08100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:28.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 shutdown_connections 2024-03-13T15:01:28.711 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:28.725+0000 7ffb0dc00700 1 -- 172.21.15.176:0/2965737150 wait complete. 2024-03-13T15:01:28.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:28 smithi176 ceph-mon[112742]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:28.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:28 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/316283606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:01:28.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:28 smithi196 ceph-mon[86193]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:28.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:28 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/316283606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:01:29.774 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:29 smithi196 ceph-mon[86193]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:29.774 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:29 smithi196 ceph-mon[86193]: pgmap v289: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:29.774 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:29 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:29.774 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:29 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:29.774 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:29 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2965737150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:01:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:29 smithi176 ceph-mon[112742]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:01:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:29 smithi176 ceph-mon[112742]: pgmap v289: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:29 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:29 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:29.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:29 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2965737150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:01:31.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:31.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:31 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:31.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:31 smithi196 ceph-mon[86193]: pgmap v290: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:31.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:31.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:31 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:31.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:31 smithi176 ceph-mon[112742]: pgmap v290: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:33.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:33 smithi196 ceph-mon[86193]: pgmap v291: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:33.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:33 smithi176 ceph-mon[112742]: pgmap v291: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-03-13T15:01:35.627 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: pgmap v292: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.3 KiB/s wr, 4 op/s 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.628 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.hjrgov"}]': finished 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.ldskqt"}]': finished 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.tcitrk"}]': finished 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.wabhcu"}]': finished 2024-03-13T15:01:35.629 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:35 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-13T15:01:35.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: pgmap v292: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.3 KiB/s wr, 4 op/s 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:35.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.hjrgov"}]: dispatch 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.hjrgov"}]': finished 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.ldskqt"}]: dispatch 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi176.ldskqt"}]': finished 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.tcitrk"}]: dispatch 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.tcitrk"}]': finished 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.wabhcu"}]: dispatch 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi196.wabhcu"}]': finished 2024-03-13T15:01:35.678 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:35 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-13T15:01:36.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all mds 2024-03-13T15:01:36.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: Upgrade: Setting filesystem cephfs Joinable 2024-03-13T15:01:36.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 3 up:standby 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:36.677 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:36 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all mds 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: Upgrade: Setting filesystem cephfs Joinable 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 3 up:standby 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:36.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:36.754 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:36.754 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:36.754 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:36.754 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:36 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:01:37.666 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:37 smithi176 ceph-mon[112742]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-13T15:01:37.666 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:37 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all rgw 2024-03-13T15:01:37.666 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:37 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all rbd-mirror 2024-03-13T15:01:37.666 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:37 smithi176 ceph-mon[112742]: pgmap v293: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 4.3 KiB/s wr, 2 op/s 2024-03-13T15:01:37.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:37 smithi196 ceph-mon[86193]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-03-13T15:01:37.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:37 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all rgw 2024-03-13T15:01:37.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:37 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all rbd-mirror 2024-03-13T15:01:37.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:37 smithi196 ceph-mon[86193]: pgmap v293: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 4.3 KiB/s wr, 2 op/s 2024-03-13T15:01:39.470 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:39 smithi176 ceph-mon[112742]: Upgrade: Updating ceph-exporter.smithi176 (1/2) 2024-03-13T15:01:39.471 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:39.471 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T15:01:39.471 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:39 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:39.471 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:39 smithi176 ceph-mon[112742]: Deploying daemon ceph-exporter.smithi176 on smithi176 2024-03-13T15:01:39.471 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:39 smithi176 ceph-mon[112742]: pgmap v294: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1024 KiB/s rd, 0 op/s 2024-03-13T15:01:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:39 smithi196 ceph-mon[86193]: Upgrade: Updating ceph-exporter.smithi176 (1/2) 2024-03-13T15:01:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi176", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T15:01:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:39 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:39 smithi196 ceph-mon[86193]: Deploying daemon ceph-exporter.smithi176 on smithi176 2024-03-13T15:01:39.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:39 smithi196 ceph-mon[86193]: pgmap v294: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1024 KiB/s rd, 0 op/s 2024-03-13T15:01:41.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:41 smithi176 ceph-mon[112742]: pgmap v295: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-13T15:01:41.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:41 smithi196 ceph-mon[86193]: pgmap v295: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-13T15:01:43.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:43 smithi176 ceph-mon[112742]: pgmap v296: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-13T15:01:43.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:43 smithi196 ceph-mon[86193]: pgmap v296: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-03-13T15:01:45.382 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:45.382 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:45.382 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:45 smithi196 ceph-mon[86193]: pgmap v297: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-13T15:01:45.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:45.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:45.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:45 smithi176 ceph-mon[112742]: pgmap v297: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-03-13T15:01:46.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:46 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:46.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:46 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:01:47.586 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:47 smithi196 ceph-mon[86193]: Upgrade: Updating ceph-exporter.smithi196 (2/2) 2024-03-13T15:01:47.586 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:47.586 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T15:01:47.586 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:47 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:47.586 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:47 smithi196 ceph-mon[86193]: Deploying daemon ceph-exporter.smithi196 on smithi196 2024-03-13T15:01:47.587 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:47 smithi196 ceph-mon[86193]: pgmap v298: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-03-13T15:01:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:47 smithi176 ceph-mon[112742]: Upgrade: Updating ceph-exporter.smithi196 (2/2) 2024-03-13T15:01:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi196", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-03-13T15:01:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:47 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:01:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:47 smithi176 ceph-mon[112742]: Deploying daemon ceph-exporter.smithi196 on smithi196 2024-03-13T15:01:47.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:47 smithi176 ceph-mon[112742]: pgmap v298: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-03-13T15:01:49.582 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:49 smithi196 ceph-mon[86193]: pgmap v299: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-03-13T15:01:49.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:49 smithi176 ceph-mon[112742]: pgmap v299: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 4 op/s 2024-03-13T15:01:51.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:51 smithi196 ceph-mon[86193]: pgmap v300: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-13T15:01:51.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:51 smithi176 ceph-mon[112742]: pgmap v300: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-03-13T15:01:53.181 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:53.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:53.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:52 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:53.182 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:52 smithi196 ceph-mon[86193]: pgmap v301: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 2 op/s 2024-03-13T15:01:53.304 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:53.304 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:53.304 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:52 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:01:53.304 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:52 smithi176 ceph-mon[112742]: pgmap v301: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 2 op/s 2024-03-13T15:01:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:55 smithi176 ceph-mon[112742]: pgmap v302: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 2 op/s 2024-03-13T15:01:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:55 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:55.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:55 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:55.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:55 smithi196 ceph-mon[86193]: pgmap v302: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s rd, 2 op/s 2024-03-13T15:01:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:55 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:55.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:55 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:57.563 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:57 smithi176 ceph-mon[112742]: pgmap v303: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 1 op/s 2024-03-13T15:01:57.563 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:57 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:57.563 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:57 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:57.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:57 smithi196 ceph-mon[86193]: pgmap v303: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 1 op/s 2024-03-13T15:01:57.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:57 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:57.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:57 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:58.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 -- 172.21.15.176:0/924383331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 msgr2=0x7fa21010ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:58.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 --2- 172.21.15.176:0/924383331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa21010ab70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009940 tx=0x7fa1fc009c50 comp rx=0 tx=0).stop 2024-03-13T15:01:58.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 -- 172.21.15.176:0/924383331 shutdown_connections 2024-03-13T15:01:58.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 --2- 172.21.15.176:0/924383331 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa21010ab70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:58.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 --2- 172.21.15.176:0/924383331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa210101d60 0x7fa210102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:58.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 -- 172.21.15.176:0/924383331 >> 172.21.15.176:0/924383331 conn(0x7fa2100fb570 msgr2=0x7fa2100fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:58.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 -- 172.21.15.176:0/924383331 shutdown_connections 2024-03-13T15:01:58.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.889+0000 7fa215400700 1 -- 172.21.15.176:0/924383331 wait complete. 2024-03-13T15:01:58.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.890+0000 7fa215400700 1 Processor -- start 2024-03-13T15:01:58.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa215400700 1 -- start start 2024-03-13T15:01:58.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa215400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa210101d60 0x7fa2100ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:58.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa215400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa2100fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:58.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa215400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa210100440 con 0x7fa210102680 2024-03-13T15:01:58.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa215400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa210100580 con 0x7fa210101d60 2024-03-13T15:01:58.877 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa20de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa2100fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:58.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa20e800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa210101d60 0x7fa2100ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:58.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa20de00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa2100fff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55874/0 (socket says 172.21.15.176:55874) 2024-03-13T15:01:58.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa20de00700 1 -- 172.21.15.176:0/3945696576 learned_addr learned my addr 172.21.15.176:0/3945696576 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:58.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.891+0000 7fa20e800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa210101d60 0x7fa2100ff9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.176:49072/0 (socket says 172.21.15.176:49072) 2024-03-13T15:01:58.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa20de00700 1 -- 172.21.15.176:0/3945696576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa210101d60 msgr2=0x7fa2100ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:58.878 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa20de00700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa210101d60 0x7fa2100ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:58.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa20de00700 1 -- 172.21.15.176:0/3945696576 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0095f0 con 0x7fa210102680 2024-03-13T15:01:58.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa20de00700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa2100fff00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0054b0 tx=0x7fa1fc00bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:58.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa207400700 1 -- 172.21.15.176:0/3945696576 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1fc01d070 con 0x7fa210102680 2024-03-13T15:01:58.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa207400700 1 -- 172.21.15.176:0/3945696576 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1fc003d10 con 0x7fa210102680 2024-03-13T15:01:58.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa207400700 1 -- 172.21.15.176:0/3945696576 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1fc017400 con 0x7fa210102680 2024-03-13T15:01:58.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.892+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa210100800 con 0x7fa210102680 2024-03-13T15:01:58.879 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.893+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2101a2c30 con 0x7fa210102680 2024-03-13T15:01:58.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.894+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa210108270 con 0x7fa210102680 2024-03-13T15:01:58.880 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.895+0000 7fa207400700 1 -- 172.21.15.176:0/3945696576 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa1fc00f460 con 0x7fa210102680 2024-03-13T15:01:58.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.895+0000 7fa207400700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa1f406dfa0 0x7fa1f4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:58.881 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.896+0000 7fa207400700 1 -- 172.21.15.176:0/3945696576 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa1fc08efe0 con 0x7fa210102680 2024-03-13T15:01:58.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.896+0000 7fa20e800700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa1f406dfa0 0x7fa1f4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:58.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.897+0000 7fa20e800700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa1f406dfa0 0x7fa1f4070460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa200009c60 tx=0x7fa200009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:58.887 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:58.901+0000 7fa207400700 1 -- 172.21.15.176:0/3945696576 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa1fc0582b0 con 0x7fa210102680 2024-03-13T15:01:59.130 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.144+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa21002cce0 con 0x7fa1f406dfa0 2024-03-13T15:01:59.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.149+0000 7fa207400700 1 -- 172.21.15.176:0/3945696576 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fa21002cce0 con 0x7fa1f406dfa0 2024-03-13T15:01:59.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa1f406dfa0 msgr2=0x7fa1f4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa1f406dfa0 0x7fa1f4070460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa200009c60 tx=0x7fa200009260 comp rx=0 tx=0).stop 2024-03-13T15:01:59.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 msgr2=0x7fa2100fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa2100fff00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0054b0 tx=0x7fa1fc00bb10 comp rx=0 tx=0).stop 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 shutdown_connections 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa1f406dfa0 0x7fa1f4070460 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa210101d60 0x7fa2100ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 --2- 172.21.15.176:0/3945696576 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa210102680 0x7fa2100fff00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 >> 172.21.15.176:0/3945696576 conn(0x7fa2100fb570 msgr2=0x7fa2101053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.152+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 shutdown_connections 2024-03-13T15:01:59.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.153+0000 7fa215400700 1 -- 172.21.15.176:0/3945696576 wait complete. 2024-03-13T15:01:59.154 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.304+0000 7f7683200700 1 -- 172.21.15.176:0/317356737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 msgr2=0x7f76840735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.304+0000 7f7683200700 1 --2- 172.21.15.176:0/317356737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f76840735a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7674009910 tx=0x7f7674009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.305+0000 7f7683200700 1 -- 172.21.15.176:0/317356737 shutdown_connections 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.305+0000 7f7683200700 1 --2- 172.21.15.176:0/317356737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7684073ae0 0x7f768410cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.305+0000 7f7683200700 1 --2- 172.21.15.176:0/317356737 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f76840735a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.305+0000 7f7683200700 1 -- 172.21.15.176:0/317356737 >> 172.21.15.176:0/317356737 conn(0x7f76840fc470 msgr2=0x7f76840fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.305+0000 7f7683200700 1 -- 172.21.15.176:0/317356737 shutdown_connections 2024-03-13T15:01:59.291 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.305+0000 7f7683200700 1 -- 172.21.15.176:0/317356737 wait complete. 2024-03-13T15:01:59.292 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.306+0000 7f7683200700 1 Processor -- start 2024-03-13T15:01:59.292 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.306+0000 7f7683200700 1 -- start start 2024-03-13T15:01:59.293 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.306+0000 7f7683200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f768419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:59.293 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7683200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7684073ae0 0x7f768419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:59.293 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7683200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768419f250 con 0x7f76840731c0 2024-03-13T15:01:59.293 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7683200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7684198700 con 0x7f7684073ae0 2024-03-13T15:01:59.293 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7681e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f768419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:59.293 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7681e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f768419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55898/0 (socket says 172.21.15.176:55898) 2024-03-13T15:01:59.293 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7681e00700 1 -- 172.21.15.176:0/2052127224 learned_addr learned my addr 172.21.15.176:0/2052127224 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7681e00700 1 -- 172.21.15.176:0/2052127224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7684073ae0 msgr2=0x7f768419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7681e00700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7684073ae0 0x7f768419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.307+0000 7f7681e00700 1 -- 172.21.15.176:0/2052127224 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76740095f0 con 0x7f76840731c0 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.308+0000 7f7681e00700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f768419e680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f767400bd70 tx=0x7f767400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.308+0000 7f767aa00700 1 -- 172.21.15.176:0/2052127224 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767401d070 con 0x7f76840731c0 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.308+0000 7f767aa00700 1 -- 172.21.15.176:0/2052127224 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7674022470 con 0x7f76840731c0 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.308+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7684198980 con 0x7f76840731c0 2024-03-13T15:01:59.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.308+0000 7f767aa00700 1 -- 172.21.15.176:0/2052127224 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767400f690 con 0x7f76840731c0 2024-03-13T15:01:59.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.308+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7684198e70 con 0x7f76840731c0 2024-03-13T15:01:59.295 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.309+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768410a380 con 0x7f76840731c0 2024-03-13T15:01:59.296 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.310+0000 7f767aa00700 1 -- 172.21.15.176:0/2052127224 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f76740225e0 con 0x7f76840731c0 2024-03-13T15:01:59.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.312+0000 7f767aa00700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f766806dfa0 0x7f7668070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:59.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.312+0000 7f767aa00700 1 -- 172.21.15.176:0/2052127224 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f767408f380 con 0x7f76840731c0 2024-03-13T15:01:59.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.312+0000 7f7681400700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f766806dfa0 0x7f7668070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:59.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.313+0000 7f7681400700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f766806dfa0 0x7f7668070460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7684199b70 tx=0x7f7670006cd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:59.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.317+0000 7f767aa00700 1 -- 172.21.15.176:0/2052127224 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f767405c330 con 0x7f76840731c0 2024-03-13T15:01:59.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.588+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7684199210 con 0x7f766806dfa0 2024-03-13T15:01:59.590 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.604+0000 7f767aa00700 1 -- 172.21.15.176:0/2052127224 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f7684199210 con 0x7f766806dfa0 2024-03-13T15:01:59.593 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.607+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f766806dfa0 msgr2=0x7f7668070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.607+0000 7f7683200700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f766806dfa0 0x7f7668070460 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7684199b70 tx=0x7f7670006cd0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.607+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 msgr2=0x7f768419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.607+0000 7f7683200700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f768419e680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f767400bd70 tx=0x7f767400be50 comp rx=0 tx=0).stop 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.608+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 shutdown_connections 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.608+0000 7f7683200700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f766806dfa0 0x7f7668070460 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.608+0000 7f7683200700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f76840731c0 0x7f768419e680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.608+0000 7f7683200700 1 --2- 172.21.15.176:0/2052127224 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7684073ae0 0x7f768419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.594 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.608+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 >> 172.21.15.176:0/2052127224 conn(0x7f76840fc470 msgr2=0x7f76841074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:59.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.608+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 shutdown_connections 2024-03-13T15:01:59.595 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.608+0000 7f7683200700 1 -- 172.21.15.176:0/2052127224 wait complete. 2024-03-13T15:01:59.645 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:59 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:59.645 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:59 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:59.646 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:01:59 smithi196 ceph-mon[86193]: pgmap v304: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:01:59.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 -- 172.21.15.176:0/3465716031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073a80 msgr2=0x7f6e2410cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 --2- 172.21.15.176:0/3465716031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073a80 0x7f6e2410cee0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6e10009910 tx=0x7f6e10009c20 comp rx=0 tx=0).stop 2024-03-13T15:01:59.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 -- 172.21.15.176:0/3465716031 shutdown_connections 2024-03-13T15:01:59.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 --2- 172.21.15.176:0/3465716031 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073a80 0x7f6e2410cee0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 --2- 172.21.15.176:0/3465716031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e24073160 0x7f6e24073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:59 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:59.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:59 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:01:59.753 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:01:59 smithi176 ceph-mon[112742]: pgmap v304: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:01:59.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 -- 172.21.15.176:0/3465716031 >> 172.21.15.176:0/3465716031 conn(0x7f6e240fc670 msgr2=0x7f6e240fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:01:59.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 -- 172.21.15.176:0/3465716031 shutdown_connections 2024-03-13T15:01:59.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.767+0000 7f6e2b400700 1 -- 172.21.15.176:0/3465716031 wait complete. 2024-03-13T15:01:59.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.768+0000 7f6e2b400700 1 Processor -- start 2024-03-13T15:01:59.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.769+0000 7f6e2b400700 1 -- start start 2024-03-13T15:01:59.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.769+0000 7f6e2b400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073160 0x7f6e2419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:59.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.769+0000 7f6e2b400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e24073a80 0x7f6e2419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:59.756 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.769+0000 7f6e2b400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2419f2d0 con 0x7f6e24073160 2024-03-13T15:01:59.756 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.769+0000 7f6e2b400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e24198780 con 0x7f6e24073a80 2024-03-13T15:01:59.756 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e28a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073160 0x7f6e2419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:59.756 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e28a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073160 0x7f6e2419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55928/0 (socket says 172.21.15.176:55928) 2024-03-13T15:01:59.756 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e28a00700 1 -- 172.21.15.176:0/2518961599 learned_addr learned my addr 172.21.15.176:0/2518961599 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:01:59.756 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e23e00700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e24073a80 0x7f6e2419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:59.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e28a00700 1 -- 172.21.15.176:0/2518961599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e24073a80 msgr2=0x7f6e2419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:01:59.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e28a00700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e24073a80 0x7f6e2419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:01:59.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e28a00700 1 -- 172.21.15.176:0/2518961599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e100095f0 con 0x7f6e24073160 2024-03-13T15:01:59.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.770+0000 7f6e23e00700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e24073a80 0x7f6e2419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-13T15:01:59.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.771+0000 7f6e28a00700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073160 0x7f6e2419e700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800d900 tx=0x7f6e1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:59.758 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.771+0000 7f6e21600700 1 -- 172.21.15.176:0/2518961599 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e180041f0 con 0x7f6e24073160 2024-03-13T15:01:59.758 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.771+0000 7f6e21600700 1 -- 172.21.15.176:0/2518961599 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e18004cf0 con 0x7f6e24073160 2024-03-13T15:01:59.758 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.771+0000 7f6e21600700 1 -- 172.21.15.176:0/2518961599 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e1800b750 con 0x7f6e24073160 2024-03-13T15:01:59.758 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.771+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e24198a60 con 0x7f6e24073160 2024-03-13T15:01:59.758 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.772+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e24198f60 con 0x7f6e24073160 2024-03-13T15:01:59.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.773+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2410a5e0 con 0x7f6e24073160 2024-03-13T15:01:59.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.774+0000 7f6e21600700 1 -- 172.21.15.176:0/2518961599 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6e18004350 con 0x7f6e24073160 2024-03-13T15:01:59.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.775+0000 7f6e21600700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e0c06dfa0 0x7f6e0c070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:01:59.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.775+0000 7f6e21600700 1 -- 172.21.15.176:0/2518961599 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f6e1808dfd0 con 0x7f6e24073160 2024-03-13T15:01:59.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.775+0000 7f6e23e00700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e0c06dfa0 0x7f6e0c070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:01:59.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.776+0000 7f6e23e00700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e0c06dfa0 0x7f6e0c070460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000b5c0 tx=0x7f6e1001a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:01:59.767 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:01:59.782+0000 7f6e21600700 1 -- 172.21.15.176:0/2518961599 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f6e1805ae50 con 0x7f6e24073160 2024-03-13T15:02:00.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.022+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e241992e0 con 0x7f6e0c06dfa0 2024-03-13T15:02:00.025 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.038+0000 7f6e21600700 1 -- 172.21.15.176:0/2518961599 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6e241992e0 con 0x7f6e0c06dfa0 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (10m) 1s ago 23m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (16s) 1s ago 23m 7818k - 19.0.0-1916-g9a75c81f 9552dbf7f115 06b476c807b4 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (8s) 4s ago 22m 6945k - 19.0.0-1916-g9a75c81f 9552dbf7f115 0f12bb84f0f6 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (6m) 1s ago 23m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (6m) 4s ago 22m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (9m) 1s ago 22m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (97s) 1s ago 16m 93.6M - 19.0.0-1916-g9a75c81f 9552dbf7f115 edc682c345ac 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (2m) 1s ago 16m 85.7M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5b6d2dc0d458 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (69s) 4s ago 16m 12.2M - 19.0.0-1916-g9a75c81f 9552dbf7f115 a9081f4c2ead 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (38s) 4s ago 16m 11.9M - 19.0.0-1916-g9a75c81f 9552dbf7f115 2e3acf7133a0 2024-03-13T15:02:00.026 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (12m) 1s ago 25m 511M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (11m) 4s ago 21m 452M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (8m) 1s ago 25m 60.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (7m) 4s ago 21m 43.9M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (11m) 1s ago 23m 8186k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (11m) 4s ago 22m 8140k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (5m) 1s ago 20m 144M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (5m) 1s ago 20m 113M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (4m) 1s ago 19m 119M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (3m) 4s ago 19m 137M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (3m) 4s ago 18m 104M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (2m) 4s ago 18m 105M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 43366d2dc8da 2024-03-13T15:02:00.027 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (10m) 1s ago 22m 72.0M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T15:02:00.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e0c06dfa0 msgr2=0x7f6e0c070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e0c06dfa0 0x7f6e0c070460 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000b5c0 tx=0x7f6e1001a040 comp rx=0 tx=0).stop 2024-03-13T15:02:00.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073160 msgr2=0x7f6e2419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073160 0x7f6e2419e700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800d900 tx=0x7f6e1800dcc0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 shutdown_connections 2024-03-13T15:02:00.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6e0c06dfa0 0x7f6e0c070460 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6e24073160 0x7f6e2419e700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 --2- 172.21.15.176:0/2518961599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e24073a80 0x7f6e2419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.041+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 >> 172.21.15.176:0/2518961599 conn(0x7f6e240fc670 msgr2=0x7f6e24107720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:00.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.042+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 shutdown_connections 2024-03-13T15:02:00.029 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.042+0000 7f6e2b400700 1 -- 172.21.15.176:0/2518961599 wait complete. 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.196+0000 7f6d15a00700 1 -- 172.21.15.176:0/2484244302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 msgr2=0x7f6d10106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.196+0000 7f6d15a00700 1 --2- 172.21.15.176:0/2484244302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10106e20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d00009910 tx=0x7f6d00009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.196+0000 7f6d15a00700 1 -- 172.21.15.176:0/2484244302 shutdown_connections 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.196+0000 7f6d15a00700 1 --2- 172.21.15.176:0/2484244302 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d10068410 0x7f6d10068890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.196+0000 7f6d15a00700 1 --2- 172.21.15.176:0/2484244302 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10106e20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.196+0000 7f6d15a00700 1 -- 172.21.15.176:0/2484244302 >> 172.21.15.176:0/2484244302 conn(0x7f6d10075330 msgr2=0x7f6d10075740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.197+0000 7f6d15a00700 1 -- 172.21.15.176:0/2484244302 shutdown_connections 2024-03-13T15:02:00.183 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.197+0000 7f6d15a00700 1 -- 172.21.15.176:0/2484244302 wait complete. 2024-03-13T15:02:00.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.198+0000 7f6d15a00700 1 Processor -- start 2024-03-13T15:02:00.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.198+0000 7f6d15a00700 1 -- start start 2024-03-13T15:02:00.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.198+0000 7f6d15a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d10068410 0x7f6d10196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:00.184 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.198+0000 7f6d15a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:00.185 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.198+0000 7f6d15a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d101971b0 con 0x7f6d10106a40 2024-03-13T15:02:00.185 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.198+0000 7f6d15a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1019af40 con 0x7f6d10068410 2024-03-13T15:02:00.185 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d07e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:00.185 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d07e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55954/0 (socket says 172.21.15.176:55954) 2024-03-13T15:02:00.185 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d07e00700 1 -- 172.21.15.176:0/1482308093 learned_addr learned my addr 172.21.15.176:0/1482308093 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d0fe00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d10068410 0x7f6d10196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d07e00700 1 -- 172.21.15.176:0/1482308093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d10068410 msgr2=0x7f6d10196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d07e00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d10068410 0x7f6d10196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d07e00700 1 -- 172.21.15.176:0/1482308093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d000095f0 con 0x7f6d10106a40 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.199+0000 7f6d07e00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10196ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800d8d0 tx=0x7f6cf800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.200+0000 7f6d0d600700 1 -- 172.21.15.176:0/1482308093 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf8009940 con 0x7f6d10106a40 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.200+0000 7f6d0d600700 1 -- 172.21.15.176:0/1482308093 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cf8010460 con 0x7f6d10106a40 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.200+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1019b220 con 0x7f6d10106a40 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.200+0000 7f6d0d600700 1 -- 172.21.15.176:0/1482308093 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cf800f5f0 con 0x7f6d10106a40 2024-03-13T15:02:00.186 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.200+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1019b770 con 0x7f6d10106a40 2024-03-13T15:02:00.187 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.201+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d10103ce0 con 0x7f6d10106a40 2024-03-13T15:02:00.188 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.202+0000 7f6d0d600700 1 -- 172.21.15.176:0/1482308093 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6cf80105d0 con 0x7f6d10106a40 2024-03-13T15:02:00.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.203+0000 7f6d0d600700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6cf406dfa0 0x7f6cf4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:00.189 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.203+0000 7f6d0d600700 1 -- 172.21.15.176:0/1482308093 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f6cf808dd80 con 0x7f6d10106a40 2024-03-13T15:02:00.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.204+0000 7f6d0fe00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6cf406dfa0 0x7f6cf4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:00.190 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.204+0000 7f6d0fe00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6cf406dfa0 0x7f6cf4070460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b5c0 tx=0x7f6d0001a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:00.198 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.213+0000 7f6d0d600700 1 -- 172.21.15.176:0/1482308093 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f6cf8057150 con 0x7f6d10106a40 2024-03-13T15:02:00.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.546+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d1004f2e0 con 0x7f6d10106a40 2024-03-13T15:02:00.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.547+0000 7f6d0d600700 1 -- 172.21.15.176:0/1482308093 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6cf805a810 con 0x7f6d10106a40 2024-03-13T15:02:00.533 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:02:00.533 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T15:02:00.533 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:02:00.533 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:00.533 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 14 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T15:02:00.534 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.549+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6cf406dfa0 msgr2=0x7f6cf4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.549+0000 7f6d15a00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6cf406dfa0 0x7f6cf4070460 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b5c0 tx=0x7f6d0001a040 comp rx=0 tx=0).stop 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.549+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 msgr2=0x7f6d10196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.549+0000 7f6d15a00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10196ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800d8d0 tx=0x7f6cf800dc90 comp rx=0 tx=0).stop 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.550+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 shutdown_connections 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.550+0000 7f6d15a00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6cf406dfa0 0x7f6cf4070460 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.550+0000 7f6d15a00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d10068410 0x7f6d10196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.550+0000 7f6d15a00700 1 --2- 172.21.15.176:0/1482308093 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6d10106a40 0x7f6d10196ad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.550+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 >> 172.21.15.176:0/1482308093 conn(0x7f6d10075330 msgr2=0x7f6d100fe970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:00.536 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.550+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 shutdown_connections 2024-03-13T15:02:00.537 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.550+0000 7f6d15a00700 1 -- 172.21.15.176:0/1482308093 wait complete. 2024-03-13T15:02:00.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:00 smithi196 ceph-mon[86193]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:00.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:00 smithi196 ceph-mon[86193]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:00.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:00.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.703+0000 7fc00a600700 1 -- 172.21.15.176:0/2959827720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 msgr2=0x7fc004079fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.703+0000 7fc00a600700 1 --2- 172.21.15.176:0/2959827720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc004079fd0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009910 tx=0x7fbff0009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.704+0000 7fc00a600700 1 -- 172.21.15.176:0/2959827720 shutdown_connections 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.704+0000 7fc00a600700 1 --2- 172.21.15.176:0/2959827720 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc004079fd0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.704+0000 7fc00a600700 1 --2- 172.21.15.176:0/2959827720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc004081820 0x7fc004079610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.704+0000 7fc00a600700 1 -- 172.21.15.176:0/2959827720 >> 172.21.15.176:0/2959827720 conn(0x7fc004076990 msgr2=0x7fc004078db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.704+0000 7fc00a600700 1 -- 172.21.15.176:0/2959827720 shutdown_connections 2024-03-13T15:02:00.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.704+0000 7fc00a600700 1 -- 172.21.15.176:0/2959827720 wait complete. 2024-03-13T15:02:00.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:00 smithi176 ceph-mon[112742]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:00.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:00 smithi176 ceph-mon[112742]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:00.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:00.691 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:00.691 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.705+0000 7fc00a600700 1 Processor -- start 2024-03-13T15:02:00.691 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.706+0000 7fc00a600700 1 -- start start 2024-03-13T15:02:00.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.706+0000 7fc00a600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc0041a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:00.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.706+0000 7fc00a600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc004081820 0x7fc0041a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:00.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.706+0000 7fc00a600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041a6160 con 0x7fc004079b50 2024-03-13T15:02:00.692 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.706+0000 7fc00a600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041a9ef0 con 0x7fc004081820 2024-03-13T15:02:00.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.706+0000 7fc003400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc0041a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:00.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.707+0000 7fc003400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc0041a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55982/0 (socket says 172.21.15.176:55982) 2024-03-13T15:02:00.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.707+0000 7fc003400700 1 -- 172.21.15.176:0/3175610045 learned_addr learned my addr 172.21.15.176:0/3175610045 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:00.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.707+0000 7fc002a00700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc004081820 0x7fc0041a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:00.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.707+0000 7fc003400700 1 -- 172.21.15.176:0/3175610045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc004081820 msgr2=0x7fc0041a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.707+0000 7fc003400700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc004081820 0x7fc0041a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.693 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.707+0000 7fc003400700 1 -- 172.21.15.176:0/3175610045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff00095f0 con 0x7fc004079b50 2024-03-13T15:02:00.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.708+0000 7fc003400700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc0041a5540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e920 tx=0x7fbff400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:00.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.708+0000 7fbffbe00700 1 -- 172.21.15.176:0/3175610045 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff400c9d0 con 0x7fc004079b50 2024-03-13T15:02:00.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.708+0000 7fbffbe00700 1 -- 172.21.15.176:0/3175610045 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbff4010430 con 0x7fc004079b50 2024-03-13T15:02:00.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.708+0000 7fbffbe00700 1 -- 172.21.15.176:0/3175610045 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff4018630 con 0x7fc004079b50 2024-03-13T15:02:00.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.708+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0041aa1d0 con 0x7fc004079b50 2024-03-13T15:02:00.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.708+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0041aa720 con 0x7fc004079b50 2024-03-13T15:02:00.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.709+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00407c9e0 con 0x7fc004079b50 2024-03-13T15:02:00.697 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.711+0000 7fbffbe00700 1 -- 172.21.15.176:0/3175610045 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fbff40105a0 con 0x7fc004079b50 2024-03-13T15:02:00.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.712+0000 7fbffbe00700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbfe806dfa0 0x7fbfe8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:00.698 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.712+0000 7fbffbe00700 1 -- 172.21.15.176:0/3175610045 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fbff4014070 con 0x7fc004079b50 2024-03-13T15:02:00.699 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.712+0000 7fc002a00700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbfe806dfa0 0x7fbfe8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:00.699 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.713+0000 7fc002a00700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbfe806dfa0 0x7fbfe8070460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc0041a6b60 tx=0x7fbff0005a90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:00.702 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.717+0000 7fbffbe00700 1 -- 172.21.15.176:0/3175610045 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fbff40531e0 con 0x7fc004079b50 2024-03-13T15:02:00.972 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.986+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc0041aaa50 con 0x7fc004079b50 2024-03-13T15:02:00.973 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.987+0000 7fbffbe00700 1 -- 172.21.15.176:0/3175610045 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1901 (secure 0 0 0) 0x7fbff4057460 con 0x7fc004079b50 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:e38 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 38 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T15:02:00.974 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T15:01:35.879487+0000 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 112 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=34462} 2024-03-13T15:02:00.975 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{0:34462} state up:active seq 24 join_fscid=1 addr [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:34494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6828/552037800,v1:172.21.15.176:6829/552037800] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:02:00.976 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{-1:44257} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.196:6824/2671769958,v1:172.21.15.196:6825/2671769958] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:02:00.977 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:44281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.196:6826/2135805809,v1:172.21.15.196:6827/2135805809] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:02:00.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbfe806dfa0 msgr2=0x7fbfe8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbfe806dfa0 0x7fbfe8070460 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc0041a6b60 tx=0x7fbff0005a90 comp rx=0 tx=0).stop 2024-03-13T15:02:00.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 msgr2=0x7fc0041a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:00.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc0041a5540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbff400e920 tx=0x7fbff400ec30 comp rx=0 tx=0).stop 2024-03-13T15:02:00.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 shutdown_connections 2024-03-13T15:02:00.977 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbfe806dfa0 0x7fbfe8070460 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc004079b50 0x7fc0041a5540 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 --2- 172.21.15.176:0/3175610045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc004081820 0x7fc0041a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:00.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.990+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 >> 172.21.15.176:0/3175610045 conn(0x7fc004076990 msgr2=0x7fc004078d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:00.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.991+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 shutdown_connections 2024-03-13T15:02:00.978 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:00.991+0000 7fc00a600700 1 -- 172.21.15.176:0/3175610045 wait complete. 2024-03-13T15:02:00.978 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 38 2024-03-13T15:02:01.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 -- 172.21.15.176:0/3519106242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c0731c0 msgr2=0x7fa07c0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 --2- 172.21.15.176:0/3519106242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c0731c0 0x7fa07c0735a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa06c009910 tx=0x7fa06c009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:01.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 -- 172.21.15.176:0/3519106242 shutdown_connections 2024-03-13T15:02:01.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 --2- 172.21.15.176:0/3519106242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07c073ae0 0x7fa07c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 --2- 172.21.15.176:0/3519106242 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c0731c0 0x7fa07c0735a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 -- 172.21.15.176:0/3519106242 >> 172.21.15.176:0/3519106242 conn(0x7fa07c0fc630 msgr2=0x7fa07c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:01.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 -- 172.21.15.176:0/3519106242 shutdown_connections 2024-03-13T15:02:01.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.147+0000 7fa081000700 1 -- 172.21.15.176:0/3519106242 wait complete. 2024-03-13T15:02:01.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.148+0000 7fa081000700 1 Processor -- start 2024-03-13T15:02:01.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.148+0000 7fa081000700 1 -- start start 2024-03-13T15:02:01.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa081000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07c0731c0 0x7fa07c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:01.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa081000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c073ae0 0x7fa07c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:01.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa081000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c19f430 con 0x7fa07c073ae0 2024-03-13T15:02:01.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa081000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c1988e0 con 0x7fa07c0731c0 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa079400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c073ae0 0x7fa07c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa079400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c073ae0 0x7fa07c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56012/0 (socket says 172.21.15.176:56012) 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa079400700 1 -- 172.21.15.176:0/577478443 learned_addr learned my addr 172.21.15.176:0/577478443 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.149+0000 7fa079e00700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07c0731c0 0x7fa07c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.150+0000 7fa079400700 1 -- 172.21.15.176:0/577478443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07c0731c0 msgr2=0x7fa07c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.150+0000 7fa079400700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07c0731c0 0x7fa07c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.150+0000 7fa079400700 1 -- 172.21.15.176:0/577478443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c0095f0 con 0x7fa07c073ae0 2024-03-13T15:02:01.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.150+0000 7fa079400700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c073ae0 0x7fa07c19eda0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa06800b4b0 tx=0x7fa06800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:01.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.150+0000 7fa072a00700 1 -- 172.21.15.176:0/577478443 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa068011800 con 0x7fa07c073ae0 2024-03-13T15:02:01.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.150+0000 7fa072a00700 1 -- 172.21.15.176:0/577478443 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa068011e20 con 0x7fa07c073ae0 2024-03-13T15:02:01.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.150+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c198bc0 con 0x7fa07c073ae0 2024-03-13T15:02:01.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.151+0000 7fa072a00700 1 -- 172.21.15.176:0/577478443 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa06800f2e0 con 0x7fa07c073ae0 2024-03-13T15:02:01.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.151+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c199110 con 0x7fa07c073ae0 2024-03-13T15:02:01.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.153+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c10a540 con 0x7fa07c073ae0 2024-03-13T15:02:01.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.154+0000 7fa072a00700 1 -- 172.21.15.176:0/577478443 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa068010380 con 0x7fa07c073ae0 2024-03-13T15:02:01.140 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.155+0000 7fa072a00700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa06006dfa0 0x7fa060070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:01.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.155+0000 7fa072a00700 1 -- 172.21.15.176:0/577478443 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa06808db20 con 0x7fa07c073ae0 2024-03-13T15:02:01.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.155+0000 7fa079e00700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa06006dfa0 0x7fa060070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:01.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.156+0000 7fa079e00700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa06006dfa0 0x7fa060070460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00b5c0 tx=0x7fa06c01a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:01.146 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.160+0000 7fa072a00700 1 -- 172.21.15.176:0/577478443 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa06805a900 con 0x7fa07c073ae0 2024-03-13T15:02:01.388 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.402+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa07c199400 con 0x7fa06006dfa0 2024-03-13T15:02:01.390 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.404+0000 7fa072a00700 1 -- 172.21.15.176:0/577478443 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fa07c199400 con 0x7fa06006dfa0 2024-03-13T15:02:01.391 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:02:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a", 2024-03-13T15:02:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": true, 2024-03-13T15:02:01.391 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "Upgrading all daemon types on all hosts", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [ 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "mds", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "osd", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "mon", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph-exporter", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "crash" 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: ], 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": "18/23 daemons upgraded", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T15:02:01.392 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:02:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa06006dfa0 msgr2=0x7fa060070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa06006dfa0 0x7fa060070460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa06c00b5c0 tx=0x7fa06c01a040 comp rx=0 tx=0).stop 2024-03-13T15:02:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c073ae0 msgr2=0x7fa07c19eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c073ae0 0x7fa07c19eda0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa06800b4b0 tx=0x7fa06800b870 comp rx=0 tx=0).stop 2024-03-13T15:02:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 shutdown_connections 2024-03-13T15:02:01.393 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa06006dfa0 0x7fa060070460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa07c0731c0 0x7fa07c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 --2- 172.21.15.176:0/577478443 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa07c073ae0 0x7fa07c19eda0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.407+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 >> 172.21.15.176:0/577478443 conn(0x7fa07c0fc630 msgr2=0x7fa07c107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:01.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.408+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 shutdown_connections 2024-03-13T15:02:01.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.408+0000 7fa081000700 1 -- 172.21.15.176:0/577478443 wait complete. 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.563+0000 7f2717400700 1 -- 172.21.15.176:0/77388667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100739d0 msgr2=0x7f271010cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.563+0000 7f2717400700 1 --2- 172.21.15.176:0/77388667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100739d0 0x7f271010cec0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009910 tx=0x7f26fc009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.563+0000 7f2717400700 1 -- 172.21.15.176:0/77388667 shutdown_connections 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.563+0000 7f2717400700 1 --2- 172.21.15.176:0/77388667 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100739d0 0x7f271010cec0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.563+0000 7f2717400700 1 --2- 172.21.15.176:0/77388667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27100730b0 0x7f2710073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.563+0000 7f2717400700 1 -- 172.21.15.176:0/77388667 >> 172.21.15.176:0/77388667 conn(0x7f27100fc630 msgr2=0x7f27100fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.564+0000 7f2717400700 1 -- 172.21.15.176:0/77388667 shutdown_connections 2024-03-13T15:02:01.550 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.564+0000 7f2717400700 1 -- 172.21.15.176:0/77388667 wait complete. 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.564+0000 7f2717400700 1 Processor -- start 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2717400700 1 -- start start 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2717400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100730b0 0x7f271019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2717400700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27100739d0 0x7f271019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2717400700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271019f2b0 con 0x7f27100730b0 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2717400700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710198760 con 0x7f27100739d0 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2714a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100730b0 0x7f271019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2714a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100730b0 0x7f271019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56022/0 (socket says 172.21.15.176:56022) 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.565+0000 7f2714a00700 1 -- 172.21.15.176:0/229192167 learned_addr learned my addr 172.21.15.176:0/229192167 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f270fe00700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27100739d0 0x7f271019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:01.552 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f2714a00700 1 -- 172.21.15.176:0/229192167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27100739d0 msgr2=0x7f271019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f2714a00700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27100739d0 0x7f271019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f2714a00700 1 -- 172.21.15.176:0/229192167 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0095f0 con 0x7f27100730b0 2024-03-13T15:02:01.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f2714a00700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100730b0 0x7f271019e6e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f270400d8d0 tx=0x7f270400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:01.553 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f270d600700 1 -- 172.21.15.176:0/229192167 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27040098a0 con 0x7f27100730b0 2024-03-13T15:02:01.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f270d600700 1 -- 172.21.15.176:0/229192167 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2704010460 con 0x7f27100730b0 2024-03-13T15:02:01.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.566+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2710198a40 con 0x7f27100730b0 2024-03-13T15:02:01.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.567+0000 7f270d600700 1 -- 172.21.15.176:0/229192167 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f270400f5f0 con 0x7f27100730b0 2024-03-13T15:02:01.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.567+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2710198f60 con 0x7f27100730b0 2024-03-13T15:02:01.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.568+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271010a5c0 con 0x7f27100730b0 2024-03-13T15:02:01.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.569+0000 7f270d600700 1 -- 172.21.15.176:0/229192167 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f2704009a00 con 0x7f27100730b0 2024-03-13T15:02:01.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.569+0000 7f270d600700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f26f806dfa0 0x7f26f8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:01.555 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.570+0000 7f270d600700 1 -- 172.21.15.176:0/229192167 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f270408dd30 con 0x7f27100730b0 2024-03-13T15:02:01.556 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.570+0000 7f270fe00700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f26f806dfa0 0x7f26f8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:01.556 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.570+0000 7f270fe00700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f26f806dfa0 0x7f26f8070460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00b5c0 tx=0x7f26fc01a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:01.560 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.574+0000 7f270d600700 1 -- 172.21.15.176:0/229192167 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f270405ab90 con 0x7f27100730b0 2024-03-13T15:02:01.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:01 smithi176 ceph-mon[112742]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:01.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:01 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:01.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:01 smithi176 ceph-mon[112742]: pgmap v305: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:01.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:01 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1482308093' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:01.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:01 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3175610045' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:02:01.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.904+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f271004ea90 con 0x7f27100730b0 2024-03-13T15:02:01.890 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.905+0000 7f270d600700 1 -- 172.21.15.176:0/229192167 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f270405a720 con 0x7f27100730b0 2024-03-13T15:02:01.890 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f26f806dfa0 msgr2=0x7f26f8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f26f806dfa0 0x7f26f8070460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f26fc00b5c0 tx=0x7f26fc01a040 comp rx=0 tx=0).stop 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100730b0 msgr2=0x7f271019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100730b0 0x7f271019e6e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f270400d8d0 tx=0x7f270400dbe0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 shutdown_connections 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f26f806dfa0 0x7f26f8070460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27100730b0 0x7f271019e6e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 --2- 172.21.15.176:0/229192167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27100739d0 0x7f271019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.907+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 >> 172.21.15.176:0/229192167 conn(0x7f27100fc630 msgr2=0x7f2710107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.908+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 shutdown_connections 2024-03-13T15:02:01.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:01.908+0000 7f2717400700 1 -- 172.21.15.176:0/229192167 wait complete. 2024-03-13T15:02:01.916 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:01 smithi196 ceph-mon[86193]: from='client.34564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:01.916 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:01 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:01.917 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:01 smithi196 ceph-mon[86193]: pgmap v305: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:01.917 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:01 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1482308093' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:01.917 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:01 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3175610045' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:02:02.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:02 smithi196 ceph-mon[86193]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:02.676 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:02 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/229192167' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:02:02.804 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:02 smithi176 ceph-mon[112742]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:02.804 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:02 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/229192167' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:02:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:03 smithi176 ceph-mon[112742]: pgmap v306: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:03.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:03 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:03.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:03 smithi196 ceph-mon[86193]: pgmap v306: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:03.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:03.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:03 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:05.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:05 smithi176 ceph-mon[112742]: pgmap v307: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:05.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:05 smithi196 ceph-mon[86193]: pgmap v307: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.177 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: Upgrade: Setting filesystem cephfs Joinable 2024-03-13T15:02:07.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-13T15:02:07.178 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:06 smithi196 ceph-mon[86193]: pgmap v308: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.376 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: Upgrade: Setting filesystem cephfs Joinable 2024-03-13T15:02:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-03-13T15:02:07.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:06 smithi176 ceph-mon[112742]: pgmap v308: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all ceph-exporter 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi176"}]: dispatch 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi176"}]': finished 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi196"}]: dispatch 2024-03-13T15:02:08.377 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi196"}]': finished 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all iscsi 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all nfs 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: Upgrade: Setting container_image for all nvmeof 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.378 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: Upgrade: Finalizing container_image settings 2024-03-13T15:02:08.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-13T15:02:08.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-13T15:02:08.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-13T15:02:08.379 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-13T15:02:08.380 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: Upgrade: Complete! 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:02:08.381 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:02:08.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:02:08.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:02:08.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:02:08.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:02:08.382 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:07 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-03-13T15:02:08.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: fsmap cephfs:1 {0=cephfs.smithi176.ldskqt=up:active} 1 up:standby-replay 2 up:standby 2024-03-13T15:02:08.426 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all ceph-exporter 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi176"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi176"}]': finished 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi196"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi196"}]': finished 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all iscsi 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all nfs 2024-03-13T15:02:08.427 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: Upgrade: Setting container_image for all nvmeof 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.428 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: Upgrade: Finalizing container_image settings 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-03-13T15:02:08.429 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.430 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: Upgrade: Complete! 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:02:08.431 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:07 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:09.375 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:08 smithi176 ceph-mon[112742]: pgmap v309: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:09.425 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:08 smithi196 ceph-mon[86193]: pgmap v309: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:11.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:11 smithi176 ceph-mon[112742]: pgmap v310: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:11.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:11 smithi196 ceph-mon[86193]: pgmap v310: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:02:13.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:13 smithi176 ceph-mon[112742]: pgmap v311: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:13.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:13 smithi196 ceph-mon[86193]: pgmap v311: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:15.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:15 smithi176 ceph-mon[112742]: pgmap v312: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:15.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:15 smithi196 ceph-mon[86193]: pgmap v312: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:15.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:17.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:17 smithi176 ceph-mon[112742]: pgmap v313: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:17.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:17 smithi196 ceph-mon[86193]: pgmap v313: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:19.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:19 smithi176 ceph-mon[112742]: pgmap v314: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:19.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:19 smithi196 ceph-mon[86193]: pgmap v314: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:21.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:21 smithi176 ceph-mon[112742]: pgmap v315: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:21.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:21 smithi196 ceph-mon[86193]: pgmap v315: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:23.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:23 smithi176 ceph-mon[112742]: pgmap v316: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:23.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:23 smithi196 ceph-mon[86193]: pgmap v316: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:25.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:25 smithi176 ceph-mon[112742]: pgmap v317: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:25.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:25 smithi196 ceph-mon[86193]: pgmap v317: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:27.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:27 smithi176 ceph-mon[112742]: pgmap v318: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:27.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:27 smithi196 ceph-mon[86193]: pgmap v318: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:29.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:29 smithi176 ceph-mon[112742]: pgmap v319: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:29.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:29 smithi196 ceph-mon[86193]: pgmap v319: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:30.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:30.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:31.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:31 smithi176 ceph-mon[112742]: pgmap v320: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:31.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:31 smithi196 ceph-mon[86193]: pgmap v320: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:32.066 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 -- 172.21.15.176:0/2226393456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073390 msgr2=0x7f4b6c073770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:32.066 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 --2- 172.21.15.176:0/2226393456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073390 0x7f4b6c073770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009940 tx=0x7f4b60009c50 comp rx=0 tx=0).stop 2024-03-13T15:02:32.066 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 -- 172.21.15.176:0/2226393456 shutdown_connections 2024-03-13T15:02:32.066 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 --2- 172.21.15.176:0/2226393456 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6c073cb0 0x7f4b6c10cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:32.066 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 --2- 172.21.15.176:0/2226393456 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073390 0x7f4b6c073770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:32.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 -- 172.21.15.176:0/2226393456 >> 172.21.15.176:0/2226393456 conn(0x7f4b6c0fc4d0 msgr2=0x7f4b6c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:32.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 -- 172.21.15.176:0/2226393456 shutdown_connections 2024-03-13T15:02:32.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.076+0000 7f4b72200700 1 -- 172.21.15.176:0/2226393456 wait complete. 2024-03-13T15:02:32.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.077+0000 7f4b72200700 1 Processor -- start 2024-03-13T15:02:32.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b72200700 1 -- start start 2024-03-13T15:02:32.067 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b72200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6c073390 0x7f4b6c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:32.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b72200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073cb0 0x7f4b6c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:32.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b72200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c19f150 con 0x7f4b6c073cb0 2024-03-13T15:02:32.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b72200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c198600 con 0x7f4b6c073390 2024-03-13T15:02:32.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b6be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073cb0 0x7f4b6c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:32.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b6be00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073cb0 0x7f4b6c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56512/0 (socket says 172.21.15.176:56512) 2024-03-13T15:02:32.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.078+0000 7f4b6be00700 1 -- 172.21.15.176:0/2357379373 learned_addr learned my addr 172.21.15.176:0/2357379373 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:32.068 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.079+0000 7f4b70e00700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6c073390 0x7f4b6c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:32.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.079+0000 7f4b6be00700 1 -- 172.21.15.176:0/2357379373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6c073390 msgr2=0x7f4b6c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:32.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.079+0000 7f4b6be00700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6c073390 0x7f4b6c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:32.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.079+0000 7f4b6be00700 1 -- 172.21.15.176:0/2357379373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b600095f0 con 0x7f4b6c073cb0 2024-03-13T15:02:32.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.079+0000 7f4b6be00700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073cb0 0x7f4b6c19eac0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800e8f0 tx=0x7f4b5800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:32.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.079+0000 7f4b69600700 1 -- 172.21.15.176:0/2357379373 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b58009790 con 0x7f4b6c073cb0 2024-03-13T15:02:32.069 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.080+0000 7f4b69600700 1 -- 172.21.15.176:0/2357379373 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b5800c810 con 0x7f4b6c073cb0 2024-03-13T15:02:32.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.080+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c1988e0 con 0x7f4b6c073cb0 2024-03-13T15:02:32.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.080+0000 7f4b69600700 1 -- 172.21.15.176:0/2357379373 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b58005580 con 0x7f4b6c073cb0 2024-03-13T15:02:32.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.080+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c198e30 con 0x7f4b6c073cb0 2024-03-13T15:02:32.070 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.081+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c10a3a0 con 0x7f4b6c073cb0 2024-03-13T15:02:32.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.082+0000 7f4b69600700 1 -- 172.21.15.176:0/2357379373 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4b5800c980 con 0x7f4b6c073cb0 2024-03-13T15:02:32.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.083+0000 7f4b69600700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b5406dff0 0x7f4b540704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:32.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.083+0000 7f4b69600700 1 -- 172.21.15.176:0/2357379373 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f4b58014070 con 0x7f4b6c073cb0 2024-03-13T15:02:32.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.083+0000 7f4b70e00700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b5406dff0 0x7f4b540704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:32.071 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.084+0000 7f4b70e00700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b5406dff0 0x7f4b540704b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006010 tx=0x7f4b6000b5a0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:32.072 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.088+0000 7f4b69600700 1 -- 172.21.15.176:0/2357379373 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4b5805af80 con 0x7f4b6c073cb0 2024-03-13T15:02:32.315 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.330+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b6c199150 con 0x7f4b5406dff0 2024-03-13T15:02:32.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.333+0000 7f4b69600700 1 -- 172.21.15.176:0/2357379373 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f4b6c199150 con 0x7f4b5406dff0 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b5406dff0 msgr2=0x7f4b540704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b5406dff0 0x7f4b540704b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006010 tx=0x7f4b6000b5a0 comp rx=0 tx=0).stop 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073cb0 msgr2=0x7f4b6c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073cb0 0x7f4b6c19eac0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4b5800e8f0 tx=0x7f4b5800ec00 comp rx=0 tx=0).stop 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 shutdown_connections 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f4b5406dff0 0x7f4b540704b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b6c073390 0x7f4b6c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:32.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 --2- 172.21.15.176:0/2357379373 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4b6c073cb0 0x7f4b6c19eac0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:32.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 >> 172.21.15.176:0/2357379373 conn(0x7f4b6c0fc4d0 msgr2=0x7f4b6c1074e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:32.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 shutdown_connections 2024-03-13T15:02:32.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:32.336+0000 7f4b72200700 1 -- 172.21.15.176:0/2357379373 wait complete. 2024-03-13T15:02:32.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:32 smithi196 ceph-mon[86193]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:32.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:32 smithi196 ceph-mon[86193]: pgmap v321: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:33.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:32 smithi176 ceph-mon[112742]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:33.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:32 smithi176 ceph-mon[112742]: pgmap v321: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:33.195 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph orch ps' 2024-03-13T15:02:33.673 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:02:35.560 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:35 smithi176 ceph-mon[112742]: pgmap v322: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 -- 172.21.15.176:0/1474562637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 msgr2=0x7f3b20107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 --2- 172.21.15.176:0/1474562637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b20107a00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009910 tx=0x7f3b0c009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 -- 172.21.15.176:0/1474562637 shutdown_connections 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 --2- 172.21.15.176:0/1474562637 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b20107a00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 --2- 172.21.15.176:0/1474562637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b20103000 0x7f3b201033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 -- 172.21.15.176:0/1474562637 >> 172.21.15.176:0/1474562637 conn(0x7f3b200fe8d0 msgr2=0x7f3b20100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 -- 172.21.15.176:0/1474562637 shutdown_connections 2024-03-13T15:02:35.800 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.814+0000 7f3b27200700 1 -- 172.21.15.176:0/1474562637 wait complete. 2024-03-13T15:02:35.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.815+0000 7f3b27200700 1 Processor -- start 2024-03-13T15:02:35.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b27200700 1 -- start start 2024-03-13T15:02:35.802 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b27200700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b20103000 0x7f3b2019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b27200700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b2019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b27200700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2019f390 con 0x7f3b201039b0 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b27200700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b20198840 con 0x7f3b20103000 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b1f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b2019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b1f400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b2019ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56526/0 (socket says 172.21.15.176:56526) 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.816+0000 7f3b1f400700 1 -- 172.21.15.176:0/2491782961 learned_addr learned my addr 172.21.15.176:0/2491782961 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.817+0000 7f3b1fe00700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b20103000 0x7f3b2019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.817+0000 7f3b1f400700 1 -- 172.21.15.176:0/2491782961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b20103000 msgr2=0x7f3b2019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.817+0000 7f3b1f400700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b20103000 0x7f3b2019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.817+0000 7f3b1f400700 1 -- 172.21.15.176:0/2491782961 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0095f0 con 0x7f3b201039b0 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.817+0000 7f3b1f400700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b2019ed00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00bda0 tx=0x7f3b0c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:35.803 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.817+0000 7f3b1cc00700 1 -- 172.21.15.176:0/2491782961 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b0c01d070 con 0x7f3b201039b0 2024-03-13T15:02:35.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.818+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b20198ac0 con 0x7f3b201039b0 2024-03-13T15:02:35.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.818+0000 7f3b1cc00700 1 -- 172.21.15.176:0/2491782961 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b0c00f460 con 0x7f3b201039b0 2024-03-13T15:02:35.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.818+0000 7f3b1cc00700 1 -- 172.21.15.176:0/2491782961 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b0c017610 con 0x7f3b201039b0 2024-03-13T15:02:35.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.818+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b20198fb0 con 0x7f3b201039b0 2024-03-13T15:02:35.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.819+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2010b400 con 0x7f3b201039b0 2024-03-13T15:02:35.804 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.820+0000 7f3b1cc00700 1 -- 172.21.15.176:0/2491782961 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f3b0c00fa40 con 0x7f3b201039b0 2024-03-13T15:02:35.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.821+0000 7f3b1cc00700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3b0406dfa0 0x7f3b04070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:35.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.821+0000 7f3b1cc00700 1 -- 172.21.15.176:0/2491782961 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f3b0c08f2f0 con 0x7f3b201039b0 2024-03-13T15:02:35.806 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.821+0000 7f3b1fe00700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3b0406dfa0 0x7f3b04070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:35.807 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.822+0000 7f3b1fe00700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3b0406dfa0 0x7f3b04070460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3b10005950 tx=0x7f3b100058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:35.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:35.826+0000 7f3b1cc00700 1 -- 172.21.15.176:0/2491782961 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f3b0c05c150 con 0x7f3b201039b0 2024-03-13T15:02:35.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:35 smithi196 ceph-mon[86193]: pgmap v322: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:36.041 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.056+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b201992e0 con 0x7f3b0406dfa0 2024-03-13T15:02:36.058 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.073+0000 7f3b1cc00700 1 -- 172.21.15.176:0/2491782961 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f3b201992e0 con 0x7f3b0406dfa0 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (10m) 37s ago 24m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (52s) 37s ago 24m 7818k - 19.0.0-1916-g9a75c81f 9552dbf7f115 06b476c807b4 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (44s) 40s ago 22m 6945k - 19.0.0-1916-g9a75c81f 9552dbf7f115 0f12bb84f0f6 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (7m) 37s ago 24m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (7m) 40s ago 22m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (9m) 37s ago 23m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (2m) 37s ago 16m 93.6M - 19.0.0-1916-g9a75c81f 9552dbf7f115 edc682c345ac 2024-03-13T15:02:36.059 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (2m) 37s ago 16m 85.7M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5b6d2dc0d458 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (105s) 40s ago 16m 12.2M - 19.0.0-1916-g9a75c81f 9552dbf7f115 a9081f4c2ead 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (74s) 40s ago 16m 11.9M - 19.0.0-1916-g9a75c81f 9552dbf7f115 2e3acf7133a0 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (13m) 37s ago 25m 511M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (12m) 40s ago 22m 452M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (8m) 37s ago 26m 60.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (8m) 40s ago 22m 43.9M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (11m) 37s ago 24m 8186k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (11m) 40s ago 22m 8140k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (6m) 37s ago 21m 144M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (5m) 37s ago 20m 113M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (5m) 37s ago 20m 119M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (4m) 40s ago 19m 137M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (3m) 40s ago 19m 104M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (3m) 40s ago 18m 105M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 43366d2dc8da 2024-03-13T15:02:36.060 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (11m) 37s ago 23m 72.0M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T15:02:36.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3b0406dfa0 msgr2=0x7f3b04070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:36.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3b0406dfa0 0x7f3b04070460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3b10005950 tx=0x7f3b100058e0 comp rx=0 tx=0).stop 2024-03-13T15:02:36.061 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 msgr2=0x7f3b2019ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b2019ed00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c00bda0 tx=0x7f3b0c00be80 comp rx=0 tx=0).stop 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 shutdown_connections 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3b0406dfa0 0x7f3b04070460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b20103000 0x7f3b2019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 --2- 172.21.15.176:0/2491782961 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3b201039b0 0x7f3b2019ed00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 >> 172.21.15.176:0/2491782961 conn(0x7f3b200fe8d0 msgr2=0x7f3b20100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.076+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 shutdown_connections 2024-03-13T15:02:36.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:36.077+0000 7f3b27200700 1 -- 172.21.15.176:0/2491782961 wait complete. 2024-03-13T15:02:37.038 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph orch upgrade status' 2024-03-13T15:02:37.438 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:02:37.796 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:37 smithi176 ceph-mon[112742]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:37.796 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:37 smithi176 ceph-mon[112742]: pgmap v323: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:37.867 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:37 smithi196 ceph-mon[86193]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:37.867 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:37 smithi196 ceph-mon[86193]: pgmap v323: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:39.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.433+0000 7fac57e00700 1 -- 172.21.15.176:0/2498848580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 msgr2=0x7fac501057a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:39.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.433+0000 7fac57e00700 1 --2- 172.21.15.176:0/2498848580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac501057a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009910 tx=0x7fac3c009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:39.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.434+0000 7fac57e00700 1 -- 172.21.15.176:0/2498848580 shutdown_connections 2024-03-13T15:02:39.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.434+0000 7fac57e00700 1 --2- 172.21.15.176:0/2498848580 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac501057a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:39.419 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.434+0000 7fac57e00700 1 --2- 172.21.15.176:0/2498848580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50100da0 0x7fac50101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:39.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.434+0000 7fac57e00700 1 -- 172.21.15.176:0/2498848580 >> 172.21.15.176:0/2498848580 conn(0x7fac500fc630 msgr2=0x7fac500fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:39.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.434+0000 7fac57e00700 1 -- 172.21.15.176:0/2498848580 shutdown_connections 2024-03-13T15:02:39.420 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.434+0000 7fac57e00700 1 -- 172.21.15.176:0/2498848580 wait complete. 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.435+0000 7fac57e00700 1 Processor -- start 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.435+0000 7fac57e00700 1 -- start start 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.435+0000 7fac57e00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50100da0 0x7fac5019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac57e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac5019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac57e00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5019d270 con 0x7fac50101750 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac57e00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac50196720 con 0x7fac50100da0 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac54a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac5019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac54a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac5019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:56548/0 (socket says 172.21.15.176:56548) 2024-03-13T15:02:39.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac54a00700 1 -- 172.21.15.176:0/1828470750 learned_addr learned my addr 172.21.15.176:0/1828470750 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:39.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac55400700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50100da0 0x7fac5019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:39.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac54a00700 1 -- 172.21.15.176:0/1828470750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50100da0 msgr2=0x7fac5019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:39.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac54a00700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50100da0 0x7fac5019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:39.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.436+0000 7fac54a00700 1 -- 172.21.15.176:0/1828470750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3c0095f0 con 0x7fac50101750 2024-03-13T15:02:39.422 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.437+0000 7fac54a00700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac5019cbe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00bda0 tx=0x7fac3c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:39.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.437+0000 7fac4e000700 1 -- 172.21.15.176:0/1828470750 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac3c01d070 con 0x7fac50101750 2024-03-13T15:02:39.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.437+0000 7fac4e000700 1 -- 172.21.15.176:0/1828470750 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac3c004fb0 con 0x7fac50101750 2024-03-13T15:02:39.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.437+0000 7fac4e000700 1 -- 172.21.15.176:0/1828470750 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac3c00f460 con 0x7fac50101750 2024-03-13T15:02:39.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.437+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac501969a0 con 0x7fac50101750 2024-03-13T15:02:39.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.437+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac50196e90 con 0x7fac50101750 2024-03-13T15:02:39.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.439+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac501090f0 con 0x7fac50101750 2024-03-13T15:02:39.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.439+0000 7fac4e000700 1 -- 172.21.15.176:0/1828470750 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fac3c022800 con 0x7fac50101750 2024-03-13T15:02:39.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.440+0000 7fac4e000700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fac3806dfa0 0x7fac38070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:39.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.440+0000 7fac4e000700 1 -- 172.21.15.176:0/1828470750 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fac3c08f2a0 con 0x7fac50101750 2024-03-13T15:02:39.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.441+0000 7fac55400700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fac3806dfa0 0x7fac38070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:39.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.441+0000 7fac55400700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fac3806dfa0 0x7fac38070460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fac44009f20 tx=0x7fac440094f0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:39.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.445+0000 7fac4e000700 1 -- 172.21.15.176:0/1828470750 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fac3c05c370 con 0x7fac50101750 2024-03-13T15:02:39.675 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.690+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac5002cce0 con 0x7fac3806dfa0 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.692+0000 7fac4e000700 1 -- 172.21.15.176:0/1828470750 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fac5002cce0 con 0x7fac3806dfa0 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout: "target_image": null, 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout: "in_progress": false, 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout: "which": "", 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout: "services_complete": [], 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout: "progress": null, 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout: "message": "", 2024-03-13T15:02:39.677 INFO:teuthology.orchestra.run.smithi176.stdout: "is_paused": false 2024-03-13T15:02:39.678 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:02:39.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fac3806dfa0 msgr2=0x7fac38070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:39.679 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fac3806dfa0 0x7fac38070460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fac44009f20 tx=0x7fac440094f0 comp rx=0 tx=0).stop 2024-03-13T15:02:39.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 msgr2=0x7fac5019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:39.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac5019cbe0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00bda0 tx=0x7fac3c00be80 comp rx=0 tx=0).stop 2024-03-13T15:02:39.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 shutdown_connections 2024-03-13T15:02:39.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fac3806dfa0 0x7fac38070460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:39.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac50100da0 0x7fac5019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:39.680 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 --2- 172.21.15.176:0/1828470750 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fac50101750 0x7fac5019cbe0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:39.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 >> 172.21.15.176:0/1828470750 conn(0x7fac500fc630 msgr2=0x7fac500fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:39.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 shutdown_connections 2024-03-13T15:02:39.681 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:39.695+0000 7fac57e00700 1 -- 172.21.15.176:0/1828470750 wait complete. 2024-03-13T15:02:39.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:39 smithi176 ceph-mon[112742]: pgmap v324: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:39.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:39 smithi196 ceph-mon[86193]: pgmap v324: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:40.425 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph health detail' 2024-03-13T15:02:40.835 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:02:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:41 smithi176 ceph-mon[112742]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:41.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:41 smithi176 ceph-mon[112742]: pgmap v325: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:41.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:41 smithi196 ceph-mon[86193]: from='client.34592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:02:41.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:41 smithi196 ceph-mon[86193]: pgmap v325: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:43.075 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 -- 172.21.15.176:0/3197632884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 msgr2=0x7f7568107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 --2- 172.21.15.176:0/3197632884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f7568107a00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7554009910 tx=0x7f7554009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 -- 172.21.15.176:0/3197632884 shutdown_connections 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 --2- 172.21.15.176:0/3197632884 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f7568107a00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 --2- 172.21.15.176:0/3197632884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7568103000 0x7f75681033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 -- 172.21.15.176:0/3197632884 >> 172.21.15.176:0/3197632884 conn(0x7f75680fe8d0 msgr2=0x7f7568100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 -- 172.21.15.176:0/3197632884 shutdown_connections 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.090+0000 7f7570000700 1 -- 172.21.15.176:0/3197632884 wait complete. 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.091+0000 7f7570000700 1 Processor -- start 2024-03-13T15:02:43.076 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.091+0000 7f7570000700 1 -- start start 2024-03-13T15:02:43.077 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.091+0000 7f7570000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7568103000 0x7f756819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f7570000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f756819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f7570000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756819f390 con 0x7f75681039b0 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f7570000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7568198840 con 0x7f7568103000 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f756cc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f756819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f756cc00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f756819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37848/0 (socket says 172.21.15.176:37848) 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f756cc00700 1 -- 172.21.15.176:0/1141195861 learned_addr learned my addr 172.21.15.176:0/1141195861 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f756cc00700 1 -- 172.21.15.176:0/1141195861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7568103000 msgr2=0x7f756819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f756cc00700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7568103000 0x7f756819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f756cc00700 1 -- 172.21.15.176:0/1141195861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75540095f0 con 0x7f75681039b0 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.092+0000 7f756cc00700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f756819ed00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f755400bda0 tx=0x7f755400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.093+0000 7f7566000700 1 -- 172.21.15.176:0/1141195861 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f755401d070 con 0x7f75681039b0 2024-03-13T15:02:43.078 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.093+0000 7f7566000700 1 -- 172.21.15.176:0/1141195861 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f755400f460 con 0x7f75681039b0 2024-03-13T15:02:43.079 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.093+0000 7f7566000700 1 -- 172.21.15.176:0/1141195861 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7554017610 con 0x7f75681039b0 2024-03-13T15:02:43.079 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.093+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7568198ac0 con 0x7f75681039b0 2024-03-13T15:02:43.079 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.093+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7568198fb0 con 0x7f75681039b0 2024-03-13T15:02:43.079 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.094+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756810b400 con 0x7f75681039b0 2024-03-13T15:02:43.079 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.095+0000 7f7566000700 1 -- 172.21.15.176:0/1141195861 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f755400fa40 con 0x7f75681039b0 2024-03-13T15:02:43.080 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.096+0000 7f7566000700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f755006dfa0 0x7f7550070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:43.081 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.096+0000 7f7566000700 1 -- 172.21.15.176:0/1141195861 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f755408f3a0 con 0x7f75681039b0 2024-03-13T15:02:43.081 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.096+0000 7f756d600700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f755006dfa0 0x7f7550070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:43.081 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.097+0000 7f756d600700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f755006dfa0 0x7f7550070460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f755c005950 tx=0x7f755c0058e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:43.086 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.101+0000 7f7566000700 1 -- 172.21.15.176:0/1141195861 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f755405c180 con 0x7f75681039b0 2024-03-13T15:02:43.394 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.409+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f756810ade0 con 0x7f75681039b0 2024-03-13T15:02:43.395 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.410+0000 7f7566000700 1 -- 172.21.15.176:0/1141195861 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f755405bd10 con 0x7f75681039b0 2024-03-13T15:02:43.395 INFO:teuthology.orchestra.run.smithi176.stdout:HEALTH_OK 2024-03-13T15:02:43.397 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f755006dfa0 msgr2=0x7f7550070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f755006dfa0 0x7f7550070460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f755c005950 tx=0x7f755c0058e0 comp rx=0 tx=0).stop 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 msgr2=0x7f756819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f756819ed00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f755400bda0 tx=0x7f755400be80 comp rx=0 tx=0).stop 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 shutdown_connections 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f755006dfa0 0x7f7550070460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7568103000 0x7f756819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 --2- 172.21.15.176:0/1141195861 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f75681039b0 0x7f756819ed00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.412+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 >> 172.21.15.176:0/1141195861 conn(0x7f75680fe8d0 msgr2=0x7f7568100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.413+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 shutdown_connections 2024-03-13T15:02:43.398 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:43.413+0000 7f7570000700 1 -- 172.21.15.176:0/1141195861 wait complete. 2024-03-13T15:02:43.782 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:43 smithi176 ceph-mon[112742]: pgmap v326: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:43.783 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:43 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1141195861' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:02:43.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:43 smithi196 ceph-mon[86193]: pgmap v326: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:43.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:43 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1141195861' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-03-13T15:02:44.143 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph versions' 2024-03-13T15:02:44.531 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:02:45.808 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:45 smithi176 ceph-mon[112742]: pgmap v327: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:45.808 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:45.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:45 smithi196 ceph-mon[86193]: pgmap v327: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:45.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:02:46.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 -- 172.21.15.176:0/1970180519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 msgr2=0x7fedf8107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:46.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 --2- 172.21.15.176:0/1970180519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8107a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fede4009910 tx=0x7fede4009c20 comp rx=0 tx=0).stop 2024-03-13T15:02:46.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 -- 172.21.15.176:0/1970180519 shutdown_connections 2024-03-13T15:02:46.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 --2- 172.21.15.176:0/1970180519 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8107a00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:46.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 --2- 172.21.15.176:0/1970180519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf8103000 0x7fedf81033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:46.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 -- 172.21.15.176:0/1970180519 >> 172.21.15.176:0/1970180519 conn(0x7fedf80fe850 msgr2=0x7fedf8100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:46.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 -- 172.21.15.176:0/1970180519 shutdown_connections 2024-03-13T15:02:46.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.646+0000 7fedfec00700 1 -- 172.21.15.176:0/1970180519 wait complete. 2024-03-13T15:02:46.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.647+0000 7fedfec00700 1 Processor -- start 2024-03-13T15:02:46.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.647+0000 7fedfec00700 1 -- start start 2024-03-13T15:02:46.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.647+0000 7fedfec00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf8103000 0x7fedf8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:46.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.647+0000 7fedfec00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.647+0000 7fedfec00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf8199520 con 0x7fedf81039b0 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.647+0000 7fedfec00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf819d330 con 0x7fedf8103000 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.648+0000 7fedf7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.648+0000 7fedf7400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:37856/0 (socket says 172.21.15.176:37856) 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.648+0000 7fedf7400700 1 -- 172.21.15.176:0/39736172 learned_addr learned my addr 172.21.15.176:0/39736172 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.648+0000 7fedf7e00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf8103000 0x7fedf8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.648+0000 7fedf7400700 1 -- 172.21.15.176:0/39736172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf8103000 msgr2=0x7fedf8198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.648+0000 7fedf7400700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf8103000 0x7fedf8198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.648+0000 7fedf7400700 1 -- 172.21.15.176:0/39736172 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede40095f0 con 0x7fedf81039b0 2024-03-13T15:02:46.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.649+0000 7fedf7400700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8198e90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fede400bda0 tx=0x7fede400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:46.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.649+0000 7fedf4c00700 1 -- 172.21.15.176:0/39736172 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede401d070 con 0x7fedf81039b0 2024-03-13T15:02:46.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.649+0000 7fedf4c00700 1 -- 172.21.15.176:0/39736172 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fede4004fb0 con 0x7fedf81039b0 2024-03-13T15:02:46.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.649+0000 7fedf4c00700 1 -- 172.21.15.176:0/39736172 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fede400f460 con 0x7fedf81039b0 2024-03-13T15:02:46.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.649+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf819d5b0 con 0x7fedf81039b0 2024-03-13T15:02:46.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.649+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf819daa0 con 0x7fedf81039b0 2024-03-13T15:02:46.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.650+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf810b310 con 0x7fedf81039b0 2024-03-13T15:02:46.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.651+0000 7fedf4c00700 1 -- 172.21.15.176:0/39736172 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fede4022800 con 0x7fedf81039b0 2024-03-13T15:02:46.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.652+0000 7fedf4c00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7feddc06dfa0 0x7feddc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:02:46.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.652+0000 7fedf4c00700 1 -- 172.21.15.176:0/39736172 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fede408f400 con 0x7fedf81039b0 2024-03-13T15:02:46.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.652+0000 7fedf7e00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7feddc06dfa0 0x7feddc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:02:46.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.653+0000 7fedf7e00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7feddc06dfa0 0x7feddc070460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fede8005950 tx=0x7fede8005eb0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:02:46.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.658+0000 7fedf4c00700 1 -- 172.21.15.176:0/39736172 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fede405c450 con 0x7fedf81039b0 2024-03-13T15:02:46.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.963+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fedf804ea90 con 0x7fedf81039b0 2024-03-13T15:02:46.949 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.964+0000 7fedf4c00700 1 -- 172.21.15.176:0/39736172 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fede405bfe0 con 0x7fedf81039b0 2024-03-13T15:02:46.949 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:02:46.949 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T15:02:46.949 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:02:46.949 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:46.949 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T15:02:46.949 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 14 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T15:02:46.950 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:02:46.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7feddc06dfa0 msgr2=0x7feddc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:46.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7feddc06dfa0 0x7feddc070460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fede8005950 tx=0x7fede8005eb0 comp rx=0 tx=0).stop 2024-03-13T15:02:46.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 msgr2=0x7fedf8198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:02:46.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8198e90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fede400bda0 tx=0x7fede400be80 comp rx=0 tx=0).stop 2024-03-13T15:02:46.952 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 shutdown_connections 2024-03-13T15:02:46.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7feddc06dfa0 0x7feddc070460 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:46.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fedf8103000 0x7fedf8198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:46.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 --2- 172.21.15.176:0/39736172 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fedf81039b0 0x7fedf8198e90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:02:46.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.967+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 >> 172.21.15.176:0/39736172 conn(0x7fedf80fe850 msgr2=0x7fedf8107270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:02:46.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.968+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 shutdown_connections 2024-03-13T15:02:46.953 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:02:46.968+0000 7fedfec00700 1 -- 172.21.15.176:0/39736172 wait complete. 2024-03-13T15:02:47.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:47 smithi176 ceph-mon[112742]: pgmap v328: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:47.626 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:47 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/39736172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:47.803 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-03-13T15:02:47.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:47 smithi196 ceph-mon[86193]: pgmap v328: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:47.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:47 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/39736172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:02:48.214 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:02:49.776 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:49 smithi176 ceph-mon[112742]: pgmap v329: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:49.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:49 smithi196 ceph-mon[86193]: pgmap v329: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:50.234 INFO:teuthology.orchestra.run.smithi176.stdout:wait for servicemap items w/ changing names to refresh 2024-03-13T15:02:50.940 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'sleep 60' 2024-03-13T15:02:51.381 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:02:51.844 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:51 smithi176 ceph-mon[112742]: pgmap v330: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:51.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:51 smithi196 ceph-mon[86193]: pgmap v330: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:53.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:53 smithi176 ceph-mon[112742]: pgmap v331: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:53.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:53 smithi196 ceph-mon[86193]: pgmap v331: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:55.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:55 smithi176 ceph-mon[112742]: pgmap v332: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:55.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:55 smithi196 ceph-mon[86193]: pgmap v332: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:02:57.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:57 smithi176 ceph-mon[112742]: pgmap v333: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:57.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:57 smithi196 ceph-mon[86193]: pgmap v333: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:02:59.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:02:59 smithi176 ceph-mon[112742]: pgmap v334: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:02:59.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:02:59 smithi196 ceph-mon[86193]: pgmap v334: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:00.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:00.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:01.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:01 smithi176 ceph-mon[112742]: pgmap v335: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:01.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:01 smithi196 ceph-mon[86193]: pgmap v335: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:03.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:03 smithi176 ceph-mon[112742]: pgmap v336: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:03.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:03 smithi196 ceph-mon[86193]: pgmap v336: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:04.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:04 smithi196 ceph-mon[86193]: pgmap v337: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:05.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:04 smithi176 ceph-mon[112742]: pgmap v337: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:07.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:07 smithi176 ceph-mon[112742]: pgmap v338: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:07.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:07 smithi196 ceph-mon[86193]: pgmap v338: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:08.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:08 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:03:08.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:08 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:03:09.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:09 smithi176 ceph-mon[112742]: pgmap v339: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:03:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:03:09.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:03:09.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:09 smithi196 ceph-mon[86193]: pgmap v339: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:03:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:03:09.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:03:11.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:11 smithi176 ceph-mon[112742]: pgmap v340: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:11 smithi196 ceph-mon[86193]: pgmap v340: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:13.865 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:13 smithi196 ceph-mon[86193]: pgmap v341: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:13.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:13 smithi176 ceph-mon[112742]: pgmap v341: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:15.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:15 smithi176 ceph-mon[112742]: pgmap v342: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:15.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:15 smithi196 ceph-mon[86193]: pgmap v342: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:17.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:17 smithi176 ceph-mon[112742]: pgmap v343: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:17.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:17 smithi196 ceph-mon[86193]: pgmap v343: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:19.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:19 smithi176 ceph-mon[112742]: pgmap v344: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:19.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:19 smithi196 ceph-mon[86193]: pgmap v344: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:21.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:21 smithi176 ceph-mon[112742]: pgmap v345: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:21.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:21 smithi196 ceph-mon[86193]: pgmap v345: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:23.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:23 smithi176 ceph-mon[112742]: pgmap v346: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:23.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:23 smithi196 ceph-mon[86193]: pgmap v346: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:25.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:25 smithi176 ceph-mon[112742]: pgmap v347: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:25.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:25 smithi196 ceph-mon[86193]: pgmap v347: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:27.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:27 smithi176 ceph-mon[112742]: pgmap v348: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:27.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:27 smithi196 ceph-mon[86193]: pgmap v348: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:29.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:29 smithi176 ceph-mon[112742]: pgmap v349: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:29.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:29 smithi196 ceph-mon[86193]: pgmap v349: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:30.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:30.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:31.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:31 smithi196 ceph-mon[86193]: pgmap v350: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:32.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:31 smithi176 ceph-mon[112742]: pgmap v350: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:32.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:32 smithi196 ceph-mon[86193]: pgmap v351: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:33.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:32 smithi176 ceph-mon[112742]: pgmap v351: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:35.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:35 smithi176 ceph-mon[112742]: pgmap v352: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:35.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:35 smithi196 ceph-mon[86193]: pgmap v352: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:37.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:37 smithi176 ceph-mon[112742]: pgmap v353: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:37.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:37 smithi196 ceph-mon[86193]: pgmap v353: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:39.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:39 smithi176 ceph-mon[112742]: pgmap v354: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:39.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:39 smithi196 ceph-mon[86193]: pgmap v354: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:41.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:41 smithi176 ceph-mon[112742]: pgmap v355: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:41.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:41 smithi196 ceph-mon[86193]: pgmap v355: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:43.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:43 smithi176 ceph-mon[112742]: pgmap v356: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:43.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:43 smithi196 ceph-mon[86193]: pgmap v356: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:45.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:45 smithi176 ceph-mon[112742]: pgmap v357: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:45.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:45.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:45 smithi196 ceph-mon[86193]: pgmap v357: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:45.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:03:47.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:46 smithi196 ceph-mon[86193]: pgmap v358: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:47.375 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:46 smithi176 ceph-mon[112742]: pgmap v358: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:49.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:49 smithi176 ceph-mon[112742]: pgmap v359: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:49.891 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:49 smithi196 ceph-mon[86193]: pgmap v359: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:51.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:51 smithi176 ceph-mon[112742]: pgmap v360: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:51.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:51 smithi196 ceph-mon[86193]: pgmap v360: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:53.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:53 smithi176 ceph-mon[112742]: pgmap v361: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:53.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:53 smithi196 ceph-mon[86193]: pgmap v361: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:03:54.112 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph orch ps' 2024-03-13T15:03:54.558 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:03:55.750 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:55 smithi176 ceph-mon[112742]: pgmap v362: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:55.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:55 smithi196 ceph-mon[86193]: pgmap v362: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:56.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 -- 172.21.15.176:0/1447109800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f01039b0 msgr2=0x7fc5f0107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:03:56.625 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 --2- 172.21.15.176:0/1447109800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f01039b0 0x7fc5f0107a00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009910 tx=0x7fc5dc009c20 comp rx=0 tx=0).stop 2024-03-13T15:03:56.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 -- 172.21.15.176:0/1447109800 shutdown_connections 2024-03-13T15:03:56.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 --2- 172.21.15.176:0/1447109800 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f01039b0 0x7fc5f0107a00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:03:56.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 --2- 172.21.15.176:0/1447109800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f0103000 0x7fc5f01033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:03:56.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 -- 172.21.15.176:0/1447109800 >> 172.21.15.176:0/1447109800 conn(0x7fc5f00fe8d0 msgr2=0x7fc5f0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:03:56.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 -- 172.21.15.176:0/1447109800 shutdown_connections 2024-03-13T15:03:56.626 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.640+0000 7fc5f7600700 1 -- 172.21.15.176:0/1447109800 wait complete. 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.642+0000 7fc5f7600700 1 Processor -- start 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.642+0000 7fc5f7600700 1 -- start start 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.642+0000 7fc5f7600700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f0103000 0x7fc5f019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.642+0000 7fc5f7600700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f01039b0 0x7fc5f019ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.642+0000 7fc5f7600700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f019f390 con 0x7fc5f0103000 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.642+0000 7fc5f7600700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0198840 con 0x7fc5f01039b0 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5f4c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f0103000 0x7fc5f019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5f4c00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f0103000 0x7fc5f019e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39462/0 (socket says 172.21.15.176:39462) 2024-03-13T15:03:56.627 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5f4c00700 1 -- 172.21.15.176:0/2977632449 learned_addr learned my addr 172.21.15.176:0/2977632449 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:03:56.628 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5efe00700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f01039b0 0x7fc5f019ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5f4c00700 1 -- 172.21.15.176:0/2977632449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f01039b0 msgr2=0x7fc5f019ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5f4c00700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f01039b0 0x7fc5f019ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5f4c00700 1 -- 172.21.15.176:0/2977632449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc0095f0 con 0x7fc5f0103000 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.643+0000 7fc5f4c00700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f0103000 0x7fc5f019e7c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400d8d0 tx=0x7fc5e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.644+0000 7fc5ed600700 1 -- 172.21.15.176:0/2977632449 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e4009940 con 0x7fc5f0103000 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.644+0000 7fc5ed600700 1 -- 172.21.15.176:0/2977632449 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5e4010460 con 0x7fc5f0103000 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.644+0000 7fc5ed600700 1 -- 172.21.15.176:0/2977632449 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e400f5f0 con 0x7fc5f0103000 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.644+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f0198b20 con 0x7fc5f0103000 2024-03-13T15:03:56.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.644+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f0199070 con 0x7fc5f0103000 2024-03-13T15:03:56.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.645+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f010b400 con 0x7fc5f0103000 2024-03-13T15:03:56.630 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.646+0000 7fc5ed600700 1 -- 172.21.15.176:0/2977632449 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc5e40105d0 con 0x7fc5f0103000 2024-03-13T15:03:56.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.647+0000 7fc5ed600700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc5d806dfa0 0x7fc5d8070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:03:56.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.647+0000 7fc5ed600700 1 -- 172.21.15.176:0/2977632449 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fc5e408de20 con 0x7fc5f0103000 2024-03-13T15:03:56.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.647+0000 7fc5efe00700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc5d806dfa0 0x7fc5d8070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:03:56.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.648+0000 7fc5efe00700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc5d806dfa0 0x7fc5d8070460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00ff0a0 tx=0x7fc5dc01a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:03:56.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.652+0000 7fc5ed600700 1 -- 172.21.15.176:0/2977632449 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc5e405ac80 con 0x7fc5f0103000 2024-03-13T15:03:56.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.884+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 --> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f01992e0 con 0x7fc5d806dfa0 2024-03-13T15:03:56.901 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.916+0000 7fc5ed600700 1 -- 172.21.15.176:0/2977632449 <== mgr.24551 v2:172.21.15.176:6800/1274703396 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc5f01992e0 con 0x7fc5d806dfa0 2024-03-13T15:03:56.902 INFO:teuthology.orchestra.run.smithi176.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-13T15:03:56.902 INFO:teuthology.orchestra.run.smithi176.stdout:alertmanager.smithi176 smithi176 *:9093,9094 running (11m) 118s ago 25m 14.7M - 0.25.0 c8568f914cd2 f203e5d6e531 2024-03-13T15:03:56.902 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi176 smithi176 running (2m) 118s ago 25m 7818k - 19.0.0-1916-g9a75c81f 9552dbf7f115 06b476c807b4 2024-03-13T15:03:56.902 INFO:teuthology.orchestra.run.smithi176.stdout:ceph-exporter.smithi196 smithi196 running (2m) 2m ago 24m 6945k - 19.0.0-1916-g9a75c81f 9552dbf7f115 0f12bb84f0f6 2024-03-13T15:03:56.902 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi176 smithi176 running (8m) 118s ago 25m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 2c27af7d9b20 2024-03-13T15:03:56.902 INFO:teuthology.orchestra.run.smithi176.stdout:crash.smithi196 smithi196 running (8m) 2m ago 24m 6665k - 19.0.0-1916-g9a75c81f 9552dbf7f115 f366b66e4b9e 2024-03-13T15:03:56.902 INFO:teuthology.orchestra.run.smithi176.stdout:grafana.smithi176 smithi176 *:3000 running (11m) 118s ago 24m 74.4M - 9.4.12 a0ea5e3c46d9 0550a44c29bd 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.hjrgov smithi176 running (3m) 118s ago 18m 93.6M - 19.0.0-1916-g9a75c81f 9552dbf7f115 edc682c345ac 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi176.ldskqt smithi176 running (3m) 118s ago 18m 85.7M - 19.0.0-1916-g9a75c81f 9552dbf7f115 5b6d2dc0d458 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.tcitrk smithi196 running (3m) 2m ago 18m 12.2M - 19.0.0-1916-g9a75c81f 9552dbf7f115 a9081f4c2ead 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mds.cephfs.smithi196.wabhcu smithi196 running (2m) 2m ago 18m 11.9M - 19.0.0-1916-g9a75c81f 9552dbf7f115 2e3acf7133a0 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi176.tzfdwd smithi176 *:8443,9283,8765 running (14m) 118s ago 27m 511M - 19.0.0-1916-g9a75c81f 9552dbf7f115 65164212d656 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mgr.smithi196.vdktbk smithi196 *:8443,9283,8765 running (13m) 2m ago 23m 452M - 19.0.0-1916-g9a75c81f 9552dbf7f115 8bcd8707d6af 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi176 smithi176 running (10m) 118s ago 27m 60.5M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 35f4ac6dca30 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:mon.smithi196 smithi196 running (9m) 2m ago 23m 43.9M 2048M 19.0.0-1916-g9a75c81f 9552dbf7f115 3f7f3fad0373 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi176 smithi176 *:9100 running (13m) 118s ago 25m 8186k - 1.5.0 0da6a335fe13 402cd441ee67 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:node-exporter.smithi196 smithi196 *:9100 running (12m) 2m ago 24m 8140k - 1.5.0 0da6a335fe13 43de7d31320b 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:osd.0 smithi176 running (7m) 118s ago 22m 144M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 9ee4d4f2265b 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:osd.1 smithi176 running (7m) 118s ago 22m 113M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 6ac5d28d6884 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:osd.2 smithi176 running (6m) 118s ago 21m 119M 1217M 19.0.0-1916-g9a75c81f 9552dbf7f115 56889da43f2b 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:osd.3 smithi196 running (5m) 2m ago 21m 137M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 a186117c65ba 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:osd.4 smithi196 running (5m) 2m ago 20m 104M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 7f510f7a757c 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:osd.5 smithi196 running (4m) 2m ago 19m 105M 2241M 19.0.0-1916-g9a75c81f 9552dbf7f115 43366d2dc8da 2024-03-13T15:03:56.903 INFO:teuthology.orchestra.run.smithi176.stdout:prometheus.smithi176 smithi176 *:9095 running (12m) 118s ago 24m 72.0M - 2.43.0 a07b618ecd1d a8953086ce16 2024-03-13T15:03:56.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.919+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc5d806dfa0 msgr2=0x7fc5d8070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:03:56.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.919+0000 7fc5f7600700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc5d806dfa0 0x7fc5d8070460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc5f00ff0a0 tx=0x7fc5dc01a040 comp rx=0 tx=0).stop 2024-03-13T15:03:56.904 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.919+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f0103000 msgr2=0x7fc5f019e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.919+0000 7fc5f7600700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f0103000 0x7fc5f019e7c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400d8d0 tx=0x7fc5e400dc90 comp rx=0 tx=0).stop 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.920+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 shutdown_connections 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.920+0000 7fc5f7600700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc5d806dfa0 0x7fc5d8070460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.920+0000 7fc5f7600700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc5f0103000 0x7fc5f019e7c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.920+0000 7fc5f7600700 1 --2- 172.21.15.176:0/2977632449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f01039b0 0x7fc5f019ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.920+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 >> 172.21.15.176:0/2977632449 conn(0x7fc5f00fe8d0 msgr2=0x7fc5f0100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.920+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 shutdown_connections 2024-03-13T15:03:56.905 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:03:56.920+0000 7fc5f7600700 1 -- 172.21.15.176:0/2977632449 wait complete. 2024-03-13T15:03:57.742 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph versions' 2024-03-13T15:03:57.828 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:57 smithi176 ceph-mon[112742]: pgmap v363: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:57.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:57 smithi196 ceph-mon[86193]: pgmap v363: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:03:58.166 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:03:58.752 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:58 smithi176 ceph-mon[112742]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:03:58.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:58 smithi196 ceph-mon[86193]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-13T15:03:59.625 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:03:59 smithi176 ceph-mon[112742]: pgmap v364: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:03:59.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:03:59 smithi196 ceph-mon[86193]: pgmap v364: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:00.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 -- 172.21.15.176:0/2730976432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c068670 msgr2=0x7f0a7c068a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:00.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 --2- 172.21.15.176:0/2730976432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c068670 0x7f0a7c068a50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c009910 tx=0x7f0a6c009c20 comp rx=0 tx=0).stop 2024-03-13T15:04:00.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 -- 172.21.15.176:0/2730976432 shutdown_connections 2024-03-13T15:04:00.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 --2- 172.21.15.176:0/2730976432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a7c069020 0x7f0a7c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:00.243 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 --2- 172.21.15.176:0/2730976432 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c068670 0x7f0a7c068a50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:00.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 -- 172.21.15.176:0/2730976432 >> 172.21.15.176:0/2730976432 conn(0x7f0a7c0755a0 msgr2=0x7f0a7c0759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:00.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 -- 172.21.15.176:0/2730976432 shutdown_connections 2024-03-13T15:04:00.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.258+0000 7f0a82000700 1 -- 172.21.15.176:0/2730976432 wait complete. 2024-03-13T15:04:00.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.260+0000 7f0a82000700 1 Processor -- start 2024-03-13T15:04:00.244 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.260+0000 7f0a82000700 1 -- start start 2024-03-13T15:04:00.245 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a82000700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a7c068670 0x7f0a7c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:00.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a82000700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c069020 0x7f0a7c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:00.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a82000700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c19f2e0 con 0x7f0a7c069020 2024-03-13T15:04:00.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a82000700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c198790 con 0x7f0a7c068670 2024-03-13T15:04:00.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a73e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c069020 0x7f0a7c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:00.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a73e00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c069020 0x7f0a7c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:39476/0 (socket says 172.21.15.176:39476) 2024-03-13T15:04:00.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a73e00700 1 -- 172.21.15.176:0/3166083291 learned_addr learned my addr 172.21.15.176:0/3166083291 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:00.246 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.261+0000 7f0a7b400700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a7c068670 0x7f0a7c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:00.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a73e00700 1 -- 172.21.15.176:0/3166083291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a7c068670 msgr2=0x7f0a7c19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:00.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a73e00700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a7c068670 0x7f0a7c19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:00.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a73e00700 1 -- 172.21.15.176:0/3166083291 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c0095f0 con 0x7f0a7c069020 2024-03-13T15:04:00.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a73e00700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c069020 0x7f0a7c19ec50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400d900 tx=0x7f0a6400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:00.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a78c00700 1 -- 172.21.15.176:0/3166083291 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a64009900 con 0x7f0a7c069020 2024-03-13T15:04:00.247 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a78c00700 1 -- 172.21.15.176:0/3166083291 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a64010460 con 0x7f0a7c069020 2024-03-13T15:04:00.248 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a78c00700 1 -- 172.21.15.176:0/3166083291 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a6400f5f0 con 0x7f0a7c069020 2024-03-13T15:04:00.248 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a7c198a70 con 0x7f0a7c069020 2024-03-13T15:04:00.248 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.262+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c198fc0 con 0x7f0a7c069020 2024-03-13T15:04:00.248 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.263+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c04ea90 con 0x7f0a7c069020 2024-03-13T15:04:00.249 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.265+0000 7f0a78c00700 1 -- 172.21.15.176:0/3166083291 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0a640105d0 con 0x7f0a7c069020 2024-03-13T15:04:00.250 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.265+0000 7f0a78c00700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0a6006dfa0 0x7f0a60070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:00.250 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.266+0000 7f0a78c00700 1 -- 172.21.15.176:0/3166083291 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f0a6408d9b0 con 0x7f0a7c069020 2024-03-13T15:04:00.250 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.266+0000 7f0a7b400700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0a6006dfa0 0x7f0a60070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:00.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.267+0000 7f0a7b400700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0a6006dfa0 0x7f0a60070460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c006010 tx=0x7f0a6c01a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:00.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.270+0000 7f0a78c00700 1 -- 172.21.15.176:0/3166083291 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f0a6405a810 con 0x7f0a7c069020 2024-03-13T15:04:00.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.585+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0a7c199390 con 0x7f0a7c069020 2024-03-13T15:04:00.570 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.586+0000 7f0a78c00700 1 -- 172.21.15.176:0/3166083291 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0a6405a3a0 con 0x7f0a7c069020 2024-03-13T15:04:00.571 INFO:teuthology.orchestra.run.smithi176.stdout:{ 2024-03-13T15:04:00.571 INFO:teuthology.orchestra.run.smithi176.stdout: "mon": { 2024-03-13T15:04:00.571 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "mgr": { 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 2 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "osd": { 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 6 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "mds": { 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 4 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: }, 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "overall": { 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)": 14 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout: } 2024-03-13T15:04:00.572 INFO:teuthology.orchestra.run.smithi176.stdout:} 2024-03-13T15:04:00.573 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0a6006dfa0 msgr2=0x7f0a60070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:00.573 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0a6006dfa0 0x7f0a60070460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c006010 tx=0x7f0a6c01a040 comp rx=0 tx=0).stop 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c069020 msgr2=0x7f0a7c19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c069020 0x7f0a7c19ec50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400d900 tx=0x7f0a6400dc10 comp rx=0 tx=0).stop 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 shutdown_connections 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0a6006dfa0 0x7f0a60070460 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a7c068670 0x7f0a7c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 --2- 172.21.15.176:0/3166083291 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0a7c069020 0x7f0a7c19ec50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 >> 172.21.15.176:0/3166083291 conn(0x7f0a7c0755a0 msgr2=0x7f0a7c102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 shutdown_connections 2024-03-13T15:04:00.574 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:00.589+0000 7f0a82000700 1 -- 172.21.15.176:0/3166083291 wait complete. 2024-03-13T15:04:00.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:00.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:00 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3166083291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:04:00.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:00.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:00 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3166083291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:04:01.345 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-03-13T15:04:01.759 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:04:01.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:01 smithi176 ceph-mon[112742]: pgmap v365: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:01.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:01 smithi196 ceph-mon[86193]: pgmap v365: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:03.789 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:03 smithi176 ceph-mon[112742]: pgmap v366: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 -- 172.21.15.176:0/1896322102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc1071d0 msgr2=0x7fa6dc107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 --2- 172.21.15.176:0/1896322102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc1071d0 0x7fa6dc107650 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009910 tx=0x7fa6cc009c20 comp rx=0 tx=0).stop 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 -- 172.21.15.176:0/1896322102 shutdown_connections 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 --2- 172.21.15.176:0/1896322102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc1071d0 0x7fa6dc107650 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 --2- 172.21.15.176:0/1896322102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc10d280 0x7fa6dc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 -- 172.21.15.176:0/1896322102 >> 172.21.15.176:0/1896322102 conn(0x7fa6dc0769e0 msgr2=0x7fa6dc078e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 -- 172.21.15.176:0/1896322102 shutdown_connections 2024-03-13T15:04:03.826 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.841+0000 7fa6e4800700 1 -- 172.21.15.176:0/1896322102 wait complete. 2024-03-13T15:04:03.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.842+0000 7fa6e4800700 1 Processor -- start 2024-03-13T15:04:03.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e4800700 1 -- start start 2024-03-13T15:04:03.827 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e4800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc1071d0 0x7fa6dc19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:03.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e4800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc10d280 0x7fa6dc19d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:03.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e4800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc19da30 con 0x7fa6dc10d280 2024-03-13T15:04:03.828 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e4800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc1a17c0 con 0x7fa6dc1071d0 2024-03-13T15:04:03.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc10d280 0x7fa6dc19d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:03.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e1400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc10d280 0x7fa6dc19d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55498/0 (socket says 172.21.15.176:55498) 2024-03-13T15:04:03.829 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.843+0000 7fa6e1400700 1 -- 172.21.15.176:0/787060851 learned_addr learned my addr 172.21.15.176:0/787060851 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6e1e00700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc1071d0 0x7fa6dc19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6e1400700 1 -- 172.21.15.176:0/787060851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc1071d0 msgr2=0x7fa6dc19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6e1400700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc1071d0 0x7fa6dc19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6e1400700 1 -- 172.21.15.176:0/787060851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc0095f0 con 0x7fa6dc10d280 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6e1400700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc10d280 0x7fa6dc19d350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00b5c0 tx=0x7fa6cc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6d6a00700 1 -- 172.21.15.176:0/787060851 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6cc01d070 con 0x7fa6dc10d280 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6d6a00700 1 -- 172.21.15.176:0/787060851 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6cc00f460 con 0x7fa6dc10d280 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.844+0000 7fa6d6a00700 1 -- 172.21.15.176:0/787060851 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6cc017610 con 0x7fa6dc10d280 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.845+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc1a1a40 con 0x7fa6dc10d280 2024-03-13T15:04:03.830 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.845+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc1a1fb0 con 0x7fa6dc10d280 2024-03-13T15:04:03.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.846+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc10f710 con 0x7fa6dc10d280 2024-03-13T15:04:03.831 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.847+0000 7fa6d6a00700 1 -- 172.21.15.176:0/787060851 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa6cc00f5d0 con 0x7fa6dc10d280 2024-03-13T15:04:03.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.848+0000 7fa6d6a00700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa6c406dfa0 0x7fa6c4070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:03.832 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.848+0000 7fa6d6a00700 1 -- 172.21.15.176:0/787060851 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa6cc08f540 con 0x7fa6dc10d280 2024-03-13T15:04:03.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.848+0000 7fa6e1e00700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa6c406dfa0 0x7fa6c4070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:03.833 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.849+0000 7fa6e1e00700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa6c406dfa0 0x7fa6c4070460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0006fd0 tx=0x7fa6d0008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:03.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:03.853+0000 7fa6d6a00700 1 -- 172.21.15.176:0/787060851 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa6cc0587f0 con 0x7fa6dc10d280 2024-03-13T15:04:03.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:03 smithi196 ceph-mon[86193]: pgmap v366: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:04.148 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.163+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa6dc1a22e0 con 0x7fa6dc10d280 2024-03-13T15:04:04.149 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.164+0000 7fa6d6a00700 1 -- 172.21.15.176:0/787060851 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa6cc05beb0 con 0x7fa6dc10d280 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa6c406dfa0 msgr2=0x7fa6c4070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa6c406dfa0 0x7fa6c4070460 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0006fd0 tx=0x7fa6d0008040 comp rx=0 tx=0).stop 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc10d280 msgr2=0x7fa6dc19d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc10d280 0x7fa6dc19d350 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc00b5c0 tx=0x7fa6cc00be00 comp rx=0 tx=0).stop 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 shutdown_connections 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa6c406dfa0 0x7fa6c4070460 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6dc1071d0 0x7fa6dc19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:04.152 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 --2- 172.21.15.176:0/787060851 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa6dc10d280 0x7fa6dc19d350 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:04.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 >> 172.21.15.176:0/787060851 conn(0x7fa6dc0769e0 msgr2=0x7fa6dc0784d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:04.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.167+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 shutdown_connections 2024-03-13T15:04:04.153 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:04.168+0000 7fa6e4800700 1 -- 172.21.15.176:0/787060851 wait complete. 2024-03-13T15:04:04.168 INFO:teuthology.orchestra.run.smithi176.stdout:true 2024-03-13T15:04:04.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:04 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/787060851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:04:04.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:04 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/787060851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:04:04.937 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -e sha1=9a75c81fd074f3cb252248b860d5cc88e5f3fb55 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-03-13T15:04:05.352 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:04:05.840 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:05 smithi176 ceph-mon[112742]: pgmap v367: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:05.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:05 smithi196 ceph-mon[86193]: pgmap v367: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.507+0000 7fa038a00700 1 -- 172.21.15.176:0/3174079700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034100da0 msgr2=0x7fa034101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.507+0000 7fa038a00700 1 --2- 172.21.15.176:0/3174079700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034100da0 0x7fa034101180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa024009910 tx=0x7fa024009c20 comp rx=0 tx=0).stop 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.508+0000 7fa038a00700 1 -- 172.21.15.176:0/3174079700 shutdown_connections 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.508+0000 7fa038a00700 1 --2- 172.21.15.176:0/3174079700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034101750 0x7fa0341057a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.508+0000 7fa038a00700 1 --2- 172.21.15.176:0/3174079700 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034100da0 0x7fa034101180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.508+0000 7fa038a00700 1 -- 172.21.15.176:0/3174079700 >> 172.21.15.176:0/3174079700 conn(0x7fa0340fc630 msgr2=0x7fa0340fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.508+0000 7fa038a00700 1 -- 172.21.15.176:0/3174079700 shutdown_connections 2024-03-13T15:04:07.493 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.508+0000 7fa038a00700 1 -- 172.21.15.176:0/3174079700 wait complete. 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.509+0000 7fa038a00700 1 Processor -- start 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.509+0000 7fa038a00700 1 -- start start 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa038a00700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034100da0 0x7fa034198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa038a00700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034101750 0x7fa034198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa038a00700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0341994d0 con 0x7fa034101750 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa038a00700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa034199640 con 0x7fa034100da0 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa029400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034101750 0x7fa034198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa029400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034101750 0x7fa034198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55518/0 (socket says 172.21.15.176:55518) 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa029400700 1 -- 172.21.15.176:0/2939460852 learned_addr learned my addr 172.21.15.176:0/2939460852 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.510+0000 7fa031e00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034100da0 0x7fa034198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:07.495 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa029400700 1 -- 172.21.15.176:0/2939460852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034100da0 msgr2=0x7fa034198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:07.496 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa029400700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034100da0 0x7fa034198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:07.496 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa029400700 1 -- 172.21.15.176:0/2939460852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240095f0 con 0x7fa034101750 2024-03-13T15:04:07.496 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa029400700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034101750 0x7fa034198ed0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00cfd0 tx=0x7fa01c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:07.496 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa02b400700 1 -- 172.21.15.176:0/2939460852 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa01c00ad70 con 0x7fa034101750 2024-03-13T15:04:07.497 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa02b400700 1 -- 172.21.15.176:0/2939460852 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa01c0128e0 con 0x7fa034101750 2024-03-13T15:04:07.497 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa03419d4c0 con 0x7fa034101750 2024-03-13T15:04:07.497 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa02b400700 1 -- 172.21.15.176:0/2939460852 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa01c011880 con 0x7fa034101750 2024-03-13T15:04:07.497 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.511+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa03419da10 con 0x7fa034101750 2024-03-13T15:04:07.497 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.512+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa034104f20 con 0x7fa034101750 2024-03-13T15:04:07.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.514+0000 7fa02b400700 1 -- 172.21.15.176:0/2939460852 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa01c012420 con 0x7fa034101750 2024-03-13T15:04:07.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.515+0000 7fa02b400700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa01806dfa0 0x7fa018070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:07.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.515+0000 7fa02b400700 1 -- 172.21.15.176:0/2939460852 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa01c08df30 con 0x7fa034101750 2024-03-13T15:04:07.499 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.515+0000 7fa031e00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa01806dfa0 0x7fa018070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:07.500 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.516+0000 7fa031e00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa01806dfa0 0x7fa018070460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa024006010 tx=0x7fa02401a040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:07.504 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.520+0000 7fa02b400700 1 -- 172.21.15.176:0/2939460852 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa01c05ad90 con 0x7fa034101750 2024-03-13T15:04:07.815 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.831+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa03404ea90 con 0x7fa034101750 2024-03-13T15:04:07.816 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:07 smithi176 ceph-mon[112742]: pgmap v368: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:07.816 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.832+0000 7fa02b400700 1 -- 172.21.15.176:0/2939460852 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa01c05a920 con 0x7fa034101750 2024-03-13T15:04:07.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa01806dfa0 msgr2=0x7fa018070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:07.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa01806dfa0 0x7fa018070460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa024006010 tx=0x7fa02401a040 comp rx=0 tx=0).stop 2024-03-13T15:04:07.819 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034101750 msgr2=0x7fa034198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034101750 0x7fa034198ed0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00cfd0 tx=0x7fa01c00a340 comp rx=0 tx=0).stop 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 shutdown_connections 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa01806dfa0 0x7fa018070460 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034100da0 0x7fa034198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 --2- 172.21.15.176:0/2939460852 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa034101750 0x7fa034198ed0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.835+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 >> 172.21.15.176:0/2939460852 conn(0x7fa0340fc630 msgr2=0x7fa0340fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.836+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 shutdown_connections 2024-03-13T15:04:07.820 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:07.836+0000 7fa038a00700 1 -- 172.21.15.176:0/2939460852 wait complete. 2024-03-13T15:04:07.837 INFO:teuthology.orchestra.run.smithi176.stdout: "ceph version 19.0.0-1916-g9a75c81f (9a75c81fd074f3cb252248b860d5cc88e5f3fb55) squid (dev)" 2024-03-13T15:04:07.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:07 smithi196 ceph-mon[86193]: pgmap v368: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:08.621 DEBUG:teuthology.parallel:result is None 2024-03-13T15:04:08.621 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-13T15:04:08.634 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T15:04:08.634 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f60e35da-e146-11ee-95c9-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-13T15:04:08.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:08 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2939460852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:04:08.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:08 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2939460852' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-03-13T15:04:09.045 INFO:teuthology.orchestra.run.smithi176.stderr:Inferring config /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/mon.smithi176/config 2024-03-13T15:04:09.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:09 smithi176 ceph-mon[112742]: pgmap v369: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:09.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:09 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:04:09.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:09 smithi196 ceph-mon[86193]: pgmap v369: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:09.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:09 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:04:10.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:10 smithi196 ceph-mon[86193]: pgmap v370: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:10.959 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:10 smithi176 ceph-mon[112742]: pgmap v370: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 -- 172.21.15.176:0/3832260252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 msgr2=0x7f534c1114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 --2- 172.21.15.176:0/3832260252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1114a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f533c009910 tx=0x7f533c009c20 comp rx=0 tx=0).stop 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 -- 172.21.15.176:0/3832260252 shutdown_connections 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 --2- 172.21.15.176:0/3832260252 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1114a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 --2- 172.21.15.176:0/3832260252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c0752c0 0x7f534c0756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 -- 172.21.15.176:0/3832260252 >> 172.21.15.176:0/3832260252 conn(0x7f534c0fe810 msgr2=0x7f534c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 -- 172.21.15.176:0/3832260252 shutdown_connections 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.145+0000 7f5354800700 1 -- 172.21.15.176:0/3832260252 wait complete. 2024-03-13T15:04:11.131 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.146+0000 7f5354800700 1 Processor -- start 2024-03-13T15:04:11.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.146+0000 7f5354800700 1 -- start start 2024-03-13T15:04:11.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5354800700 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c0752c0 0x7f534c1a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:11.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5354800700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1a2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:11.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5354800700 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c1a3620 con 0x7f534c075be0 2024-03-13T15:04:11.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5354800700 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c19cae0 con 0x7f534c0752c0 2024-03-13T15:04:11.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5351400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1a2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:11.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5351400700 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1a2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:55528/0 (socket says 172.21.15.176:55528) 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5351400700 1 -- 172.21.15.176:0/292102200 learned_addr learned my addr 172.21.15.176:0/292102200 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.147+0000 7f5351e00700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c0752c0 0x7f534c1a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5351400700 1 -- 172.21.15.176:0/292102200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c0752c0 msgr2=0x7f534c1a2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5351400700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c0752c0 0x7f534c1a2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5351400700 1 -- 172.21.15.176:0/292102200 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c0095f0 con 0x7f534c075be0 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5351400700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1a2f40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f533c0092e0 tx=0x7f533c00b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5346a00700 1 -- 172.21.15.176:0/292102200 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f533c01d070 con 0x7f534c075be0 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5346a00700 1 -- 172.21.15.176:0/292102200 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f533c0052a0 con 0x7f534c075be0 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c19cdc0 con 0x7f534c075be0 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5346a00700 1 -- 172.21.15.176:0/292102200 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f533c017400 con 0x7f534c075be0 2024-03-13T15:04:11.134 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.148+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c19d310 con 0x7f534c075be0 2024-03-13T15:04:11.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.150+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c10ec10 con 0x7f534c075be0 2024-03-13T15:04:11.135 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.151+0000 7f5346a00700 1 -- 172.21.15.176:0/292102200 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f533c00f820 con 0x7f534c075be0 2024-03-13T15:04:11.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.152+0000 7f5346a00700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f533406dfa0 0x7f5334070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:11.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.152+0000 7f5346a00700 1 -- 172.21.15.176:0/292102200 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f533c08f150 con 0x7f534c075be0 2024-03-13T15:04:11.136 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.152+0000 7f5351e00700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f533406dfa0 0x7f5334070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:11.137 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.153+0000 7f5351e00700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f533406dfa0 0x7f5334070460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5340009c60 tx=0x7f5340009260 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:11.141 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.157+0000 7f5346a00700 1 -- 172.21.15.176:0/292102200 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f533c05bfd0 con 0x7f534c075be0 2024-03-13T15:04:11.410 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.425+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f534c04ea90 con 0x7f534c075be0 2024-03-13T15:04:11.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.426+0000 7f5346a00700 1 -- 172.21.15.176:0/292102200 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1901 (secure 0 0 0) 0x7f533c05bb60 con 0x7f534c075be0 2024-03-13T15:04:11.412 INFO:teuthology.orchestra.run.smithi176.stdout:e39 2024-03-13T15:04:11.412 INFO:teuthology.orchestra.run.smithi176.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-13T15:04:11.412 INFO:teuthology.orchestra.run.smithi176.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-03-13T15:04:11.412 INFO:teuthology.orchestra.run.smithi176.stdout:legacy client fscid: 1 2024-03-13T15:04:11.412 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:11.412 INFO:teuthology.orchestra.run.smithi176.stdout:Filesystem 'cephfs' (1) 2024-03-13T15:04:11.412 INFO:teuthology.orchestra.run.smithi176.stdout:fs_name cephfs 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:epoch 39 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:created 2024-03-13T14:45:38.361828+0000 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:modified 2024-03-13T15:02:06.329079+0000 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:tableserver 0 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:root 0 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:session_timeout 60 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:session_autoclose 300 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:max_file_size 1099511627776 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:max_xattr_size 65536 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:required_client_features {} 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure 0 2024-03-13T15:04:11.413 INFO:teuthology.orchestra.run.smithi176.stdout:last_failure_osd_epoch 112 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:max_mds 1 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:in 0 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:up {0=34462} 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:failed 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:damaged 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:stopped 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:data_pools [3] 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:metadata_pool 2 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:inline_data disabled 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:balancer 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:bal_rank_mask -1 2024-03-13T15:04:11.414 INFO:teuthology.orchestra.run.smithi176.stdout:standby_count_wanted 1 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.ldskqt{0:34462} state up:active seq 24 join_fscid=1 addr [v2:172.21.15.176:6826/648979087,v1:172.21.15.176:6827/648979087] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi176.hjrgov{0:34494} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.176:6828/552037800,v1:172.21.15.176:6829/552037800] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout:Standby daemons: 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.tcitrk{-1:44257} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.196:6824/2671769958,v1:172.21.15.196:6825/2671769958] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:04:11.415 INFO:teuthology.orchestra.run.smithi176.stdout:[mds.cephfs.smithi196.wabhcu{-1:44281} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.196:6826/2135805809,v1:172.21.15.196:6827/2135805809] compat {c=[1],r=[1],i=[fff]}] 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f533406dfa0 msgr2=0x7f5334070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f533406dfa0 0x7f5334070460 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5340009c60 tx=0x7f5340009260 comp rx=0 tx=0).stop 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 msgr2=0x7f534c1a2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1a2f40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f533c0092e0 tx=0x7f533c00b680 comp rx=0 tx=0).stop 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 shutdown_connections 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f533406dfa0 0x7f5334070460 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f534c0752c0 0x7f534c1a2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 --2- 172.21.15.176:0/292102200 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f534c075be0 0x7f534c1a2f40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.429+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 >> 172.21.15.176:0/292102200 conn(0x7f534c0fe810 msgr2=0x7f534c0ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.430+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 shutdown_connections 2024-03-13T15:04:11.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:11.430+0000 7f5354800700 1 -- 172.21.15.176:0/292102200 wait complete. 2024-03-13T15:04:11.417 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 39 2024-03-13T15:04:11.788 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:04:11.788 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:04:11.788 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:04:11.788 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:11 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/292102200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:04:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:04:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:04:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:04:11.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:11 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/292102200' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-13T15:04:12.190 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-03-13T15:04:12.201 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T15:04:12.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 -- 172.21.15.176:0/990930743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 msgr2=0x7f0e301028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:12.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 --2- 172.21.15.176:0/990930743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e301028c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009880 tx=0x7f0e2002bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 -- 172.21.15.176:0/990930743 shutdown_connections 2024-03-13T15:04:12.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 --2- 172.21.15.176:0/990930743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e30102e90 0x7f0e30106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 --2- 172.21.15.176:0/990930743 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e301028c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 -- 172.21.15.176:0/990930743 >> 172.21.15.176:0/990930743 conn(0x7f0e300fdd00 msgr2=0x7f0e30100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:12.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 -- 172.21.15.176:0/990930743 shutdown_connections 2024-03-13T15:04:12.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.441+0000 7f0e36a00640 1 -- 172.21.15.176:0/990930743 wait complete. 2024-03-13T15:04:12.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.442+0000 7f0e36a00640 1 Processor -- start 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.442+0000 7f0e36a00640 1 -- start start 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e30197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e36a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e30198510 0x7f0e3019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e36a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30198b10 con 0x7f0e301024e0 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e36a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30198c80 con 0x7f0e30198510 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e30197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e2fe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e30197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35444/0 (socket says 172.21.15.176:35444) 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e2fe00640 1 -- 172.21.15.176:0/1629949023 learned_addr learned my addr 172.21.15.176:0/1629949023 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e2fe00640 1 -- 172.21.15.176:0/1629949023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e30198510 msgr2=0x7f0e3019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e27400640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e30198510 0x7f0e3019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e2fe00640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e30198510 0x7f0e3019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.443+0000 7f0e2fe00640 1 -- 172.21.15.176:0/1629949023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20009560 con 0x7f0e301024e0 2024-03-13T15:04:12.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.444+0000 7f0e2fe00640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e30197fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e20004270 tx=0x7f0e200042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:12.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.444+0000 7f0e2d600640 1 -- 172.21.15.176:0/1629949023 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e200035f0 con 0x7f0e301024e0 2024-03-13T15:04:12.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.444+0000 7f0e2d600640 1 -- 172.21.15.176:0/1629949023 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e20003c10 con 0x7f0e301024e0 2024-03-13T15:04:12.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.444+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3019cf70 con 0x7f0e301024e0 2024-03-13T15:04:12.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.444+0000 7f0e2d600640 1 -- 172.21.15.176:0/1629949023 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e20034440 con 0x7f0e301024e0 2024-03-13T15:04:12.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.444+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3019d4d0 con 0x7f0e301024e0 2024-03-13T15:04:12.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.445+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3004e040 con 0x7f0e301024e0 2024-03-13T15:04:12.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.446+0000 7f0e2d600640 1 -- 172.21.15.176:0/1629949023 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0e20005060 con 0x7f0e301024e0 2024-03-13T15:04:12.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.447+0000 7f0e2d600640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e1406e020 0x7f0e140704e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:12.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.447+0000 7f0e2d600640 1 -- 172.21.15.176:0/1629949023 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f0e200b03a0 con 0x7f0e301024e0 2024-03-13T15:04:12.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.447+0000 7f0e27400640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e1406e020 0x7f0e140704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:12.432 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.448+0000 7f0e27400640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e1406e020 0x7f0e140704e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e300694d0 tx=0x7f0e180072f0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:12.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.452+0000 7f0e2d600640 1 -- 172.21.15.176:0/1629949023 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f0e20079650 con 0x7f0e301024e0 2024-03-13T15:04:12.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.650+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e3010a7e0 con 0x7f0e301024e0 2024-03-13T15:04:12.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.652+0000 7f0e2d600640 1 -- 172.21.15.176:0/1629949023 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17190 (secure 0 0 0) 0x7f0e2007cd10 con 0x7f0e301024e0 2024-03-13T15:04:12.637 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:12.637 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":112,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","created":"2024-03-13T14:36:32.120003+0000","modified":"2024-03-13T15:01:08.961154+0000","last_up_change":"2024-03-13T14:59:42.380775+0000","last_in_change":"2024-03-13T14:43:42.351790+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T14:42:22.643268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-13T14:45:37.278783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-13T14:45:37.880653+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c570fcfa-ceda-4e39-b200-5c8b0b7fab19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":63,"up_thru":105,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6803","nonce":468040948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6805","nonce":468040948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6809","nonce":468040948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6807","nonce":468040948}]},"public_addr":"172.21.15.176:6803/468040948","cluster_addr":"172.21.15.176:6805/468040948","heartbeat_back_addr":"172.21.15.176:6809/468040948","heartbeat_front_addr":"172.21.15.176:6807/468040948","state":["exists","up"]},{"osd":1,"uuid":"5a973902-7ecc-4a05-b917-a74e108df4db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":71,"up_from":75,"up_thru":105,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6811","nonce":2654405375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6813","nonce":2654405375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6817","nonce":2654405375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6815","nonce":2654405375}]},"public_addr":"172.21.15.176:6811/2654405375","cluster_addr":"172.21.15.176:6813/2654405375","heartbeat_back_addr":"172.21.15.176:6817/2654405375","heartbeat_front_addr":"172.21.15.176:6815/2654405375","state":["exists","up"]},{"osd":2,"uuid":"f72757d1-b949-45fb-abcd-c8649cec14f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":76,"up_from":80,"up_thru":105,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6819","nonce":2651949103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6821","nonce":2651949103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6825","nonce":2651949103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6823","nonce":2651949103}]},"public_addr":"172.21.15.176:6819/2651949103","cluster_addr":"172.21.15.176:6821/2651949103","heartbeat_back_addr":"172.21.15.176:6825/2651949103","heartbeat_front_addr":"172.21.15.176:6823/2651949103","state":["exists","up"]},{"osd":3,"uuid":"e1c42f2a-0efa-4c44-a965-9e5904aeace0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":81,"up_from":89,"up_thru":105,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6801","nonce":1554416376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6803","nonce":1554416376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6807","nonce":1554416376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6805","nonce":1554416376}]},"public_addr":"172.21.15.196:6801/1554416376","cluster_addr":"172.21.15.196:6803/1554416376","heartbeat_back_addr":"172.21.15.196:6807/1554416376","heartbeat_front_addr":"172.21.15.196:6805/1554416376","state":["exists","up"]},{"osd":4,"uuid":"b4c7c89d-a349-4b39-b64a-e1080d213acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":96,"up_from":100,"up_thru":105,"down_at":97,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6809","nonce":1908907228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6811","nonce":1908907228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6815","nonce":1908907228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6813","nonce":1908907228}]},"public_addr":"172.21.15.196:6809/1908907228","cluster_addr":"172.21.15.196:6811/1908907228","heartbeat_back_addr":"172.21.15.196:6815/1908907228","heartbeat_front_addr":"172.21.15.196:6813/1908907228","state":["exists","up"]},{"osd":5,"uuid":"efb96c7b-7a78-4c40-996f-f8141029c310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":101,"up_from":105,"up_thru":105,"down_at":102,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6817","nonce":3411147075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6819","nonce":3411147075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6823","nonce":3411147075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6821","nonce":3411147075}]},"public_addr":"172.21.15.196:6817/3411147075","cluster_addr":"172.21.15.196:6819/3411147075","heartbeat_back_addr":"172.21.15.196:6823/3411147075","heartbeat_front_addr":"172.21.15.196:6821/3411147075","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:06.529418+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:43.356501+0000","dead_epoch":71},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:19.060195+0000","dead_epoch":76},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:53.551733+0000","dead_epoch":81},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:43:28.434416+0000","dead_epoch":96},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:44:05.009478+0000","dead_epoch":101}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.9","mappings":[{"from":0,"to":3}]},{"pgid":"2.15","mappings":[{"from":0,"to":2}]},{"pgid":"2.1e","mappings":[{"from":0,"to":3}]},{"pgid":"3.6","mappings":[{"from":1,"to":2}]},{"pgid":"3.b","mappings":[{"from":4,"to":2}]},{"pgid":"3.12","mappings":[{"from":1,"to":5}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6827/1275358571":"2024-03-14T15:01:08.961104+0000","172.21.15.196:6826/1275358571":"2024-03-14T15:01:08.961104+0000","172.21.15.176:6800/2698132085":"2024-03-14T14:48:28.652138+0000","172.21.15.176:0/2786891974":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/2461806366":"2024-03-14T14:54:00.256736+0000","172.21.15.176:6801/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.196:6828/1297073737":"2024-03-14T14:49:27.654006+0000","172.21.15.196:6829/1297073737":"2024-03-14T14:49:27.654006+0000","172.21.15.176:6800/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/3527709806":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/1319558216":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1541570168":"2024-03-14T14:48:28.652138+0000","172.21.15.176:0/248607826":"2024-03-14T14:37:47.494508+0000","172.21.15.196:0/1871453830":"2024-03-14T14:49:27.654006+0000","172.21.15.176:0/1825335586":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1477585567":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1947716670":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/1567438135":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2108157434":"2024-03-14T14:37:47.494508+0000","172.21.15.196:0/4213061990":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/3434817823":"2024-03-14T14:49:27.654006+0000","172.21.15.176:6801/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1665184531":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6800/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.196:6824/152396394":"2024-03-14T15:00:43.199506+0000","172.21.15.176:0/665298320":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6801/2698132085":"2024-03-14T14:48:28.652138+0000","172.21.15.196:0/3486069398":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/945284252":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/3034338010":"2024-03-14T14:49:27.654006+0000","172.21.15.196:6825/152396394":"2024-03-14T15:00:43.199506+0000","172.21.15.176:0/698195875":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1930887076":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6827/3592546199":"2024-03-14T14:45:59.134341+0000","172.21.15.176:0/3215654415":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/258177446":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6800/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6826/3592546199":"2024-03-14T14:45:59.134341+0000","172.21.15.176:0/2332303582":"2024-03-14T14:37:05.490145+0000","172.21.15.176:6801/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6801/3527709806":"2024-03-14T14:54:00.256736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e1406e020 msgr2=0x7f0e140704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e1406e020 0x7f0e140704e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e300694d0 tx=0x7f0e180072f0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 msgr2=0x7f0e30197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e30197fd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e20004270 tx=0x7f0e200042a0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 shutdown_connections 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e1406e020 0x7f0e140704e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e30198510 0x7f0e3019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 --2- 172.21.15.176:0/1629949023 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e301024e0 0x7f0e30197fd0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 >> 172.21.15.176:0/1629949023 conn(0x7f0e300fdd00 msgr2=0x7f0e300ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 shutdown_connections 2024-03-13T15:04:12.640 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.655+0000 7f0e36a00640 1 -- 172.21.15.176:0/1629949023 wait complete. 2024-03-13T15:04:12.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-13T14:42:22.643268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-13T14:45:37.278783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-13T14:45:37.880653+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-13T15:04:12.661 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-13T15:04:12.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.884+0000 7fb8a1000640 1 -- 172.21.15.176:0/2838629289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 msgr2=0x7fb89c1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:12.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.884+0000 7fb8a1000640 1 --2- 172.21.15.176:0/2838629289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c1026c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009880 tx=0x7fb88c02bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.884+0000 7fb8a1000640 1 -- 172.21.15.176:0/2838629289 shutdown_connections 2024-03-13T15:04:12.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.884+0000 7fb8a1000640 1 --2- 172.21.15.176:0/2838629289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c102c90 0x7fb89c106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.884+0000 7fb8a1000640 1 --2- 172.21.15.176:0/2838629289 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c1026c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.885+0000 7fb8a1000640 1 -- 172.21.15.176:0/2838629289 >> 172.21.15.176:0/2838629289 conn(0x7fb89c0fdb20 msgr2=0x7fb89c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:12.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.885+0000 7fb8a1000640 1 -- 172.21.15.176:0/2838629289 shutdown_connections 2024-03-13T15:04:12.870 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.885+0000 7fb8a1000640 1 -- 172.21.15.176:0/2838629289 wait complete. 2024-03-13T15:04:12.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb8a1000640 1 Processor -- start 2024-03-13T15:04:12.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb8a1000640 1 -- start start 2024-03-13T15:04:12.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb8a1000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:12.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb8a1000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c1983c0 0x7fb89c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb8a1000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1989c0 con 0x7fb89c1022e0 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb8a1000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c198b30 con 0x7fb89c1983c0 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb89b400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb89b400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35458/0 (socket says 172.21.15.176:35458) 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.886+0000 7fb89b400640 1 -- 172.21.15.176:0/516012676 learned_addr learned my addr 172.21.15.176:0/516012676 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb89aa00640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c1983c0 0x7fb89c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb89b400640 1 -- 172.21.15.176:0/516012676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c1983c0 msgr2=0x7fb89c19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb89b400640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c1983c0 0x7fb89c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb89b400640 1 -- 172.21.15.176:0/516012676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c009560 con 0x7fb89c1022e0 2024-03-13T15:04:12.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb89b400640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c197e80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb88c036040 tx=0x7fb88c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:12.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb893e00640 1 -- 172.21.15.176:0/516012676 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88c0037c0 con 0x7fb89c1022e0 2024-03-13T15:04:12.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb893e00640 1 -- 172.21.15.176:0/516012676 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb88c003de0 con 0x7fb89c1022e0 2024-03-13T15:04:12.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.887+0000 7fb893e00640 1 -- 172.21.15.176:0/516012676 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb88c034440 con 0x7fb89c1022e0 2024-03-13T15:04:12.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.888+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c19cdc0 con 0x7fb89c1022e0 2024-03-13T15:04:12.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.888+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c19d320 con 0x7fb89c1022e0 2024-03-13T15:04:12.874 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.889+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c04e040 con 0x7fb89c1022e0 2024-03-13T15:04:12.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.890+0000 7fb893e00640 1 -- 172.21.15.176:0/516012676 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb88c003920 con 0x7fb89c1022e0 2024-03-13T15:04:12.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.891+0000 7fb893e00640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb88006df10 0x7fb8800703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:12.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.891+0000 7fb893e00640 1 -- 172.21.15.176:0/516012676 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fb88c0b0410 con 0x7fb89c1022e0 2024-03-13T15:04:12.875 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.891+0000 7fb89aa00640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb88006df10 0x7fb8800703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:12.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.892+0000 7fb89aa00640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb88006df10 0x7fb8800703d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8880058c0 tx=0x7fb888005c60 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:12.882 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:12.896+0000 7fb893e00640 1 -- 172.21.15.176:0/516012676 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fb88c079780 con 0x7fb89c1022e0 2024-03-13T15:04:12.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:12 smithi196 ceph-mon[86193]: pgmap v371: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:12.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:12 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1629949023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T15:04:13.077 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.093+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb89c1991d0 con 0x7fb89c1022e0 2024-03-13T15:04:13.079 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.094+0000 7fb893e00640 1 -- 172.21.15.176:0/516012676 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb88c07ce40 con 0x7fb89c1022e0 2024-03-13T15:04:13.079 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 1 2024-03-13T15:04:13.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb88006df10 msgr2=0x7fb8800703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb88006df10 0x7fb8800703d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb8880058c0 tx=0x7fb888005c60 comp rx=0 tx=0).stop 2024-03-13T15:04:13.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 msgr2=0x7fb89c197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c197e80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb88c036040 tx=0x7fb88c004270 comp rx=0 tx=0).stop 2024-03-13T15:04:13.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 shutdown_connections 2024-03-13T15:04:13.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb88006df10 0x7fb8800703d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.082 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb89c1983c0 0x7fb89c19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.083 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 --2- 172.21.15.176:0/516012676 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb89c1022e0 0x7fb89c197e80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.083 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.097+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 >> 172.21.15.176:0/516012676 conn(0x7fb89c0fdb20 msgr2=0x7fb89c0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:13.083 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.098+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 shutdown_connections 2024-03-13T15:04:13.083 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.098+0000 7fb8a1000640 1 -- 172.21.15.176:0/516012676 wait complete. 2024-03-13T15:04:13.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:12 smithi176 ceph-mon[112742]: pgmap v371: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:13.098 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:12 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1629949023' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T15:04:13.102 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 -- 172.21.15.176:0/2869131562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0ff750 msgr2=0x7f27cc103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 --2- 172.21.15.176:0/2869131562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0ff750 0x7f27cc103c10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009880 tx=0x7f27b802bd80 comp rx=0 tx=0).stop 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 -- 172.21.15.176:0/2869131562 shutdown_connections 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 --2- 172.21.15.176:0/2869131562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0ff750 0x7f27cc103c10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 --2- 172.21.15.176:0/2869131562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27cc0fee30 0x7f27cc0ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 -- 172.21.15.176:0/2869131562 >> 172.21.15.176:0/2869131562 conn(0x7f27cc0fa880 msgr2=0x7f27cc0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 -- 172.21.15.176:0/2869131562 shutdown_connections 2024-03-13T15:04:13.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.321+0000 7f27d2200640 1 -- 172.21.15.176:0/2869131562 wait complete. 2024-03-13T15:04:13.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.322+0000 7f27d2200640 1 Processor -- start 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.322+0000 7f27d2200640 1 -- start start 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.322+0000 7f27d2200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0fee30 0x7f27cc19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.322+0000 7f27d2200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27cc0ff750 0x7f27cc19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d2200640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc19cd10 con 0x7f27cc0fee30 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d2200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc19ce80 con 0x7f27cc0ff750 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d0e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0fee30 0x7f27cc19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d0e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0fee30 0x7f27cc19c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35472/0 (socket says 172.21.15.176:35472) 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d0e00640 1 -- 172.21.15.176:0/4275358887 learned_addr learned my addr 172.21.15.176:0/4275358887 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d0e00640 1 -- 172.21.15.176:0/4275358887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27cc0ff750 msgr2=0x7f27cc19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d0e00640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27cc0ff750 0x7f27cc19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.323+0000 7f27d0e00640 1 -- 172.21.15.176:0/4275358887 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8009560 con 0x7f27cc0fee30 2024-03-13T15:04:13.308 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.324+0000 7f27d0e00640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0fee30 0x7f27cc19c1d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c8a0 tx=0x7f27c000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:13.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.324+0000 7f27c9600640 1 -- 172.21.15.176:0/4275358887 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27c0007b80 con 0x7f27cc0fee30 2024-03-13T15:04:13.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.324+0000 7f27c9600640 1 -- 172.21.15.176:0/4275358887 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27c0007ce0 con 0x7f27cc0fee30 2024-03-13T15:04:13.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.324+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc1a0d10 con 0x7f27cc0fee30 2024-03-13T15:04:13.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.324+0000 7f27c9600640 1 -- 172.21.15.176:0/4275358887 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27c0018810 con 0x7f27cc0fee30 2024-03-13T15:04:13.309 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.324+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc1a1270 con 0x7f27cc0fee30 2024-03-13T15:04:13.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.325+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc04e040 con 0x7f27cc0fee30 2024-03-13T15:04:13.310 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.326+0000 7f27c9600640 1 -- 172.21.15.176:0/4275358887 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f27c0018a80 con 0x7f27cc0fee30 2024-03-13T15:04:13.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.327+0000 7f27c9600640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27b406df10 0x7f27b40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:13.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.327+0000 7f27c9600640 1 -- 172.21.15.176:0/4275358887 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f27c008e040 con 0x7f27cc0fee30 2024-03-13T15:04:13.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.327+0000 7f27cbe00640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27b406df10 0x7f27b40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:13.312 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.328+0000 7f27cbe00640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27b406df10 0x7f27b40703d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27b8036040 tx=0x7f27b8000bc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:13.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.332+0000 7f27c9600640 1 -- 172.21.15.176:0/4275358887 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f27c0057390 con 0x7f27cc0fee30 2024-03-13T15:04:13.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.542+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f27cc1012e0 con 0x7f27cc0fee30 2024-03-13T15:04:13.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.543+0000 7f27c9600640 1 -- 172.21.15.176:0/4275358887 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v112) v1 ==== 107+0+11 (secure 0 0 0) 0x7f27c005aa50 con 0x7f27cc0fee30 2024-03-13T15:04:13.527 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27b406df10 msgr2=0x7f27b40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27b406df10 0x7f27b40703d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27b8036040 tx=0x7f27b8000bc0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0fee30 msgr2=0x7f27cc19c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0fee30 0x7f27cc19c1d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f27c000c8a0 tx=0x7f27c000cd60 comp rx=0 tx=0).stop 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 shutdown_connections 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27b406df10 0x7f27b40703d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27cc0ff750 0x7f27cc19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 --2- 172.21.15.176:0/4275358887 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27cc0fee30 0x7f27cc19c1d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 >> 172.21.15.176:0/4275358887 conn(0x7f27cc0fa880 msgr2=0x7f27cc110290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.545+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 shutdown_connections 2024-03-13T15:04:13.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.546+0000 7f27d2200640 1 -- 172.21.15.176:0/4275358887 wait complete. 2024-03-13T15:04:13.548 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 -- 172.21.15.176:0/3374728598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c067f70 msgr2=0x7fc24c104c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 --2- 172.21.15.176:0/3374728598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c067f70 0x7fc24c104c70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc238009880 tx=0x7fc23802bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 -- 172.21.15.176:0/3374728598 shutdown_connections 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 --2- 172.21.15.176:0/3374728598 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c067f70 0x7fc24c104c70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 --2- 172.21.15.176:0/3374728598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0675c0 0x7fc24c0679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 -- 172.21.15.176:0/3374728598 >> 172.21.15.176:0/3374728598 conn(0x7fc24c074a50 msgr2=0x7fc24c074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 -- 172.21.15.176:0/3374728598 shutdown_connections 2024-03-13T15:04:13.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.753+0000 7fc252000640 1 -- 172.21.15.176:0/3374728598 wait complete. 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.754+0000 7fc252000640 1 Processor -- start 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.754+0000 7fc252000640 1 -- start start 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc252000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0675c0 0x7fc24c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc252000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c198490 0x7fc24c19c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc252000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c198a90 con 0x7fc24c198490 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc252000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c198c00 con 0x7fc24c0675c0 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc24aa00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c198490 0x7fc24c19c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc24aa00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c198490 0x7fc24c19c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35498/0 (socket says 172.21.15.176:35498) 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc24aa00640 1 -- 172.21.15.176:0/2365178629 learned_addr learned my addr 172.21.15.176:0/2365178629 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:13.741 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc24aa00640 1 -- 172.21.15.176:0/2365178629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0675c0 msgr2=0x7fc24c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc24b400640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0675c0 0x7fc24c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc24aa00640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0675c0 0x7fc24c197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.755+0000 7fc24aa00640 1 -- 172.21.15.176:0/2365178629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238009560 con 0x7fc24c198490 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.756+0000 7fc24aa00640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c198490 0x7fc24c19c8f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc238036040 tx=0x7fc23802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.756+0000 7fc243e00640 1 -- 172.21.15.176:0/2365178629 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2380035f0 con 0x7fc24c198490 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.756+0000 7fc243e00640 1 -- 172.21.15.176:0/2365178629 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc238003c10 con 0x7fc24c198490 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.756+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c19ce90 con 0x7fc24c198490 2024-03-13T15:04:13.742 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.756+0000 7fc243e00640 1 -- 172.21.15.176:0/2365178629 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc238034440 con 0x7fc24c198490 2024-03-13T15:04:13.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.756+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c19d3f0 con 0x7fc24c198490 2024-03-13T15:04:13.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.757+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c19d720 con 0x7fc24c198490 2024-03-13T15:04:13.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.758+0000 7fc243e00640 1 -- 172.21.15.176:0/2365178629 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc238003750 con 0x7fc24c198490 2024-03-13T15:04:13.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.759+0000 7fc243e00640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc23006df10 0x7fc2300703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:13.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.759+0000 7fc243e00640 1 -- 172.21.15.176:0/2365178629 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fc2380b0240 con 0x7fc24c198490 2024-03-13T15:04:13.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.759+0000 7fc24b400640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc23006df10 0x7fc2300703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:13.744 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.760+0000 7fc24b400640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc23006df10 0x7fc2300703d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005fd0 tx=0x7fc23c005850 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:13.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:13 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/516012676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T15:04:13.748 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:13 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/4275358887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T15:04:13.749 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.764+0000 7fc243e00640 1 -- 172.21.15.176:0/2365178629 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc238079570 con 0x7fc24c198490 2024-03-13T15:04:13.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:13 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/516012676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T15:04:13.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:13 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/4275358887' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T15:04:13.944 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.960+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fc24c04e040 con 0x7fc24c198490 2024-03-13T15:04:13.945 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.960+0000 7fc243e00640 1 -- 172.21.15.176:0/2365178629 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v112) v1 ==== 107+0+11 (secure 0 0 0) 0x7fc23807cc30 con 0x7fc24c198490 2024-03-13T15:04:13.945 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T15:04:13.947 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc23006df10 msgr2=0x7fc2300703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc23006df10 0x7fc2300703d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc23c005fd0 tx=0x7fc23c005850 comp rx=0 tx=0).stop 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c198490 msgr2=0x7fc24c19c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c198490 0x7fc24c19c8f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc238036040 tx=0x7fc23802c850 comp rx=0 tx=0).stop 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 shutdown_connections 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc24c198490 0x7fc24c19c8f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc23006df10 0x7fc2300703d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 --2- 172.21.15.176:0/2365178629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0675c0 0x7fc24c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 >> 172.21.15.176:0/2365178629 conn(0x7fc24c074a50 msgr2=0x7fc24c101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 shutdown_connections 2024-03-13T15:04:13.948 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:13.963+0000 7fc252000640 1 -- 172.21.15.176:0/2365178629 wait complete. 2024-03-13T15:04:13.965 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-13T15:04:14.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.172+0000 7f69e7e00640 1 -- 172.21.15.176:0/1274341080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8100a60 msgr2=0x7f69e8104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.172+0000 7f69e7e00640 1 --2- 172.21.15.176:0/1274341080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8100a60 0x7f69e8104aa0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009880 tx=0x7f69d002bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.172+0000 7f69e7e00640 1 -- 172.21.15.176:0/1274341080 shutdown_connections 2024-03-13T15:04:14.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.172+0000 7f69e7e00640 1 --2- 172.21.15.176:0/1274341080 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e8100a60 0x7f69e8104aa0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.172+0000 7f69e7e00640 1 --2- 172.21.15.176:0/1274341080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e81000b0 0x7f69e8100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.157 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.172+0000 7f69e7e00640 1 -- 172.21.15.176:0/1274341080 >> 172.21.15.176:0/1274341080 conn(0x7f69e80fb930 msgr2=0x7f69e80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:14.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.172+0000 7f69e7e00640 1 -- 172.21.15.176:0/1274341080 shutdown_connections 2024-03-13T15:04:14.158 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.173+0000 7f69e7e00640 1 -- 172.21.15.176:0/1274341080 wait complete. 2024-03-13T15:04:14.159 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e7e00640 1 Processor -- start 2024-03-13T15:04:14.159 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e7e00640 1 -- start start 2024-03-13T15:04:14.159 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e7e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e81000b0 0x7f69e819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:14.159 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e7e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e819bec0 0x7f69e81959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:14.159 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e7e00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e819c400 con 0x7f69e81000b0 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e7e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e819c540 con 0x7f69e819bec0 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e6a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e81000b0 0x7f69e819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e6a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e81000b0 0x7f69e819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35502/0 (socket says 172.21.15.176:35502) 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.174+0000 7f69e6a00640 1 -- 172.21.15.176:0/3830495351 learned_addr learned my addr 172.21.15.176:0/3830495351 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69de000640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e819bec0 0x7f69e81959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69e6a00640 1 -- 172.21.15.176:0/3830495351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e819bec0 msgr2=0x7f69e81959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69e6a00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e819bec0 0x7f69e81959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69e6a00640 1 -- 172.21.15.176:0/3830495351 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0009560 con 0x7f69e81000b0 2024-03-13T15:04:14.160 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69de000640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e819bec0 0x7f69e81959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:14.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69e6a00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e81000b0 0x7f69e819b980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f69d800c8a0 tx=0x7f69d800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:14.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69dfe00640 1 -- 172.21.15.176:0/3830495351 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d8007b80 con 0x7f69e81000b0 2024-03-13T15:04:14.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69dfe00640 1 -- 172.21.15.176:0/3830495351 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69d8007ce0 con 0x7f69e81000b0 2024-03-13T15:04:14.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69dfe00640 1 -- 172.21.15.176:0/3830495351 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d8018810 con 0x7f69e81000b0 2024-03-13T15:04:14.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.175+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e8195ff0 con 0x7f69e81000b0 2024-03-13T15:04:14.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.176+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e8196550 con 0x7f69e81000b0 2024-03-13T15:04:14.161 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.177+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e8196830 con 0x7f69e81000b0 2024-03-13T15:04:14.162 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.178+0000 7f69dfe00640 1 -- 172.21.15.176:0/3830495351 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f69d8018a80 con 0x7f69e81000b0 2024-03-13T15:04:14.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.179+0000 7f69dfe00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f69cc06dfd0 0x7f69cc070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:14.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.179+0000 7f69dfe00640 1 -- 172.21.15.176:0/3830495351 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f69d808e0c0 con 0x7f69e81000b0 2024-03-13T15:04:14.163 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.179+0000 7f69de000640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f69cc06dfd0 0x7f69cc070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:14.164 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.180+0000 7f69de000640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f69cc06dfd0 0x7f69cc070490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69d0036040 tx=0x7f69d0000bc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:14.167 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.183+0000 7f69dfe00640 1 -- 172.21.15.176:0/3830495351 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f69d8057410 con 0x7f69e81000b0 2024-03-13T15:04:14.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.418+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69e8108410 con 0x7f69e81000b0 2024-03-13T15:04:14.404 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.420+0000 7f69dfe00640 1 -- 172.21.15.176:0/3830495351 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5011 (secure 0 0 0) 0x7f69d805aad0 con 0x7f69e81000b0 2024-03-13T15:04:14.405 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:14.405 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44281,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6827/2135805809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2135805809},{"type":"v1","addr":"172.21.15.196:6827","nonce":2135805809}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:02:06.329079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34494":{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:14.405 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 39 2024-03-13T15:04:14.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f69cc06dfd0 msgr2=0x7f69cc070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.407 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f69cc06dfd0 0x7f69cc070490 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f69d0036040 tx=0x7f69d0000bc0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e81000b0 msgr2=0x7f69e819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e81000b0 0x7f69e819b980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f69d800c8a0 tx=0x7f69d800cd60 comp rx=0 tx=0).stop 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 shutdown_connections 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f69cc06dfd0 0x7f69cc070490 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f69e819bec0 0x7f69e81959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 --2- 172.21.15.176:0/3830495351 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f69e81000b0 0x7f69e819b980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 >> 172.21.15.176:0/3830495351 conn(0x7f69e80fb930 msgr2=0x7f69e80fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 shutdown_connections 2024-03-13T15:04:14.408 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.423+0000 7f69e7e00640 1 -- 172.21.15.176:0/3830495351 wait complete. 2024-03-13T15:04:14.424 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 1, 'flags': 50} 2024-03-13T15:04:14.424 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-03-13T15:04:14.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.645+0000 7f465cc00640 1 -- 172.21.15.176:0/2876081290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101b20 msgr2=0x7f465810a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.645+0000 7f465cc00640 1 --2- 172.21.15.176:0/2876081290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101b20 0x7f465810a000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4644009880 tx=0x7f464402bd70 comp rx=0 tx=0).stop 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.645+0000 7f465cc00640 1 -- 172.21.15.176:0/2876081290 shutdown_connections 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.645+0000 7f465cc00640 1 --2- 172.21.15.176:0/2876081290 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101b20 0x7f465810a000 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.645+0000 7f465cc00640 1 --2- 172.21.15.176:0/2876081290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4658101200 0x7f46581015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.645+0000 7f465cc00640 1 -- 172.21.15.176:0/2876081290 >> 172.21.15.176:0/2876081290 conn(0x7f46580faa20 msgr2=0x7f46580fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.645+0000 7f465cc00640 1 -- 172.21.15.176:0/2876081290 shutdown_connections 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.646+0000 7f465cc00640 1 -- 172.21.15.176:0/2876081290 wait complete. 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.646+0000 7f465cc00640 1 Processor -- start 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.646+0000 7f465cc00640 1 -- start start 2024-03-13T15:04:14.632 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f465cc00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101200 0x7f4658198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:14.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f465cc00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4658101b20 0x7f4658198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:14.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f465cc00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4658198bd0 con 0x7f4658101200 2024-03-13T15:04:14.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f465cc00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465819c9a0 con 0x7f4658101b20 2024-03-13T15:04:14.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f4655e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101200 0x7f4658198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:14.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f4655e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101200 0x7f4658198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35520/0 (socket says 172.21.15.176:35520) 2024-03-13T15:04:14.633 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f4655e00640 1 -- 172.21.15.176:0/1074428045 learned_addr learned my addr 172.21.15.176:0/1074428045 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:14.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f4655400640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4658101b20 0x7f4658198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:14.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f4655e00640 1 -- 172.21.15.176:0/1074428045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4658101b20 msgr2=0x7f4658198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f4655e00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4658101b20 0x7f4658198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.647+0000 7f4655e00640 1 -- 172.21.15.176:0/1074428045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644009560 con 0x7f4658101200 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.648+0000 7f4655e00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101200 0x7f4658198000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f464800d6f0 tx=0x7f464800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.648+0000 7f464ea00640 1 -- 172.21.15.176:0/1074428045 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4648009a30 con 0x7f4658101200 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.648+0000 7f464ea00640 1 -- 172.21.15.176:0/1074428045 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46480054e0 con 0x7f4658101200 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.648+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465819cc80 con 0x7f4658101200 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.648+0000 7f464ea00640 1 -- 172.21.15.176:0/1074428045 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4648010460 con 0x7f4658101200 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.648+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465819d1e0 con 0x7f4658101200 2024-03-13T15:04:14.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.649+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465804e040 con 0x7f4658101200 2024-03-13T15:04:14.636 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.651+0000 7f464ea00640 1 -- 172.21.15.176:0/1074428045 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f4648009b90 con 0x7f4658101200 2024-03-13T15:04:14.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.653+0000 7f464ea00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f463c06df10 0x7f463c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:14.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.653+0000 7f464ea00640 1 -- 172.21.15.176:0/1074428045 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f464808dac0 con 0x7f4658101200 2024-03-13T15:04:14.637 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.653+0000 7f4655400640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f463c06df10 0x7f463c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:14.638 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.654+0000 7f4655400640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f463c06df10 0x7f463c0703d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4644036040 tx=0x7f4644005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:14.642 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.658+0000 7f464ea00640 1 -- 172.21.15.176:0/1074428045 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f4648056d70 con 0x7f4658101200 2024-03-13T15:04:14.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.880+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f465810d2e0 con 0x7f4658101200 2024-03-13T15:04:14.866 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.882+0000 7f464ea00640 1 -- 172.21.15.176:0/1074428045 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v39) v1 ==== 107+0+4050 (secure 0 0 0) 0x7f464805a430 con 0x7f4658101200 2024-03-13T15:04:14.866 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:14.866 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/668575330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":668575330},{"type":"v1","addr":"172.21.15.176:6827","nonce":668575330}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T14:59:47.444191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi196.tcitrk","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6825/152396394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":152396394},{"type":"v1","addr":"172.21.15.196:6825","nonce":152396394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:14.867 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 19 2024-03-13T15:04:14.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f463c06df10 msgr2=0x7f463c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f463c06df10 0x7f463c0703d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4644036040 tx=0x7f4644005fd0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101200 msgr2=0x7f4658198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:14.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101200 0x7f4658198000 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f464800d6f0 tx=0x7f464800dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 shutdown_connections 2024-03-13T15:04:14.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f463c06df10 0x7f463c0703d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4658101b20 0x7f4658198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 --2- 172.21.15.176:0/1074428045 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f4658101200 0x7f4658198000 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:14.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 >> 172.21.15.176:0/1074428045 conn(0x7f46580faa20 msgr2=0x7f46580ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:14.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 shutdown_connections 2024-03-13T15:04:14.869 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:14.884+0000 7f465cc00640 1 -- 172.21.15.176:0/1074428045 wait complete. 2024-03-13T15:04:14.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:14 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2365178629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T15:04:14.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:14 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3830495351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-13T15:04:14.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:14 smithi176 ceph-mon[112742]: pgmap v372: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:14.886 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 19 2024-03-13T15:04:14.886 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-03-13T15:04:14.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:14 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2365178629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T15:04:14.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:14 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3830495351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-13T15:04:14.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:14 smithi196 ceph-mon[86193]: pgmap v372: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:15.098 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 -- 172.21.15.176:0/2158493636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 msgr2=0x7f952810c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 --2- 172.21.15.176:0/2158493636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f952810c180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9510009880 tx=0x7f951002bd80 comp rx=0 tx=0).stop 2024-03-13T15:04:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 -- 172.21.15.176:0/2158493636 shutdown_connections 2024-03-13T15:04:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 --2- 172.21.15.176:0/2158493636 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f952810c180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 --2- 172.21.15.176:0/2158493636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95280726d0 0x7f9528072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 -- 172.21.15.176:0/2158493636 >> 172.21.15.176:0/2158493636 conn(0x7f95280fb970 msgr2=0x7f95280fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 -- 172.21.15.176:0/2158493636 shutdown_connections 2024-03-13T15:04:15.099 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.114+0000 7f9527e00640 1 -- 172.21.15.176:0/2158493636 wait complete. 2024-03-13T15:04:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.115+0000 7f9527e00640 1 Processor -- start 2024-03-13T15:04:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.115+0000 7f9527e00640 1 -- start start 2024-03-13T15:04:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.115+0000 7f9527e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95280726d0 0x7f9528197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.115+0000 7f9527e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f95281982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:15.101 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.115+0000 7f9527e00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9528198940 con 0x7f9528072ff0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.115+0000 7f9527e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952819c710 con 0x7f95280726d0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f95281982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f95281982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35528/0 (socket says 172.21.15.176:35528) 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526000640 1 -- 172.21.15.176:0/3810705721 learned_addr learned my addr 172.21.15.176:0/3810705721 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526a00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95280726d0 0x7f9528197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526000640 1 -- 172.21.15.176:0/3810705721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95280726d0 msgr2=0x7f9528197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526000640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95280726d0 0x7f9528197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526000640 1 -- 172.21.15.176:0/3810705721 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9510009560 con 0x7f9528072ff0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526a00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95280726d0 0x7f9528197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.116+0000 7f9526000640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f95281982b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9510036040 tx=0x7f9510004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.117+0000 7f952ca00640 1 -- 172.21.15.176:0/3810705721 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95100037c0 con 0x7f9528072ff0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.117+0000 7f952ca00640 1 -- 172.21.15.176:0/3810705721 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9510003de0 con 0x7f9528072ff0 2024-03-13T15:04:15.102 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.117+0000 7f952ca00640 1 -- 172.21.15.176:0/3810705721 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951003e450 con 0x7f9528072ff0 2024-03-13T15:04:15.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.117+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952819c990 con 0x7f9528072ff0 2024-03-13T15:04:15.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.117+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952819cef0 con 0x7f9528072ff0 2024-03-13T15:04:15.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.118+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952804e040 con 0x7f9528072ff0 2024-03-13T15:04:15.103 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.119+0000 7f952ca00640 1 -- 172.21.15.176:0/3810705721 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f9510005060 con 0x7f9528072ff0 2024-03-13T15:04:15.104 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.120+0000 7f952ca00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f950c06df10 0x7f950c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:15.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.120+0000 7f952ca00640 1 -- 172.21.15.176:0/3810705721 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f95100b0b50 con 0x7f9528072ff0 2024-03-13T15:04:15.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.121+0000 7f9526a00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f950c06df10 0x7f950c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:15.105 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.121+0000 7f9526a00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f950c06df10 0x7f950c0703d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9518005c40 tx=0x7f95180082f0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:15.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.125+0000 7f952ca00640 1 -- 172.21.15.176:0/3810705721 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f9510079fa0 con 0x7f9528072ff0 2024-03-13T15:04:15.336 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.352+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f95281097f0 con 0x7f9528072ff0 2024-03-13T15:04:15.337 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.353+0000 7f952ca00640 1 -- 172.21.15.176:0/3810705721 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v39) v1 ==== 107+0+4826 (secure 0 0 0) 0x7f951007d660 con 0x7f9528072ff0 2024-03-13T15:04:15.338 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:15.338 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14498,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/668575330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":668575330},{"type":"v1","addr":"172.21.15.176:6827","nonce":668575330}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34450,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/46228150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":46228150},{"type":"v1","addr":"172.21.15.176:6829","nonce":46228150}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T14:59:47.444191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi196.tcitrk","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6825/152396394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":152396394},{"type":"v1","addr":"172.21.15.196:6825","nonce":152396394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:15.338 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 20 2024-03-13T15:04:15.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f950c06df10 msgr2=0x7f950c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f950c06df10 0x7f950c0703d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9518005c40 tx=0x7f95180082f0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 msgr2=0x7f95281982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f95281982b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9510036040 tx=0x7f9510004270 comp rx=0 tx=0).stop 2024-03-13T15:04:15.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 shutdown_connections 2024-03-13T15:04:15.340 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9528072ff0 0x7f95281982b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f950c06df10 0x7f950c0703d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 --2- 172.21.15.176:0/3810705721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95280726d0 0x7f9528197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.355+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 >> 172.21.15.176:0/3810705721 conn(0x7f95280fb970 msgr2=0x7f95281069c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:15.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.356+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 shutdown_connections 2024-03-13T15:04:15.341 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.356+0000 7f9527e00640 1 -- 172.21.15.176:0/3810705721 wait complete. 2024-03-13T15:04:15.358 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-03-13T15:04:15.358 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 -- 172.21.15.176:0/1654339725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481024d0 msgr2=0x7fa8481028b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 --2- 172.21.15.176:0/1654339725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481024d0 0x7fa8481028b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa838009880 tx=0x7fa83802bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 -- 172.21.15.176:0/1654339725 shutdown_connections 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 --2- 172.21.15.176:0/1654339725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa848102e80 0x7fa848106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 --2- 172.21.15.176:0/1654339725 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481024d0 0x7fa8481028b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 -- 172.21.15.176:0/1654339725 >> 172.21.15.176:0/1654339725 conn(0x7fa8480fdd10 msgr2=0x7fa848100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 -- 172.21.15.176:0/1654339725 shutdown_connections 2024-03-13T15:04:15.564 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.579+0000 7fa84d800640 1 -- 172.21.15.176:0/1654339725 wait complete. 2024-03-13T15:04:15.566 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.580+0000 7fa84d800640 1 Processor -- start 2024-03-13T15:04:15.566 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.580+0000 7fa84d800640 1 -- start start 2024-03-13T15:04:15.566 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.580+0000 7fa84d800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481024d0 0x7fa848197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:15.566 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.580+0000 7fa84d800640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481983a0 0x7fa84819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:15.566 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.580+0000 7fa84d800640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8481989a0 con 0x7fa8481983a0 2024-03-13T15:04:15.566 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.580+0000 7fa84d800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa848198b10 con 0x7fa8481024d0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.581+0000 7fa846000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481983a0 0x7fa84819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.581+0000 7fa846000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481983a0 0x7fa84819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35550/0 (socket says 172.21.15.176:35550) 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.581+0000 7fa846000640 1 -- 172.21.15.176:0/3043965787 learned_addr learned my addr 172.21.15.176:0/3043965787 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.581+0000 7fa846000640 1 -- 172.21.15.176:0/3043965787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481024d0 msgr2=0x7fa848197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.581+0000 7fa846000640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481024d0 0x7fa848197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.581+0000 7fa846000640 1 -- 172.21.15.176:0/3043965787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa838009560 con 0x7fa8481983a0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.581+0000 7fa846000640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481983a0 0x7fa84819c800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c920 tx=0x7fa83400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.582+0000 7fa83f400640 1 -- 172.21.15.176:0/3043965787 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa834007b80 con 0x7fa8481983a0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.582+0000 7fa83f400640 1 -- 172.21.15.176:0/3043965787 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8340054e0 con 0x7fa8481983a0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.582+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84819ce00 con 0x7fa8481983a0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.582+0000 7fa83f400640 1 -- 172.21.15.176:0/3043965787 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa83400f450 con 0x7fa8481983a0 2024-03-13T15:04:15.567 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.582+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84819d360 con 0x7fa8481983a0 2024-03-13T15:04:15.568 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.583+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84819d640 con 0x7fa8481983a0 2024-03-13T15:04:15.568 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.584+0000 7fa83f400640 1 -- 172.21.15.176:0/3043965787 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa8340046c0 con 0x7fa8481983a0 2024-03-13T15:04:15.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.584+0000 7fa83f400640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa82c06dfd0 0x7fa82c070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:15.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.585+0000 7fa83f400640 1 -- 172.21.15.176:0/3043965787 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa83408dd60 con 0x7fa8481983a0 2024-03-13T15:04:15.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.585+0000 7fa846a00640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa82c06dfd0 0x7fa82c070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:15.569 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.585+0000 7fa846a00640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa82c06dfd0 0x7fa82c070490 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa838036040 tx=0x7fa838000bc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:15.573 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.589+0000 7fa83f400640 1 -- 172.21.15.176:0/3043965787 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa834092070 con 0x7fa8481983a0 2024-03-13T15:04:15.808 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.824+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7fa84804e040 con 0x7fa8481983a0 2024-03-13T15:04:15.810 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.825+0000 7fa83f400640 1 -- 172.21.15.176:0/3043965787 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v39) v1 ==== 107+0+4047 (secure 0 0 0) 0x7fa834057010 con 0x7fa8481983a0 2024-03-13T15:04:15.810 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:15.810 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34450,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/46228150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":46228150},{"type":"v1","addr":"172.21.15.176:6829","nonce":46228150}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T14:59:47.444191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi196.tcitrk","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6825/152396394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":152396394},{"type":"v1","addr":"172.21.15.196:6825","nonce":152396394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:15.810 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 21 2024-03-13T15:04:15.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa82c06dfd0 msgr2=0x7fa82c070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa82c06dfd0 0x7fa82c070490 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa838036040 tx=0x7fa838000bc0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481983a0 msgr2=0x7fa84819c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:15.812 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481983a0 0x7fa84819c800 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c920 tx=0x7fa83400cde0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 shutdown_connections 2024-03-13T15:04:15.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa8481983a0 0x7fa84819c800 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa82c06dfd0 0x7fa82c070490 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 --2- 172.21.15.176:0/3043965787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8481024d0 0x7fa848197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:15.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 >> 172.21.15.176:0/3043965787 conn(0x7fa8480fdd10 msgr2=0x7fa8481000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:15.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 shutdown_connections 2024-03-13T15:04:15.813 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:15.828+0000 7fa84d800640 1 -- 172.21.15.176:0/3043965787 wait complete. 2024-03-13T15:04:15.831 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-03-13T15:04:15.831 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-03-13T15:04:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:15 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1074428045' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-13T15:04:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:15 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3810705721' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-13T15:04:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 -- 172.21.15.176:0/1296215108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae8102e70 msgr2=0x7fdae8106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 --2- 172.21.15.176:0/1296215108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae8102e70 0x7fdae8106eb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009880 tx=0x7fdad002bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 -- 172.21.15.176:0/1296215108 shutdown_connections 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 --2- 172.21.15.176:0/1296215108 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae8102e70 0x7fdae8106eb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 --2- 172.21.15.176:0/1296215108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae81024c0 0x7fdae81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 -- 172.21.15.176:0/1296215108 >> 172.21.15.176:0/1296215108 conn(0x7fdae80fdd40 msgr2=0x7fdae8100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 -- 172.21.15.176:0/1296215108 shutdown_connections 2024-03-13T15:04:16.045 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.060+0000 7fdaef400640 1 -- 172.21.15.176:0/1296215108 wait complete. 2024-03-13T15:04:16.047 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.061+0000 7fdaef400640 1 Processor -- start 2024-03-13T15:04:16.047 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.061+0000 7fdaef400640 1 -- start start 2024-03-13T15:04:16.047 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdaef400640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae81024c0 0x7fdae8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:16.047 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdaef400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae81984b0 0x7fdae819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdaef400640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae8198ab0 con 0x7fdae81984b0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdaef400640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae8198c20 con 0x7fdae81024c0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae81984b0 0x7fdae819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdadfe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae81984b0 0x7fdae819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35570/0 (socket says 172.21.15.176:35570) 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdadfe00640 1 -- 172.21.15.176:0/1628558346 learned_addr learned my addr 172.21.15.176:0/1628558346 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdaeca00640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae81024c0 0x7fdae8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdadfe00640 1 -- 172.21.15.176:0/1628558346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae81024c0 msgr2=0x7fdae8197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdadfe00640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae81024c0 0x7fdae8197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdadfe00640 1 -- 172.21.15.176:0/1628558346 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad0009560 con 0x7fdae81984b0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.062+0000 7fdaeca00640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae81024c0 0x7fdae8197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.063+0000 7fdadfe00640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae81984b0 0x7fdae819c910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036040 tx=0x7fdad002c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.063+0000 7fdae6000640 1 -- 172.21.15.176:0/1628558346 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad0004520 con 0x7fdae81984b0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.063+0000 7fdae6000640 1 -- 172.21.15.176:0/1628558346 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdad0003ab0 con 0x7fdae81984b0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.063+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae819ceb0 con 0x7fdae81984b0 2024-03-13T15:04:16.048 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.063+0000 7fdae6000640 1 -- 172.21.15.176:0/1628558346 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdad0034440 con 0x7fdae81984b0 2024-03-13T15:04:16.049 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.063+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae819d410 con 0x7fdae81984b0 2024-03-13T15:04:16.049 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.064+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae804e040 con 0x7fdae81984b0 2024-03-13T15:04:16.049 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.065+0000 7fdae6000640 1 -- 172.21.15.176:0/1628558346 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fdad0003c20 con 0x7fdae81984b0 2024-03-13T15:04:16.050 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.066+0000 7fdae6000640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdacc06df10 0x7fdacc0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:16.050 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.066+0000 7fdae6000640 1 -- 172.21.15.176:0/1628558346 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fdad00b0240 con 0x7fdae81984b0 2024-03-13T15:04:16.051 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.066+0000 7fdaeca00640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdacc06df10 0x7fdacc0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:16.051 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.067+0000 7fdaeca00640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdacc06df10 0x7fdacc0703d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009b40 tx=0x7fdad8005e50 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:16.055 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.071+0000 7fdae6000640 1 -- 172.21.15.176:0/1628558346 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fdad0079590 con 0x7fdae81984b0 2024-03-13T15:04:16.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:15 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1074428045' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-03-13T15:04:16.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:15 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3810705721' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-03-13T15:04:16.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:16.292 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.308+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7fdae810a7c0 con 0x7fdae81984b0 2024-03-13T15:04:16.294 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.309+0000 7fdae6000640 1 -- 172.21.15.176:0/1628558346 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v39) v1 ==== 107+0+4860 (secure 0 0 0) 0x7fdad0077e60 con 0x7fdae81984b0 2024-03-13T15:04:16.294 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:16.294 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34450,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/46228150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":46228150},{"type":"v1","addr":"172.21.15.176:6829","nonce":46228150}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T14:59:47.444191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi196.tcitrk","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6825/152396394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":152396394},{"type":"v1","addr":"172.21.15.196:6825","nonce":152396394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:16.295 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 22 2024-03-13T15:04:16.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdacc06df10 msgr2=0x7fdacc0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdacc06df10 0x7fdacc0703d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009b40 tx=0x7fdad8005e50 comp rx=0 tx=0).stop 2024-03-13T15:04:16.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae81984b0 msgr2=0x7fdae819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae81984b0 0x7fdae819c910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdad0036040 tx=0x7fdad002c7a0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 shutdown_connections 2024-03-13T15:04:16.297 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fdae81984b0 0x7fdae819c910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fdacc06df10 0x7fdacc0703d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 --2- 172.21.15.176:0/1628558346 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdae81024c0 0x7fdae8197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.312+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 >> 172.21.15.176:0/1628558346 conn(0x7fdae80fdd40 msgr2=0x7fdae80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:16.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.313+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 shutdown_connections 2024-03-13T15:04:16.298 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.313+0000 7fdaef400640 1 -- 172.21.15.176:0/1628558346 wait complete. 2024-03-13T15:04:16.316 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-03-13T15:04:16.316 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-03-13T15:04:16.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.541+0000 7fbc15200640 1 -- 172.21.15.176:0/584550617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10102e70 msgr2=0x7fbc10106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.541+0000 7fbc15200640 1 --2- 172.21.15.176:0/584550617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10102e70 0x7fbc10106eb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009880 tx=0x7fbbf802bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.541+0000 7fbc15200640 1 -- 172.21.15.176:0/584550617 shutdown_connections 2024-03-13T15:04:16.526 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.541+0000 7fbc15200640 1 --2- 172.21.15.176:0/584550617 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10102e70 0x7fbc10106eb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.541+0000 7fbc15200640 1 --2- 172.21.15.176:0/584550617 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc101024c0 0x7fbc101028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.541+0000 7fbc15200640 1 -- 172.21.15.176:0/584550617 >> 172.21.15.176:0/584550617 conn(0x7fbc100fdd40 msgr2=0x7fbc10100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:16.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.541+0000 7fbc15200640 1 -- 172.21.15.176:0/584550617 shutdown_connections 2024-03-13T15:04:16.527 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.542+0000 7fbc15200640 1 -- 172.21.15.176:0/584550617 wait complete. 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.542+0000 7fbc15200640 1 Processor -- start 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc15200640 1 -- start start 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc15200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc101024c0 0x7fbc10198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc15200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10198560 0x7fbc1019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc15200640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10198b60 con 0x7fbc10198560 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc15200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10198cd0 con 0x7fbc101024c0 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc05400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10198560 0x7fbc1019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc05400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10198560 0x7fbc1019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35596/0 (socket says 172.21.15.176:35596) 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.543+0000 7fbc05400640 1 -- 172.21.15.176:0/3836221469 learned_addr learned my addr 172.21.15.176:0/3836221469 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:16.529 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc0de00640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc101024c0 0x7fbc10198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc05400640 1 -- 172.21.15.176:0/3836221469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc101024c0 msgr2=0x7fbc10198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc05400640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc101024c0 0x7fbc10198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc05400640 1 -- 172.21.15.176:0/3836221469 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8009560 con 0x7fbc10198560 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc0de00640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc101024c0 0x7fbc10198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc05400640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10198560 0x7fbc1019c9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8036040 tx=0x7fbbf802c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc07400640 1 -- 172.21.15.176:0/3836221469 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbf8004520 con 0x7fbc10198560 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc07400640 1 -- 172.21.15.176:0/3836221469 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbf8003ab0 con 0x7fbc10198560 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc07400640 1 -- 172.21.15.176:0/3836221469 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbf8034440 con 0x7fbc10198560 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.544+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1019cf60 con 0x7fbc10198560 2024-03-13T15:04:16.530 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.545+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1019d4c0 con 0x7fbc10198560 2024-03-13T15:04:16.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.546+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1004e040 con 0x7fbc10198560 2024-03-13T15:04:16.531 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.546+0000 7fbc07400640 1 -- 172.21.15.176:0/3836221469 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fbbf8003c20 con 0x7fbc10198560 2024-03-13T15:04:16.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.548+0000 7fbc07400640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbbf406df10 0x7fbbf40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:16.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.548+0000 7fbc07400640 1 -- 172.21.15.176:0/3836221469 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fbbf80b0260 con 0x7fbc10198560 2024-03-13T15:04:16.532 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.548+0000 7fbc0de00640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbbf406df10 0x7fbbf40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:16.533 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.549+0000 7fbc0de00640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbbf406df10 0x7fbbf40703d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009b40 tx=0x7fbc00005e50 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:16.537 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.553+0000 7fbc07400640 1 -- 172.21.15.176:0/3836221469 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fbbf8079510 con 0x7fbc10198560 2024-03-13T15:04:16.775 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.791+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fbc1019d730 con 0x7fbc10198560 2024-03-13T15:04:16.777 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.792+0000 7fbc07400640 1 -- 172.21.15.176:0/3836221469 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v39) v1 ==== 107+0+4084 (secure 0 0 0) 0x7fbbf807cbd0 con 0x7fbc10198560 2024-03-13T15:04:16.777 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:16.777 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T14:59:47.444191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi196.tcitrk","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6825/152396394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":152396394},{"type":"v1","addr":"172.21.15.196:6825","nonce":152396394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:16.777 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 23 2024-03-13T15:04:16.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.795+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbbf406df10 msgr2=0x7fbbf40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.795+0000 7fbc15200640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbbf406df10 0x7fbbf40703d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009b40 tx=0x7fbc00005e50 comp rx=0 tx=0).stop 2024-03-13T15:04:16.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.795+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10198560 msgr2=0x7fbc1019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:16.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.795+0000 7fbc15200640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10198560 0x7fbc1019c9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8036040 tx=0x7fbbf802c7a0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.795+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 shutdown_connections 2024-03-13T15:04:16.780 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.795+0000 7fbc15200640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fbc10198560 0x7fbc1019c9c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.796+0000 7fbc15200640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fbbf406df10 0x7fbbf40703d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.796+0000 7fbc15200640 1 --2- 172.21.15.176:0/3836221469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc101024c0 0x7fbc10198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:16.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.796+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 >> 172.21.15.176:0/3836221469 conn(0x7fbc100fdd40 msgr2=0x7fbc100ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:16.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.796+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 shutdown_connections 2024-03-13T15:04:16.781 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:16.796+0000 7fbc15200640 1 -- 172.21.15.176:0/3836221469 wait complete. 2024-03-13T15:04:16.799 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-03-13T15:04:16.800 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 -- 172.21.15.176:0/872186049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 msgr2=0x7f6a3c10ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 --2- 172.21.15.176:0/872186049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c10ca80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009880 tx=0x7f6a2c02bd90 comp rx=0 tx=0).stop 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 -- 172.21.15.176:0/872186049 shutdown_connections 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 --2- 172.21.15.176:0/872186049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c1066a0 0x7f6a3c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 --2- 172.21.15.176:0/872186049 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c10ca80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 -- 172.21.15.176:0/872186049 >> 172.21.15.176:0/872186049 conn(0x7f6a3c075e50 msgr2=0x7f6a3c078270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 -- 172.21.15.176:0/872186049 shutdown_connections 2024-03-13T15:04:17.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.026+0000 7f6a40a00640 1 -- 172.21.15.176:0/872186049 wait complete. 2024-03-13T15:04:17.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.027+0000 7f6a40a00640 1 Processor -- start 2024-03-13T15:04:17.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.027+0000 7f6a40a00640 1 -- start start 2024-03-13T15:04:17.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a40a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c1066a0 0x7f6a3c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a40a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a40a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c19cde0 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a40a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1a0bb0 con 0x7f6a3c1066a0 2024-03-13T15:04:17.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a31400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a31400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c19c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35616/0 (socket says 172.21.15.176:35616) 2024-03-13T15:04:17.013 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a31400640 1 -- 172.21.15.176:0/600051562 learned_addr learned my addr 172.21.15.176:0/600051562 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a31400640 1 -- 172.21.15.176:0/600051562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c1066a0 msgr2=0x7f6a3c19c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.028+0000 7f6a31400640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c1066a0 0x7f6a3c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.029+0000 7f6a31400640 1 -- 172.21.15.176:0/600051562 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c009560 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.029+0000 7f6a31400640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c19c750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400d6f0 tx=0x7f6a2400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.029+0000 7f6a33400640 1 -- 172.21.15.176:0/600051562 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a24009a30 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.029+0000 7f6a33400640 1 -- 172.21.15.176:0/600051562 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a240054e0 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.029+0000 7f6a33400640 1 -- 172.21.15.176:0/600051562 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a24010460 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.029+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c1a0e90 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.029+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c1a13f0 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.030+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c04e040 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.015 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.031+0000 7f6a33400640 1 -- 172.21.15.176:0/600051562 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6a24009b90 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.016 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.032+0000 7f6a33400640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a2006dfd0 0x7f6a20070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.032+0000 7f6a33400640 1 -- 172.21.15.176:0/600051562 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f6a2408d920 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.032+0000 7f6a39e00640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a2006dfd0 0x7f6a20070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.017 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.033+0000 7f6a39e00640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a2006dfd0 0x7f6a20070490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c036040 tx=0x7f6a2c000bc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:17.021 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.037+0000 7f6a33400640 1 -- 172.21.15.176:0/600051562 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f6a24056cf0 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:16 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3043965787' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-13T15:04:17.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:16 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1628558346' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-13T15:04:17.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:16 smithi176 ceph-mon[112742]: pgmap v373: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:17.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:16 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3043965787' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-03-13T15:04:17.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:16 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1628558346' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-03-13T15:04:17.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:16 smithi196 ceph-mon[86193]: pgmap v373: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:17.258 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.274+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f6a3c19d560 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.259 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.275+0000 7f6a33400640 1 -- 172.21.15.176:0/600051562 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v39) v1 ==== 107+0+4897 (secure 0 0 0) 0x7f6a2405a3b0 con 0x7f6a3c10c6a0 2024-03-13T15:04:17.260 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:17.260 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T14:59:47.444191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24249},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24249":{"gid":24249,"name":"cephfs.smithi196.tcitrk","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.196:6825/152396394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":152396394},{"type":"v1","addr":"172.21.15.196:6825","nonce":152396394}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:17.260 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 24 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a2006dfd0 msgr2=0x7f6a20070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a2006dfd0 0x7f6a20070490 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c036040 tx=0x7f6a2c000bc0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 msgr2=0x7f6a3c19c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c19c750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400d6f0 tx=0x7f6a2400dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 shutdown_connections 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a3c10c6a0 0x7f6a3c19c750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a2006dfd0 0x7f6a20070490 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 --2- 172.21.15.176:0/600051562 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c1066a0 0x7f6a3c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.263 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.278+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 >> 172.21.15.176:0/600051562 conn(0x7f6a3c075e50 msgr2=0x7f6a3c0777a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:17.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.279+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 shutdown_connections 2024-03-13T15:04:17.264 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.279+0000 7f6a40a00640 1 -- 172.21.15.176:0/600051562 wait complete. 2024-03-13T15:04:17.280 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-03-13T15:04:17.280 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 -- 172.21.15.176:0/1011781889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c0102e70 msgr2=0x7f68c0106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 --2- 172.21.15.176:0/1011781889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c0102e70 0x7f68c0106eb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68ac009880 tx=0x7f68ac02bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 -- 172.21.15.176:0/1011781889 shutdown_connections 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 --2- 172.21.15.176:0/1011781889 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c0102e70 0x7f68c0106eb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 --2- 172.21.15.176:0/1011781889 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68c01024c0 0x7f68c01028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 -- 172.21.15.176:0/1011781889 >> 172.21.15.176:0/1011781889 conn(0x7f68c00fdd40 msgr2=0x7f68c0100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 -- 172.21.15.176:0/1011781889 shutdown_connections 2024-03-13T15:04:17.485 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.500+0000 7f68c8200640 1 -- 172.21.15.176:0/1011781889 wait complete. 2024-03-13T15:04:17.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.501+0000 7f68c8200640 1 Processor -- start 2024-03-13T15:04:17.486 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c8200640 1 -- start start 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c8200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c01024c0 0x7f68c0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c8200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68c0198490 0x7f68c019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c8200640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0198a90 con 0x7f68c01024c0 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c8200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0198c00 con 0x7f68c0198490 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c5800640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c01024c0 0x7f68c0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c5800640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c01024c0 0x7f68c0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35638/0 (socket says 172.21.15.176:35638) 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.502+0000 7f68c5800640 1 -- 172.21.15.176:0/1921706912 learned_addr learned my addr 172.21.15.176:0/1921706912 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68c5800640 1 -- 172.21.15.176:0/1921706912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68c0198490 msgr2=0x7f68c019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68c4e00640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68c0198490 0x7f68c019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68c5800640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68c0198490 0x7f68c019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68c5800640 1 -- 172.21.15.176:0/1921706912 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac009560 con 0x7f68c01024c0 2024-03-13T15:04:17.488 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68c4e00640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68c0198490 0x7f68c019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:17.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68c5800640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c01024c0 0x7f68c0197f50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f68b000ea60 tx=0x7f68b000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:17.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68be000640 1 -- 172.21.15.176:0/1921706912 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b000cdb0 con 0x7f68c01024c0 2024-03-13T15:04:17.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68be000640 1 -- 172.21.15.176:0/1921706912 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b0011930 con 0x7f68c01024c0 2024-03-13T15:04:17.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68be000640 1 -- 172.21.15.176:0/1921706912 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b001aa30 con 0x7f68c01024c0 2024-03-13T15:04:17.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.503+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c019cef0 con 0x7f68c01024c0 2024-03-13T15:04:17.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.504+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c019d450 con 0x7f68c01024c0 2024-03-13T15:04:17.489 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.505+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c019d720 con 0x7f68c01024c0 2024-03-13T15:04:17.490 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.506+0000 7f68be000640 1 -- 172.21.15.176:0/1921706912 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f68b001abf0 con 0x7f68c01024c0 2024-03-13T15:04:17.491 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.507+0000 7f68be000640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68a406df10 0x7f68a40703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.491 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.507+0000 7f68be000640 1 -- 172.21.15.176:0/1921706912 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f68b008e510 con 0x7f68c01024c0 2024-03-13T15:04:17.491 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.507+0000 7f68c4e00640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68a406df10 0x7f68a40703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.492 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.508+0000 7f68c4e00640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68a406df10 0x7f68a40703d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68ac036040 tx=0x7f68ac005fd0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:17.496 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.512+0000 7f68be000640 1 -- 172.21.15.176:0/1921706912 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f68b00578e0 con 0x7f68c01024c0 2024-03-13T15:04:17.725 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.741+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f68c01992d0 con 0x7f68c01024c0 2024-03-13T15:04:17.726 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.742+0000 7f68be000640 1 -- 172.21.15.176:0/1921706912 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v39) v1 ==== 107+0+4109 (secure 0 0 0) 0x7f68b005afa0 con 0x7f68c01024c0 2024-03-13T15:04:17.726 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:17.726 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:00:43.199531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:17.727 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 25 2024-03-13T15:04:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68a406df10 msgr2=0x7f68a40703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68a406df10 0x7f68a40703d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68ac036040 tx=0x7f68ac005fd0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c01024c0 msgr2=0x7f68c0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.729 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c01024c0 0x7f68c0197f50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f68b000ea60 tx=0x7f68b000ef20 comp rx=0 tx=0).stop 2024-03-13T15:04:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 shutdown_connections 2024-03-13T15:04:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f68a406df10 0x7f68a40703d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68c0198490 0x7f68c019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 --2- 172.21.15.176:0/1921706912 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f68c01024c0 0x7f68c0197f50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 >> 172.21.15.176:0/1921706912 conn(0x7f68c00fdd40 msgr2=0x7f68c00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 shutdown_connections 2024-03-13T15:04:17.730 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.745+0000 7f68c8200640 1 -- 172.21.15.176:0/1921706912 wait complete. 2024-03-13T15:04:17.747 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-03-13T15:04:17.747 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-03-13T15:04:17.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.971+0000 7f3d7b000640 1 -- 172.21.15.176:0/2382381388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d741022d0 msgr2=0x7f3d741026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.971+0000 7f3d7b000640 1 --2- 172.21.15.176:0/2382381388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d741022d0 0x7f3d741026b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009880 tx=0x7f3d6802bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.971+0000 7f3d7b000640 1 -- 172.21.15.176:0/2382381388 shutdown_connections 2024-03-13T15:04:17.956 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.972+0000 7f3d7b000640 1 --2- 172.21.15.176:0/2382381388 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d74102c80 0x7f3d74106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.972+0000 7f3d7b000640 1 --2- 172.21.15.176:0/2382381388 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d741022d0 0x7f3d741026b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.972+0000 7f3d7b000640 1 -- 172.21.15.176:0/2382381388 >> 172.21.15.176:0/2382381388 conn(0x7f3d740fdb50 msgr2=0x7f3d740fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:17.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.972+0000 7f3d7b000640 1 -- 172.21.15.176:0/2382381388 shutdown_connections 2024-03-13T15:04:17.957 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.972+0000 7f3d7b000640 1 -- 172.21.15.176:0/2382381388 wait complete. 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d7b000640 1 Processor -- start 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d7b000640 1 -- start start 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d7b000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d741022d0 0x7f3d74197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d7b000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d74198320 0x7f3d7419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d7b000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74198920 con 0x7f3d74198320 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d7b000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74198a90 con 0x7f3d741022d0 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d79200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d74198320 0x7f3d7419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d79200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d74198320 0x7f3d7419c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35664/0 (socket says 172.21.15.176:35664) 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.973+0000 7f3d79200640 1 -- 172.21.15.176:0/1487596202 learned_addr learned my addr 172.21.15.176:0/1487596202 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d79c00640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d741022d0 0x7f3d74197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.959 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d79200640 1 -- 172.21.15.176:0/1487596202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d741022d0 msgr2=0x7f3d74197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d79200640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d741022d0 0x7f3d74197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d79200640 1 -- 172.21.15.176:0/1487596202 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68009560 con 0x7f3d74198320 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d79c00640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d741022d0 0x7f3d74197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d79200640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d74198320 0x7f3d7419c780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000d6f0 tx=0x7f3d6000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d72000640 1 -- 172.21.15.176:0/1487596202 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d600051f0 con 0x7f3d74198320 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d72000640 1 -- 172.21.15.176:0/1487596202 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d60009ce0 con 0x7f3d74198320 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d72000640 1 -- 172.21.15.176:0/1487596202 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d60004e80 con 0x7f3d74198320 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7419cd80 con 0x7f3d74198320 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.974+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7419d2e0 con 0x7f3d74198320 2024-03-13T15:04:17.960 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.975+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7419d5b0 con 0x7f3d74198320 2024-03-13T15:04:17.961 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.976+0000 7f3d72000640 1 -- 172.21.15.176:0/1487596202 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f3d600046c0 con 0x7f3d74198320 2024-03-13T15:04:17.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.978+0000 7f3d72000640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3d5c06df60 0x7f3d5c070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:17.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.978+0000 7f3d72000640 1 -- 172.21.15.176:0/1487596202 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f3d6008d890 con 0x7f3d74198320 2024-03-13T15:04:17.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.978+0000 7f3d79c00640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3d5c06df60 0x7f3d5c070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:17.963 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.979+0000 7f3d79c00640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3d5c06df60 0x7f3d5c070420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d68005ab0 tx=0x7f3d6802cf90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:17.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:17.982+0000 7f3d72000640 1 -- 172.21.15.176:0/1487596202 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f3d60056bc0 con 0x7f3d74198320 2024-03-13T15:04:18.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:17 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3836221469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-13T15:04:18.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:17 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/600051562' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-13T15:04:18.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:17 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3836221469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-03-13T15:04:18.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:17 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/600051562' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-03-13T15:04:18.203 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.218+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f3d74199190 con 0x7f3d74198320 2024-03-13T15:04:18.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.220+0000 7f3d72000640 1 -- 172.21.15.176:0/1487596202 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v39) v1 ==== 107+0+4120 (secure 0 0 0) 0x7f3d6005a280 con 0x7f3d74198320 2024-03-13T15:04:18.204 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:18.205 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:00:43.212732+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:18.205 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 26 2024-03-13T15:04:18.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.222+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3d5c06df60 msgr2=0x7f3d5c070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.222+0000 7f3d7b000640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3d5c06df60 0x7f3d5c070420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d68005ab0 tx=0x7f3d6802cf90 comp rx=0 tx=0).stop 2024-03-13T15:04:18.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d74198320 msgr2=0x7f3d7419c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d74198320 0x7f3d7419c780 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000d6f0 tx=0x7f3d6000dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 shutdown_connections 2024-03-13T15:04:18.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3d74198320 0x7f3d7419c780 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f3d5c06df60 0x7f3d5c070420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 --2- 172.21.15.176:0/1487596202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d741022d0 0x7f3d74197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 >> 172.21.15.176:0/1487596202 conn(0x7f3d740fdb50 msgr2=0x7f3d740ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:18.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 shutdown_connections 2024-03-13T15:04:18.208 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.223+0000 7f3d7b000640 1 -- 172.21.15.176:0/1487596202 wait complete. 2024-03-13T15:04:18.227 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-03-13T15:04:18.227 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 -- 172.21.15.176:0/1468579498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 msgr2=0x7f0e80102880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 --2- 172.21.15.176:0/1468579498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80102880 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0e74009c20 tx=0x7f0e7402c1b0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 -- 172.21.15.176:0/1468579498 shutdown_connections 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 --2- 172.21.15.176:0/1468579498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e80102e50 0x7f0e80106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 --2- 172.21.15.176:0/1468579498 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80102880 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 -- 172.21.15.176:0/1468579498 >> 172.21.15.176:0/1468579498 conn(0x7f0e800fdd00 msgr2=0x7f0e80100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 -- 172.21.15.176:0/1468579498 shutdown_connections 2024-03-13T15:04:18.436 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.451+0000 7f0e88600640 1 -- 172.21.15.176:0/1468579498 wait complete. 2024-03-13T15:04:18.438 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.452+0000 7f0e88600640 1 Processor -- start 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e88600640 1 -- start start 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e88600640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e88600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e801985a0 0x7f0e8019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e88600640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80198b10 con 0x7f0e801024a0 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e88600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80198c80 con 0x7f0e801985a0 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e85c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e85c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35676/0 (socket says 172.21.15.176:35676) 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.453+0000 7f0e85c00640 1 -- 172.21.15.176:0/2671683365 learned_addr learned my addr 172.21.15.176:0/2671683365 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e85200640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e801985a0 0x7f0e8019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e85c00640 1 -- 172.21.15.176:0/2671683365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e801985a0 msgr2=0x7f0e8019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e85c00640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e801985a0 0x7f0e8019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e85c00640 1 -- 172.21.15.176:0/2671683365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74009900 con 0x7f0e801024a0 2024-03-13T15:04:18.439 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e85200640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e801985a0 0x7f0e8019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:18.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e85c00640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80198060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0e74037040 tx=0x7f0e74032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:18.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e7e000640 1 -- 172.21.15.176:0/2671683365 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e74034820 con 0x7f0e801024a0 2024-03-13T15:04:18.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e7e000640 1 -- 172.21.15.176:0/2671683365 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e74003e30 con 0x7f0e801024a0 2024-03-13T15:04:18.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e7e000640 1 -- 172.21.15.176:0/2671683365 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e7403f440 con 0x7f0e801024a0 2024-03-13T15:04:18.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.454+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8019cf30 con 0x7f0e801024a0 2024-03-13T15:04:18.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.455+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8019d440 con 0x7f0e801024a0 2024-03-13T15:04:18.440 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.456+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8004e040 con 0x7f0e801024a0 2024-03-13T15:04:18.441 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.457+0000 7f0e7e000640 1 -- 172.21.15.176:0/2671683365 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f0e7403b040 con 0x7f0e801024a0 2024-03-13T15:04:18.442 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.458+0000 7f0e7e000640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e6806df10 0x7f0e680703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:18.442 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.458+0000 7f0e7e000640 1 -- 172.21.15.176:0/2671683365 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f0e740b0ed0 con 0x7f0e801024a0 2024-03-13T15:04:18.442 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.458+0000 7f0e85200640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e6806df10 0x7f0e680703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:18.443 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.459+0000 7f0e85200640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e6806df10 0x7f0e680703d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e80069580 tx=0x7f0e6c005d90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:18.446 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.462+0000 7f0e7e000640 1 -- 172.21.15.176:0/2671683365 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f0e7407a2c0 con 0x7f0e801024a0 2024-03-13T15:04:18.689 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.705+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f0e8019d6b0 con 0x7f0e801024a0 2024-03-13T15:04:18.690 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.706+0000 7f0e7e000640 1 -- 172.21.15.176:0/2671683365 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v39) v1 ==== 107+0+4936 (secure 0 0 0) 0x7f0e7407d980 con 0x7f0e801024a0 2024-03-13T15:04:18.691 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:18.691 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:00:43.212732+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:18.691 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 27 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e6806df10 msgr2=0x7f0e680703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e6806df10 0x7f0e680703d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0e80069580 tx=0x7f0e6c005d90 comp rx=0 tx=0).stop 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 msgr2=0x7f0e80198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80198060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0e74037040 tx=0x7f0e74032b60 comp rx=0 tx=0).stop 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 shutdown_connections 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f0e6806df10 0x7f0e680703d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e801985a0 0x7f0e8019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 --2- 172.21.15.176:0/2671683365 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f0e801024a0 0x7f0e80198060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.694 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 >> 172.21.15.176:0/2671683365 conn(0x7f0e800fdd00 msgr2=0x7f0e800ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:18.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.709+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 shutdown_connections 2024-03-13T15:04:18.695 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.710+0000 7f0e88600640 1 -- 172.21.15.176:0/2671683365 wait complete. 2024-03-13T15:04:18.717 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-03-13T15:04:18.717 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-03-13T15:04:18.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:18 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1921706912' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-13T15:04:18.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:18 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1487596202' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-13T15:04:18.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:18 smithi176 ceph-mon[112742]: pgmap v374: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 -- 172.21.15.176:0/2538149430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072f60 msgr2=0x7f27e4110950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 --2- 172.21.15.176:0/2538149430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072f60 0x7f27e4110950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009880 tx=0x7f27cc02bd90 comp rx=0 tx=0).stop 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 -- 172.21.15.176:0/2538149430 shutdown_connections 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 --2- 172.21.15.176:0/2538149430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072f60 0x7f27e4110950 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 --2- 172.21.15.176:0/2538149430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e4072640 0x7f27e4072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 -- 172.21.15.176:0/2538149430 >> 172.21.15.176:0/2538149430 conn(0x7f27e40fb970 msgr2=0x7f27e40fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 -- 172.21.15.176:0/2538149430 shutdown_connections 2024-03-13T15:04:18.923 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.938+0000 7f27e3200640 1 -- 172.21.15.176:0/2538149430 wait complete. 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.939+0000 7f27e3200640 1 Processor -- start 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.939+0000 7f27e3200640 1 -- start start 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.939+0000 7f27e3200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072640 0x7f27e419c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.939+0000 7f27e3200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e4072f60 0x7f27e419c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.939+0000 7f27e3200640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419cd10 con 0x7f27e4072640 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.939+0000 7f27e3200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41a0ae0 con 0x7f27e4072f60 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.940+0000 7f27e1e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072640 0x7f27e419c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:18.925 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.940+0000 7f27e1e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072640 0x7f27e419c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35708/0 (socket says 172.21.15.176:35708) 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.940+0000 7f27e1e00640 1 -- 172.21.15.176:0/1170491724 learned_addr learned my addr 172.21.15.176:0/1170491724 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.940+0000 7f27e1e00640 1 -- 172.21.15.176:0/1170491724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e4072f60 msgr2=0x7f27e419c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.940+0000 7f27e1e00640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e4072f60 0x7f27e419c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.940+0000 7f27e1e00640 1 -- 172.21.15.176:0/1170491724 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc009560 con 0x7f27e4072640 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.940+0000 7f27e1e00640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072640 0x7f27e419c140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f27d400d6f0 tx=0x7f27d400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.941+0000 7f27db400640 1 -- 172.21.15.176:0/1170491724 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d4009a30 con 0x7f27e4072640 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.941+0000 7f27db400640 1 -- 172.21.15.176:0/1170491724 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f27d40054e0 con 0x7f27e4072640 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.941+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e41a0dc0 con 0x7f27e4072640 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.941+0000 7f27db400640 1 -- 172.21.15.176:0/1170491724 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d4010460 con 0x7f27e4072640 2024-03-13T15:04:18.926 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.941+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e41a1320 con 0x7f27e4072640 2024-03-13T15:04:18.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.942+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e404e040 con 0x7f27e4072640 2024-03-13T15:04:18.927 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.943+0000 7f27db400640 1 -- 172.21.15.176:0/1170491724 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f27d4009b90 con 0x7f27e4072640 2024-03-13T15:04:18.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.944+0000 7f27db400640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27c806df10 0x7f27c80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:18.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.944+0000 7f27db400640 1 -- 172.21.15.176:0/1170491724 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f27d408d9c0 con 0x7f27e4072640 2024-03-13T15:04:18.928 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.944+0000 7f27d9400640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27c806df10 0x7f27c80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:18.929 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.945+0000 7f27d9400640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27c806df10 0x7f27c80703d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f27cc036040 tx=0x7f27cc005d50 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:18.932 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:18.948+0000 7f27db400640 1 -- 172.21.15.176:0/1170491724 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f27d4056cf0 con 0x7f27e4072640 2024-03-13T15:04:19.174 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.190+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f27e410e040 con 0x7f27e4072640 2024-03-13T15:04:19.175 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.191+0000 7f27db400640 1 -- 172.21.15.176:0/1170491724 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v39) v1 ==== 107+0+4941 (secure 0 0 0) 0x7f27d405a3b0 con 0x7f27e4072640 2024-03-13T15:04:19.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:18 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1921706912' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-03-13T15:04:19.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:18 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1487596202' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-03-13T15:04:19.176 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:18 smithi196 ceph-mon[86193]: pgmap v374: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:19.176 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:19.176 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:00:52.403117+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":226,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:19.176 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 28 2024-03-13T15:04:19.178 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.193+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27c806df10 msgr2=0x7f27c80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.178 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.193+0000 7f27e3200640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27c806df10 0x7f27c80703d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f27cc036040 tx=0x7f27cc005d50 comp rx=0 tx=0).stop 2024-03-13T15:04:19.178 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.193+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072640 msgr2=0x7f27e419c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.178 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.193+0000 7f27e3200640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072640 0x7f27e419c140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f27d400d6f0 tx=0x7f27d400dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.179 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.194+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 shutdown_connections 2024-03-13T15:04:19.179 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.194+0000 7f27e3200640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f27c806df10 0x7f27c80703d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.179 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.194+0000 7f27e3200640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e4072f60 0x7f27e419c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.179 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.194+0000 7f27e3200640 1 --2- 172.21.15.176:0/1170491724 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f27e4072640 0x7f27e419c140 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.179 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.194+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 >> 172.21.15.176:0/1170491724 conn(0x7f27e40fb970 msgr2=0x7f27e4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:19.179 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.194+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 shutdown_connections 2024-03-13T15:04:19.179 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.194+0000 7f27e3200640 1 -- 172.21.15.176:0/1170491724 wait complete. 2024-03-13T15:04:19.197 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-03-13T15:04:19.198 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-03-13T15:04:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 -- 172.21.15.176:0/824870065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc102e70 msgr2=0x7fc7bc106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 --2- 172.21.15.176:0/824870065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc102e70 0x7fc7bc106eb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009880 tx=0x7fc7a402bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 -- 172.21.15.176:0/824870065 shutdown_connections 2024-03-13T15:04:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 --2- 172.21.15.176:0/824870065 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc102e70 0x7fc7bc106eb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 --2- 172.21.15.176:0/824870065 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7bc1024c0 0x7fc7bc1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.411 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 -- 172.21.15.176:0/824870065 >> 172.21.15.176:0/824870065 conn(0x7fc7bc0fdd40 msgr2=0x7fc7bc100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:19.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 -- 172.21.15.176:0/824870065 shutdown_connections 2024-03-13T15:04:19.412 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.426+0000 7fc7c1800640 1 -- 172.21.15.176:0/824870065 wait complete. 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.427+0000 7fc7c1800640 1 Processor -- start 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.427+0000 7fc7c1800640 1 -- start start 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7c1800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7bc1024c0 0x7fc7bc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7c1800640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc1984b0 0x7fc7bc19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7c1800640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc198ab0 con 0x7fc7bc1984b0 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7c1800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc198c20 con 0x7fc7bc1024c0 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7b2000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc1984b0 0x7fc7bc19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7b2000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc1984b0 0x7fc7bc19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35712/0 (socket says 172.21.15.176:35712) 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7b2000640 1 -- 172.21.15.176:0/3601995337 learned_addr learned my addr 172.21.15.176:0/3601995337 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7baa00640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7bc1024c0 0x7fc7bc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7b2000640 1 -- 172.21.15.176:0/3601995337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7bc1024c0 msgr2=0x7fc7bc197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.414 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7b2000640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7bc1024c0 0x7fc7bc197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.428+0000 7fc7b2000640 1 -- 172.21.15.176:0/3601995337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4009560 con 0x7fc7bc1984b0 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.429+0000 7fc7baa00640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7bc1024c0 0x7fc7bc197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.429+0000 7fc7b2000640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc1984b0 0x7fc7bc19c910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4036040 tx=0x7fc7a402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.429+0000 7fc7b3e00640 1 -- 172.21.15.176:0/3601995337 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a40035f0 con 0x7fc7bc1984b0 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.429+0000 7fc7b3e00640 1 -- 172.21.15.176:0/3601995337 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7a4003c10 con 0x7fc7bc1984b0 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.429+0000 7fc7b3e00640 1 -- 172.21.15.176:0/3601995337 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a4034440 con 0x7fc7bc1984b0 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.429+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc19ceb0 con 0x7fc7bc1984b0 2024-03-13T15:04:19.415 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.429+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc19d410 con 0x7fc7bc1984b0 2024-03-13T15:04:19.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.430+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc04e040 con 0x7fc7bc1984b0 2024-03-13T15:04:19.416 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.431+0000 7fc7b3e00640 1 -- 172.21.15.176:0/3601995337 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fc7a4003750 con 0x7fc7bc1984b0 2024-03-13T15:04:19.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.433+0000 7fc7b3e00640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc7a006df10 0x7fc7a00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:19.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.433+0000 7fc7b3e00640 1 -- 172.21.15.176:0/3601995337 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fc7a40b0230 con 0x7fc7bc1984b0 2024-03-13T15:04:19.417 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.433+0000 7fc7baa00640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc7a006df10 0x7fc7a00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:19.418 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.433+0000 7fc7baa00640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc7a006df10 0x7fc7a00703d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005fd0 tx=0x7fc7ac0093e0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:19.421 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.437+0000 7fc7b3e00640 1 -- 172.21.15.176:0/3601995337 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fc7a4079560 con 0x7fc7bc1984b0 2024-03-13T15:04:19.662 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.678+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fc7bc19d680 con 0x7fc7bc1984b0 2024-03-13T15:04:19.663 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.679+0000 7fc7b3e00640 1 -- 172.21.15.176:0/3601995337 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v39) v1 ==== 107+0+4938 (secure 0 0 0) 0x7fc7a407cc20 con 0x7fc7bc1984b0 2024-03-13T15:04:19.663 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:19.664 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:00:52.689487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":227,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:19.664 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 29 2024-03-13T15:04:19.666 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc7a006df10 msgr2=0x7fc7a00703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.666 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc7a006df10 0x7fc7a00703d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac005fd0 tx=0x7fc7ac0093e0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc1984b0 msgr2=0x7fc7bc19c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc1984b0 0x7fc7bc19c910 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4036040 tx=0x7fc7a402c850 comp rx=0 tx=0).stop 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 shutdown_connections 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fc7bc1984b0 0x7fc7bc19c910 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fc7a006df10 0x7fc7a00703d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 --2- 172.21.15.176:0/3601995337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7bc1024c0 0x7fc7bc197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 >> 172.21.15.176:0/3601995337 conn(0x7fc7bc0fdd40 msgr2=0x7fc7bc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 shutdown_connections 2024-03-13T15:04:19.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.682+0000 7fc7c1800640 1 -- 172.21.15.176:0/3601995337 wait complete. 2024-03-13T15:04:19.686 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-03-13T15:04:19.686 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 -- 172.21.15.176:0/1424723611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34102e70 msgr2=0x7f1f34106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 --2- 172.21.15.176:0/1424723611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34102e70 0x7f1f34106eb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009880 tx=0x7f1f2002bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 -- 172.21.15.176:0/1424723611 shutdown_connections 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 --2- 172.21.15.176:0/1424723611 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34102e70 0x7f1f34106eb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 --2- 172.21.15.176:0/1424723611 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f341024c0 0x7f1f341028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 -- 172.21.15.176:0/1424723611 >> 172.21.15.176:0/1424723611 conn(0x7f1f340fdd40 msgr2=0x7f1f34100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 -- 172.21.15.176:0/1424723611 shutdown_connections 2024-03-13T15:04:19.889 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.904+0000 7f1f3a200640 1 -- 172.21.15.176:0/1424723611 wait complete. 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.905+0000 7f1f3a200640 1 Processor -- start 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.905+0000 7f1f3a200640 1 -- start start 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f3a200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f341024c0 0x7f1f34198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f3a200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34198540 0x7f1f3419c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f3a200640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34198ab0 con 0x7f1f34198540 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f3a200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34198c20 con 0x7f1f341024c0 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f32a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34198540 0x7f1f3419c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f32a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34198540 0x7f1f3419c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35724/0 (socket says 172.21.15.176:35724) 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f32a00640 1 -- 172.21.15.176:0/3425009333 learned_addr learned my addr 172.21.15.176:0/3425009333 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f33400640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f341024c0 0x7f1f34198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f32a00640 1 -- 172.21.15.176:0/3425009333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f341024c0 msgr2=0x7f1f34198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:19.891 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f32a00640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f341024c0 0x7f1f34198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:19.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f32a00640 1 -- 172.21.15.176:0/3425009333 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20009560 con 0x7f1f34198540 2024-03-13T15:04:19.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.906+0000 7f1f33400640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f341024c0 0x7f1f34198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:19.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.907+0000 7f1f32a00640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34198540 0x7f1f3419c9a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1f20036040 tx=0x7f1f20004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:19.892 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.907+0000 7f1f2be00640 1 -- 172.21.15.176:0/3425009333 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f200035f0 con 0x7f1f34198540 2024-03-13T15:04:19.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.907+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3419cf40 con 0x7f1f34198540 2024-03-13T15:04:19.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.907+0000 7f1f2be00640 1 -- 172.21.15.176:0/3425009333 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f20003c10 con 0x7f1f34198540 2024-03-13T15:04:19.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.907+0000 7f1f2be00640 1 -- 172.21.15.176:0/3425009333 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f20034440 con 0x7f1f34198540 2024-03-13T15:04:19.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.907+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3419d450 con 0x7f1f34198540 2024-03-13T15:04:19.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.908+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3404e040 con 0x7f1f34198540 2024-03-13T15:04:19.893 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.909+0000 7f1f2be00640 1 -- 172.21.15.176:0/3425009333 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f1f20003750 con 0x7f1f34198540 2024-03-13T15:04:19.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.910+0000 7f1f2be00640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f1f1806df10 0x7f1f180703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:19.894 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.910+0000 7f1f2be00640 1 -- 172.21.15.176:0/3425009333 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f1f200b03d0 con 0x7f1f34198540 2024-03-13T15:04:19.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.911+0000 7f1f33400640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f1f1806df10 0x7f1f180703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:19.895 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.911+0000 7f1f33400640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f1f1806df10 0x7f1f180703d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f24008e00 tx=0x7f1f24008350 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:19.900 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:19.916+0000 7f1f2be00640 1 -- 172.21.15.176:0/3425009333 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f1f20079680 con 0x7f1f34198540 2024-03-13T15:04:20.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:19 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2671683365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-13T15:04:20.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:19 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1170491724' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-13T15:04:20.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:19 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3601995337' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-13T15:04:20.138 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.153+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f1f34199350 con 0x7f1f34198540 2024-03-13T15:04:20.139 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.155+0000 7f1f2be00640 1 -- 172.21.15.176:0/3425009333 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v39) v1 ==== 107+0+4938 (secure 0 0 0) 0x7f1f2007cd40 con 0x7f1f34198540 2024-03-13T15:04:20.140 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:20.140 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:00:53.816937+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24255},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24255":{"gid":24255,"name":"cephfs.smithi196.wabhcu","rank":0,"incarnation":26,"state":"up:active","state_seq":228,"addr":"172.21.15.196:6827/1275358571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":1275358571},{"type":"v1","addr":"172.21.15.196:6827","nonce":1275358571}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:20.140 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 30 2024-03-13T15:04:20.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f1f1806df10 msgr2=0x7f1f180703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f1f1806df10 0x7f1f180703d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f24008e00 tx=0x7f1f24008350 comp rx=0 tx=0).stop 2024-03-13T15:04:20.142 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34198540 msgr2=0x7f1f3419c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34198540 0x7f1f3419c9a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1f20036040 tx=0x7f1f20004040 comp rx=0 tx=0).stop 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 shutdown_connections 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f1f34198540 0x7f1f3419c9a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f1f1806df10 0x7f1f180703d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 --2- 172.21.15.176:0/3425009333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f341024c0 0x7f1f34198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 >> 172.21.15.176:0/3425009333 conn(0x7f1f340fdd40 msgr2=0x7f1f340ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 shutdown_connections 2024-03-13T15:04:20.143 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.158+0000 7f1f3a200640 1 -- 172.21.15.176:0/3425009333 wait complete. 2024-03-13T15:04:20.161 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-03-13T15:04:20.161 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-03-13T15:04:20.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:19 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2671683365' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-03-13T15:04:20.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:19 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1170491724' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-03-13T15:04:20.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:19 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3601995337' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-03-13T15:04:20.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 -- 172.21.15.176:0/3241887386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a901066a0 msgr2=0x7f6a90106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3241887386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a901066a0 0x7f6a90106b20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009880 tx=0x7f6a7802bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 -- 172.21.15.176:0/3241887386 shutdown_connections 2024-03-13T15:04:20.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3241887386 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a901066a0 0x7f6a90106b20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3241887386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a9010c660 0x7f6a9010ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.375 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 -- 172.21.15.176:0/3241887386 >> 172.21.15.176:0/3241887386 conn(0x7f6a90075e80 msgr2=0x7f6a900782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:20.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 -- 172.21.15.176:0/3241887386 shutdown_connections 2024-03-13T15:04:20.376 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.390+0000 7f6a96c00640 1 -- 172.21.15.176:0/3241887386 wait complete. 2024-03-13T15:04:20.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.391+0000 7f6a96c00640 1 Processor -- start 2024-03-13T15:04:20.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a96c00640 1 -- start start 2024-03-13T15:04:20.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a96c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a901066a0 0x7f6a9019c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:20.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a96c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a9010c660 0x7f6a9019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:20.377 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a96c00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9019cee0 con 0x7f6a9010c660 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a96c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a901a0cb0 con 0x7f6a901066a0 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a87400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a9010c660 0x7f6a9019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a87400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a9010c660 0x7f6a9019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35734/0 (socket says 172.21.15.176:35734) 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.392+0000 7f6a87400640 1 -- 172.21.15.176:0/3712696215 learned_addr learned my addr 172.21.15.176:0/3712696215 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a8fe00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a901066a0 0x7f6a9019c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a87400640 1 -- 172.21.15.176:0/3712696215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a901066a0 msgr2=0x7f6a9019c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a87400640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a901066a0 0x7f6a9019c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a87400640 1 -- 172.21.15.176:0/3712696215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a78009560 con 0x7f6a9010c660 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a8fe00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a901066a0 0x7f6a9019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:20.378 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a87400640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a9010c660 0x7f6a9019c850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000c00 tx=0x7f6a7802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a8d600640 1 -- 172.21.15.176:0/3712696215 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a78004520 con 0x7f6a9010c660 2024-03-13T15:04:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a8d600640 1 -- 172.21.15.176:0/3712696215 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a78003ab0 con 0x7f6a9010c660 2024-03-13T15:04:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a901a0f30 con 0x7f6a9010c660 2024-03-13T15:04:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.393+0000 7f6a8d600640 1 -- 172.21.15.176:0/3712696215 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a78034440 con 0x7f6a9010c660 2024-03-13T15:04:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.394+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a901a1430 con 0x7f6a9010c660 2024-03-13T15:04:20.379 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.395+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9004e040 con 0x7f6a9010c660 2024-03-13T15:04:20.380 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.396+0000 7f6a8d600640 1 -- 172.21.15.176:0/3712696215 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f6a78003c20 con 0x7f6a9010c660 2024-03-13T15:04:20.381 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.397+0000 7f6a8d600640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a7406df10 0x7f6a740703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:20.381 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.397+0000 7f6a8d600640 1 -- 172.21.15.176:0/3712696215 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f6a780b0260 con 0x7f6a9010c660 2024-03-13T15:04:20.382 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.397+0000 7f6a8fe00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a7406df10 0x7f6a740703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:20.382 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.398+0000 7f6a8fe00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a7406df10 0x7f6a740703d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009b40 tx=0x7f6a80005e50 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:20.386 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.402+0000 7f6a8d600640 1 -- 172.21.15.176:0/3712696215 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f6a780795b0 con 0x7f6a9010c660 2024-03-13T15:04:20.629 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.645+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f6a9019d660 con 0x7f6a9010c660 2024-03-13T15:04:20.631 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.646+0000 7f6a8d600640 1 -- 172.21.15.176:0/3712696215 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v39) v1 ==== 107+0+4143 (secure 0 0 0) 0x7f6a7807cc70 con 0x7f6a9010c660 2024-03-13T15:04:20.631 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:20.631 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:08.961127+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:20.631 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 31 2024-03-13T15:04:20.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.649+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a7406df10 msgr2=0x7f6a740703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.649+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a7406df10 0x7f6a740703d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009b40 tx=0x7f6a80005e50 comp rx=0 tx=0).stop 2024-03-13T15:04:20.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.649+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a9010c660 msgr2=0x7f6a9019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.649+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a9010c660 0x7f6a9019c850 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6a78000c00 tx=0x7f6a7802c850 comp rx=0 tx=0).stop 2024-03-13T15:04:20.634 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.650+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 shutdown_connections 2024-03-13T15:04:20.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.650+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f6a9010c660 0x7f6a9019c850 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.650+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f6a7406df10 0x7f6a740703d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.650+0000 7f6a96c00640 1 --2- 172.21.15.176:0/3712696215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a901066a0 0x7f6a9019c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.650+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 >> 172.21.15.176:0/3712696215 conn(0x7f6a90075e80 msgr2=0x7f6a90076960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:20.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.650+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 shutdown_connections 2024-03-13T15:04:20.635 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.650+0000 7f6a96c00640 1 -- 172.21.15.176:0/3712696215 wait complete. 2024-03-13T15:04:20.652 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-03-13T15:04:20.652 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-03-13T15:04:20.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.876+0000 7f8076a00640 1 -- 172.21.15.176:0/3503934761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f807010c6a0 msgr2=0x7f807010ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.876+0000 7f8076a00640 1 --2- 172.21.15.176:0/3503934761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f807010c6a0 0x7f807010ca80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8060009880 tx=0x7f806002bd90 comp rx=0 tx=0).stop 2024-03-13T15:04:20.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.876+0000 7f8076a00640 1 -- 172.21.15.176:0/3503934761 shutdown_connections 2024-03-13T15:04:20.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.876+0000 7f8076a00640 1 --2- 172.21.15.176:0/3503934761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80701066a0 0x7f8070106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.861 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.876+0000 7f8076a00640 1 --2- 172.21.15.176:0/3503934761 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f807010c6a0 0x7f807010ca80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.877+0000 7f8076a00640 1 -- 172.21.15.176:0/3503934761 >> 172.21.15.176:0/3503934761 conn(0x7f8070075e50 msgr2=0x7f8070078270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:20.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.877+0000 7f8076a00640 1 -- 172.21.15.176:0/3503934761 shutdown_connections 2024-03-13T15:04:20.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.877+0000 7f8076a00640 1 -- 172.21.15.176:0/3503934761 wait complete. 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.877+0000 7f8076a00640 1 Processor -- start 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f8076a00640 1 -- start start 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f8076a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80701066a0 0x7f807019c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f8076a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807010c6a0 0x7f807019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f8076a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807019cdf0 con 0x7f80701066a0 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f8076a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701a0bc0 con 0x7f807010c6a0 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f806fe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80701066a0 0x7f807019c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f806fe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80701066a0 0x7f807019c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35758/0 (socket says 172.21.15.176:35758) 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.878+0000 7f806fe00640 1 -- 172.21.15.176:0/2176175025 learned_addr learned my addr 172.21.15.176:0/2176175025 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f8067e00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807010c6a0 0x7f807019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f806fe00640 1 -- 172.21.15.176:0/2176175025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807010c6a0 msgr2=0x7f807019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f806fe00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807010c6a0 0x7f807019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:20.864 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f806fe00640 1 -- 172.21.15.176:0/2176175025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009560 con 0x7f80701066a0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f8067e00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807010c6a0 0x7f807019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f806fe00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80701066a0 0x7f807019c220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8060036040 tx=0x7f8060004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f806d600640 1 -- 172.21.15.176:0/2176175025 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80600035f0 con 0x7f80701066a0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80701a0e40 con 0x7f80701066a0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f806d600640 1 -- 172.21.15.176:0/2176175025 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8060003c10 con 0x7f80701066a0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80701a13a0 con 0x7f80701066a0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.879+0000 7f806d600640 1 -- 172.21.15.176:0/2176175025 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f806003e450 con 0x7f80701066a0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.881+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807004e040 con 0x7f80701066a0 2024-03-13T15:04:20.865 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.881+0000 7f806d600640 1 -- 172.21.15.176:0/2176175025 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f8060004f60 con 0x7f80701066a0 2024-03-13T15:04:20.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.883+0000 7f806d600640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f805406dfd0 0x7f8054070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:20.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.883+0000 7f806d600640 1 -- 172.21.15.176:0/2176175025 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f80600b0b40 con 0x7f80701066a0 2024-03-13T15:04:20.867 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.883+0000 7f8067e00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f805406dfd0 0x7f8054070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:20.868 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.884+0000 7f8067e00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f805406dfd0 0x7f8054070490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f807019d830 tx=0x7f8058005d30 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:20.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:20.888+0000 7f806d600640 1 -- 172.21.15.176:0/2176175025 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f8060079df0 con 0x7f80701066a0 2024-03-13T15:04:21.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:20 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3425009333' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-13T15:04:21.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:20 smithi176 ceph-mon[112742]: pgmap v375: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:21.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:20 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3712696215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-13T15:04:21.128 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.143+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f80700008d0 con 0x7f80701066a0 2024-03-13T15:04:21.129 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.144+0000 7f806d600640 1 -- 172.21.15.176:0/2176175025 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v39) v1 ==== 107+0+4188 (secure 0 0 0) 0x7f806007d4b0 con 0x7f80701066a0 2024-03-13T15:04:21.129 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:21.129 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:08.974218+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:21.129 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 32 2024-03-13T15:04:21.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f805406dfd0 msgr2=0x7f8054070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f805406dfd0 0x7f8054070490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f807019d830 tx=0x7f8058005d30 comp rx=0 tx=0).stop 2024-03-13T15:04:21.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80701066a0 msgr2=0x7f807019c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80701066a0 0x7f807019c220 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8060036040 tx=0x7f8060004040 comp rx=0 tx=0).stop 2024-03-13T15:04:21.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 shutdown_connections 2024-03-13T15:04:21.132 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f805406dfd0 0x7f8054070490 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f807010c6a0 0x7f807019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 --2- 172.21.15.176:0/2176175025 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f80701066a0 0x7f807019c220 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.147+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 >> 172.21.15.176:0/2176175025 conn(0x7f8070075e50 msgr2=0x7f8070077820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:21.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.148+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 shutdown_connections 2024-03-13T15:04:21.133 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.148+0000 7f8076a00640 1 -- 172.21.15.176:0/2176175025 wait complete. 2024-03-13T15:04:21.149 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-03-13T15:04:21.149 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-03-13T15:04:21.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:20 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3425009333' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-03-13T15:04:21.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:20 smithi196 ceph-mon[86193]: pgmap v375: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:21.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:20 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3712696215' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-03-13T15:04:21.357 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 -- 172.21.15.176:0/1749798454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c102e70 msgr2=0x7efd9c106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 --2- 172.21.15.176:0/1749798454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c102e70 0x7efd9c106eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efd88009880 tx=0x7efd8802bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 -- 172.21.15.176:0/1749798454 shutdown_connections 2024-03-13T15:04:21.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 --2- 172.21.15.176:0/1749798454 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c102e70 0x7efd9c106eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 --2- 172.21.15.176:0/1749798454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd9c1024c0 0x7efd9c1028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 -- 172.21.15.176:0/1749798454 >> 172.21.15.176:0/1749798454 conn(0x7efd9c0fdce0 msgr2=0x7efd9c100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:21.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 -- 172.21.15.176:0/1749798454 shutdown_connections 2024-03-13T15:04:21.358 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.373+0000 7efda4000640 1 -- 172.21.15.176:0/1749798454 wait complete. 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.374+0000 7efda4000640 1 Processor -- start 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.374+0000 7efda4000640 1 -- start start 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.374+0000 7efda4000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd9c1024c0 0x7efd9c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.374+0000 7efda4000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1986c0 0x7efd9c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.374+0000 7efda4000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c198cc0 con 0x7efd9c1986c0 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.374+0000 7efda4000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c198e30 con 0x7efd9c1024c0 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda0c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1986c0 0x7efd9c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:21.360 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda0c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1986c0 0x7efd9c19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35782/0 (socket says 172.21.15.176:35782) 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda0c00640 1 -- 172.21.15.176:0/3936368495 learned_addr learned my addr 172.21.15.176:0/3936368495 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda1600640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd9c1024c0 0x7efd9c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda0c00640 1 -- 172.21.15.176:0/3936368495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd9c1024c0 msgr2=0x7efd9c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda0c00640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd9c1024c0 0x7efd9c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda0c00640 1 -- 172.21.15.176:0/3936368495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88009560 con 0x7efd9c1986c0 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.375+0000 7efda0c00640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1986c0 0x7efd9c19cb20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd88036040 tx=0x7efd88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.376+0000 7efd9a000640 1 -- 172.21.15.176:0/3936368495 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd880037c0 con 0x7efd9c1986c0 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.376+0000 7efd9a000640 1 -- 172.21.15.176:0/3936368495 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd88003de0 con 0x7efd9c1986c0 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.376+0000 7efd9a000640 1 -- 172.21.15.176:0/3936368495 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd8803e460 con 0x7efd9c1986c0 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.376+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c19d120 con 0x7efd9c1986c0 2024-03-13T15:04:21.361 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.376+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c19d680 con 0x7efd9c1986c0 2024-03-13T15:04:21.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.377+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c04e040 con 0x7efd9c1986c0 2024-03-13T15:04:21.362 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.378+0000 7efd9a000640 1 -- 172.21.15.176:0/3936368495 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7efd88004fd0 con 0x7efd9c1986c0 2024-03-13T15:04:21.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.379+0000 7efd9a000640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efd8406df10 0x7efd840703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:21.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.379+0000 7efd9a000640 1 -- 172.21.15.176:0/3936368495 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7efd880b0780 con 0x7efd9c1986c0 2024-03-13T15:04:21.363 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.379+0000 7efda1600640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efd8406df10 0x7efd840703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:21.364 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.380+0000 7efda1600640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efd8406df10 0x7efd840703d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd90009890 tx=0x7efd90008040 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:21.368 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.384+0000 7efd9a000640 1 -- 172.21.15.176:0/3936368495 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7efd88079ab0 con 0x7efd9c1986c0 2024-03-13T15:04:21.606 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.622+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7efd9c10a7c0 con 0x7efd9c1986c0 2024-03-13T15:04:21.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.623+0000 7efd9a000640 1 -- 172.21.15.176:0/3936368495 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v39) v1 ==== 107+0+4192 (secure 0 0 0) 0x7efd8807d170 con 0x7efd9c1986c0 2024-03-13T15:04:21.607 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:21.607 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:17.821882+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":22,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:21.608 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 33 2024-03-13T15:04:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.625+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efd8406df10 msgr2=0x7efd840703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.625+0000 7efda4000640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efd8406df10 0x7efd840703d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efd90009890 tx=0x7efd90008040 comp rx=0 tx=0).stop 2024-03-13T15:04:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.625+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1986c0 msgr2=0x7efd9c19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.610 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.625+0000 7efda4000640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1986c0 0x7efd9c19cb20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd88036040 tx=0x7efd88004270 comp rx=0 tx=0).stop 2024-03-13T15:04:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.626+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 shutdown_connections 2024-03-13T15:04:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.626+0000 7efda4000640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7efd9c1986c0 0x7efd9c19cb20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.626+0000 7efda4000640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efd8406df10 0x7efd840703d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.626+0000 7efda4000640 1 --2- 172.21.15.176:0/3936368495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd9c1024c0 0x7efd9c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.626+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 >> 172.21.15.176:0/3936368495 conn(0x7efd9c0fdce0 msgr2=0x7efd9c0ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.626+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 shutdown_connections 2024-03-13T15:04:21.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.626+0000 7efda4000640 1 -- 172.21.15.176:0/3936368495 wait complete. 2024-03-13T15:04:21.628 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-03-13T15:04:21.629 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-03-13T15:04:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.852+0000 7f3496000640 1 -- 172.21.15.176:0/3534250586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f349010b400 msgr2=0x7f349010b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.852+0000 7f3496000640 1 --2- 172.21.15.176:0/3534250586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f349010b400 0x7f349010b7e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f347c009880 tx=0x7f347c02bd70 comp rx=0 tx=0).stop 2024-03-13T15:04:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.852+0000 7f3496000640 1 -- 172.21.15.176:0/3534250586 shutdown_connections 2024-03-13T15:04:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.852+0000 7f3496000640 1 --2- 172.21.15.176:0/3534250586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3490106490 0x7f34901068f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.852+0000 7f3496000640 1 --2- 172.21.15.176:0/3534250586 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f349010b400 0x7f349010b7e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.837 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.852+0000 7f3496000640 1 -- 172.21.15.176:0/3534250586 >> 172.21.15.176:0/3534250586 conn(0x7f3490075c50 msgr2=0x7f3490078070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:21.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.853+0000 7f3496000640 1 -- 172.21.15.176:0/3534250586 shutdown_connections 2024-03-13T15:04:21.838 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.853+0000 7f3496000640 1 -- 172.21.15.176:0/3534250586 wait complete. 2024-03-13T15:04:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3496000640 1 Processor -- start 2024-03-13T15:04:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3496000640 1 -- start start 2024-03-13T15:04:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3496000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3490106490 0x7f349019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3496000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f349010b400 0x7f349019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3496000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349019cd20 con 0x7f3490106490 2024-03-13T15:04:21.839 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3496000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34901a0af0 con 0x7f349010b400 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3494c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3490106490 0x7f349019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3494c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3490106490 0x7f349019c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:35806/0 (socket says 172.21.15.176:35806) 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.854+0000 7f3494c00640 1 -- 172.21.15.176:0/470047790 learned_addr learned my addr 172.21.15.176:0/470047790 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f3494c00640 1 -- 172.21.15.176:0/470047790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f349010b400 msgr2=0x7f349019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f348fe00640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f349010b400 0x7f349019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f3494c00640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f349010b400 0x7f349019c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f3494c00640 1 -- 172.21.15.176:0/470047790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c009560 con 0x7f3490106490 2024-03-13T15:04:21.840 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f3494c00640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3490106490 0x7f349019c150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f347c009230 tx=0x7f347c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:21.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f348d600640 1 -- 172.21.15.176:0/470047790 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f347c0035f0 con 0x7f3490106490 2024-03-13T15:04:21.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34901a0d70 con 0x7f3490106490 2024-03-13T15:04:21.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f348d600640 1 -- 172.21.15.176:0/470047790 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f347c003c10 con 0x7f3490106490 2024-03-13T15:04:21.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.855+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34901a12d0 con 0x7f3490106490 2024-03-13T15:04:21.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.856+0000 7f348d600640 1 -- 172.21.15.176:0/470047790 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f347c005690 con 0x7f3490106490 2024-03-13T15:04:21.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.857+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349004e040 con 0x7f3490106490 2024-03-13T15:04:21.841 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.857+0000 7f348d600640 1 -- 172.21.15.176:0/470047790 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f347c004f60 con 0x7f3490106490 2024-03-13T15:04:21.842 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.858+0000 7f348d600640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f347806df10 0x7f34780703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:21.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.858+0000 7f348d600640 1 -- 172.21.15.176:0/470047790 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f347c0b0b10 con 0x7f3490106490 2024-03-13T15:04:21.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.859+0000 7f348fe00640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f347806df10 0x7f34780703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:21.843 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.859+0000 7f348fe00640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f347806df10 0x7f34780703d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3484005fd0 tx=0x7f3484009470 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:21.847 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:21.863+0000 7f348d600640 1 -- 172.21.15.176:0/470047790 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f347c079dc0 con 0x7f3490106490 2024-03-13T15:04:22.084 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.099+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f349010e940 con 0x7f3490106490 2024-03-13T15:04:22.084 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.100+0000 7f348d600640 1 -- 172.21.15.176:0/470047790 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v39) v1 ==== 107+0+4189 (secure 0 0 0) 0x7f347c07d480 con 0x7f3490106490 2024-03-13T15:04:22.085 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:22.085 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:18.546424+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":23,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:22.085 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 34 2024-03-13T15:04:22.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f347806df10 msgr2=0x7f34780703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f347806df10 0x7f34780703d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3484005fd0 tx=0x7f3484009470 comp rx=0 tx=0).stop 2024-03-13T15:04:22.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3490106490 msgr2=0x7f349019c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.087 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3490106490 0x7f349019c150 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f347c009230 tx=0x7f347c004040 comp rx=0 tx=0).stop 2024-03-13T15:04:22.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 shutdown_connections 2024-03-13T15:04:22.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f347806df10 0x7f34780703d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f349010b400 0x7f349019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 --2- 172.21.15.176:0/470047790 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f3490106490 0x7f349019c150 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 >> 172.21.15.176:0/470047790 conn(0x7f3490075c50 msgr2=0x7f3490076760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:22.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 shutdown_connections 2024-03-13T15:04:22.088 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.103+0000 7f3496000640 1 -- 172.21.15.176:0/470047790 wait complete. 2024-03-13T15:04:22.102 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:21 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2176175025' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-13T15:04:22.103 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:21 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3936368495' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-13T15:04:22.106 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-03-13T15:04:22.106 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-03-13T15:04:22.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:21 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2176175025' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-03-13T15:04:22.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:21 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3936368495' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-03-13T15:04:22.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.331+0000 7fa254c00640 1 -- 172.21.15.176:0/4222914993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 msgr2=0x7fa25010ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.331+0000 7fa254c00640 1 --2- 172.21.15.176:0/4222914993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25010ca40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa240009880 tx=0x7fa24002bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.332+0000 7fa254c00640 1 -- 172.21.15.176:0/4222914993 shutdown_connections 2024-03-13T15:04:22.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.332+0000 7fa254c00640 1 --2- 172.21.15.176:0/4222914993 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2501066a0 0x7fa250106b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.332+0000 7fa254c00640 1 --2- 172.21.15.176:0/4222914993 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25010ca40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.332+0000 7fa254c00640 1 -- 172.21.15.176:0/4222914993 >> 172.21.15.176:0/4222914993 conn(0x7fa250075e80 msgr2=0x7fa2500782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:22.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.332+0000 7fa254c00640 1 -- 172.21.15.176:0/4222914993 shutdown_connections 2024-03-13T15:04:22.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.332+0000 7fa254c00640 1 -- 172.21.15.176:0/4222914993 wait complete. 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.333+0000 7fa254c00640 1 Processor -- start 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.333+0000 7fa254c00640 1 -- start start 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.333+0000 7fa254c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2501066a0 0x7fa25019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.333+0000 7fa254c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.333+0000 7fa254c00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25019cf70 con 0x7fa25010c660 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.333+0000 7fa254c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2501a0d40 con 0x7fa2501066a0 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa245400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa245400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58556/0 (socket says 172.21.15.176:58556) 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa245400640 1 -- 172.21.15.176:0/3958153625 learned_addr learned my addr 172.21.15.176:0/3958153625 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa24de00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2501066a0 0x7fa25019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa245400640 1 -- 172.21.15.176:0/3958153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2501066a0 msgr2=0x7fa25019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.319 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa245400640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2501066a0 0x7fa25019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa245400640 1 -- 172.21.15.176:0/3958153625 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa240009560 con 0x7fa25010c660 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa245400640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25019c8e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa23800d6f0 tx=0x7fa23800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa247400640 1 -- 172.21.15.176:0/3958153625 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa238009a30 con 0x7fa25010c660 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa247400640 1 -- 172.21.15.176:0/3958153625 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2380054e0 con 0x7fa25010c660 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.334+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2501a1020 con 0x7fa25010c660 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.335+0000 7fa247400640 1 -- 172.21.15.176:0/3958153625 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa238010460 con 0x7fa25010c660 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.335+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2501a1580 con 0x7fa25010c660 2024-03-13T15:04:22.320 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.336+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25004e040 con 0x7fa25010c660 2024-03-13T15:04:22.321 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.337+0000 7fa247400640 1 -- 172.21.15.176:0/3958153625 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa238009b90 con 0x7fa25010c660 2024-03-13T15:04:22.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.338+0000 7fa247400640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa23406df10 0x7fa2340703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:22.322 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.338+0000 7fa247400640 1 -- 172.21.15.176:0/3958153625 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa23808d9c0 con 0x7fa25010c660 2024-03-13T15:04:22.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.338+0000 7fa24de00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa23406df10 0x7fa2340703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:22.323 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.339+0000 7fa24de00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa23406df10 0x7fa2340703d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa24002cf90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:22.327 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.343+0000 7fa247400640 1 -- 172.21.15.176:0/3958153625 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa238056cf0 con 0x7fa25010c660 2024-03-13T15:04:22.554 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.570+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fa25010ebc0 con 0x7fa25010c660 2024-03-13T15:04:22.556 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.571+0000 7fa247400640 1 -- 172.21.15.176:0/3958153625 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v39) v1 ==== 107+0+4189 (secure 0 0 0) 0x7fa23805a3b0 con 0x7fa25010c660 2024-03-13T15:04:22.556 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:22.556 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:19.671648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:22.556 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 35 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa23406df10 msgr2=0x7fa2340703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa23406df10 0x7fa2340703d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa240000c00 tx=0x7fa24002cf90 comp rx=0 tx=0).stop 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 msgr2=0x7fa25019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25019c8e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa23800d6f0 tx=0x7fa23800dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 shutdown_connections 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa25010c660 0x7fa25019c8e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa23406df10 0x7fa2340703d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 --2- 172.21.15.176:0/3958153625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2501066a0 0x7fa25019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 >> 172.21.15.176:0/3958153625 conn(0x7fa250075e80 msgr2=0x7fa250076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 shutdown_connections 2024-03-13T15:04:22.559 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.574+0000 7fa254c00640 1 -- 172.21.15.176:0/3958153625 wait complete. 2024-03-13T15:04:22.578 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-03-13T15:04:22.578 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 -- 172.21.15.176:0/1520191102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c100290 msgr2=0x7f000c100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 --2- 172.21.15.176:0/1520191102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c100290 0x7f000c100670 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7efffc009880 tx=0x7efffc02bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 -- 172.21.15.176:0/1520191102 shutdown_connections 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 --2- 172.21.15.176:0/1520191102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c100c40 0x7f000c104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 --2- 172.21.15.176:0/1520191102 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c100290 0x7f000c100670 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 -- 172.21.15.176:0/1520191102 >> 172.21.15.176:0/1520191102 conn(0x7f000c0fbaf0 msgr2=0x7f000c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 -- 172.21.15.176:0/1520191102 shutdown_connections 2024-03-13T15:04:22.790 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.805+0000 7f0010a00640 1 -- 172.21.15.176:0/1520191102 wait complete. 2024-03-13T15:04:22.792 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.806+0000 7f0010a00640 1 Processor -- start 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.806+0000 7f0010a00640 1 -- start start 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0010a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c100290 0x7f000c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0010a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c198410 0x7f000c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0010a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c198a10 con 0x7f000c198410 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0010a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c198b80 con 0x7f000c100290 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0009400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c198410 0x7f000c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0009400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c198410 0x7f000c19c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58578/0 (socket says 172.21.15.176:58578) 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0009400640 1 -- 172.21.15.176:0/1913624169 learned_addr learned my addr 172.21.15.176:0/1913624169 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.807+0000 7f0009e00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c100290 0x7f000c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0009400640 1 -- 172.21.15.176:0/1913624169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c100290 msgr2=0x7f000c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0009400640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c100290 0x7f000c197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0009400640 1 -- 172.21.15.176:0/1913624169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc009560 con 0x7f000c198410 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0009e00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c100290 0x7f000c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:22.793 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0009400640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c198410 0x7f000c19c870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efff800d6f0 tx=0x7efff800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:22.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0002a00640 1 -- 172.21.15.176:0/1913624169 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff80051f0 con 0x7f000c198410 2024-03-13T15:04:22.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c19ce70 con 0x7f000c198410 2024-03-13T15:04:22.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0002a00640 1 -- 172.21.15.176:0/1913624169 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efff8009ce0 con 0x7f000c198410 2024-03-13T15:04:22.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0002a00640 1 -- 172.21.15.176:0/1913624169 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efff8004e80 con 0x7f000c198410 2024-03-13T15:04:22.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.808+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c19d3d0 con 0x7f000c198410 2024-03-13T15:04:22.794 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.809+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c19d6a0 con 0x7f000c198410 2024-03-13T15:04:22.795 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.811+0000 7f0002a00640 1 -- 172.21.15.176:0/1913624169 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7efff80046c0 con 0x7f000c198410 2024-03-13T15:04:22.796 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.812+0000 7f0002a00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efff006df80 0x7efff0070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:22.796 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.812+0000 7f0002a00640 1 -- 172.21.15.176:0/1913624169 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7efff808d9a0 con 0x7f000c198410 2024-03-13T15:04:22.796 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.812+0000 7f0009e00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efff006df80 0x7efff0070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:22.797 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.813+0000 7f0009e00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efff006df80 0x7efff0070440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efffc036040 tx=0x7efffc02cf90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:22.801 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:22.817+0000 7f0002a00640 1 -- 172.21.15.176:0/1913624169 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7efff8056c50 con 0x7f000c198410 2024-03-13T15:04:22.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:22 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/470047790' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-13T15:04:22.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:22 smithi176 ceph-mon[112742]: pgmap v376: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:22.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:22 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3958153625' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-13T15:04:23.024 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.039+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f000c199300 con 0x7f000c198410 2024-03-13T15:04:23.025 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.040+0000 7f0002a00640 1 -- 172.21.15.176:0/1913624169 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v39) v1 ==== 107+0+5005 (secure 0 0 0) 0x7f000c199300 con 0x7f000c198410 2024-03-13T15:04:23.025 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:23.025 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44281,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6827/2135805809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2135805809},{"type":"v1","addr":"172.21.15.196:6827","nonce":2135805809}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:19.671648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:23.025 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 36 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efff006df80 msgr2=0x7efff0070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efff006df80 0x7efff0070440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efffc036040 tx=0x7efffc02cf90 comp rx=0 tx=0).stop 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c198410 msgr2=0x7f000c19c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c198410 0x7f000c19c870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efff800d6f0 tx=0x7efff800dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 shutdown_connections 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f000c198410 0x7f000c19c870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7efff006df80 0x7efff0070440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 --2- 172.21.15.176:0/1913624169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f000c100290 0x7f000c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 >> 172.21.15.176:0/1913624169 conn(0x7f000c0fbaf0 msgr2=0x7f000c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 shutdown_connections 2024-03-13T15:04:23.028 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.043+0000 7f0010a00640 1 -- 172.21.15.176:0/1913624169 wait complete. 2024-03-13T15:04:23.045 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-03-13T15:04:23.045 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-03-13T15:04:23.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:22 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/470047790' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-03-13T15:04:23.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:22 smithi196 ceph-mon[86193]: pgmap v376: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:23.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:22 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3958153625' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-03-13T15:04:23.250 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.265+0000 7faa7a000640 1 -- 172.21.15.176:0/1328191099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 msgr2=0x7faa74106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.265+0000 7faa7a000640 1 --2- 172.21.15.176:0/1328191099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa74106b00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faa5c009880 tx=0x7faa5c02bd80 comp rx=0 tx=0).stop 2024-03-13T15:04:23.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.266+0000 7faa7a000640 1 -- 172.21.15.176:0/1328191099 shutdown_connections 2024-03-13T15:04:23.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.266+0000 7faa7a000640 1 --2- 172.21.15.176:0/1328191099 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa74106b00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.266+0000 7faa7a000640 1 --2- 172.21.15.176:0/1328191099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa7410c6a0 0x7faa7410ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.266+0000 7faa7a000640 1 -- 172.21.15.176:0/1328191099 >> 172.21.15.176:0/1328191099 conn(0x7faa74075e50 msgr2=0x7faa74078270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:23.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.266+0000 7faa7a000640 1 -- 172.21.15.176:0/1328191099 shutdown_connections 2024-03-13T15:04:23.251 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.266+0000 7faa7a000640 1 -- 172.21.15.176:0/1328191099 wait complete. 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.267+0000 7faa7a000640 1 Processor -- start 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.267+0000 7faa7a000640 1 -- start start 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.267+0000 7faa7a000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa7419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.267+0000 7faa7a000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa7410c6a0 0x7faa7419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.267+0000 7faa7a000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7419ce20 con 0x7faa741066a0 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.267+0000 7faa7a000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa741a0bf0 con 0x7faa7410c6a0 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa73400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa7419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa73400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa7419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58602/0 (socket says 172.21.15.176:58602) 2024-03-13T15:04:23.253 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa73400640 1 -- 172.21.15.176:0/3689367812 learned_addr learned my addr 172.21.15.176:0/3689367812 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa73400640 1 -- 172.21.15.176:0/3689367812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa7410c6a0 msgr2=0x7faa7419c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa73400640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa7410c6a0 0x7faa7419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa73400640 1 -- 172.21.15.176:0/3689367812 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5c009560 con 0x7faa741066a0 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa73400640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa7419c250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c8f0 tx=0x7faa6400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa70c00640 1 -- 172.21.15.176:0/3689367812 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa64007b80 con 0x7faa741066a0 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa70c00640 1 -- 172.21.15.176:0/3689367812 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa640054e0 con 0x7faa741066a0 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa741a0ed0 con 0x7faa741066a0 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.268+0000 7faa70c00640 1 -- 172.21.15.176:0/3689367812 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6400f450 con 0x7faa741066a0 2024-03-13T15:04:23.254 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.269+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa741a1430 con 0x7faa741066a0 2024-03-13T15:04:23.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.270+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7404e040 con 0x7faa741066a0 2024-03-13T15:04:23.255 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.271+0000 7faa70c00640 1 -- 172.21.15.176:0/3689367812 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7faa64004f60 con 0x7faa741066a0 2024-03-13T15:04:23.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.271+0000 7faa70c00640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7faa5806dfd0 0x7faa58070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:23.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.271+0000 7faa70c00640 1 -- 172.21.15.176:0/3689367812 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7faa6408dd30 con 0x7faa741066a0 2024-03-13T15:04:23.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.272+0000 7faa6aa00640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7faa5806dfd0 0x7faa58070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:23.256 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.272+0000 7faa6aa00640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7faa5806dfd0 0x7faa58070490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa5c036040 tx=0x7faa5c000bc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:23.261 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.277+0000 7faa70c00640 1 -- 172.21.15.176:0/3689367812 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7faa64057020 con 0x7faa741066a0 2024-03-13T15:04:23.496 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.512+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7faa74113b90 con 0x7faa741066a0 2024-03-13T15:04:23.498 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.514+0000 7faa70c00640 1 -- 172.21.15.176:0/3689367812 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v39) v1 ==== 107+0+5005 (secure 0 0 0) 0x7faa6405a6e0 con 0x7faa741066a0 2024-03-13T15:04:23.498 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:23.498 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44281,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6827/2135805809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2135805809},{"type":"v1","addr":"172.21.15.196:6827","nonce":2135805809}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:34.858248+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:23.499 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 37 2024-03-13T15:04:23.501 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7faa5806dfd0 msgr2=0x7faa58070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7faa5806dfd0 0x7faa58070490 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7faa5c036040 tx=0x7faa5c000bc0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 msgr2=0x7faa7419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa7419c250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c8f0 tx=0x7faa6400cdb0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 shutdown_connections 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7faa5806dfd0 0x7faa58070490 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa7410c6a0 0x7faa7419c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 --2- 172.21.15.176:0/3689367812 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7faa741066a0 0x7faa7419c250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 >> 172.21.15.176:0/3689367812 conn(0x7faa74075e50 msgr2=0x7faa740777d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 shutdown_connections 2024-03-13T15:04:23.502 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.517+0000 7faa7a000640 1 -- 172.21.15.176:0/3689367812 wait complete. 2024-03-13T15:04:23.520 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 37 2024-03-13T15:04:23.520 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-03-13T15:04:23.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.747+0000 7f24bec00640 1 -- 172.21.15.176:0/2270705760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8102e50 msgr2=0x7f24b8106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.747+0000 7f24bec00640 1 --2- 172.21.15.176:0/2270705760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8102e50 0x7f24b8106e90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009880 tx=0x7f24a002bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.748+0000 7f24bec00640 1 -- 172.21.15.176:0/2270705760 shutdown_connections 2024-03-13T15:04:23.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.748+0000 7f24bec00640 1 --2- 172.21.15.176:0/2270705760 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8102e50 0x7f24b8106e90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.732 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.748+0000 7f24bec00640 1 --2- 172.21.15.176:0/2270705760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24b81024a0 0x7f24b8102880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.748+0000 7f24bec00640 1 -- 172.21.15.176:0/2270705760 >> 172.21.15.176:0/2270705760 conn(0x7f24b80fdd00 msgr2=0x7f24b8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:23.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.748+0000 7f24bec00640 1 -- 172.21.15.176:0/2270705760 shutdown_connections 2024-03-13T15:04:23.733 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.748+0000 7f24bec00640 1 -- 172.21.15.176:0/2270705760 wait complete. 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.749+0000 7f24bec00640 1 Processor -- start 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.749+0000 7f24bec00640 1 -- start start 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.749+0000 7f24bec00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24b81024a0 0x7f24b8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.749+0000 7f24bec00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8198440 0x7f24b819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.749+0000 7f24bec00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b81989b0 con 0x7f24b8198440 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.749+0000 7f24bec00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b8198b20 con 0x7f24b81024a0 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.749+0000 7f24afe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8198440 0x7f24b819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24afe00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8198440 0x7f24b819c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58612/0 (socket says 172.21.15.176:58612) 2024-03-13T15:04:23.734 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24afe00640 1 -- 172.21.15.176:0/4045230300 learned_addr learned my addr 172.21.15.176:0/4045230300 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:23.735 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24b7e00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24b81024a0 0x7f24b8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:23.735 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24afe00640 1 -- 172.21.15.176:0/4045230300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24b81024a0 msgr2=0x7f24b8197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:23.735 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24afe00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24b81024a0 0x7f24b8197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24afe00640 1 -- 172.21.15.176:0/4045230300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0009560 con 0x7f24b8198440 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24afe00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8198440 0x7f24b819c8a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009230 tx=0x7f24a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.750+0000 7f24b5600640 1 -- 172.21.15.176:0/4045230300 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a00037c0 con 0x7f24b8198440 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.751+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b819ce40 con 0x7f24b8198440 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.751+0000 7f24b5600640 1 -- 172.21.15.176:0/4045230300 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24a0003de0 con 0x7f24b8198440 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.751+0000 7f24b5600640 1 -- 172.21.15.176:0/4045230300 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a003e460 con 0x7f24b8198440 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.751+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b819d350 con 0x7f24b8198440 2024-03-13T15:04:23.736 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.752+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b804e040 con 0x7f24b8198440 2024-03-13T15:04:23.737 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.753+0000 7f24b5600640 1 -- 172.21.15.176:0/4045230300 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f24a0004fd0 con 0x7f24b8198440 2024-03-13T15:04:23.738 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.754+0000 7f24b5600640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f249c06dfd0 0x7f249c070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:23.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.754+0000 7f24b5600640 1 -- 172.21.15.176:0/4045230300 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f24a00b0780 con 0x7f24b8198440 2024-03-13T15:04:23.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.754+0000 7f24b7e00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f249c06dfd0 0x7f249c070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:23.739 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.755+0000 7f24b7e00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f249c06dfd0 0x7f249c070490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a8005e10 tx=0x7f24a8005da0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:23.743 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:23.759+0000 7f24b5600640 1 -- 172.21.15.176:0/4045230300 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f24a0079af0 con 0x7f24b8198440 2024-03-13T15:04:23.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:23 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1913624169' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-13T15:04:23.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:23 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3689367812' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-13T15:04:24.006 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.022+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f24b8102e50 con 0x7f24b8198440 2024-03-13T15:04:24.008 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.023+0000 7f24b5600640 1 -- 172.21.15.176:0/4045230300 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v39) v1 ==== 107+0+5011 (secure 0 0 0) 0x7f24a007d1b0 con 0x7f24b8198440 2024-03-13T15:04:24.008 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:24.008 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44257,"name":"cephfs.smithi196.tcitrk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6825/2671769958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2671769958},{"type":"v1","addr":"172.21.15.196:6825","nonce":2671769958}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":44281,"name":"cephfs.smithi196.wabhcu","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6827/2135805809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6826","nonce":2135805809},{"type":"v1","addr":"172.21.15.196:6827","nonce":2135805809}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-13T14:45:38.361828+0000","modified":"2024-03-13T15:01:35.879487+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34462":{"gid":34462,"name":"cephfs.smithi176.ldskqt","rank":0,"incarnation":32,"state":"up:active","state_seq":24,"addr":"172.21.15.176:6827/648979087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6826","nonce":648979087},{"type":"v1","addr":"172.21.15.176:6827","nonce":648979087}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34494":{"gid":34494,"name":"cephfs.smithi176.hjrgov","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.176:6829/552037800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6828","nonce":552037800},{"type":"v1","addr":"172.21.15.176:6829","nonce":552037800}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-03-13T15:04:24.009 INFO:teuthology.orchestra.run.smithi176.stderr:dumped fsmap epoch 38 2024-03-13T15:04:24.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f249c06dfd0 msgr2=0x7f249c070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f249c06dfd0 0x7f249c070490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a8005e10 tx=0x7f24a8005da0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8198440 msgr2=0x7f24b819c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8198440 0x7f24b819c8a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009230 tx=0x7f24a0004270 comp rx=0 tx=0).stop 2024-03-13T15:04:24.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 shutdown_connections 2024-03-13T15:04:24.011 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f24b8198440 0x7f24b819c8a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f249c06dfd0 0x7f249c070490 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 --2- 172.21.15.176:0/4045230300 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24b81024a0 0x7f24b8197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 >> 172.21.15.176:0/4045230300 conn(0x7f24b80fdd00 msgr2=0x7f24b8100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:24.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 shutdown_connections 2024-03-13T15:04:24.012 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.027+0000 7f24bec00640 1 -- 172.21.15.176:0/4045230300 wait complete. 2024-03-13T15:04:24.030 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-13T15:04:24.041 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-13T15:04:24.041 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T15:04:24.042 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-13T15:04:24.095 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T15:04:24.095 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-13T15:04:24.143 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T15:04:24.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:23 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1913624169' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-03-13T15:04:24.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:23 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3689367812' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-03-13T15:04:24.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.414+0000 7fd47e200640 1 -- 172.21.15.176:0/2221301784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd478102e90 msgr2=0x7fd478106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.414+0000 7fd47e200640 1 --2- 172.21.15.176:0/2221301784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd478102e90 0x7fd478106ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd460009880 tx=0x7fd46002bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.414+0000 7fd47e200640 1 -- 172.21.15.176:0/2221301784 shutdown_connections 2024-03-13T15:04:24.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.414+0000 7fd47e200640 1 --2- 172.21.15.176:0/2221301784 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd478102e90 0x7fd478106ed0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.414+0000 7fd47e200640 1 --2- 172.21.15.176:0/2221301784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4781024e0 0x7fd4781028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.414+0000 7fd47e200640 1 -- 172.21.15.176:0/2221301784 >> 172.21.15.176:0/2221301784 conn(0x7fd4780fdd00 msgr2=0x7fd478100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:24.399 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.414+0000 7fd47e200640 1 -- 172.21.15.176:0/2221301784 shutdown_connections 2024-03-13T15:04:24.400 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.415+0000 7fd47e200640 1 -- 172.21.15.176:0/2221301784 wait complete. 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.415+0000 7fd47e200640 1 Processor -- start 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd47e200640 1 -- start start 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd47e200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4781024e0 0x7fd478197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd47e200640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4781983f0 0x7fd47819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd47e200640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4781989f0 con 0x7fd4781024e0 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd47e200640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd478198b60 con 0x7fd4781983f0 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd477400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4781024e0 0x7fd478197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd477400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4781024e0 0x7fd478197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58630/0 (socket says 172.21.15.176:58630) 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.416+0000 7fd477400640 1 -- 172.21.15.176:0/1461387240 learned_addr learned my addr 172.21.15.176:0/1461387240 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd477400640 1 -- 172.21.15.176:0/1461387240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4781983f0 msgr2=0x7fd47819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd46ea00640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4781983f0 0x7fd47819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:24.401 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd477400640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4781983f0 0x7fd47819c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd477400640 1 -- 172.21.15.176:0/1461387240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd460009560 con 0x7fd4781024e0 2024-03-13T15:04:24.402 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd477400640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4781024e0 0x7fd478197eb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd46800d900 tx=0x7fd46800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:24.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd474c00640 1 -- 172.21.15.176:0/1461387240 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd468009c00 con 0x7fd4781024e0 2024-03-13T15:04:24.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd474c00640 1 -- 172.21.15.176:0/1461387240 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd468009d60 con 0x7fd4781024e0 2024-03-13T15:04:24.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd474c00640 1 -- 172.21.15.176:0/1461387240 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd46800f710 con 0x7fd4781024e0 2024-03-13T15:04:24.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47819ce50 con 0x7fd4781024e0 2024-03-13T15:04:24.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.417+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47819d3b0 con 0x7fd4781024e0 2024-03-13T15:04:24.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.418+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47819d680 con 0x7fd4781024e0 2024-03-13T15:04:24.403 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.419+0000 7fd474c00640 1 -- 172.21.15.176:0/1461387240 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fd468010460 con 0x7fd4781024e0 2024-03-13T15:04:24.404 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.420+0000 7fd474c00640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fd45c06dfd0 0x7fd45c070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:24.405 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.420+0000 7fd474c00640 1 -- 172.21.15.176:0/1461387240 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fd46808dc60 con 0x7fd4781024e0 2024-03-13T15:04:24.405 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.421+0000 7fd46ea00640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fd45c06dfd0 0x7fd45c070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:24.405 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.421+0000 7fd46ea00640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fd45c06dfd0 0x7fd45c070490 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd460036040 tx=0x7fd46002cf90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:24.409 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.425+0000 7fd474c00640 1 -- 172.21.15.176:0/1461387240 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fd468056f90 con 0x7fd4781024e0 2024-03-13T15:04:24.607 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.623+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd47804e040 con 0x7fd4781024e0 2024-03-13T15:04:24.611 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.626+0000 7fd474c00640 1 -- 172.21.15.176:0/1461387240 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17190 (secure 0 0 0) 0x7fd46805a650 con 0x7fd4781024e0 2024-03-13T15:04:24.611 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:24.611 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":112,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","created":"2024-03-13T14:36:32.120003+0000","modified":"2024-03-13T15:01:08.961154+0000","last_up_change":"2024-03-13T14:59:42.380775+0000","last_in_change":"2024-03-13T14:43:42.351790+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T14:42:22.643268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-13T14:45:37.278783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-13T14:45:37.880653+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c570fcfa-ceda-4e39-b200-5c8b0b7fab19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":63,"up_thru":105,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6803","nonce":468040948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6805","nonce":468040948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6809","nonce":468040948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6807","nonce":468040948}]},"public_addr":"172.21.15.176:6803/468040948","cluster_addr":"172.21.15.176:6805/468040948","heartbeat_back_addr":"172.21.15.176:6809/468040948","heartbeat_front_addr":"172.21.15.176:6807/468040948","state":["exists","up"]},{"osd":1,"uuid":"5a973902-7ecc-4a05-b917-a74e108df4db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":71,"up_from":75,"up_thru":105,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6811","nonce":2654405375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6813","nonce":2654405375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6817","nonce":2654405375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6815","nonce":2654405375}]},"public_addr":"172.21.15.176:6811/2654405375","cluster_addr":"172.21.15.176:6813/2654405375","heartbeat_back_addr":"172.21.15.176:6817/2654405375","heartbeat_front_addr":"172.21.15.176:6815/2654405375","state":["exists","up"]},{"osd":2,"uuid":"f72757d1-b949-45fb-abcd-c8649cec14f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":76,"up_from":80,"up_thru":105,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6819","nonce":2651949103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6821","nonce":2651949103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6825","nonce":2651949103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6823","nonce":2651949103}]},"public_addr":"172.21.15.176:6819/2651949103","cluster_addr":"172.21.15.176:6821/2651949103","heartbeat_back_addr":"172.21.15.176:6825/2651949103","heartbeat_front_addr":"172.21.15.176:6823/2651949103","state":["exists","up"]},{"osd":3,"uuid":"e1c42f2a-0efa-4c44-a965-9e5904aeace0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":81,"up_from":89,"up_thru":105,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6801","nonce":1554416376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6803","nonce":1554416376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6807","nonce":1554416376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6805","nonce":1554416376}]},"public_addr":"172.21.15.196:6801/1554416376","cluster_addr":"172.21.15.196:6803/1554416376","heartbeat_back_addr":"172.21.15.196:6807/1554416376","heartbeat_front_addr":"172.21.15.196:6805/1554416376","state":["exists","up"]},{"osd":4,"uuid":"b4c7c89d-a349-4b39-b64a-e1080d213acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":96,"up_from":100,"up_thru":105,"down_at":97,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6809","nonce":1908907228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6811","nonce":1908907228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6815","nonce":1908907228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6813","nonce":1908907228}]},"public_addr":"172.21.15.196:6809/1908907228","cluster_addr":"172.21.15.196:6811/1908907228","heartbeat_back_addr":"172.21.15.196:6815/1908907228","heartbeat_front_addr":"172.21.15.196:6813/1908907228","state":["exists","up"]},{"osd":5,"uuid":"efb96c7b-7a78-4c40-996f-f8141029c310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":101,"up_from":105,"up_thru":105,"down_at":102,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6817","nonce":3411147075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6819","nonce":3411147075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6823","nonce":3411147075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6821","nonce":3411147075}]},"public_addr":"172.21.15.196:6817/3411147075","cluster_addr":"172.21.15.196:6819/3411147075","heartbeat_back_addr":"172.21.15.196:6823/3411147075","heartbeat_front_addr":"172.21.15.196:6821/3411147075","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:06.529418+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:43.356501+0000","dead_epoch":71},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:19.060195+0000","dead_epoch":76},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:53.551733+0000","dead_epoch":81},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:43:28.434416+0000","dead_epoch":96},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:44:05.009478+0000","dead_epoch":101}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.9","mappings":[{"from":0,"to":3}]},{"pgid":"2.15","mappings":[{"from":0,"to":2}]},{"pgid":"2.1e","mappings":[{"from":0,"to":3}]},{"pgid":"3.6","mappings":[{"from":1,"to":2}]},{"pgid":"3.b","mappings":[{"from":4,"to":2}]},{"pgid":"3.12","mappings":[{"from":1,"to":5}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6827/1275358571":"2024-03-14T15:01:08.961104+0000","172.21.15.196:6826/1275358571":"2024-03-14T15:01:08.961104+0000","172.21.15.176:6800/2698132085":"2024-03-14T14:48:28.652138+0000","172.21.15.176:0/2786891974":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/2461806366":"2024-03-14T14:54:00.256736+0000","172.21.15.176:6801/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.196:6828/1297073737":"2024-03-14T14:49:27.654006+0000","172.21.15.196:6829/1297073737":"2024-03-14T14:49:27.654006+0000","172.21.15.176:6800/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/3527709806":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/1319558216":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1541570168":"2024-03-14T14:48:28.652138+0000","172.21.15.176:0/248607826":"2024-03-14T14:37:47.494508+0000","172.21.15.196:0/1871453830":"2024-03-14T14:49:27.654006+0000","172.21.15.176:0/1825335586":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1477585567":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1947716670":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/1567438135":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2108157434":"2024-03-14T14:37:47.494508+0000","172.21.15.196:0/4213061990":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/3434817823":"2024-03-14T14:49:27.654006+0000","172.21.15.176:6801/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1665184531":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6800/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.196:6824/152396394":"2024-03-14T15:00:43.199506+0000","172.21.15.176:0/665298320":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6801/2698132085":"2024-03-14T14:48:28.652138+0000","172.21.15.196:0/3486069398":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/945284252":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/3034338010":"2024-03-14T14:49:27.654006+0000","172.21.15.196:6825/152396394":"2024-03-14T15:00:43.199506+0000","172.21.15.176:0/698195875":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1930887076":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6827/3592546199":"2024-03-14T14:45:59.134341+0000","172.21.15.176:0/3215654415":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/258177446":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6800/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6826/3592546199":"2024-03-14T14:45:59.134341+0000","172.21.15.176:0/2332303582":"2024-03-14T14:37:05.490145+0000","172.21.15.176:6801/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6801/3527709806":"2024-03-14T14:54:00.256736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fd45c06dfd0 msgr2=0x7fd45c070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fd45c06dfd0 0x7fd45c070490 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd460036040 tx=0x7fd46002cf90 comp rx=0 tx=0).stop 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4781024e0 msgr2=0x7fd478197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4781024e0 0x7fd478197eb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd46800d900 tx=0x7fd46800ddc0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 shutdown_connections 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fd45c06dfd0 0x7fd45c070490 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4781983f0 0x7fd47819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 --2- 172.21.15.176:0/1461387240 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fd4781024e0 0x7fd478197eb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.629+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 >> 172.21.15.176:0/1461387240 conn(0x7fd4780fdd00 msgr2=0x7fd4780ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.630+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 shutdown_connections 2024-03-13T15:04:24.614 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.630+0000 7fd47e200640 1 -- 172.21.15.176:0/1461387240 wait complete. 2024-03-13T15:04:24.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-13T14:42:22.643268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-13T14:45:37.278783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-13T14:45:37.880653+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-13T15:04:24.634 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-13T15:04:24.851 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2677050786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 msgr2=0x7fe1e4110920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2677050786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e4110920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009880 tx=0x7fe1d002bd80 comp rx=0 tx=0).stop 2024-03-13T15:04:24.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2677050786 shutdown_connections 2024-03-13T15:04:24.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2677050786 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e4110920 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2677050786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1e4072640 0x7fe1e4072a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2677050786 >> 172.21.15.176:0/2677050786 conn(0x7fe1e40fb920 msgr2=0x7fe1e40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:24.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2677050786 shutdown_connections 2024-03-13T15:04:24.852 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.867+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2677050786 wait complete. 2024-03-13T15:04:24.853 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.868+0000 7fe1e8e00640 1 Processor -- start 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.868+0000 7fe1e8e00640 1 -- start start 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e8e00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1e4072640 0x7fe1e419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e8e00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e419c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e8e00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e419cc30 con 0x7fe1e4072f60 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e8e00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e419cda0 con 0x7fe1e4072640 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e2a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e419c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e2a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e419c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58658/0 (socket says 172.21.15.176:58658) 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e2a00640 1 -- 172.21.15.176:0/2955664215 learned_addr learned my addr 172.21.15.176:0/2955664215 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e2a00640 1 -- 172.21.15.176:0/2955664215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1e4072640 msgr2=0x7fe1e419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e2a00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1e4072640 0x7fe1e419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.869+0000 7fe1e2a00640 1 -- 172.21.15.176:0/2955664215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0009560 con 0x7fe1e4072f60 2024-03-13T15:04:24.854 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.870+0000 7fe1e2a00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e419c630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00099a0 tx=0x7fe1d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:24.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.870+0000 7fe1dbe00640 1 -- 172.21.15.176:0/2955664215 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d0004b00 con 0x7fe1e4072f60 2024-03-13T15:04:24.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.870+0000 7fe1dbe00640 1 -- 172.21.15.176:0/2955664215 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1d0003c80 con 0x7fe1e4072f60 2024-03-13T15:04:24.855 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.870+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e41a0bd0 con 0x7fe1e4072f60 2024-03-13T15:04:24.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.870+0000 7fe1dbe00640 1 -- 172.21.15.176:0/2955664215 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1d003e450 con 0x7fe1e4072f60 2024-03-13T15:04:24.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.870+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e41a1130 con 0x7fe1e4072f60 2024-03-13T15:04:24.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.871+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e404e040 con 0x7fe1e4072f60 2024-03-13T15:04:24.856 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.872+0000 7fe1dbe00640 1 -- 172.21.15.176:0/2955664215 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fe1d0004fb0 con 0x7fe1e4072f60 2024-03-13T15:04:24.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.873+0000 7fe1dbe00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1c806df10 0x7fe1c80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:24.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.873+0000 7fe1dbe00640 1 -- 172.21.15.176:0/2955664215 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fe1d00b0b70 con 0x7fe1e4072f60 2024-03-13T15:04:24.857 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.873+0000 7fe1e3400640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1c806df10 0x7fe1c80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:24.858 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.874+0000 7fe1e3400640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1c806df10 0x7fe1c80703d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400a990 tx=0x7fe1d4005c70 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:24.862 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:24.877+0000 7fe1dbe00640 1 -- 172.21.15.176:0/2955664215 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fe1d0079ea0 con 0x7fe1e4072f60 2024-03-13T15:04:25.059 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.074+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1e4113680 con 0x7fe1e4072f60 2024-03-13T15:04:25.059 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.075+0000 7fe1dbe00640 1 -- 172.21.15.176:0/2955664215 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1d007d560 con 0x7fe1e4072f60 2024-03-13T15:04:25.059 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 1 2024-03-13T15:04:25.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1c806df10 msgr2=0x7fe1c80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.062 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1c806df10 0x7fe1c80703d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400a990 tx=0x7fe1d4005c70 comp rx=0 tx=0).stop 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 msgr2=0x7fe1e419c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e419c630 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00099a0 tx=0x7fe1d0004270 comp rx=0 tx=0).stop 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 shutdown_connections 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fe1e4072f60 0x7fe1e419c630 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fe1c806df10 0x7fe1c80703d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 --2- 172.21.15.176:0/2955664215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1e4072640 0x7fe1e419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 >> 172.21.15.176:0/2955664215 conn(0x7fe1e40fb920 msgr2=0x7fe1e4102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 shutdown_connections 2024-03-13T15:04:25.063 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.078+0000 7fe1e8e00640 1 -- 172.21.15.176:0/2955664215 wait complete. 2024-03-13T15:04:25.077 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:24 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/4045230300' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-13T15:04:25.078 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:24 smithi176 ceph-mon[112742]: pgmap v377: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:25.078 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:24 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1461387240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T15:04:25.081 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-13T15:04:25.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:24 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/4045230300' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-03-13T15:04:25.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:24 smithi196 ceph-mon[86193]: pgmap v377: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:25.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:24 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1461387240' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T15:04:25.300 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 -- 172.21.15.176:0/3112988430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581024c0 msgr2=0x7fa0581028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 --2- 172.21.15.176:0/3112988430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581024c0 0x7fa0581028a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa048009880 tx=0x7fa04802bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 -- 172.21.15.176:0/3112988430 shutdown_connections 2024-03-13T15:04:25.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 --2- 172.21.15.176:0/3112988430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa058102e70 0x7fa058106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 --2- 172.21.15.176:0/3112988430 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581024c0 0x7fa0581028a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 -- 172.21.15.176:0/3112988430 >> 172.21.15.176:0/3112988430 conn(0x7fa0580fdd40 msgr2=0x7fa058100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:25.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 -- 172.21.15.176:0/3112988430 shutdown_connections 2024-03-13T15:04:25.301 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.316+0000 7fa05de00640 1 -- 172.21.15.176:0/3112988430 wait complete. 2024-03-13T15:04:25.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.317+0000 7fa05de00640 1 Processor -- start 2024-03-13T15:04:25.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.317+0000 7fa05de00640 1 -- start start 2024-03-13T15:04:25.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.317+0000 7fa05de00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0581024c0 0x7fa058197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:25.302 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.317+0000 7fa05de00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581984d0 0x7fa05819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.317+0000 7fa05de00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058198ad0 con 0x7fa0581984d0 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.317+0000 7fa05de00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058198c40 con 0x7fa0581024c0 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa04e000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581984d0 0x7fa05819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa04e000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581984d0 0x7fa05819c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58676/0 (socket says 172.21.15.176:58676) 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa04e000640 1 -- 172.21.15.176:0/3328332262 learned_addr learned my addr 172.21.15.176:0/3328332262 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa056a00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0581024c0 0x7fa058197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa04e000640 1 -- 172.21.15.176:0/3328332262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0581024c0 msgr2=0x7fa058197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa04e000640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0581024c0 0x7fa058197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.303 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa04e000640 1 -- 172.21.15.176:0/3328332262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009560 con 0x7fa0581984d0 2024-03-13T15:04:25.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.318+0000 7fa04e000640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581984d0 0x7fa05819c930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa04000c920 tx=0x7fa04000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:25.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.319+0000 7fa05ca00640 1 -- 172.21.15.176:0/3328332262 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa040007b80 con 0x7fa0581984d0 2024-03-13T15:04:25.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.319+0000 7fa05ca00640 1 -- 172.21.15.176:0/3328332262 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa040007ce0 con 0x7fa0581984d0 2024-03-13T15:04:25.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.319+0000 7fa05ca00640 1 -- 172.21.15.176:0/3328332262 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0400187d0 con 0x7fa0581984d0 2024-03-13T15:04:25.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.319+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05819cf30 con 0x7fa0581984d0 2024-03-13T15:04:25.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.319+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05819d490 con 0x7fa0581984d0 2024-03-13T15:04:25.304 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.320+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05804e040 con 0x7fa0581984d0 2024-03-13T15:04:25.305 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.321+0000 7fa05ca00640 1 -- 172.21.15.176:0/3328332262 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fa04000f450 con 0x7fa0581984d0 2024-03-13T15:04:25.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.322+0000 7fa05ca00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa03c06df10 0x7fa03c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:25.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.322+0000 7fa05ca00640 1 -- 172.21.15.176:0/3328332262 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fa04008dff0 con 0x7fa0581984d0 2024-03-13T15:04:25.306 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.322+0000 7fa056a00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa03c06df10 0x7fa03c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:25.307 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.323+0000 7fa056a00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa03c06df10 0x7fa03c0703d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa048036040 tx=0x7fa04802cf90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:25.311 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.327+0000 7fa05ca00640 1 -- 172.21.15.176:0/3328332262 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fa040057340 con 0x7fa0581984d0 2024-03-13T15:04:25.510 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.525+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fa0581993d0 con 0x7fa0581984d0 2024-03-13T15:04:25.511 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.526+0000 7fa05ca00640 1 -- 172.21.15.176:0/3328332262 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v112) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa04005aa00 con 0x7fa0581984d0 2024-03-13T15:04:25.511 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T15:04:25.514 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa03c06df10 msgr2=0x7fa03c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa03c06df10 0x7fa03c0703d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa048036040 tx=0x7fa04802cf90 comp rx=0 tx=0).stop 2024-03-13T15:04:25.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581984d0 msgr2=0x7fa05819c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581984d0 0x7fa05819c930 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa04000c920 tx=0x7fa04000cde0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 shutdown_connections 2024-03-13T15:04:25.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fa0581984d0 0x7fa05819c930 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.515 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fa03c06df10 0x7fa03c0703d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 --2- 172.21.15.176:0/3328332262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0581024c0 0x7fa058197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 >> 172.21.15.176:0/3328332262 conn(0x7fa0580fdd40 msgr2=0x7fa0580ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:25.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 shutdown_connections 2024-03-13T15:04:25.516 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.529+0000 7fa05de00640 1 -- 172.21.15.176:0/3328332262 wait complete. 2024-03-13T15:04:25.534 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 -- 172.21.15.176:0/245393940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 msgr2=0x7fb3c40679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 --2- 172.21.15.176:0/245393940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c40679a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009880 tx=0x7fb3b402bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 -- 172.21.15.176:0/245393940 shutdown_connections 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 --2- 172.21.15.176:0/245393940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c4067f70 0x7fb3c4104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 --2- 172.21.15.176:0/245393940 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c40679a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 -- 172.21.15.176:0/245393940 >> 172.21.15.176:0/245393940 conn(0x7fb3c4074a50 msgr2=0x7fb3c4074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 -- 172.21.15.176:0/245393940 shutdown_connections 2024-03-13T15:04:25.751 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.766+0000 7fb3ca000640 1 -- 172.21.15.176:0/245393940 wait complete. 2024-03-13T15:04:25.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.767+0000 7fb3ca000640 1 Processor -- start 2024-03-13T15:04:25.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.767+0000 7fb3ca000640 1 -- start start 2024-03-13T15:04:25.753 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3ca000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3ca000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c41984c0 0x7fb3c419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3ca000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4198a30 con 0x7fb3c40675c0 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3ca000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c4198ba0 con 0x7fb3c41984c0 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3c3400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3c3400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c4197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58700/0 (socket says 172.21.15.176:58700) 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3c3400640 1 -- 172.21.15.176:0/688015196 learned_addr learned my addr 172.21.15.176:0/688015196 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3c2a00640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c41984c0 0x7fb3c419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3c3400640 1 -- 172.21.15.176:0/688015196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c41984c0 msgr2=0x7fb3c419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3c3400640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c41984c0 0x7fb3c419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.768+0000 7fb3c3400640 1 -- 172.21.15.176:0/688015196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b4009560 con 0x7fb3c40675c0 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.769+0000 7fb3c3400640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c4197f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009230 tx=0x7fb3b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.769+0000 7fb3bbe00640 1 -- 172.21.15.176:0/688015196 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b40035f0 con 0x7fb3c40675c0 2024-03-13T15:04:25.754 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.769+0000 7fb3bbe00640 1 -- 172.21.15.176:0/688015196 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3b4003c10 con 0x7fb3c40675c0 2024-03-13T15:04:25.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.769+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c419cec0 con 0x7fb3c40675c0 2024-03-13T15:04:25.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.769+0000 7fb3bbe00640 1 -- 172.21.15.176:0/688015196 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3b4034440 con 0x7fb3c40675c0 2024-03-13T15:04:25.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.769+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c419d3d0 con 0x7fb3c40675c0 2024-03-13T15:04:25.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.770+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c404e040 con 0x7fb3c40675c0 2024-03-13T15:04:25.755 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.771+0000 7fb3bbe00640 1 -- 172.21.15.176:0/688015196 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7fb3b4004fd0 con 0x7fb3c40675c0 2024-03-13T15:04:25.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.772+0000 7fb3bbe00640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb3a806df10 0x7fb3a80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:25.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.773+0000 7fb3bbe00640 1 -- 172.21.15.176:0/688015196 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7fb3b40b0230 con 0x7fb3c40675c0 2024-03-13T15:04:25.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.773+0000 7fb3c2a00640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb3a806df10 0x7fb3a80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:25.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.773+0000 7fb3c2a00640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb3a806df10 0x7fb3a80703d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4072e40 tx=0x7fb3b0005cc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:25.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.777+0000 7fb3bbe00640 1 -- 172.21.15.176:0/688015196 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7fb3b4079600 con 0x7fb3c40675c0 2024-03-13T15:04:25.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:25 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2955664215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T15:04:25.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:25 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/3328332262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T15:04:25.962 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.978+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb3c419d640 con 0x7fb3c40675c0 2024-03-13T15:04:25.963 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.979+0000 7fb3bbe00640 1 -- 172.21.15.176:0/688015196 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v112) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb3b407ccc0 con 0x7fb3c40675c0 2024-03-13T15:04:25.963 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T15:04:25.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.981+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb3a806df10 msgr2=0x7fb3a80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.981+0000 7fb3ca000640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb3a806df10 0x7fb3a80703d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4072e40 tx=0x7fb3b0005cc0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.981+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 msgr2=0x7fb3c4197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:25.966 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.981+0000 7fb3ca000640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c4197f80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009230 tx=0x7fb3b4004040 comp rx=0 tx=0).stop 2024-03-13T15:04:25.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.981+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 shutdown_connections 2024-03-13T15:04:25.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.981+0000 7fb3ca000640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7fb3a806df10 0x7fb3a80703d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.982+0000 7fb3ca000640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3c41984c0 0x7fb3c419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.982+0000 7fb3ca000640 1 --2- 172.21.15.176:0/688015196 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7fb3c40675c0 0x7fb3c4197f80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:25.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.982+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 >> 172.21.15.176:0/688015196 conn(0x7fb3c4074a50 msgr2=0x7fb3c4101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:25.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.982+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 shutdown_connections 2024-03-13T15:04:25.967 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:25.982+0000 7fb3ca000640 1 -- 172.21.15.176:0/688015196 wait complete. 2024-03-13T15:04:25.986 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-13T15:04:26.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:25 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2955664215' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T15:04:26.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:25 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/3328332262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 -- 172.21.15.176:0/902802100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100c20 msgr2=0x7f616c104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 --2- 172.21.15.176:0/902802100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100c20 0x7f616c104c60 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6154009880 tx=0x7f615402bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 -- 172.21.15.176:0/902802100 shutdown_connections 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 --2- 172.21.15.176:0/902802100 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100c20 0x7f616c104c60 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 --2- 172.21.15.176:0/902802100 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f616c100270 0x7f616c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 -- 172.21.15.176:0/902802100 >> 172.21.15.176:0/902802100 conn(0x7f616c0fbad0 msgr2=0x7f616c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 -- 172.21.15.176:0/902802100 shutdown_connections 2024-03-13T15:04:26.201 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.216+0000 7f6173a00640 1 -- 172.21.15.176:0/902802100 wait complete. 2024-03-13T15:04:26.203 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.217+0000 7f6173a00640 1 Processor -- start 2024-03-13T15:04:26.203 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.217+0000 7f6173a00640 1 -- start start 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6173a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100270 0x7f616c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6173a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f616c198430 0x7f616c19c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6173a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c198a30 con 0x7f616c100270 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6173a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c198ba0 con 0x7f616c198430 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6171000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100270 0x7f616c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6171000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100270 0x7f616c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58716/0 (socket says 172.21.15.176:58716) 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6171000640 1 -- 172.21.15.176:0/2394918175 learned_addr learned my addr 172.21.15.176:0/2394918175 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6163e00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f616c198430 0x7f616c19c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6171000640 1 -- 172.21.15.176:0/2394918175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f616c198430 msgr2=0x7f616c19c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6171000640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f616c198430 0x7f616c19c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.218+0000 7f6171000640 1 -- 172.21.15.176:0/2394918175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6154009560 con 0x7f616c100270 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.219+0000 7f6171000640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100270 0x7f616c197ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f615c00d720 tx=0x7f615c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.219+0000 7f616a000640 1 -- 172.21.15.176:0/2394918175 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f615c0051f0 con 0x7f616c100270 2024-03-13T15:04:26.204 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.219+0000 7f616a000640 1 -- 172.21.15.176:0/2394918175 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f615c009ce0 con 0x7f616c100270 2024-03-13T15:04:26.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.219+0000 7f616a000640 1 -- 172.21.15.176:0/2394918175 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f615c010460 con 0x7f616c100270 2024-03-13T15:04:26.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.219+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c19ce90 con 0x7f616c100270 2024-03-13T15:04:26.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.219+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c19d3f0 con 0x7f616c100270 2024-03-13T15:04:26.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.220+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c04e040 con 0x7f616c100270 2024-03-13T15:04:26.205 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.221+0000 7f616a000640 1 -- 172.21.15.176:0/2394918175 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f615c005350 con 0x7f616c100270 2024-03-13T15:04:26.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.222+0000 7f616a000640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f615006dfd0 0x7f6150070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:26.206 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.222+0000 7f616a000640 1 -- 172.21.15.176:0/2394918175 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f615c08da20 con 0x7f616c100270 2024-03-13T15:04:26.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.222+0000 7f6163e00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f615006dfd0 0x7f6150070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:26.207 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.223+0000 7f6163e00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f615006dfd0 0x7f6150070490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6154036040 tx=0x7f615402cf90 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:26.212 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.228+0000 7f616a000640 1 -- 172.21.15.176:0/2394918175 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f615c056d70 con 0x7f616c100270 2024-03-13T15:04:26.423 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.439+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f616c199240 con 0x7f616c100270 2024-03-13T15:04:26.424 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.440+0000 7f616a000640 1 -- 172.21.15.176:0/2394918175 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v112) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f615c05a430 con 0x7f616c100270 2024-03-13T15:04:26.424 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6827/1275358571 2024-03-14T15:01:08.961104+0000 2024-03-13T15:04:26.424 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6826/1275358571 2024-03-14T15:01:08.961104+0000 2024-03-13T15:04:26.424 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/2698132085 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:26.424 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2786891974 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:26.424 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2461806366 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:26.424 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/922386027 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6828/1297073737 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6829/1297073737 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/922386027 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/3527709806 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1319558216 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1541570168 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/248607826 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/1871453830 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1825335586 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1477585567 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1947716670 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1567438135 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2108157434 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:26.425 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/4213061990 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/3434817823 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/1711793734 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1665184531 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/1711793734 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6824/152396394 2024-03-14T15:00:43.199506+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/665298320 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/2698132085 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/3486069398 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/945284252 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/3034338010 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6825/152396394 2024-03-14T15:00:43.199506+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/698195875 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1930887076 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:26.426 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6827/3592546199 2024-03-14T14:45:59.134341+0000 2024-03-13T15:04:26.427 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/3215654415 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:26.427 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/258177446 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:26.427 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/456564682 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:26.427 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6826/3592546199 2024-03-14T14:45:59.134341+0000 2024-03-13T15:04:26.427 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2332303582 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:26.427 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/456564682 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:26.427 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/3527709806 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:listed 41 entries 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.442+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f615006dfd0 msgr2=0x7f6150070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.442+0000 7f6173a00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f615006dfd0 0x7f6150070490 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6154036040 tx=0x7f615402cf90 comp rx=0 tx=0).stop 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.442+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100270 msgr2=0x7f616c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.442+0000 7f6173a00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100270 0x7f616c197ef0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f615c00d720 tx=0x7f615c00dbe0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.443+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 shutdown_connections 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.443+0000 7f6173a00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f615006dfd0 0x7f6150070490 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.428 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.443+0000 7f6173a00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f616c198430 0x7f616c19c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.443+0000 7f6173a00640 1 --2- 172.21.15.176:0/2394918175 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f616c100270 0x7f616c197ef0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.443+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 >> 172.21.15.176:0/2394918175 conn(0x7f616c0fbad0 msgr2=0x7f616c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:26.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.443+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 shutdown_connections 2024-03-13T15:04:26.429 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.443+0000 7f6173a00640 1 -- 172.21.15.176:0/2394918175 wait complete. 2024-03-13T15:04:26.447 DEBUG:teuthology.orchestra.run.smithi176:> set -ex 2024-03-13T15:04:26.447 DEBUG:teuthology.orchestra.run.smithi176:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-13T15:04:26.495 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-13T15:04:26.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.772+0000 7f958e800640 1 -- 172.21.15.176:0/2002919810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067c70 msgr2=0x7f958810c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.772+0000 7f958e800640 1 --2- 172.21.15.176:0/2002919810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067c70 0x7f958810c650 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9570009880 tx=0x7f957002bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.772+0000 7f958e800640 1 -- 172.21.15.176:0/2002919810 shutdown_connections 2024-03-13T15:04:26.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.772+0000 7f958e800640 1 --2- 172.21.15.176:0/2002919810 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067c70 0x7f958810c650 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.772+0000 7f958e800640 1 --2- 172.21.15.176:0/2002919810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9588067350 0x7f9588067730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.757 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.772+0000 7f958e800640 1 -- 172.21.15.176:0/2002919810 >> 172.21.15.176:0/2002919810 conn(0x7f9588075b40 msgr2=0x7f9588075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:26.758 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.773+0000 7f958e800640 1 -- 172.21.15.176:0/2002919810 shutdown_connections 2024-03-13T15:04:26.758 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.773+0000 7f958e800640 1 -- 172.21.15.176:0/2002919810 wait complete. 2024-03-13T15:04:26.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.773+0000 7f958e800640 1 Processor -- start 2024-03-13T15:04:26.759 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f958e800640 1 -- start start 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f958e800640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067350 0x7f9588197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f958e800640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9588067c70 0x7f9588198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f958e800640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9588198b10 con 0x7f9588067350 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f958e800640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9588198c80 con 0x7f9588067c70 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f9587400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067350 0x7f9588197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f9587400640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067350 0x7f9588197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58726/0 (socket says 172.21.15.176:58726) 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f9587400640 1 -- 172.21.15.176:0/1739056995 learned_addr learned my addr 172.21.15.176:0/1739056995 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.774+0000 7f9586a00640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9588067c70 0x7f9588198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f9587400640 1 -- 172.21.15.176:0/1739056995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9588067c70 msgr2=0x7f9588198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f9587400640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9588067c70 0x7f9588198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f9587400640 1 -- 172.21.15.176:0/1739056995 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9570009560 con 0x7f9588067350 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f9586a00640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9588067c70 0x7f9588198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-13T15:04:26.760 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f9587400640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067350 0x7f9588197fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f958000cc60 tx=0x7f958000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:26.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f957fe00640 1 -- 172.21.15.176:0/1739056995 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958000f450 con 0x7f9588067350 2024-03-13T15:04:26.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f957fe00640 1 -- 172.21.15.176:0/1739056995 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f958000fa70 con 0x7f9588067350 2024-03-13T15:04:26.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958819cb10 con 0x7f9588067350 2024-03-13T15:04:26.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f957fe00640 1 -- 172.21.15.176:0/1739056995 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9580018b80 con 0x7f9588067350 2024-03-13T15:04:26.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.775+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958819d070 con 0x7f9588067350 2024-03-13T15:04:26.761 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.776+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958804e040 con 0x7f9588067350 2024-03-13T15:04:26.762 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.777+0000 7f957fe00640 1 -- 172.21.15.176:0/1739056995 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f9580018d00 con 0x7f9588067350 2024-03-13T15:04:26.763 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.778+0000 7f957fe00640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f956c06dfd0 0x7f956c070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:26.763 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.779+0000 7f957fe00640 1 -- 172.21.15.176:0/1739056995 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f958008e3d0 con 0x7f9588067350 2024-03-13T15:04:26.763 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.779+0000 7f9586a00640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f956c06dfd0 0x7f956c070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:26.763 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.779+0000 7f9586a00640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f956c06dfd0 0x7f956c070490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9570036040 tx=0x7f957002cb50 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:26.767 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.783+0000 7f957fe00640 1 -- 172.21.15.176:0/1739056995 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f9580057720 con 0x7f9588067350 2024-03-13T15:04:26.968 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.983+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9588109d30 con 0x7f9588067350 2024-03-13T15:04:26.971 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.987+0000 7f957fe00640 1 -- 172.21.15.176:0/1739056995 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+17190 (secure 0 0 0) 0x7f958005ade0 con 0x7f9588067350 2024-03-13T15:04:26.972 INFO:teuthology.orchestra.run.smithi176.stdout: 2024-03-13T15:04:26.972 INFO:teuthology.orchestra.run.smithi176.stdout:{"epoch":112,"fsid":"f60e35da-e146-11ee-95c9-87774f69a715","created":"2024-03-13T14:36:32.120003+0000","modified":"2024-03-13T15:01:08.961154+0000","last_up_change":"2024-03-13T14:59:42.380775+0000","last_in_change":"2024-03-13T14:43:42.351790+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-13T14:42:22.643268+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-03-13T14:45:37.278783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-03-13T14:45:37.880653+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c570fcfa-ceda-4e39-b200-5c8b0b7fab19","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":63,"up_thru":105,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6802","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6803","nonce":468040948}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6804","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6805","nonce":468040948}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6808","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6809","nonce":468040948}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6806","nonce":468040948},{"type":"v1","addr":"172.21.15.176:6807","nonce":468040948}]},"public_addr":"172.21.15.176:6803/468040948","cluster_addr":"172.21.15.176:6805/468040948","heartbeat_back_addr":"172.21.15.176:6809/468040948","heartbeat_front_addr":"172.21.15.176:6807/468040948","state":["exists","up"]},{"osd":1,"uuid":"5a973902-7ecc-4a05-b917-a74e108df4db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":71,"up_from":75,"up_thru":105,"down_at":72,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6810","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6811","nonce":2654405375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6812","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6813","nonce":2654405375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6816","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6817","nonce":2654405375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6814","nonce":2654405375},{"type":"v1","addr":"172.21.15.176:6815","nonce":2654405375}]},"public_addr":"172.21.15.176:6811/2654405375","cluster_addr":"172.21.15.176:6813/2654405375","heartbeat_back_addr":"172.21.15.176:6817/2654405375","heartbeat_front_addr":"172.21.15.176:6815/2654405375","state":["exists","up"]},{"osd":2,"uuid":"f72757d1-b949-45fb-abcd-c8649cec14f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":76,"up_from":80,"up_thru":105,"down_at":77,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6818","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6819","nonce":2651949103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6820","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6821","nonce":2651949103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6824","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6825","nonce":2651949103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.176:6822","nonce":2651949103},{"type":"v1","addr":"172.21.15.176:6823","nonce":2651949103}]},"public_addr":"172.21.15.176:6819/2651949103","cluster_addr":"172.21.15.176:6821/2651949103","heartbeat_back_addr":"172.21.15.176:6825/2651949103","heartbeat_front_addr":"172.21.15.176:6823/2651949103","state":["exists","up"]},{"osd":3,"uuid":"e1c42f2a-0efa-4c44-a965-9e5904aeace0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":27,"last_clean_end":81,"up_from":89,"up_thru":105,"down_at":82,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6801","nonce":1554416376}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6802","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6803","nonce":1554416376}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6807","nonce":1554416376}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1554416376},{"type":"v1","addr":"172.21.15.196:6805","nonce":1554416376}]},"public_addr":"172.21.15.196:6801/1554416376","cluster_addr":"172.21.15.196:6803/1554416376","heartbeat_back_addr":"172.21.15.196:6807/1554416376","heartbeat_front_addr":"172.21.15.196:6805/1554416376","state":["exists","up"]},{"osd":4,"uuid":"b4c7c89d-a349-4b39-b64a-e1080d213acd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":35,"last_clean_end":96,"up_from":100,"up_thru":105,"down_at":97,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6808","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6809","nonce":1908907228}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6811","nonce":1908907228}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6815","nonce":1908907228}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6812","nonce":1908907228},{"type":"v1","addr":"172.21.15.196:6813","nonce":1908907228}]},"public_addr":"172.21.15.196:6809/1908907228","cluster_addr":"172.21.15.196:6811/1908907228","heartbeat_back_addr":"172.21.15.196:6815/1908907228","heartbeat_front_addr":"172.21.15.196:6813/1908907228","state":["exists","up"]},{"osd":5,"uuid":"efb96c7b-7a78-4c40-996f-f8141029c310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":40,"last_clean_end":101,"up_from":105,"up_thru":105,"down_at":102,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6817","nonce":3411147075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6819","nonce":3411147075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6823","nonce":3411147075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":3411147075},{"type":"v1","addr":"172.21.15.196:6821","nonce":3411147075}]},"public_addr":"172.21.15.196:6817/3411147075","cluster_addr":"172.21.15.196:6819/3411147075","heartbeat_back_addr":"172.21.15.196:6823/3411147075","heartbeat_front_addr":"172.21.15.196:6821/3411147075","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:06.529418+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:41:43.356501+0000","dead_epoch":71},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:19.060195+0000","dead_epoch":76},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:42:53.551733+0000","dead_epoch":81},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:43:28.434416+0000","dead_epoch":96},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-13T14:44:05.009478+0000","dead_epoch":101}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.9","mappings":[{"from":0,"to":3}]},{"pgid":"2.15","mappings":[{"from":0,"to":2}]},{"pgid":"2.1e","mappings":[{"from":0,"to":3}]},{"pgid":"3.6","mappings":[{"from":1,"to":2}]},{"pgid":"3.b","mappings":[{"from":4,"to":2}]},{"pgid":"3.12","mappings":[{"from":1,"to":5}]},{"pgid":"3.1c","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.196:6827/1275358571":"2024-03-14T15:01:08.961104+0000","172.21.15.196:6826/1275358571":"2024-03-14T15:01:08.961104+0000","172.21.15.176:6800/2698132085":"2024-03-14T14:48:28.652138+0000","172.21.15.176:0/2786891974":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/2461806366":"2024-03-14T14:54:00.256736+0000","172.21.15.176:6801/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.196:6828/1297073737":"2024-03-14T14:49:27.654006+0000","172.21.15.196:6829/1297073737":"2024-03-14T14:49:27.654006+0000","172.21.15.176:6800/922386027":"2024-03-14T14:37:47.494508+0000","172.21.15.176:6800/3527709806":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/1319558216":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1541570168":"2024-03-14T14:48:28.652138+0000","172.21.15.176:0/248607826":"2024-03-14T14:37:47.494508+0000","172.21.15.196:0/1871453830":"2024-03-14T14:49:27.654006+0000","172.21.15.176:0/1825335586":"2024-03-14T14:39:27.974888+0000","172.21.15.176:0/1477585567":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1947716670":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/1567438135":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/2108157434":"2024-03-14T14:37:47.494508+0000","172.21.15.196:0/4213061990":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/3434817823":"2024-03-14T14:49:27.654006+0000","172.21.15.176:6801/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.176:0/1665184531":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6800/1711793734":"2024-03-14T14:37:05.490145+0000","172.21.15.196:6824/152396394":"2024-03-14T15:00:43.199506+0000","172.21.15.176:0/665298320":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6801/2698132085":"2024-03-14T14:48:28.652138+0000","172.21.15.196:0/3486069398":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/945284252":"2024-03-14T14:49:27.654006+0000","172.21.15.196:0/3034338010":"2024-03-14T14:49:27.654006+0000","172.21.15.196:6825/152396394":"2024-03-14T15:00:43.199506+0000","172.21.15.176:0/698195875":"2024-03-14T14:37:47.494508+0000","172.21.15.176:0/1930887076":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6827/3592546199":"2024-03-14T14:45:59.134341+0000","172.21.15.176:0/3215654415":"2024-03-14T14:54:00.256736+0000","172.21.15.176:0/258177446":"2024-03-14T14:48:28.652138+0000","172.21.15.176:6800/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6826/3592546199":"2024-03-14T14:45:59.134341+0000","172.21.15.176:0/2332303582":"2024-03-14T14:37:05.490145+0000","172.21.15.176:6801/456564682":"2024-03-14T14:39:27.974888+0000","172.21.15.176:6801/3527709806":"2024-03-14T14:54:00.256736+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f956c06dfd0 msgr2=0x7f956c070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f956c06dfd0 0x7f956c070490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9570036040 tx=0x7f957002cb50 comp rx=0 tx=0).stop 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067350 msgr2=0x7f9588197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067350 0x7f9588197fd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f958000cc60 tx=0x7f958000a620 comp rx=0 tx=0).stop 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 shutdown_connections 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f956c06dfd0 0x7f956c070490 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9588067c70 0x7f9588198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 --2- 172.21.15.176:0/1739056995 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9588067350 0x7f9588197fd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 >> 172.21.15.176:0/1739056995 conn(0x7f9588075b40 msgr2=0x7f95880fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 shutdown_connections 2024-03-13T15:04:26.975 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:26.990+0000 7f958e800640 1 -- 172.21.15.176:0/1739056995 wait complete. 2024-03-13T15:04:26.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-13T14:42:22.643268+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-03-13T14:45:37.278783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-03-13T14:45:37.880653+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-13T15:04:26.994 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-13T15:04:27.076 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:26 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/688015196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T15:04:27.077 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:26 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2394918175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-13T15:04:27.077 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:26 smithi176 ceph-mon[112742]: pgmap v378: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:27.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:26 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/688015196' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T15:04:27.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:26 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2394918175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-13T15:04:27.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:26 smithi196 ceph-mon[86193]: pgmap v378: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:27.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 -- 172.21.15.176:0/4003151967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2101022e0 msgr2=0x7ff2101026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.216 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 --2- 172.21.15.176:0/4003151967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2101022e0 0x7ff2101026c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff204009880 tx=0x7ff20402bce0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 -- 172.21.15.176:0/4003151967 shutdown_connections 2024-03-13T15:04:27.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 --2- 172.21.15.176:0/4003151967 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff210102c90 0x7ff210106cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 --2- 172.21.15.176:0/4003151967 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff2101022e0 0x7ff2101026c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 -- 172.21.15.176:0/4003151967 >> 172.21.15.176:0/4003151967 conn(0x7ff2100fdb60 msgr2=0x7ff2100fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:27.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 -- 172.21.15.176:0/4003151967 shutdown_connections 2024-03-13T15:04:27.217 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.232+0000 7ff216a00640 1 -- 172.21.15.176:0/4003151967 wait complete. 2024-03-13T15:04:27.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.233+0000 7ff216a00640 1 Processor -- start 2024-03-13T15:04:27.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.233+0000 7ff216a00640 1 -- start start 2024-03-13T15:04:27.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.233+0000 7ff216a00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2101022e0 0x7ff210197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:27.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.233+0000 7ff216a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff210198310 0x7ff21019c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:27.218 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.233+0000 7ff216a00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff210198910 con 0x7ff210198310 2024-03-13T15:04:27.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.233+0000 7ff216a00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff210198a80 con 0x7ff2101022e0 2024-03-13T15:04:27.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.234+0000 7ff214c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff210198310 0x7ff21019c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:27.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.234+0000 7ff214c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff210198310 0x7ff21019c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58742/0 (socket says 172.21.15.176:58742) 2024-03-13T15:04:27.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.234+0000 7ff214c00640 1 -- 172.21.15.176:0/2442855649 learned_addr learned my addr 172.21.15.176:0/2442855649 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:27.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.234+0000 7ff214c00640 1 -- 172.21.15.176:0/2442855649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2101022e0 msgr2=0x7ff210197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.234+0000 7ff214c00640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2101022e0 0x7ff210197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.219 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.234+0000 7ff214c00640 1 -- 172.21.15.176:0/2442855649 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204009560 con 0x7ff210198310 2024-03-13T15:04:27.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.234+0000 7ff214c00640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff210198310 0x7ff21019c770 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff20000d6f0 tx=0x7ff20000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:27.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.235+0000 7ff20e000640 1 -- 172.21.15.176:0/2442855649 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2000051f0 con 0x7ff210198310 2024-03-13T15:04:27.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.235+0000 7ff20e000640 1 -- 172.21.15.176:0/2442855649 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff200009ce0 con 0x7ff210198310 2024-03-13T15:04:27.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.235+0000 7ff20e000640 1 -- 172.21.15.176:0/2442855649 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff200004e80 con 0x7ff210198310 2024-03-13T15:04:27.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.235+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21019cd70 con 0x7ff210198310 2024-03-13T15:04:27.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.235+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21019d2d0 con 0x7ff210198310 2024-03-13T15:04:27.220 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.236+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21019d5a0 con 0x7ff210198310 2024-03-13T15:04:27.221 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.237+0000 7ff20e000640 1 -- 172.21.15.176:0/2442855649 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7ff2000046c0 con 0x7ff210198310 2024-03-13T15:04:27.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.238+0000 7ff20e000640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff1f806deb0 0x7ff1f8070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:27.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.238+0000 7ff20e000640 1 -- 172.21.15.176:0/2442855649 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7ff20008d9a0 con 0x7ff210198310 2024-03-13T15:04:27.222 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.238+0000 7ff215600640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff1f806deb0 0x7ff1f8070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:27.223 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.239+0000 7ff215600640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff1f806deb0 0x7ff1f8070370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff20402efd0 tx=0x7ff204000bc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:27.227 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.243+0000 7ff20e000640 1 -- 172.21.15.176:0/2442855649 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7ff200056c50 con 0x7ff210198310 2024-03-13T15:04:27.426 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.441+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff210199180 con 0x7ff210198310 2024-03-13T15:04:27.427 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.443+0000 7ff20e000640 1 -- 172.21.15.176:0/2442855649 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff20005a310 con 0x7ff210198310 2024-03-13T15:04:27.427 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 1 2024-03-13T15:04:27.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff1f806deb0 msgr2=0x7ff1f8070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff1f806deb0 0x7ff1f8070370 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff20402efd0 tx=0x7ff204000bc0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff210198310 msgr2=0x7ff21019c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff210198310 0x7ff21019c770 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff20000d6f0 tx=0x7ff20000dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 shutdown_connections 2024-03-13T15:04:27.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7ff210198310 0x7ff21019c770 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.430 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7ff1f806deb0 0x7ff1f8070370 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 --2- 172.21.15.176:0/2442855649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2101022e0 0x7ff210197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.445+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 >> 172.21.15.176:0/2442855649 conn(0x7ff2100fdb60 msgr2=0x7ff2100ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:27.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.446+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 shutdown_connections 2024-03-13T15:04:27.431 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.446+0000 7ff216a00640 1 -- 172.21.15.176:0/2442855649 wait complete. 2024-03-13T15:04:27.448 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-03-13T15:04:27.666 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 -- 172.21.15.176:0/382220528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92881024c0 msgr2=0x7f92881028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.666 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 --2- 172.21.15.176:0/382220528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92881024c0 0x7f92881028a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9278009c20 tx=0x7f927802c1b0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 -- 172.21.15.176:0/382220528 shutdown_connections 2024-03-13T15:04:27.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 --2- 172.21.15.176:0/382220528 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9288102e70 0x7f9288106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 --2- 172.21.15.176:0/382220528 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f92881024c0 0x7f92881028a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 -- 172.21.15.176:0/382220528 >> 172.21.15.176:0/382220528 conn(0x7f92880fdd40 msgr2=0x7f9288100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:27.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 -- 172.21.15.176:0/382220528 shutdown_connections 2024-03-13T15:04:27.667 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.682+0000 7f928e000640 1 -- 172.21.15.176:0/382220528 wait complete. 2024-03-13T15:04:27.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.683+0000 7f928e000640 1 Processor -- start 2024-03-13T15:04:27.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.683+0000 7f928e000640 1 -- start start 2024-03-13T15:04:27.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f928e000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92881024c0 0x7f9288198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:27.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f928e000640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9288198560 0x7f928819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:27.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f928e000640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288198b60 con 0x7f9288198560 2024-03-13T15:04:27.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f928e000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9288198cd0 con 0x7f92881024c0 2024-03-13T15:04:27.668 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f927ea00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9288198560 0x7f928819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:27.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f927ea00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9288198560 0x7f928819c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58766/0 (socket says 172.21.15.176:58766) 2024-03-13T15:04:27.669 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f927ea00640 1 -- 172.21.15.176:0/1450332695 learned_addr learned my addr 172.21.15.176:0/1450332695 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.684+0000 7f9287400640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92881024c0 0x7f9288198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f927ea00640 1 -- 172.21.15.176:0/1450332695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92881024c0 msgr2=0x7f9288198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f927ea00640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92881024c0 0x7f9288198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f927ea00640 1 -- 172.21.15.176:0/1450332695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9278009900 con 0x7f9288198560 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f927ea00640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9288198560 0x7f928819c9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f927000c920 tx=0x7f927000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f9284c00640 1 -- 172.21.15.176:0/1450332695 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9270007b80 con 0x7f9288198560 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f9284c00640 1 -- 172.21.15.176:0/1450332695 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92700054e0 con 0x7f9288198560 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928819cfc0 con 0x7f9288198560 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f9284c00640 1 -- 172.21.15.176:0/1450332695 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f927000f450 con 0x7f9288198560 2024-03-13T15:04:27.670 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.685+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928819d520 con 0x7f9288198560 2024-03-13T15:04:27.671 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.687+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928819d800 con 0x7f9288198560 2024-03-13T15:04:27.672 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.688+0000 7f9284c00640 1 -- 172.21.15.176:0/1450332695 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f92700046c0 con 0x7f9288198560 2024-03-13T15:04:27.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.688+0000 7f9284c00640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f926c06df10 0x7f926c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:27.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.689+0000 7f9284c00640 1 -- 172.21.15.176:0/1450332695 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f927008dcc0 con 0x7f9288198560 2024-03-13T15:04:27.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.689+0000 7f9287400640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f926c06df10 0x7f926c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:27.673 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.689+0000 7f9287400640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f926c06df10 0x7f926c0703d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9278000f80 tx=0x7f9278034d30 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:27.677 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.693+0000 7f9284c00640 1 -- 172.21.15.176:0/1450332695 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f9270057010 con 0x7f9288198560 2024-03-13T15:04:27.872 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.888+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f9288199370 con 0x7f9288198560 2024-03-13T15:04:27.873 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.888+0000 7f9284c00640 1 -- 172.21.15.176:0/1450332695 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v112) v1 ==== 107+0+11 (secure 0 0 0) 0x7f927005a6d0 con 0x7f9288198560 2024-03-13T15:04:27.873 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f926c06df10 msgr2=0x7f926c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f926c06df10 0x7f926c0703d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9278000f80 tx=0x7f9278034d30 comp rx=0 tx=0).stop 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9288198560 msgr2=0x7f928819c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9288198560 0x7f928819c9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f927000c920 tx=0x7f927000cde0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 shutdown_connections 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f9288198560 0x7f928819c9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f926c06df10 0x7f926c0703d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 --2- 172.21.15.176:0/1450332695 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92881024c0 0x7f9288198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 >> 172.21.15.176:0/1450332695 conn(0x7f92880fdd40 msgr2=0x7f92880ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 shutdown_connections 2024-03-13T15:04:27.876 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:27.891+0000 7f928e000640 1 -- 172.21.15.176:0/1450332695 wait complete. 2024-03-13T15:04:27.893 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-03-13T15:04:28.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 -- 172.21.15.176:0/2244118854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 msgr2=0x7f343c106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 --2- 172.21.15.176:0/2244118854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c106b20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f342c009880 tx=0x7f342c02bcf0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 -- 172.21.15.176:0/2244118854 shutdown_connections 2024-03-13T15:04:28.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 --2- 172.21.15.176:0/2244118854 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c106b20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 --2- 172.21.15.176:0/2244118854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f343c10c660 0x7f343c10ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 -- 172.21.15.176:0/2244118854 >> 172.21.15.176:0/2244118854 conn(0x7f343c075e80 msgr2=0x7f343c0782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:28.107 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 -- 172.21.15.176:0/2244118854 shutdown_connections 2024-03-13T15:04:28.108 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.122+0000 7f3443c00640 1 -- 172.21.15.176:0/2244118854 wait complete. 2024-03-13T15:04:28.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.123+0000 7f3443c00640 1 Processor -- start 2024-03-13T15:04:28.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.123+0000 7f3443c00640 1 -- start start 2024-03-13T15:04:28.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3443c00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:28.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3443c00640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f343c10c660 0x7f343c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:28.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3443c00640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c19ced0 con 0x7f343c1066a0 2024-03-13T15:04:28.109 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3443c00640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c1a0ca0 con 0x7f343c10c660 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3441200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3441200640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58780/0 (socket says 172.21.15.176:58780) 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3441200640 1 -- 172.21.15.176:0/878235318 learned_addr learned my addr 172.21.15.176:0/878235318 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3441200640 1 -- 172.21.15.176:0/878235318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f343c10c660 msgr2=0x7f343c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3441200640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f343c10c660 0x7f343c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.124+0000 7f3441200640 1 -- 172.21.15.176:0/878235318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c009560 con 0x7f343c1066a0 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.125+0000 7f3441200640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c19c300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f342400d8d0 tx=0x7f342400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.125+0000 7f343a000640 1 -- 172.21.15.176:0/878235318 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342400bea0 con 0x7f343c1066a0 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.125+0000 7f343a000640 1 -- 172.21.15.176:0/878235318 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3424009c00 con 0x7f343c1066a0 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.125+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c1a0f80 con 0x7f343c1066a0 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.125+0000 7f343a000640 1 -- 172.21.15.176:0/878235318 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f342400f710 con 0x7f343c1066a0 2024-03-13T15:04:28.110 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.125+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c1a14e0 con 0x7f343c1066a0 2024-03-13T15:04:28.111 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.126+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c04e040 con 0x7f343c1066a0 2024-03-13T15:04:28.111 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.127+0000 7f343a000640 1 -- 172.21.15.176:0/878235318 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f342400f990 con 0x7f343c1066a0 2024-03-13T15:04:28.112 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.128+0000 7f343a000640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f341c06df10 0x7f341c0703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:28.112 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.128+0000 7f343a000640 1 -- 172.21.15.176:0/878235318 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f342408dbe0 con 0x7f343c1066a0 2024-03-13T15:04:28.112 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.128+0000 7f3433e00640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f341c06df10 0x7f341c0703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:28.113 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.129+0000 7f3433e00640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f341c06df10 0x7f341c0703d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f342c036040 tx=0x7f342c000bc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:28.117 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.133+0000 7f343a000640 1 -- 172.21.15.176:0/878235318 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f3424056f30 con 0x7f343c1066a0 2024-03-13T15:04:28.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:27 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1739056995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T15:04:28.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:27 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/2442855649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T15:04:28.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:27 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1739056995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-13T15:04:28.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:27 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/2442855649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-13T15:04:28.313 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.328+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f343c10ebc0 con 0x7f343c1066a0 2024-03-13T15:04:28.313 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.329+0000 7f343a000640 1 -- 172.21.15.176:0/878235318 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v112) v1 ==== 107+0+11 (secure 0 0 0) 0x7f342405a5f0 con 0x7f343c1066a0 2024-03-13T15:04:28.313 INFO:teuthology.orchestra.run.smithi176.stdout:pg_num: 32 2024-03-13T15:04:28.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.331+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f341c06df10 msgr2=0x7f341c0703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.331+0000 7f3443c00640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f341c06df10 0x7f341c0703d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f342c036040 tx=0x7f342c000bc0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.316 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.331+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 msgr2=0x7f343c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.331+0000 7f3443c00640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c19c300 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f342400d8d0 tx=0x7f342400dd90 comp rx=0 tx=0).stop 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.332+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 shutdown_connections 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.332+0000 7f3443c00640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f341c06df10 0x7f341c0703d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.332+0000 7f3443c00640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f343c10c660 0x7f343c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.332+0000 7f3443c00640 1 --2- 172.21.15.176:0/878235318 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f343c1066a0 0x7f343c19c300 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.332+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 >> 172.21.15.176:0/878235318 conn(0x7f343c075e80 msgr2=0x7f343c076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.332+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 shutdown_connections 2024-03-13T15:04:28.317 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.332+0000 7f3443c00640 1 -- 172.21.15.176:0/878235318 wait complete. 2024-03-13T15:04:28.334 DEBUG:teuthology.orchestra.run.smithi176:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-13T15:04:28.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 -- 172.21.15.176:0/3906516358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 msgr2=0x7f86d4104300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 --2- 172.21.15.176:0/3906516358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4104300 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f86c4009c20 tx=0x7f86c402c1b0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 -- 172.21.15.176:0/3906516358 shutdown_connections 2024-03-13T15:04:28.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 --2- 172.21.15.176:0/3906516358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86d40fefe0 0x7f86d40ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 --2- 172.21.15.176:0/3906516358 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4104300 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.539 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 -- 172.21.15.176:0/3906516358 >> 172.21.15.176:0/3906516358 conn(0x7f86d4075b40 msgr2=0x7f86d4075f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:28.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 -- 172.21.15.176:0/3906516358 shutdown_connections 2024-03-13T15:04:28.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.554+0000 7f86da600640 1 -- 172.21.15.176:0/3906516358 wait complete. 2024-03-13T15:04:28.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.556+0000 7f86da600640 1 Processor -- start 2024-03-13T15:04:28.540 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.556+0000 7f86da600640 1 -- start start 2024-03-13T15:04:28.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.556+0000 7f86da600640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86d40fefe0 0x7f86d4197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:28.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.556+0000 7f86da600640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:28.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.556+0000 7f86da600640 1 -- --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d4198b00 con 0x7f86d4103f20 2024-03-13T15:04:28.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.556+0000 7f86da600640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d419c8d0 con 0x7f86d40fefe0 2024-03-13T15:04:28.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d2a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:28.541 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d2a00640 1 --2- >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.176:3300/0 says I am v2:172.21.15.176:58794/0 (socket says 172.21.15.176:58794) 2024-03-13T15:04:28.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d2a00640 1 -- 172.21.15.176:0/27225665 learned_addr learned my addr 172.21.15.176:0/27225665 (peer_addr_for_me v2:172.21.15.176:0/0) 2024-03-13T15:04:28.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d3400640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86d40fefe0 0x7f86d4197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:28.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d2a00640 1 -- 172.21.15.176:0/27225665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86d40fefe0 msgr2=0x7f86d4197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d2a00640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86d40fefe0 0x7f86d4197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.542 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d2a00640 1 -- 172.21.15.176:0/27225665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4009900 con 0x7f86d4103f20 2024-03-13T15:04:28.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.557+0000 7f86d2a00640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4198470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00d6f0 tx=0x7f86bc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:28.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.558+0000 7f86cbe00640 1 -- 172.21.15.176:0/27225665 <== mon.0 v2:172.21.15.176:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc009a30 con 0x7f86d4103f20 2024-03-13T15:04:28.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.558+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d419cbb0 con 0x7f86d4103f20 2024-03-13T15:04:28.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.558+0000 7f86cbe00640 1 -- 172.21.15.176:0/27225665 <== mon.0 v2:172.21.15.176:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86bc0054e0 con 0x7f86d4103f20 2024-03-13T15:04:28.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.558+0000 7f86cbe00640 1 -- 172.21.15.176:0/27225665 <== mon.0 v2:172.21.15.176:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86bc010460 con 0x7f86d4103f20 2024-03-13T15:04:28.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.558+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d419d110 con 0x7f86d4103f20 2024-03-13T15:04:28.543 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.559+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d404e040 con 0x7f86d4103f20 2024-03-13T15:04:28.544 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.560+0000 7f86cbe00640 1 -- 172.21.15.176:0/27225665 <== mon.0 v2:172.21.15.176:3300/0 4 ==== mgrmap(e 39) v1 ==== 91843+0+0 (secure 0 0 0) 0x7f86bc009b90 con 0x7f86d4103f20 2024-03-13T15:04:28.545 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.561+0000 7f86cbe00640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f86b806dfd0 0x7f86b8070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-13T15:04:28.545 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.561+0000 7f86cbe00640 1 -- 172.21.15.176:0/27225665 <== mon.0 v2:172.21.15.176:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6789+0+0 (secure 0 0 0) 0x7f86bc08dac0 con 0x7f86d4103f20 2024-03-13T15:04:28.546 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.561+0000 7f86d3400640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f86b806dfd0 0x7f86b8070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-13T15:04:28.546 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.562+0000 7f86d3400640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f86b806dfd0 0x7f86b8070490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f86c4000f80 tx=0x7f86c4032fc0 comp rx=0 tx=0).ready entity=mgr.24551 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-13T15:04:28.551 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.566+0000 7f86cbe00640 1 -- 172.21.15.176:0/27225665 <== mon.0 v2:172.21.15.176:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184781 (secure 0 0 0) 0x7f86bc056d70 con 0x7f86d4103f20 2024-03-13T15:04:28.763 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.779+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 --> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f86d4199280 con 0x7f86d4103f20 2024-03-13T15:04:28.768 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.783+0000 7f86cbe00640 1 -- 172.21.15.176:0/27225665 <== mon.0 v2:172.21.15.176:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v112) v1 ==== 81+0+2462 (secure 0 0 0) 0x7f86bc05a430 con 0x7f86d4103f20 2024-03-13T15:04:28.768 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6827/1275358571 2024-03-14T15:01:08.961104+0000 2024-03-13T15:04:28.768 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6826/1275358571 2024-03-14T15:01:08.961104+0000 2024-03-13T15:04:28.768 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/2698132085 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:28.768 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2786891974 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:28.768 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2461806366 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:28.768 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/922386027 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6828/1297073737 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6829/1297073737 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/922386027 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/3527709806 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1319558216 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1541570168 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/248607826 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/1871453830 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1825335586 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1477585567 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1947716670 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1567438135 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2108157434 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/4213061990 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/3434817823 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.769 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/1711793734 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1665184531 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/1711793734 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6824/152396394 2024-03-14T15:00:43.199506+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/665298320 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/2698132085 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/3486069398 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/945284252 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:0/3034338010 2024-03-14T14:49:27.654006+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.196:6825/152396394 2024-03-14T15:00:43.199506+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/698195875 2024-03-14T14:37:47.494508+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/1930887076 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6827/3592546199 2024-03-14T14:45:59.134341+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/3215654415 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:28.770 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/258177446 2024-03-14T14:48:28.652138+0000 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6800/456564682 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6826/3592546199 2024-03-14T14:45:59.134341+0000 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:0/2332303582 2024-03-14T14:37:05.490145+0000 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/456564682 2024-03-14T14:39:27.974888+0000 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stdout:172.21.15.176:6801/3527709806 2024-03-14T14:54:00.256736+0000 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stderr:listed 41 entries 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f86b806dfd0 msgr2=0x7f86b8070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.771 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f86b806dfd0 0x7f86b8070490 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f86c4000f80 tx=0x7f86c4032fc0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 msgr2=0x7f86d4198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4198470 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00d6f0 tx=0x7f86bc00dbb0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 shutdown_connections 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:3300/0,v1:172.21.15.176:6789/0] conn(0x7f86d4103f20 0x7f86d4198470 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.176:6800/1274703396,v1:172.21.15.176:6801/1274703396] conn(0x7f86b806dfd0 0x7f86b8070490 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 --2- 172.21.15.176:0/27225665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f86d40fefe0 0x7f86d4197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 >> 172.21.15.176:0/27225665 conn(0x7f86d4075b40 msgr2=0x7f86d4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.786+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 shutdown_connections 2024-03-13T15:04:28.772 INFO:teuthology.orchestra.run.smithi176.stderr:2024-03-13T15:04:28.787+0000 7f86da600640 1 -- 172.21.15.176:0/27225665 wait complete. 2024-03-13T15:04:28.790 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi176.front.sepia.ceph.com... 2024-03-13T15:04:28.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-13T15:04:28.790 DEBUG:teuthology.orchestra.run.smithi176:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-13T15:04:28.884 INFO:teuthology.orchestra.run:waiting for 300 2024-03-13T15:04:29.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:28 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/1450332695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T15:04:29.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:28 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/878235318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T15:04:29.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:28 smithi176 ceph-mon[112742]: pgmap v379: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:29.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:28 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/1450332695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-03-13T15:04:29.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:28 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/878235318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-03-13T15:04:29.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:28 smithi196 ceph-mon[86193]: pgmap v379: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:30.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:29 smithi176 ceph-mon[112742]: from='client.? 172.21.15.176:0/27225665' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-13T15:04:30.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:29 smithi196 ceph-mon[86193]: from='client.? 172.21.15.176:0/27225665' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-13T15:04:31.077 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:31.077 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:30 smithi196 ceph-mon[86193]: pgmap v380: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:31.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:31.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:30 smithi176 ceph-mon[112742]: pgmap v380: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:33.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:33 smithi176 ceph-mon[112742]: pgmap v381: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:33.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:33 smithi196 ceph-mon[86193]: pgmap v381: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:35.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:35 smithi176 ceph-mon[112742]: pgmap v382: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:35.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:35 smithi196 ceph-mon[86193]: pgmap v382: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:04:37.375 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:36 smithi176 ceph-mon[112742]: pgmap v383: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:37.425 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:36 smithi196 ceph-mon[86193]: pgmap v383: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:39.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:39 smithi176 ceph-mon[112742]: pgmap v384: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:39.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:39 smithi196 ceph-mon[86193]: pgmap v384: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:41.675 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:41 smithi196 ceph-mon[86193]: pgmap v385: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:41.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:41 smithi176 ceph-mon[112742]: pgmap v385: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:43.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:43 smithi176 ceph-mon[112742]: pgmap v386: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:43.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:43 smithi196 ceph-mon[86193]: pgmap v386: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:45.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:45 smithi176 ceph-mon[112742]: pgmap v387: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:45.876 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:45.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:45 smithi196 ceph-mon[86193]: pgmap v387: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:45.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:04:47.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:47 smithi176 ceph-mon[112742]: pgmap v388: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:47.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:47 smithi196 ceph-mon[86193]: pgmap v388: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:49.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:49 smithi176 ceph-mon[112742]: pgmap v389: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:49.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:49 smithi196 ceph-mon[86193]: pgmap v389: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:51.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:51 smithi176 ceph-mon[112742]: pgmap v390: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:51.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:51 smithi196 ceph-mon[86193]: pgmap v390: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:53.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:53 smithi176 ceph-mon[112742]: pgmap v391: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:53.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:53 smithi196 ceph-mon[86193]: pgmap v391: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:55.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:55 smithi176 ceph-mon[112742]: pgmap v392: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:55.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:55 smithi196 ceph-mon[86193]: pgmap v392: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:04:57.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:57 smithi196 ceph-mon[86193]: pgmap v393: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:58.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:57 smithi176 ceph-mon[112742]: pgmap v393: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:04:59.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:04:59 smithi196 ceph-mon[86193]: pgmap v394: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:00.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:04:59 smithi176 ceph-mon[112742]: pgmap v394: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:00.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:01.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:01 smithi196 ceph-mon[86193]: pgmap v395: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:02.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:01 smithi176 ceph-mon[112742]: pgmap v395: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:02.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:02 smithi196 ceph-mon[86193]: pgmap v396: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:03.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:02 smithi176 ceph-mon[112742]: pgmap v396: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:05.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:05 smithi176 ceph-mon[112742]: pgmap v397: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:05.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:05 smithi196 ceph-mon[86193]: pgmap v397: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:07.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:07 smithi176 ceph-mon[112742]: pgmap v398: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:07.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:07 smithi196 ceph-mon[86193]: pgmap v398: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:09.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:09 smithi176 ceph-mon[112742]: pgmap v399: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:09.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:09 smithi196 ceph-mon[86193]: pgmap v399: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:11.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:11 smithi176 ceph-mon[112742]: pgmap v400: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:11.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:11 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:05:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:11 smithi196 ceph-mon[86193]: pgmap v400: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:11 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:05:12.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:12 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:05:12.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:12 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:05:12.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:12 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:05:12.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:12 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:05:12.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:12 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:05:12.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:12 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:05:13.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:13 smithi176 ceph-mon[112742]: pgmap v401: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:13.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:13 smithi196 ceph-mon[86193]: pgmap v401: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:15.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:15 smithi176 ceph-mon[112742]: pgmap v402: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:15.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:15 smithi196 ceph-mon[86193]: pgmap v402: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:17.675 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:17 smithi196 ceph-mon[86193]: pgmap v403: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:18.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:17 smithi176 ceph-mon[112742]: pgmap v403: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:19.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:19 smithi196 ceph-mon[86193]: pgmap v404: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:20.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:19 smithi176 ceph-mon[112742]: pgmap v404: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:21.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:21 smithi196 ceph-mon[86193]: pgmap v405: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:22.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:21 smithi176 ceph-mon[112742]: pgmap v405: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:23.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:23 smithi196 ceph-mon[86193]: pgmap v406: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:05:24.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:23 smithi176 ceph-mon[112742]: pgmap v406: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:05:25.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:24 smithi176 ceph-mon[112742]: pgmap v407: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:05:25.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:24 smithi196 ceph-mon[86193]: pgmap v407: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:05:27.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:27 smithi176 ceph-mon[112742]: pgmap v408: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:27.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:27 smithi196 ceph-mon[86193]: pgmap v408: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:29.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:29 smithi176 ceph-mon[112742]: pgmap v409: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:05:29.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:29 smithi196 ceph-mon[86193]: pgmap v409: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 767 B/s rd, 1 op/s 2024-03-13T15:05:30.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:30.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:31.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:31 smithi176 ceph-mon[112742]: pgmap v410: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:31.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:31 smithi196 ceph-mon[86193]: pgmap v410: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:33.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:33 smithi176 ceph-mon[112742]: pgmap v411: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:33.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:33 smithi196 ceph-mon[86193]: pgmap v411: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:35.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:35 smithi196 ceph-mon[86193]: pgmap v412: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:36.009 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:35 smithi176 ceph-mon[112742]: pgmap v412: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:37.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:37 smithi196 ceph-mon[86193]: pgmap v413: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:38.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:37 smithi176 ceph-mon[112742]: pgmap v413: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:39.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:39 smithi196 ceph-mon[86193]: pgmap v414: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:40.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:39 smithi176 ceph-mon[112742]: pgmap v414: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:41.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:41 smithi196 ceph-mon[86193]: pgmap v415: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:42.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:41 smithi176 ceph-mon[112742]: pgmap v415: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:43.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:43 smithi196 ceph-mon[86193]: pgmap v416: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:44.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:43 smithi176 ceph-mon[112742]: pgmap v416: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:45.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:44 smithi176 ceph-mon[112742]: pgmap v417: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:45.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:44 smithi196 ceph-mon[86193]: pgmap v417: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:05:46.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:46.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:05:47.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:46 smithi176 ceph-mon[112742]: pgmap v418: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:47.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:46 smithi196 ceph-mon[86193]: pgmap v418: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:49.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:49 smithi176 ceph-mon[112742]: pgmap v419: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:49.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:49 smithi196 ceph-mon[86193]: pgmap v419: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:51.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:51 smithi176 ceph-mon[112742]: pgmap v420: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:51.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:51 smithi196 ceph-mon[86193]: pgmap v420: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:53.675 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:53 smithi196 ceph-mon[86193]: pgmap v421: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:53.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:53 smithi176 ceph-mon[112742]: pgmap v421: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:55.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:55 smithi176 ceph-mon[112742]: pgmap v422: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:55.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:55 smithi196 ceph-mon[86193]: pgmap v422: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:05:57.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:57 smithi196 ceph-mon[86193]: pgmap v423: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:58.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:57 smithi176 ceph-mon[112742]: pgmap v423: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:05:59.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:05:59 smithi196 ceph-mon[86193]: pgmap v424: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:00.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:05:59 smithi176 ceph-mon[112742]: pgmap v424: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:00.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:01.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:01.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:01 smithi196 ceph-mon[86193]: pgmap v425: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:02.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:01 smithi176 ceph-mon[112742]: pgmap v425: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:03.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:03 smithi196 ceph-mon[86193]: pgmap v426: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:04.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:03 smithi176 ceph-mon[112742]: pgmap v426: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:05.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:05 smithi196 ceph-mon[86193]: pgmap v427: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:06.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:05 smithi176 ceph-mon[112742]: pgmap v427: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:06.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:06 smithi196 ceph-mon[86193]: pgmap v428: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:07.038 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:06 smithi176 ceph-mon[112742]: pgmap v428: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:09.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:09 smithi176 ceph-mon[112742]: pgmap v429: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:09.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:09 smithi196 ceph-mon[86193]: pgmap v429: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:11.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:11 smithi176 ceph-mon[112742]: pgmap v430: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:11 smithi196 ceph-mon[86193]: pgmap v430: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:13.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:13 smithi176 ceph-mon[112742]: pgmap v431: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:13.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:13 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:06:13.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:13 smithi196 ceph-mon[86193]: pgmap v431: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:13.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:13 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:06:14.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:14 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:06:14.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:14 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:06:14.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:14 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:06:15.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:14 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:06:15.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:14 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:06:15.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:14 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:06:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:15 smithi196 ceph-mon[86193]: pgmap v432: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:16.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:15 smithi176 ceph-mon[112742]: pgmap v432: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:16.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:17.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:17 smithi196 ceph-mon[86193]: pgmap v433: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:18.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:17 smithi176 ceph-mon[112742]: pgmap v433: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:19.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:19 smithi196 ceph-mon[86193]: pgmap v434: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:20.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:19 smithi176 ceph-mon[112742]: pgmap v434: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:21.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:21 smithi196 ceph-mon[86193]: pgmap v435: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:22.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:21 smithi176 ceph-mon[112742]: pgmap v435: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:23.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:23 smithi196 ceph-mon[86193]: pgmap v436: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:24.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:23 smithi176 ceph-mon[112742]: pgmap v436: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:25.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:25 smithi196 ceph-mon[86193]: pgmap v437: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:26.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:25 smithi176 ceph-mon[112742]: pgmap v437: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:27.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:26 smithi176 ceph-mon[112742]: pgmap v438: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:27.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:26 smithi196 ceph-mon[86193]: pgmap v438: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:29.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:29 smithi176 ceph-mon[112742]: pgmap v439: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:29.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:29 smithi196 ceph-mon[86193]: pgmap v439: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:30.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:31.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:31.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:31 smithi196 ceph-mon[86193]: pgmap v440: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:32.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:31 smithi176 ceph-mon[112742]: pgmap v440: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:33.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:33 smithi196 ceph-mon[86193]: pgmap v441: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:34.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:33 smithi176 ceph-mon[112742]: pgmap v441: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:35.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:35 smithi196 ceph-mon[86193]: pgmap v442: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:36.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:35 smithi176 ceph-mon[112742]: pgmap v442: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:37.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:37 smithi196 ceph-mon[86193]: pgmap v443: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:38.066 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:37 smithi176 ceph-mon[112742]: pgmap v443: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:39.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:39 smithi196 ceph-mon[86193]: pgmap v444: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:40.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:39 smithi176 ceph-mon[112742]: pgmap v444: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:41.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:41 smithi196 ceph-mon[86193]: pgmap v445: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:42.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:41 smithi176 ceph-mon[112742]: pgmap v445: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:43.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:43 smithi196 ceph-mon[86193]: pgmap v446: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:44.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:43 smithi176 ceph-mon[112742]: pgmap v446: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:45.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:44 smithi176 ceph-mon[112742]: pgmap v447: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:45.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:44 smithi196 ceph-mon[86193]: pgmap v447: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:46.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:46.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:06:47.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:46 smithi176 ceph-mon[112742]: pgmap v448: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:47.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:46 smithi196 ceph-mon[86193]: pgmap v448: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:49.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:49 smithi196 ceph-mon[86193]: pgmap v449: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:50.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:49 smithi176 ceph-mon[112742]: pgmap v449: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:51.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:51 smithi196 ceph-mon[86193]: pgmap v450: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:52.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:51 smithi176 ceph-mon[112742]: pgmap v450: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:53.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:53 smithi176 ceph-mon[112742]: pgmap v451: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:53.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:53 smithi196 ceph-mon[86193]: pgmap v451: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:06:55.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:55 smithi196 ceph-mon[86193]: pgmap v452: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:56.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:55 smithi176 ceph-mon[112742]: pgmap v452: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:06:57.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:57 smithi196 ceph-mon[86193]: pgmap v453: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:58.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:57 smithi176 ceph-mon[112742]: pgmap v453: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:06:59.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:06:59 smithi196 ceph-mon[86193]: pgmap v454: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:00.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:06:59 smithi176 ceph-mon[112742]: pgmap v454: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:00.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:01.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:01 smithi196 ceph-mon[86193]: pgmap v455: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:02.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:01 smithi176 ceph-mon[112742]: pgmap v455: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:03.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:02 smithi176 ceph-mon[112742]: pgmap v456: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:03.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:02 smithi196 ceph-mon[86193]: pgmap v456: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:05.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:05 smithi196 ceph-mon[86193]: pgmap v457: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:06.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:05 smithi176 ceph-mon[112742]: pgmap v457: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:07.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:07 smithi196 ceph-mon[86193]: pgmap v458: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:08.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:07 smithi176 ceph-mon[112742]: pgmap v458: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:09.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:09 smithi196 ceph-mon[86193]: pgmap v459: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:10.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:09 smithi176 ceph-mon[112742]: pgmap v459: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:11.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:11 smithi196 ceph-mon[86193]: pgmap v460: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:12.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:11 smithi176 ceph-mon[112742]: pgmap v460: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:13.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:13 smithi196 ceph-mon[86193]: pgmap v461: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:13.987 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:13 smithi176 ceph-mon[112742]: pgmap v461: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:14.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:14 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:07:15.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:14 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:07:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:15 smithi196 ceph-mon[86193]: pgmap v462: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:15.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:16.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:15 smithi176 ceph-mon[112742]: pgmap v462: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:16.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:16.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:07:16.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:07:16.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:07:17.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:07:17.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:07:17.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:07:17.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:17 smithi196 ceph-mon[86193]: pgmap v463: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:18.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:17 smithi176 ceph-mon[112742]: pgmap v463: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:19.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:18 smithi176 ceph-mon[112742]: pgmap v464: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:19.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:18 smithi196 ceph-mon[86193]: pgmap v464: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:21.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:21 smithi196 ceph-mon[86193]: pgmap v465: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:22.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:21 smithi176 ceph-mon[112742]: pgmap v465: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:23.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:23 smithi196 ceph-mon[86193]: pgmap v466: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:24.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:23 smithi176 ceph-mon[112742]: pgmap v466: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:25.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:25 smithi196 ceph-mon[86193]: pgmap v467: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:26.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:25 smithi176 ceph-mon[112742]: pgmap v467: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:27.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:27 smithi196 ceph-mon[86193]: pgmap v468: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:28.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:27 smithi176 ceph-mon[112742]: pgmap v468: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:29.926 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:29 smithi196 ceph-mon[86193]: pgmap v469: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:30.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:29 smithi176 ceph-mon[112742]: pgmap v469: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:30.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:31.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:32.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:31 smithi176 ceph-mon[112742]: pgmap v470: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:32.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:31 smithi196 ceph-mon[86193]: pgmap v470: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:33.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:32 smithi176 ceph-mon[112742]: pgmap v471: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:33.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:32 smithi196 ceph-mon[86193]: pgmap v471: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:35.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:35 smithi196 ceph-mon[86193]: pgmap v472: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:36.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:35 smithi176 ceph-mon[112742]: pgmap v472: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:37.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:37 smithi196 ceph-mon[86193]: pgmap v473: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:38.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:37 smithi176 ceph-mon[112742]: pgmap v473: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:39.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:39 smithi196 ceph-mon[86193]: pgmap v474: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:40.117 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:39 smithi176 ceph-mon[112742]: pgmap v474: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:41.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:41 smithi196 ceph-mon[86193]: pgmap v475: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:42.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:41 smithi176 ceph-mon[112742]: pgmap v475: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:43.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:43 smithi196 ceph-mon[86193]: pgmap v476: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:44.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:43 smithi176 ceph-mon[112742]: pgmap v476: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:46.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:45 smithi176 ceph-mon[112742]: pgmap v477: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:46.126 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:46.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:45 smithi196 ceph-mon[86193]: pgmap v477: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:46.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:07:48.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:47 smithi176 ceph-mon[112742]: pgmap v478: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:48.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:47 smithi196 ceph-mon[86193]: pgmap v478: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:49.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:48 smithi176 ceph-mon[112742]: pgmap v479: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:49.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:48 smithi196 ceph-mon[86193]: pgmap v479: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:07:51.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:51 smithi196 ceph-mon[86193]: pgmap v480: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:52.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:51 smithi176 ceph-mon[112742]: pgmap v480: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:53.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:53 smithi196 ceph-mon[86193]: pgmap v481: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:54.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:53 smithi176 ceph-mon[112742]: pgmap v481: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:55.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:55 smithi196 ceph-mon[86193]: pgmap v482: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:56.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:55 smithi176 ceph-mon[112742]: pgmap v482: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:07:57.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:57 smithi196 ceph-mon[86193]: pgmap v483: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:58.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:57 smithi176 ceph-mon[112742]: pgmap v483: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:07:59.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:07:59 smithi196 ceph-mon[86193]: pgmap v484: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:00.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:07:59 smithi176 ceph-mon[112742]: pgmap v484: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:01.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:02.076 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:01 smithi196 ceph-mon[86193]: pgmap v485: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:02.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:01 smithi176 ceph-mon[112742]: pgmap v485: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:03.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:02 smithi176 ceph-mon[112742]: pgmap v486: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:03.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:02 smithi196 ceph-mon[86193]: pgmap v486: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:05.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:05 smithi196 ceph-mon[86193]: pgmap v487: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:05.973 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:05 smithi176 ceph-mon[112742]: pgmap v487: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:07.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:07 smithi196 ceph-mon[86193]: pgmap v488: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:08.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:07 smithi176 ceph-mon[112742]: pgmap v488: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:09.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:09 smithi196 ceph-mon[86193]: pgmap v489: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:10.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:09 smithi176 ceph-mon[112742]: pgmap v489: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:11.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:11 smithi196 ceph-mon[86193]: pgmap v490: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:12.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:11 smithi176 ceph-mon[112742]: pgmap v490: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:14.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:13 smithi176 ceph-mon[112742]: pgmap v491: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:14.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:13 smithi196 ceph-mon[86193]: pgmap v491: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:16.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:15 smithi176 ceph-mon[112742]: pgmap v492: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:16.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:16.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:15 smithi196 ceph-mon[86193]: pgmap v492: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:16.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:17.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:16 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:08:17.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:16 smithi176 ceph-mon[112742]: pgmap v493: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:17.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:16 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:08:17.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:16 smithi196 ceph-mon[86193]: pgmap v493: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:17.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:17 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:08:17.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:17 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:08:17.925 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:17 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:08:18.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:17 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:08:18.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:17 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:08:18.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:17 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:08:19.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:18 smithi176 ceph-mon[112742]: pgmap v494: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:19.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:18 smithi196 ceph-mon[86193]: pgmap v494: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:21.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:21 smithi176 ceph-mon[112742]: pgmap v495: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:21.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:21 smithi196 ceph-mon[86193]: pgmap v495: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:23.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:23 smithi196 ceph-mon[86193]: pgmap v496: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:24.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:23 smithi176 ceph-mon[112742]: pgmap v496: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:26.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:25 smithi176 ceph-mon[112742]: pgmap v497: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:26.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:25 smithi196 ceph-mon[86193]: pgmap v497: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:28.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:27 smithi176 ceph-mon[112742]: pgmap v498: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:28.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:27 smithi196 ceph-mon[86193]: pgmap v498: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:30.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:29 smithi176 ceph-mon[112742]: pgmap v499: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:30.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:29 smithi196 ceph-mon[86193]: pgmap v499: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:31.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:30 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:31.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:30 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:32.076 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:31 smithi196 ceph-mon[86193]: pgmap v500: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:32.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:31 smithi176 ceph-mon[112742]: pgmap v500: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:33.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:32 smithi176 ceph-mon[112742]: pgmap v501: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:33.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:32 smithi196 ceph-mon[86193]: pgmap v501: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:08:35.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:35 smithi196 ceph-mon[86193]: pgmap v502: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:36.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:35 smithi176 ceph-mon[112742]: pgmap v502: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:37.374 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:37 smithi176 ceph-mon[112742]: pgmap v503: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:37.424 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:37 smithi196 ceph-mon[86193]: pgmap v503: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:39.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:39 smithi196 ceph-mon[86193]: pgmap v504: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:40.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:39 smithi176 ceph-mon[112742]: pgmap v504: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:42.116 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:41 smithi176 ceph-mon[112742]: pgmap v505: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:42.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:41 smithi196 ceph-mon[86193]: pgmap v505: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:44.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:43 smithi176 ceph-mon[112742]: pgmap v506: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:44.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:43 smithi196 ceph-mon[86193]: pgmap v506: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:45.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:44 smithi176 ceph-mon[112742]: pgmap v507: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:45.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:44 smithi196 ceph-mon[86193]: pgmap v507: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:46.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:45 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:46.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:45 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:08:47.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:46 smithi176 ceph-mon[112742]: pgmap v508: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:47.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:46 smithi196 ceph-mon[86193]: pgmap v508: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:49.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:49 smithi196 ceph-mon[86193]: pgmap v509: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:50.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:49 smithi176 ceph-mon[112742]: pgmap v509: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:52.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:51 smithi176 ceph-mon[112742]: pgmap v510: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:52.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:51 smithi196 ceph-mon[86193]: pgmap v510: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:53.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:53 smithi196 ceph-mon[86193]: pgmap v511: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:54.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:53 smithi176 ceph-mon[112742]: pgmap v511: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:56.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:55 smithi176 ceph-mon[112742]: pgmap v512: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:56.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:55 smithi196 ceph-mon[86193]: pgmap v512: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:08:57.374 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:57 smithi176 ceph-mon[112742]: pgmap v513: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:08:57.424 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:57 smithi196 ceph-mon[86193]: pgmap v513: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:00.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:08:59 smithi176 ceph-mon[112742]: pgmap v514: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:00.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:08:59 smithi196 ceph-mon[86193]: pgmap v514: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:01.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:00 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:09:01.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:00 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:09:02.076 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:01 smithi196 ceph-mon[86193]: pgmap v515: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:02.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:01 smithi176 ceph-mon[112742]: pgmap v515: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:03.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:02 smithi176 ceph-mon[112742]: pgmap v516: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:03.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:02 smithi196 ceph-mon[86193]: pgmap v516: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:06.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:05 smithi176 ceph-mon[112742]: pgmap v517: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:06.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:05 smithi196 ceph-mon[86193]: pgmap v517: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:07.979 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:07 smithi176 ceph-mon[112742]: pgmap v518: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:08.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:07 smithi196 ceph-mon[86193]: pgmap v518: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:10.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:09 smithi176 ceph-mon[112742]: pgmap v519: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:10.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:09 smithi196 ceph-mon[86193]: pgmap v519: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:12.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:11 smithi176 ceph-mon[112742]: pgmap v520: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:12.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:11 smithi196 ceph-mon[86193]: pgmap v520: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:13.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:12 smithi176 ceph-mon[112742]: pgmap v521: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:13.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:12 smithi196 ceph-mon[86193]: pgmap v521: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:16.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:15 smithi176 ceph-mon[112742]: pgmap v522: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:16.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:15 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:09:16.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:15 smithi196 ceph-mon[86193]: pgmap v522: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:16.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:15 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-13T15:09:18.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:17 smithi176 ceph-mon[112742]: pgmap v523: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:18.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:17 smithi196 ceph-mon[86193]: pgmap v523: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 852 B/s rd, 1 op/s 2024-03-13T15:09:19.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:09:19.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:09:19.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:18 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:09:19.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-13T15:09:19.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:09:19.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:18 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:09:20.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:19 smithi176 ceph-mon[112742]: pgmap v524: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:20.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:19 smithi196 ceph-mon[86193]: pgmap v524: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:21.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:09:21.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:09:21.125 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:20 smithi176 ceph-mon[112742]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:09:21.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-13T15:09:21.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-13T15:09:21.175 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:20 smithi196 ceph-mon[86193]: from='mgr.24551 172.21.15.176:0/3529411348' entity='mgr.smithi176.tzfdwd' 2024-03-13T15:09:22.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:21 smithi176 ceph-mon[112742]: pgmap v525: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:22.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:21 smithi196 ceph-mon[86193]: pgmap v525: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:23.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:22 smithi176 ceph-mon[112742]: pgmap v526: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:23.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:22 smithi196 ceph-mon[86193]: pgmap v526: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:26.124 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:25 smithi176 ceph-mon[112742]: pgmap v527: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:26.174 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:25 smithi196 ceph-mon[86193]: pgmap v527: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-03-13T15:09:27.956 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-03-13T15:09:27.956 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-13T15:09:27.959 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-13T15:09:27.973 INFO:tasks.cephadm:Teardown begin 2024-03-13T15:09:27.973 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/cephadm.py", line 1867, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-13T15:09:27.975 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-13T15:09:28.064 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:27 smithi176 ceph-mon[112742]: pgmap v528: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:28.081 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-13T15:09:28.168 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:27 smithi196 ceph-mon[86193]: pgmap v528: 65 pgs: 65 active+clean; 204 MiB data, 867 MiB used, 536 GiB / 536 GiB avail; 853 B/s rd, 1 op/s 2024-03-13T15:09:28.185 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-13T15:09:28.185 DEBUG:teuthology.orchestra.run.smithi176:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-13T15:09:28.242 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-13T15:09:28.301 INFO:tasks.cephadm:Stopping all daemons... 2024-03-13T15:09:28.301 INFO:tasks.cephadm.mon.smithi176:Stopping mon.smithi176... 2024-03-13T15:09:28.301 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176 2024-03-13T15:09:28.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:28 smithi176 systemd[1]: Stopping Ceph mon.smithi176 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:09:29.255 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:28 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[112738]: 2024-03-13T15:09:28.909+0000 7f67a8200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi176 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:09:29.255 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:28 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176[112738]: 2024-03-13T15:09:28.909+0000 7f67a8200700 -1 mon.smithi176@0(leader) e3 *** Got Signal Terminated *** 2024-03-13T15:09:29.255 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:28 smithi176 podman[160239]: 2024-03-13 15:09:28.983044388 +0000 UTC m=+0.235656596 container died 35f4ac6dca3064405c7650c6475f1f4692ccd31856e7a7c14c488c0454ead7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-13T15:09:29.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:29 smithi176 podman[160239]: 2024-03-13 15:09:29.546272993 +0000 UTC m=+0.798885244 container cleanup 35f4ac6dca3064405c7650c6475f1f4692ccd31856e7a7c14c488c0454ead7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T15:09:29.874 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:29 smithi176 bash[160239]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176 2024-03-13T15:09:29.875 INFO:journalctl@ceph.mon.smithi176.smithi176.stdout:Mar 13 15:09:29 smithi176 podman[160263]: 2024-03-13 15:09:29.756084897 +0000 UTC m=+0.763824737 container remove 35f4ac6dca3064405c7650c6475f1f4692ccd31856e7a7c14c488c0454ead7dc (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi176, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T15:09:30.241 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi176.service' 2024-03-13T15:09:30.397 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:09:30.398 INFO:tasks.cephadm.mon.smithi176:Stopped mon.smithi176 2024-03-13T15:09:30.398 INFO:tasks.cephadm.mon.smithi196:Stopping mon.smithi196... 2024-03-13T15:09:30.398 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi196 2024-03-13T15:09:30.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:30 smithi196 systemd[1]: Stopping Ceph mon.smithi196 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:09:31.210 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:30 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196[86189]: 2024-03-13T15:09:30.960+0000 7ff033200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi196 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:09:31.211 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:30 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196[86189]: 2024-03-13T15:09:30.960+0000 7ff033200700 -1 mon.smithi196@1(peon) e3 *** Got Signal Terminated *** 2024-03-13T15:09:31.534 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:31 smithi196 podman[123492]: 2024-03-13 15:09:31.227856039 +0000 UTC m=+0.402601291 container died 3f7f3fad03733d993fa7623f7c7228f3c39607a4f671d4834051974a382afa5a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-13T15:09:31.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:31 smithi196 podman[123492]: 2024-03-13 15:09:31.754631965 +0000 UTC m=+0.929377227 container cleanup 3f7f3fad03733d993fa7623f7c7228f3c39607a4f671d4834051974a382afa5a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, io.buildah.version=1.33.5) 2024-03-13T15:09:31.924 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:31 smithi196 bash[123492]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196 2024-03-13T15:09:32.424 INFO:journalctl@ceph.mon.smithi196.smithi196.stdout:Mar 13 15:09:32 smithi196 podman[123506]: 2024-03-13 15:09:32.056130702 +0000 UTC m=+0.811412961 container remove 3f7f3fad03733d993fa7623f7c7228f3c39607a4f671d4834051974a382afa5a (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-mon-smithi196, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-13T15:09:32.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@mon.smithi196.service' 2024-03-13T15:09:32.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:09:32.616 INFO:tasks.cephadm.mon.smithi196:Stopped mon.smithi196 2024-03-13T15:09:32.616 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-13T15:09:32.616 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.0 2024-03-13T15:09:33.124 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:32 smithi176 systemd[1]: Stopping Ceph osd.0 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:09:33.624 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:33 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[125051]: 2024-03-13T15:09:33.223+0000 7f4ac6800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:09:33.624 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:33 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[125051]: 2024-03-13T15:09:33.223+0000 7f4ac6800700 -1 osd.0 112 *** Got signal Terminated *** 2024-03-13T15:09:33.624 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:33 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0[125051]: 2024-03-13T15:09:33.223+0000 7f4ac6800700 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T15:09:39.016 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:38 smithi176 podman[160369]: 2024-03-13 15:09:38.731906161 +0000 UTC m=+5.655624565 container stop 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-03-13T15:09:39.016 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:38 smithi176 podman[160369]: 2024-03-13 15:09:38.734076647 +0000 UTC m=+5.657795065 container died 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5) 2024-03-13T15:09:39.369 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:39 smithi176 podman[160369]: 2024-03-13 15:09:39.209218111 +0000 UTC m=+6.132936582 container cleanup 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-13T15:09:39.369 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:39 smithi176 bash[160369]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0 2024-03-13T15:09:39.624 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:39 smithi176 podman[160388]: 2024-03-13 15:09:39.385256924 +0000 UTC m=+0.629947130 container remove 9ee4d4f2265b52bdc8e70cd989b72b111b0f80505df1d5a930a1decc876b9741 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-13T15:09:40.498 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:40 smithi176 podman[160475]: 2024-03-13 15:09:40.09003861 +0000 UTC m=+0.035020000 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T15:09:40.499 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:40 smithi176 podman[160475]: 2024-03-13 15:09:40.261361239 +0000 UTC m=+0.206342666 container create a5298728a452b650389d44367a6914ff7166c08a9798ce24102b232a2d4e581d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240222, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-13T15:09:40.499 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:40 smithi176 podman[160475]: 2024-03-13 15:09:40.515236025 +0000 UTC m=+0.460217462 container init a5298728a452b650389d44367a6914ff7166c08a9798ce24102b232a2d4e581d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T15:09:40.874 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:40 smithi176 podman[160475]: 2024-03-13 15:09:40.527308599 +0000 UTC m=+0.472289994 container start a5298728a452b650389d44367a6914ff7166c08a9798ce24102b232a2d4e581d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD) 2024-03-13T15:09:40.874 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:40 smithi176 podman[160475]: 2024-03-13 15:09:40.613582289 +0000 UTC m=+0.558563739 container attach a5298728a452b650389d44367a6914ff7166c08a9798ce24102b232a2d4e581d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T15:09:41.304 INFO:journalctl@ceph.osd.0.smithi176.stdout:Mar 13 15:09:40 smithi176 podman[160475]: 2024-03-13 15:09:40.962252657 +0000 UTC m=+0.907234112 container died a5298728a452b650389d44367a6914ff7166c08a9798ce24102b232a2d4e581d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-13T15:09:41.716 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.0.service' 2024-03-13T15:09:41.837 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:09:41.837 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-13T15:09:41.838 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-13T15:09:41.838 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.1 2024-03-13T15:09:42.213 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:41 smithi176 systemd[1]: Stopping Ceph osd.1 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:09:42.624 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:42 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[130480]: 2024-03-13T15:09:42.468+0000 7f4e92c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:09:42.624 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:42 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[130480]: 2024-03-13T15:09:42.468+0000 7f4e92c00700 -1 osd.1 112 *** Got signal Terminated *** 2024-03-13T15:09:42.624 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:42 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1[130480]: 2024-03-13T15:09:42.468+0000 7f4e92c00700 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T15:09:48.202 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:47 smithi176 podman[160581]: 2024-03-13 15:09:47.949123443 +0000 UTC m=+5.658870298 container died 6ac5d28d6884134ff7a1e40b9b59de6e9e096f7766591e2eb45809493f08a79e (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-03-13T15:09:48.504 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:48 smithi176 podman[160581]: 2024-03-13 15:09:48.32835908 +0000 UTC m=+6.038105970 container cleanup 6ac5d28d6884134ff7a1e40b9b59de6e9e096f7766591e2eb45809493f08a79e (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-03-13T15:09:48.505 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:48 smithi176 bash[160581]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1 2024-03-13T15:09:48.874 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:48 smithi176 podman[160612]: 2024-03-13 15:09:48.521293864 +0000 UTC m=+0.555868934 container remove 6ac5d28d6884134ff7a1e40b9b59de6e9e096f7766591e2eb45809493f08a79e (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222) 2024-03-13T15:09:49.374 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:49 smithi176 podman[160700]: 2024-03-13 15:09:49.188781356 +0000 UTC m=+0.034860931 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T15:09:49.374 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:49 smithi176 podman[160700]: 2024-03-13 15:09:49.297256853 +0000 UTC m=+0.143336464 container create 6e73638ce4671cbfedac3f255bc99accc24f99dd857f7d324b04897afd4fb00d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222) 2024-03-13T15:09:49.874 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:49 smithi176 podman[160700]: 2024-03-13 15:09:49.61947243 +0000 UTC m=+0.465552075 container init 6e73638ce4671cbfedac3f255bc99accc24f99dd857f7d324b04897afd4fb00d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222) 2024-03-13T15:09:49.874 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:49 smithi176 podman[160700]: 2024-03-13 15:09:49.630969939 +0000 UTC m=+0.477049532 container start 6e73638ce4671cbfedac3f255bc99accc24f99dd857f7d324b04897afd4fb00d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-13T15:09:49.874 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:49 smithi176 podman[160700]: 2024-03-13 15:09:49.716542414 +0000 UTC m=+0.562622045 container attach 6e73638ce4671cbfedac3f255bc99accc24f99dd857f7d324b04897afd4fb00d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-13T15:09:50.374 INFO:journalctl@ceph.osd.1.smithi176.stdout:Mar 13 15:09:50 smithi176 podman[160700]: 2024-03-13 15:09:50.089299017 +0000 UTC m=+0.935378605 container died 6e73638ce4671cbfedac3f255bc99accc24f99dd857f7d324b04897afd4fb00d (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-03-13T15:09:50.801 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.1.service' 2024-03-13T15:09:50.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:09:50.924 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-13T15:09:50.924 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-13T15:09:50.924 DEBUG:teuthology.orchestra.run.smithi176:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.2 2024-03-13T15:09:51.374 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:51 smithi176 systemd[1]: Stopping Ceph osd.2 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:09:51.874 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:51 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[136042]: 2024-03-13T15:09:51.521+0000 7f7a97e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:09:51.874 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:51 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[136042]: 2024-03-13T15:09:51.521+0000 7f7a97e00700 -1 osd.2 112 *** Got signal Terminated *** 2024-03-13T15:09:51.874 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:51 smithi176 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2[136042]: 2024-03-13T15:09:51.521+0000 7f7a97e00700 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T15:09:57.279 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:56 smithi176 podman[160808]: 2024-03-13 15:09:56.960367628 +0000 UTC m=+5.576627359 container died 56889da43f2b88a34f545c9413178c87454bcaa8e82322ef66e8ee59625f0a77 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-03-13T15:09:57.624 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:57 smithi176 podman[160808]: 2024-03-13 15:09:57.414515009 +0000 UTC m=+6.030774778 container cleanup 56889da43f2b88a34f545c9413178c87454bcaa8e82322ef66e8ee59625f0a77 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-13T15:09:57.624 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:57 smithi176 bash[160808]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2 2024-03-13T15:09:58.124 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:57 smithi176 podman[160828]: 2024-03-13 15:09:57.707933142 +0000 UTC m=+0.729557059 container remove 56889da43f2b88a34f545c9413178c87454bcaa8e82322ef66e8ee59625f0a77 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T15:09:58.770 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:58 smithi176 podman[160914]: 2024-03-13 15:09:58.40824195 +0000 UTC m=+0.037172155 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T15:09:58.771 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:58 smithi176 podman[160914]: 2024-03-13 15:09:58.533606756 +0000 UTC m=+0.162536995 container create 89abea6a9154b5c812b66ece74cdd2879716bc29e5afeb20833f0758c7ef3854 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-13T15:09:58.771 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:58 smithi176 podman[160914]: 2024-03-13 15:09:58.78734562 +0000 UTC m=+0.416275866 container init 89abea6a9154b5c812b66ece74cdd2879716bc29e5afeb20833f0758c7ef3854 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T15:09:59.124 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:58 smithi176 podman[160914]: 2024-03-13 15:09:58.799640573 +0000 UTC m=+0.428570796 container start 89abea6a9154b5c812b66ece74cdd2879716bc29e5afeb20833f0758c7ef3854 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240222, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-13T15:09:59.124 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:58 smithi176 podman[160914]: 2024-03-13 15:09:58.894504026 +0000 UTC m=+0.523434328 container attach 89abea6a9154b5c812b66ece74cdd2879716bc29e5afeb20833f0758c7ef3854 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-03-13T15:09:59.529 INFO:journalctl@ceph.osd.2.smithi176.stdout:Mar 13 15:09:59 smithi176 podman[160942]: 2024-03-13 15:09:59.289340594 +0000 UTC m=+0.038910561 container died 89abea6a9154b5c812b66ece74cdd2879716bc29e5afeb20833f0758c7ef3854 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.build-date=20240222, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-03-13T15:09:59.902 DEBUG:teuthology.orchestra.run.smithi176:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.2.service' 2024-03-13T15:10:00.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:10:00.014 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-13T15:10:00.014 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-13T15:10:00.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.3 2024-03-13T15:10:00.404 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:00 smithi196 systemd[1]: Stopping Ceph osd.3 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:10:00.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:00 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:00.421+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:00.675 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:00 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T15:10:00.589+0000 7f4e47200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:10:00.675 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:00 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T15:10:00.589+0000 7f4e47200700 -1 osd.3 112 *** Got signal Terminated *** 2024-03-13T15:10:00.675 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:00 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T15:10:00.589+0000 7f4e47200700 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T15:10:01.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:01 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:01.474+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:01.925 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:01 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:01.456+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:02.826 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:02 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:02.496+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:02.826 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:02 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:02.448+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:03.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:03 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:03.466+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:03.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:03 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:03.436+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:04.174 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:03 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T15:10:03.816+0000 7f4e43400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:37.875424+0000 front 2024-03-13T15:09:37.875555+0000 (oldest deadline 2024-03-13T15:10:03.775413+0000) 2024-03-13T15:10:04.456 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:04.399+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:04.781 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:04.472+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:04.782 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:04 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3[96081]: 2024-03-13T15:10:04.798+0000 7f4e43400700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:37.875424+0000 front 2024-03-13T15:09:37.875555+0000 (oldest deadline 2024-03-13T15:10:03.775413+0000) 2024-03-13T15:10:05.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:05 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:05.404+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:05.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:05 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:05.477+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:06.424 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:06 smithi196 podman[123627]: 2024-03-13 15:10:06.152566331 +0000 UTC m=+5.721253434 container died a186117c65ba776ba9db2f1af9ed12450ee0bff9100fff3f67e26616c4330b62 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240222, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-13T15:10:06.424 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:06 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:06.395+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:06.825 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:06 smithi196 podman[123627]: 2024-03-13 15:10:06.607119008 +0000 UTC m=+6.175806157 container cleanup a186117c65ba776ba9db2f1af9ed12450ee0bff9100fff3f67e26616c4330b62 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-13T15:10:06.825 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:06 smithi196 bash[123627]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3 2024-03-13T15:10:06.825 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:06 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:06.492+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:07.174 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:06 smithi196 podman[123659]: 2024-03-13 15:10:06.841568293 +0000 UTC m=+0.678518890 container remove a186117c65ba776ba9db2f1af9ed12450ee0bff9100fff3f67e26616c4330b62 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T15:10:07.607 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:07 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:07.510+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:07.607 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:07 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:07.366+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:07.912 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:07 smithi196 podman[123748]: 2024-03-13 15:10:07.530068767 +0000 UTC m=+0.035131028 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T15:10:07.913 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:07 smithi196 podman[123748]: 2024-03-13 15:10:07.667393375 +0000 UTC m=+0.172455696 container create 3840de5f40cf48a562fbf453cd341f627f03813562af18bbe05a504f9f31e5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-03-13T15:10:07.913 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:07 smithi196 podman[123748]: 2024-03-13 15:10:07.929547728 +0000 UTC m=+0.434610033 container init 3840de5f40cf48a562fbf453cd341f627f03813562af18bbe05a504f9f31e5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81) 2024-03-13T15:10:08.174 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:07 smithi196 podman[123748]: 2024-03-13 15:10:07.943162471 +0000 UTC m=+0.448224746 container start 3840de5f40cf48a562fbf453cd341f627f03813562af18bbe05a504f9f31e5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-03-13T15:10:08.174 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:08 smithi196 podman[123748]: 2024-03-13 15:10:08.086780978 +0000 UTC m=+0.591843333 container attach 3840de5f40cf48a562fbf453cd341f627f03813562af18bbe05a504f9f31e5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240222, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T15:10:08.675 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:08 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:08.483+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:08.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:08 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:08.381+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:08.676 INFO:journalctl@ceph.osd.3.smithi196.stdout:Mar 13 15:10:08 smithi196 podman[123748]: 2024-03-13 15:10:08.379324784 +0000 UTC m=+0.884387112 container died 3840de5f40cf48a562fbf453cd341f627f03813562af18bbe05a504f9f31e5e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T15:10:09.116 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.3.service' 2024-03-13T15:10:09.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:10:09.238 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-13T15:10:09.238 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-13T15:10:09.238 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.4 2024-03-13T15:10:09.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:09 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:09.403+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:09.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:09 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:09.403+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:09.675 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:09 smithi196 systemd[1]: Stopping Ceph osd.4 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:10:09.675 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:09 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:09.487+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:09.675 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:09 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:09.487+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.309279+0000 front 2024-03-13T15:09:47.308964+0000 (oldest deadline 2024-03-13T15:10:09.008555+0000) 2024-03-13T15:10:10.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:09 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:09.899+0000 7f3f62400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:10:10.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:09 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:09.899+0000 7f3f62400700 -1 osd.4 112 *** Got signal Terminated *** 2024-03-13T15:10:10.174 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:09 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:09.899+0000 7f3f62400700 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T15:10:10.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:10 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:10.468+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:10.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:10 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:10.468+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.309279+0000 front 2024-03-13T15:09:47.308964+0000 (oldest deadline 2024-03-13T15:10:09.008555+0000) 2024-03-13T15:10:10.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:10 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:10.404+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:10.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:10 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:10.404+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:11.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:11 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:11.465+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:11.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:11 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:11.465+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.309279+0000 front 2024-03-13T15:09:47.308964+0000 (oldest deadline 2024-03-13T15:10:09.008555+0000) 2024-03-13T15:10:11.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:11 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:11.398+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:11.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:11 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:11.398+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:12.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:12 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:12.419+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:12.675 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:12 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:12.419+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.309279+0000 front 2024-03-13T15:09:47.308964+0000 (oldest deadline 2024-03-13T15:10:09.008555+0000) 2024-03-13T15:10:12.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:12 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:12.427+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:12.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:12 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:12.427+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:13.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:13 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:13.433+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:13.675 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:13 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:13.433+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.309279+0000 front 2024-03-13T15:09:47.308964+0000 (oldest deadline 2024-03-13T15:10:09.008555+0000) 2024-03-13T15:10:13.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:13 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:13.434+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:13.675 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:13 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:13.434+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:14.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:14 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:14.432+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:14.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:14 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:14.432+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:14.674 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:14 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:14.401+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.607021+0000 front 2024-03-13T15:09:35.607157+0000 (oldest deadline 2024-03-13T15:10:00.906880+0000) 2024-03-13T15:10:14.675 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:14 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4[101406]: 2024-03-13T15:10:14.401+0000 7f3f59600700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.309279+0000 front 2024-03-13T15:09:47.308964+0000 (oldest deadline 2024-03-13T15:10:09.008555+0000) 2024-03-13T15:10:15.589 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:15 smithi196 podman[123855]: 2024-03-13 15:10:15.259081582 +0000 UTC m=+5.515839211 container died 7f510f7a757cc86d57b2a153291fccf2aec1f2dbaa2c6d17ad8b73a8fa115285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240222, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-13T15:10:15.589 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:15 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:15.444+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:15.589 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:15 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:15.444+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:15.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:15 smithi196 podman[123855]: 2024-03-13 15:10:15.740128348 +0000 UTC m=+5.996886033 container cleanup 7f510f7a757cc86d57b2a153291fccf2aec1f2dbaa2c6d17ad8b73a8fa115285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-13T15:10:15.924 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:15 smithi196 bash[123855]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4 2024-03-13T15:10:16.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:15 smithi196 podman[123873]: 2024-03-13 15:10:15.99121693 +0000 UTC m=+0.715975261 container remove 7f510f7a757cc86d57b2a153291fccf2aec1f2dbaa2c6d17ad8b73a8fa115285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T15:10:16.805 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:16 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:16.488+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:16.805 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:16 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:16.488+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:17.103 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:16 smithi196 podman[123965]: 2024-03-13 15:10:16.727303906 +0000 UTC m=+0.034795814 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T15:10:17.104 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:16 smithi196 podman[123965]: 2024-03-13 15:10:16.866924574 +0000 UTC m=+0.174416539 container create 6196ebcd0a537cd9c931f3e9d92121aef464cb2c10a9d073e552a268f01a8285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-13T15:10:17.104 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:17 smithi196 podman[123965]: 2024-03-13 15:10:17.120718921 +0000 UTC m=+0.428210916 container init 6196ebcd0a537cd9c931f3e9d92121aef464cb2c10a9d073e552a268f01a8285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, CEPH_POINT_RELEASE=) 2024-03-13T15:10:17.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:17 smithi196 podman[123965]: 2024-03-13 15:10:17.131374903 +0000 UTC m=+0.438866855 container start 6196ebcd0a537cd9c931f3e9d92121aef464cb2c10a9d073e552a268f01a8285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-03-13T15:10:17.424 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:17 smithi196 podman[123965]: 2024-03-13 15:10:17.219384803 +0000 UTC m=+0.526876798 container attach 6196ebcd0a537cd9c931f3e9d92121aef464cb2c10a9d073e552a268f01a8285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b) 2024-03-13T15:10:17.910 INFO:journalctl@ceph.osd.4.smithi196.stdout:Mar 13 15:10:17 smithi196 podman[123991]: 2024-03-13 15:10:17.603451495 +0000 UTC m=+0.030821568 container died 6196ebcd0a537cd9c931f3e9d92121aef464cb2c10a9d073e552a268f01a8285 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-03-13T15:10:17.910 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:17 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:17.476+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:17.910 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:17 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:17.476+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:18.307 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.4.service' 2024-03-13T15:10:18.429 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:10:18.429 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-13T15:10:18.429 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-13T15:10:18.430 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.5 2024-03-13T15:10:18.556 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:18 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:18.468+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:18.557 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:18 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:18.469+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:18.557 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:18 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:18.469+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6822 osd.2 since back 2024-03-13T15:09:56.043885+0000 front 2024-03-13T15:09:56.044000+0000 (oldest deadline 2024-03-13T15:10:17.743451+0000) 2024-03-13T15:10:18.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:18 smithi196 systemd[1]: Stopping Ceph osd.5 for f60e35da-e146-11ee-95c9-87774f69a715... 2024-03-13T15:10:19.424 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:18 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:18.981+0000 7f5836600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-13T15:10:19.424 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:18 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:18.981+0000 7f5836600700 -1 osd.5 112 *** Got signal Terminated *** 2024-03-13T15:10:19.424 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:18 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:18.981+0000 7f5836600700 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-13T15:10:19.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:19 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:19.476+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:19.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:19 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:19.476+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:19.925 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:19 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:19.476+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6822 osd.2 since back 2024-03-13T15:09:56.043885+0000 front 2024-03-13T15:09:56.044000+0000 (oldest deadline 2024-03-13T15:10:17.743451+0000) 2024-03-13T15:10:20.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:20 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:20.469+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:20.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:20 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:20.469+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:20.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:20 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:20.469+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6822 osd.2 since back 2024-03-13T15:09:56.043885+0000 front 2024-03-13T15:09:56.044000+0000 (oldest deadline 2024-03-13T15:10:17.743451+0000) 2024-03-13T15:10:21.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:21 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:21.460+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:21.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:21 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:21.460+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:21.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:21 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:21.460+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6822 osd.2 since back 2024-03-13T15:09:56.043885+0000 front 2024-03-13T15:09:56.044000+0000 (oldest deadline 2024-03-13T15:10:17.743451+0000) 2024-03-13T15:10:22.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:22 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:22.505+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:22.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:22 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:22.505+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:22.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:22 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:22.505+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6822 osd.2 since back 2024-03-13T15:09:56.043885+0000 front 2024-03-13T15:09:56.044000+0000 (oldest deadline 2024-03-13T15:10:17.743451+0000) 2024-03-13T15:10:23.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:23 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:23.546+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6806 osd.0 since back 2024-03-13T15:09:35.338383+0000 front 2024-03-13T15:09:35.338109+0000 (oldest deadline 2024-03-13T15:09:59.938581+0000) 2024-03-13T15:10:23.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:23 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:23.546+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6814 osd.1 since back 2024-03-13T15:09:47.441009+0000 front 2024-03-13T15:09:47.441099+0000 (oldest deadline 2024-03-13T15:10:08.541094+0000) 2024-03-13T15:10:23.924 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:23 smithi196 ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5[106752]: 2024-03-13T15:10:23.546+0000 7f582d800700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.176:6822 osd.2 since back 2024-03-13T15:09:56.043885+0000 front 2024-03-13T15:09:56.044000+0000 (oldest deadline 2024-03-13T15:10:17.743451+0000) 2024-03-13T15:10:24.890 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:24 smithi196 podman[124072]: 2024-03-13 15:10:24.539822986 +0000 UTC m=+5.682910995 container died 43366d2dc8dacc75d72d1bfe86c5007ee5c7563c5f45c31476ea48f38d1ae452 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240222, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-03-13T15:10:25.174 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:25 smithi196 podman[124072]: 2024-03-13 15:10:25.016262085 +0000 UTC m=+6.159350124 container cleanup 43366d2dc8dacc75d72d1bfe86c5007ee5c7563c5f45c31476ea48f38d1ae452 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240222, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-13T15:10:25.175 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:25 smithi196 bash[124072]: ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5 2024-03-13T15:10:25.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:25 smithi196 podman[124091]: 2024-03-13 15:10:25.250783192 +0000 UTC m=+0.698840828 container remove 43366d2dc8dacc75d72d1bfe86c5007ee5c7563c5f45c31476ea48f38d1ae452 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5, org.label-schema.build-date=20240222, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-03-13T15:10:26.397 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:26 smithi196 podman[124179]: 2024-03-13 15:10:26.036065048 +0000 UTC m=+0.035045321 image pull 9552dbf7f1150d1b2a06d61271f6e8a442468b837e4d87e78677059ba6282398 quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a 2024-03-13T15:10:26.397 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:26 smithi196 podman[124179]: 2024-03-13 15:10:26.143313999 +0000 UTC m=+0.142294335 container create 8a55b9f9a1bae3b94a5704901a5bc6b0bd0e1f5dfed889bb96277ce138d524b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-13T15:10:26.397 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:26 smithi196 podman[124179]: 2024-03-13 15:10:26.413846953 +0000 UTC m=+0.412827269 container init 8a55b9f9a1bae3b94a5704901a5bc6b0bd0e1f5dfed889bb96277ce138d524b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.build-date=20240222, org.label-schema.schema-version=1.0) 2024-03-13T15:10:26.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:26 smithi196 podman[124179]: 2024-03-13 15:10:26.424677769 +0000 UTC m=+0.423658054 container start 8a55b9f9a1bae3b94a5704901a5bc6b0bd0e1f5dfed889bb96277ce138d524b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240222, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-13T15:10:26.674 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:26 smithi196 podman[124179]: 2024-03-13 15:10:26.569358892 +0000 UTC m=+0.568339193 container attach 8a55b9f9a1bae3b94a5704901a5bc6b0bd0e1f5dfed889bb96277ce138d524b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240222, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-13T15:10:27.174 INFO:journalctl@ceph.osd.5.smithi196.stdout:Mar 13 15:10:26 smithi196 podman[124179]: 2024-03-13 15:10:26.853583456 +0000 UTC m=+0.852563772 container died 8a55b9f9a1bae3b94a5704901a5bc6b0bd0e1f5dfed889bb96277ce138d524b7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f370a9f4a21a182b102262afad8b3849a4ad8eebb2fa178834e2609268df332a, name=ceph-f60e35da-e146-11ee-95c9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=6d53bc8ece767735a31d32bfc429296668de9e0b, RELEASE=wip-vshankar-testing-20240307.013758-9a75c81, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240222, maintainer=Guillaume Abrioux ) 2024-03-13T15:10:27.642 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f60e35da-e146-11ee-95c9-87774f69a715@osd.5.service' 2024-03-13T15:10:27.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-13T15:10:27.758 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-13T15:10:27.758 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f60e35da-e146-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-13T15:10:28.042 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T15:10:35.011 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi176.stderr:ceph-fuse[76677]: fuse finished with error 0 and tester_r 0 2024-03-13T15:10:57.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f60e35da-e146-11ee-95c9-87774f69a715 --force --keep-logs 2024-03-13T15:10:57.634 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T15:11:16.421 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-13T15:11:16.522 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-13T15:11:16.628 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-13T15:11:16.630 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645/remote/smithi176/crash 2024-03-13T15:11:16.631 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/crash -- . 2024-03-13T15:11:16.723 INFO:teuthology.orchestra.run.smithi176.stderr:tar: /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-13T15:11:16.724 INFO:teuthology.orchestra.run.smithi176.stderr:tar: Error is not recoverable: exiting now 2024-03-13T15:11:16.729 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645/remote/smithi196/crash 2024-03-13T15:11:16.729 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/crash -- . 2024-03-13T15:11:16.826 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/f60e35da-e146-11ee-95c9-87774f69a715/crash: Cannot open: No such file or directory 2024-03-13T15:11:16.826 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-03-13T15:11:16.830 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-13T15:11:16.831 DEBUG:teuthology.orchestra.run.smithi176:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-13T15:11:16.942 INFO:tasks.cephadm:Compressing logs... 2024-03-13T15:11:16.942 DEBUG:teuthology.orchestra.run.smithi176:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-13T15:11:16.947 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-13T15:11:17.010 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-13T15:11:17.014 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-13T15:11:17.015 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-03-13T15:11:17.016 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi176.log 2024-03-13T15:11:17.017 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mon.smithi176.log 2024-03-13T15:11:17.018 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi176.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-13T15:11:17.019 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mon.smithi176.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.1.log 2024-03-13T15:11:17.021 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mgr.smithi176.tzfdwd.log 2024-03-13T15:11:17.021 INFO:teuthology.orchestra.run.smithi176.stderr: 94.2% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi176.log.gz 2024-03-13T15:11:17.022 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.0.log 2024-03-13T15:11:17.025 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mgr.smithi176.tzfdwd.log: /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-13T15:11:17.027 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.cephadm.log: 91.4% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-13T15:11:17.029 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.log 2024-03-13T15:11:17.029 INFO:teuthology.orchestra.run.smithi176.stderr: 85.8% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-13T15:11:17.032 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.2.log 2024-03-13T15:11:17.036 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.2.log: /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.log: 88.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-13T15:11:17.036 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-13T15:11:17.038 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi176.hjrgov.log 2024-03-13T15:11:17.039 INFO:teuthology.orchestra.run.smithi176.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi176.ldskqt.log 2024-03-13T15:11:17.040 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-13T15:11:17.040 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-03-13T15:11:17.042 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mgr.smithi196.vdktbk.log 2024-03-13T15:11:17.043 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.audit.log 2024-03-13T15:11:17.044 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mgr.smithi196.vdktbk.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.4.log 2024-03-13T15:11:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.audit.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-13T15:11:17.045 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.5.log 2024-03-13T15:11:17.047 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.3.log 2024-03-13T15:11:17.047 INFO:teuthology.orchestra.run.smithi176.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-volume.log: /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi176.hjrgov.log: /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi176.ldskqt.log: 88.6% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-13T15:11:17.048 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi196.tcitrk.log 2024-03-13T15:11:17.050 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi196.wabhcu.log 2024-03-13T15:11:17.051 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi196.tcitrk.log: 91.6% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.audit.log.gz 2024-03-13T15:11:17.051 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.cephadm.log 2024-03-13T15:11:17.053 INFO:teuthology.orchestra.run.smithi196.stderr:gzip/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi196.wabhcu.log: -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.log 2024-03-13T15:11:17.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.cephadm.log: 85.8% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.cephadm.log.gz 2024-03-13T15:11:17.056 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mon.smithi196.log 2024-03-13T15:11:17.058 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mon.smithi196.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-volume.log 2024-03-13T15:11:17.060 INFO:teuthology.orchestra.run.smithi196.stderr: 88.6% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph.log.gz 2024-03-13T15:11:17.061 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi196.log 2024-03-13T15:11:17.063 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-03-13T15:11:17.063 INFO:teuthology.orchestra.run.smithi176.stderr: 90.8% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-13T15:11:17.081 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi196.log: /var/log/ceph/ceph-client.1.log: 94.1% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-client.ceph-exporter.smithi196.log.gz 2024-03-13T15:11:17.101 INFO:teuthology.orchestra.run.smithi196.stderr: 89.2% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mgr.smithi196.vdktbk.log.gz 2024-03-13T15:11:17.107 INFO:teuthology.orchestra.run.smithi196.stderr: 90.8% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-volume.log.gz 2024-03-13T15:11:17.641 INFO:teuthology.orchestra.run.smithi196.stderr: 92.5% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mon.smithi196.log.gz 2024-03-13T15:11:17.763 INFO:teuthology.orchestra.run.smithi176.stderr: 88.3% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mgr.smithi176.tzfdwd.log.gz 2024-03-13T15:11:18.674 INFO:teuthology.orchestra.run.smithi176.stderr: 90.4% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mon.smithi176.log.gz 2024-03-13T15:11:23.109 INFO:teuthology.orchestra.run.smithi196.stderr: 94.7% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi196.wabhcu.log.gz 2024-03-13T15:11:24.364 INFO:teuthology.orchestra.run.smithi176.stderr: 93.2% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.1.log.gz 2024-03-13T15:11:24.584 INFO:teuthology.orchestra.run.smithi176.stderr: 93.6% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.2.log.gz 2024-03-13T15:11:24.748 INFO:teuthology.orchestra.run.smithi196.stderr: 93.2% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.4.log.gz 2024-03-13T15:11:25.518 INFO:teuthology.orchestra.run.smithi176.stderr: 93.3% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.0.log.gz 2024-03-13T15:11:25.661 INFO:teuthology.orchestra.run.smithi176.stderr: 94.7% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi176.ldskqt.log.gz 2024-03-13T15:11:26.465 INFO:teuthology.orchestra.run.smithi196.stderr: 93.1% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.5.log.gz 2024-03-13T15:11:27.443 INFO:teuthology.orchestra.run.smithi196.stderr: 93.4% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-osd.3.log.gz 2024-03-13T15:11:30.905 INFO:teuthology.orchestra.run.smithi176.stderr: 94.7% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi176.hjrgov.log.gz 2024-03-13T15:11:31.934 INFO:teuthology.orchestra.run.smithi196.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-03-13T15:11:31.934 INFO:teuthology.orchestra.run.smithi196.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-03-13T15:11:34.345 INFO:teuthology.orchestra.run.smithi176.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-03-13T15:11:34.348 INFO:teuthology.orchestra.run.smithi176.stderr: 2024-03-13T15:11:34.349 INFO:teuthology.orchestra.run.smithi176.stderr:real 0m17.370s 2024-03-13T15:11:34.349 INFO:teuthology.orchestra.run.smithi176.stderr:user 1m0.306s 2024-03-13T15:11:34.349 INFO:teuthology.orchestra.run.smithi176.stderr:sys 0m4.651s 2024-03-13T15:12:48.032 INFO:teuthology.orchestra.run.smithi196.stderr: 92.8% -- replaced with /var/log/ceph/f60e35da-e146-11ee-95c9-87774f69a715/ceph-mds.cephfs.smithi196.tcitrk.log.gz 2024-03-13T15:12:48.035 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-03-13T15:12:48.035 INFO:teuthology.orchestra.run.smithi196.stderr:real 1m31.035s 2024-03-13T15:12:48.035 INFO:teuthology.orchestra.run.smithi196.stderr:user 2m10.822s 2024-03-13T15:12:48.035 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m9.239s 2024-03-13T15:12:48.037 INFO:tasks.cephadm:Archiving logs... 2024-03-13T15:12:48.037 DEBUG:teuthology.misc:Transferring archived files from smithi176:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645/remote/smithi176/log 2024-03-13T15:12:48.039 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-13T15:12:55.583 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645/remote/smithi196/log 2024-03-13T15:12:55.584 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-13T15:13:15.200 INFO:tasks.cephadm:Removing cluster... 2024-03-13T15:13:15.200 DEBUG:teuthology.orchestra.run.smithi176:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f60e35da-e146-11ee-95c9-87774f69a715 --force 2024-03-13T15:13:15.474 INFO:teuthology.orchestra.run.smithi176.stdout:Deleting cluster with fsid: f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T15:13:16.246 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f60e35da-e146-11ee-95c9-87774f69a715 --force 2024-03-13T15:13:16.522 INFO:teuthology.orchestra.run.smithi196.stdout:Deleting cluster with fsid: f60e35da-e146-11ee-95c9-87774f69a715 2024-03-13T15:13:17.512 INFO:tasks.cephadm:Removing cephadm ... 2024-03-13T15:13:17.513 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-13T15:13:17.570 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-13T15:13:17.623 INFO:tasks.cephadm:Teardown complete 2024-03-13T15:13:17.624 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-13T15:13:17.639 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-13T15:13:17.640 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-13T15:13:17.640 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-13T15:13:17.648 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-13T15:13:17.795 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-13T15:13:17.806 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-13T15:13:17.807 DEBUG:teuthology.orchestra.run.smithi176:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T15:13:17.814 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-13T15:13:17.860 INFO:teuthology.orchestra.run.smithi176.stderr:bash: line 1: ntpq: command not found 2024-03-13T15:13:17.868 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-03-13T15:13:17.870 INFO:teuthology.orchestra.run.smithi176.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T15:13:17.870 INFO:teuthology.orchestra.run.smithi176.stdout:=============================================================================== 2024-03-13T15:13:17.870 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 34 +2339us[+2339us] +/- 101ms 2024-03-13T15:13:17.870 INFO:teuthology.orchestra.run.smithi176.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -1496us[-1494us] +/- 59ms 2024-03-13T15:13:17.870 INFO:teuthology.orchestra.run.smithi176.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +274us[ +276us] +/- 75ms 2024-03-13T15:13:17.870 INFO:teuthology.orchestra.run.smithi176.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T15:13:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-13T15:13:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-03-13T15:13:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +2324us[+2315us] +/- 101ms 2024-03-13T15:13:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -1514us[-1523us] +/- 59ms 2024-03-13T15:13:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +245us[ +245us] +/- 75ms 2024-03-13T15:13:17.878 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-13T15:13:17.879 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-13T15:13:17.890 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-13T15:13:17.891 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-13T15:13:17.903 DEBUG:teuthology.orchestra.run.smithi176:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-13T15:13:18.123 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-13T15:13:18.355 DEBUG:teuthology.orchestra.run.smithi176:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T15:13:18.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T15:13:18.452 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-13T15:13:18.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T15:13:18.556 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-13T15:13:18.569 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-13T15:13:18.581 INFO:teuthology.task.internal:Duration was 2795.295641 seconds 2024-03-13T15:13:18.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-13T15:13:18.598 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-13T15:13:18.598 DEBUG:teuthology.orchestra.run.smithi176:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-13T15:13:18.604 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-13T15:13:18.744 INFO:teuthology.orchestra.run.smithi176.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T15:13:18.770 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-13T15:13:19.203 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-13T15:13:19.203 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi176.front.sepia.ceph.com 2024-03-13T15:13:19.203 DEBUG:teuthology.orchestra.run.smithi176:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-13T15:13:19.259 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-03-13T15:13:19.259 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-13T15:13:19.337 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-13T15:13:19.337 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-13T15:13:19.345 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-13T15:13:19.427 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-13T15:13:19.428 DEBUG:teuthology.orchestra.run.smithi176:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-13T15:13:19.432 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-13T15:13:20.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-13T15:13:20.248 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-13T15:13:20.248 DEBUG:teuthology.orchestra.run.smithi176:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-13T15:13:20.335 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-13T15:13:20.433 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-13T15:13:20.445 DEBUG:teuthology.orchestra.run.smithi176:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-13T15:13:20.453 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-13T15:13:20.548 INFO:teuthology.orchestra.run.smithi176.stdout:kernel.core_pattern = core 2024-03-13T15:13:20.564 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-03-13T15:13:20.644 DEBUG:teuthology.orchestra.run.smithi176:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-13T15:13:20.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T15:13:20.699 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-13T15:13:20.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T15:13:20.754 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-13T15:13:20.767 INFO:teuthology.task.internal:Transferring archived files... 2024-03-13T15:13:20.768 DEBUG:teuthology.misc:Transferring archived files from smithi176:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645/remote/smithi176 2024-03-13T15:13:20.769 DEBUG:teuthology.orchestra.run.smithi176:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-13T15:13:20.894 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645/remote/smithi196 2024-03-13T15:13:20.895 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-13T15:13:21.007 INFO:teuthology.task.internal:Removing archive directory... 2024-03-13T15:13:21.007 DEBUG:teuthology.orchestra.run.smithi176:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-13T15:13:21.015 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-13T15:13:21.071 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-13T15:13:21.087 INFO:teuthology.task.internal:Not uploading archives. 2024-03-13T15:13:21.087 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-13T15:13:21.098 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-13T15:13:21.099 DEBUG:teuthology.orchestra.run.smithi176:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-13T15:13:21.106 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-13T15:13:21.155 INFO:teuthology.orchestra.run.smithi176.stdout: 393249 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Mar 13 15:13 /home/ubuntu/cephtest 2024-03-13T15:13:21.156 INFO:teuthology.orchestra.run.smithi176.stdout: 681386 4 d--------- 2 ubuntu ubuntu 4096 Mar 13 14:46 /home/ubuntu/cephtest/mnt.0 2024-03-13T15:13:21.156 INFO:teuthology.orchestra.run.smithi176.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-03-13T15:13:21.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-13T15:13:21.159 INFO:teuthology.orchestra.run.smithi176.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-03-13T15:13:21.159 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_9a75c81fd074f3cb252248b860d5cc88e5f3fb55/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi176 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-03-13T15:13:21.160 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-13T15:13:21.173 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-13T15:13:21.186 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-03-13T15:13:21.223 INFO:teuthology.nuke:Checking targets against current locks 2024-03-13T15:13:21.245 DEBUG:teuthology.nuke:shortname: smithi176 2024-03-13T15:13:21.245 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-13T15:13:21.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi176.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 14:17:55.269420', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJYixE/TymHz03c2MtipOqrDOT/TE1wqB3Z88lRmj6FOWijYZm6wZF6ZbWYc2BzehhIX/LdXGmQ6uX7b3J+Hq/0='} 2024-03-13T15:13:21.276 DEBUG:teuthology.nuke:shortname: smithi196 2024-03-13T15:13:21.276 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-13T15:13:21.301 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-13_13:59:32-fs-wip-vshankar-testing-20240307.013758-testing-default-smithi/7596645', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-13 14:17:55.271188', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6dkZT6B+RIDW36bsfZxql7s/g7tLqsKK84frAp/Kbp1jAMQfqcwGneZIsKaMmaG9O225hbvZCTcpVJz9sEoWU='} 2024-03-13T15:13:21.319 INFO:teuthology.orchestra.console.smithi176:Power off 2024-03-13T15:13:21.320 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T15:13:21.347 DEBUG:teuthology.orchestra.console.smithi176:power off output: Chassis Power Control: Down/Off 2024-03-13T15:13:21.347 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T15:13:21.363 INFO:teuthology.orchestra.console.smithi196:Power off 2024-03-13T15:13:21.363 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-13T15:13:21.374 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-03-13T15:13:21.385 DEBUG:teuthology.orchestra.console.smithi196:power off output: Chassis Power Control: Down/Off 2024-03-13T15:13:21.385 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T15:13:21.408 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-13T15:13:25.376 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T15:13:25.409 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T15:13:25.486 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-03-13T15:13:25.519 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is on 2024-03-13T15:13:29.490 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T15:13:29.520 DEBUG:teuthology.orchestra.console.smithi196:pexpect command: ipmitool -H smithi196.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T15:13:29.602 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is on 2024-03-13T15:13:29.630 DEBUG:teuthology.orchestra.console.smithi196:check power output: Chassis Power is off 2024-03-13T15:13:29.730 INFO:teuthology.orchestra.console.smithi196:Power off completed 2024-03-13T15:13:33.606 DEBUG:teuthology.orchestra.console.smithi176:pexpect command: ipmitool -H smithi176.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-13T15:13:33.717 DEBUG:teuthology.orchestra.console.smithi176:check power output: Chassis Power is off 2024-03-13T15:13:33.818 INFO:teuthology.orchestra.console.smithi176:Power off completed 2024-03-13T15:13:33.923 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.1} 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2795.2956414222717 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default host.a-kernel-sha1: a87fde6e633694345631fcca2fd8a5f190eef61c host.b-kernel-sha1: a87fde6e633694345631fcca2fd8a5f190eef61c owner: scheduled_vshankar@teuthology status: fail success: false 2024-03-13T15:13:33.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-13T15:13:34.005 INFO:teuthology.run:FAIL